策划审批的两个 GUI 入口等过项目锁瞬时争用,不再把成功的审批画成失败

线上症状:点「批准」后卡里弹出 `项目正在被其他写操作占用`,但 agent.db 显示
`plan.gdd_decided action=approve` 已落盘、approvals/v1.json 已写、index 里
approvedVersion=1、run 随后正常收束,且没有任何 gdd_projection_gap ——
审批本身是成功的,报错来自它之后那一次刷新。

成因是决定放行 run 续跑,而 GUI 在 decide 返回后紧接着 hydrate:
`decide → await hydratePlanGddState()` 与 runner 的 `agent.schedule_ready`、
会话写入、planning 投影同时伸手拿 `.agent/project.lock`。hydrate 那一支是
一次性取锁、不重试,撞上就返回 PLAN_STORAGE_IO,前端原样贴进审批卡。

仓库里本来就有这个约定:project_gates 的 `..._with_wait` 只认
`项目正在被其他写操作占用:` 这一个前缀并退避重试,运行时侧 18 个点都在用。
策划的两个 GUI 入口是漏网的。按「丢了这一次要付什么代价」分档接上:

- planning.gdd-decision 是一次性用户意图,丢了用户得重新找到卡片 → 等满窗口。
- planning.hydrate 是轮询刷新,下一拍还会来 → 只等 1 秒(新增 short wait 档),
  免得为一次刷新把面板卡住整个窗口。

前端再兜一层:hydrate 撞上这条错误时保留上一份状态、不写错误位。后端等过短
窗口仍拿不到,只说明此刻运行时正在写盘;这条 effect 每次监工状态变化都会重跑。
用 includes 而不是 startsWith,因为 hydrate 的错误带 `PLAN_STORAGE_IO: ` 前缀。

两条新测试都做过 A/B(换回一次性取锁即失败):
- decision_rides_out_a_briefly_held_project_lock
- hydrate_rides_out_a_briefly_held_project_lock
锁由后台线程持有 120ms 再释放;pid 与 createdAt 都写真值,否则会被失效锁回收
顺手删掉、根本占不住。

hydrate 原有的争用用例(考脱敏与错误码形状)保持红,只更新了那条已失效的
「这一支不重试」注释。

做游戏链路未触及:改的两个函数都是策划专属入口,共用的
`acquire_project_write_lock` 语义一行未动。

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-23 05:13:11 +00:00
parent 07ab58d2b9
commit 1337ce03e1
5 changed files with 169 additions and 15 deletions
@@ -1779,18 +1779,31 @@ pub(crate) fn project_verification_completion_blocker_at(
project_verification_completion_blocker_at_locked(root, agent_id, run_id, observations)
}
pub(crate) fn acquire_game_creator_agent_runtime_project_write_lock_with_wait(
const AGENT_RUNTIME_PROJECT_WRITE_LOCK_RETRY_INTERVAL: Duration = Duration::from_millis(5);
const AGENT_RUNTIME_PROJECT_WRITE_LOCK_WAIT_ATTEMPTS: usize = 2_000;
const AGENT_RUNTIME_PROJECT_WRITE_LOCK_SHORT_WAIT_ATTEMPTS: usize = 200;
/// Take the project write lock, riding out transient contention for at most
/// `max_attempts` polls.
///
/// `项目正在被其他写操作占用:` is the one lock error that means
/// "nothing is broken, the current holder is mid-write" — every other variant
/// (a torn lock file, a denied path) is returned immediately. Callers pick the
/// budget from what a lost race costs them: a one-shot user intent waits out the
/// full window, a poll that will run again shortly waits far less.
fn acquire_game_creator_agent_runtime_project_write_lock_within(
root: &Path,
command_id: &str,
max_attempts: usize,
) -> Result<ProjectWriteLock, String> {
const MAX_ATTEMPTS: usize = 2_000;
for attempt in 0..MAX_ATTEMPTS {
let max_attempts = max_attempts.max(1);
for attempt in 0..max_attempts {
match acquire_project_write_lock(root, command_id) {
Err(error)
if error.starts_with("项目正在被其他写操作占用:")
&& attempt + 1 < MAX_ATTEMPTS =>
&& attempt + 1 < max_attempts =>
{
std::thread::sleep(Duration::from_millis(5));
std::thread::sleep(AGENT_RUNTIME_PROJECT_WRITE_LOCK_RETRY_INTERVAL);
}
result => return result,
}
@@ -1798,6 +1811,32 @@ pub(crate) fn acquire_game_creator_agent_runtime_project_write_lock_with_wait(
unreachable!("project write lock retry loop always returns")
}
pub(crate) fn acquire_game_creator_agent_runtime_project_write_lock_with_wait(
root: &Path,
command_id: &str,
) -> Result<ProjectWriteLock, String> {
acquire_game_creator_agent_runtime_project_write_lock_within(
root,
command_id,
AGENT_RUNTIME_PROJECT_WRITE_LOCK_WAIT_ATTEMPTS,
)
}
/// Same wait, sized for a caller that re-runs on its own — a GUI refresh poll
/// rather than a user's one-shot decision. Blocking such a caller for the full
/// window would stall the panel it feeds; losing the race only costs it the
/// current tick.
pub(crate) fn acquire_game_creator_agent_runtime_project_write_lock_with_short_wait(
root: &Path,
command_id: &str,
) -> Result<ProjectWriteLock, String> {
acquire_game_creator_agent_runtime_project_write_lock_within(
root,
command_id,
AGENT_RUNTIME_PROJECT_WRITE_LOCK_SHORT_WAIT_ATTEMPTS,
)
}
pub(crate) fn acquire_game_creator_agent_provider_plan_project_write_lock_with_wait(
root: &Path,
command_id: &str,
@@ -1358,7 +1358,15 @@ pub(crate) fn decide_plan_gdd_at(
));
}
validate_decision_transport(input)?;
let _lock = acquire_project_write_lock(root, "planning.gdd-decision").map_err(|error| {
// A decision is a one-shot user intent: the click either lands or the user
// has to find the card again. The runner is writing concurrently for the
// whole life of the plan run, so a one-shot acquire hands the button a
// failure whenever it happens to land mid-write. Wait the full window.
let _lock = acquire_game_creator_agent_runtime_project_write_lock_with_wait(
root,
"planning.gdd-decision",
)
.map_err(|error| {
approval_error(
"PLAN_DURABILITY_FAILED",
redact_agent_runtime_project_paths(root, &error, 500),
@@ -469,13 +469,23 @@ pub(crate) fn hydrate_game_creator_plan_gdd_state_at(
// §18.3:先在同一把项目锁内只读校验所有 authority 的 projectId,之后才允许
// session/index/pending recovery 写入。这样复制到另一个项目的 sidecar 只能失败关闭,
// 不会在发现错绑前改写任何投影。
let _lock =
crate::project::acquire_project_write_lock(root, "planning.hydrate").map_err(|error| {
plan_gdd_state_error(
"PLAN_STORAGE_IO",
redact_agent_runtime_project_paths(root, &error, 500),
)
})?;
// Ride out transient contention instead of failing the caller. The GUI
// re-hydrates right after a decision lands, and the decision is exactly what
// releases the run to resume writing, so the refresh and the resumed runner
// reach for this lock at the same moment. Losing that race used to paint
// `项目正在被其他写操作占用` into the approval card of an approval that had
// already committed. The wait is the short one: this call runs again on the
// next supervisor poll, so it must never stall the panel for the full window.
let _lock = acquire_game_creator_agent_runtime_project_write_lock_with_short_wait(
root,
"planning.hydrate",
)
.map_err(|error| {
plan_gdd_state_error(
"PLAN_STORAGE_IO",
redact_agent_runtime_project_paths(root, &error, 500),
)
})?;
let gdds = read_plan_gdd_chain_locked(root)?;
let approvals = read_plan_gdd_approvals_locked(root)?;
let session_read_only = read_plan_session_read_only_locked(root)?;
@@ -675,6 +685,41 @@ mod tests {
);
}
/// GUI 在决定落盘后紧接着重灌卡片,而决定本身正是放行 run 继续写盘的那一下——
/// 刷新和续跑的 runner 会同时伸手拿同一把项目锁。这条断言的是短窗口内的争用要被
/// 等过去:否则一次刚成功的审批会在卡里显示成 `项目正在被其他写操作占用`。
#[test]
fn hydrate_rides_out_a_briefly_held_project_lock() {
let temporary = tempfile::tempdir().expect("create hydrate wait fixture");
let root = temporary.path().join("project");
crate::project::init_local_game_project_at(&root, "hydrate-wait", "锁等待项目")
.expect("initialize hydrate wait fixture");
// 两个字段都必须写真值,否则会被失效锁回收顺手删掉、锁根本占不住。
let lock_path = root.join(".agent/project.lock");
let held = serde_json::json!({
"commandId": "test.hold",
"pid": std::process::id(),
"createdAt": unix_timestamp(),
"nonce": 0,
});
std::fs::write(
&lock_path,
serde_json::to_vec(&held).expect("serialize held lock"),
)
.expect("hold project lock");
let holder = std::thread::spawn(move || {
std::thread::sleep(std::time::Duration::from_millis(120));
std::fs::remove_file(&lock_path).expect("release project lock");
});
let view = hydrate_game_creator_plan_gdd_state_at(&root)
.expect("hydrate 必须等过瞬时锁争用,而不是把失败画进审批卡");
holder.join().expect("lock holder thread");
assert_eq!(view.state, "not_started");
}
#[test]
fn hydrate_redacts_project_paths_from_contended_project_lock_errors() {
let temporary = tempfile::tempdir().expect("create hydrate lock fixture");
@@ -682,8 +727,9 @@ mod tests {
crate::project::init_local_game_project_at(&root, "hydrate-lock", "锁竞争项目")
.expect("initialize hydrate lock fixture");
// `.agent/project.lock` 是 `create_new(true)` 的文件锁且这一支不重试,预先占住它,
// hydrate 的第一次取锁(`reconcile` 内那次)就确定性失败。
// `.agent/project.lock` 是 `create_new(true)` 的文件锁。hydrate 现在会等一个短
// 窗口再放弃(见 `..._with_short_wait`),所以这里占住不放:等窗口耗尽,hydrate
// 的取锁确定性失败,这条用例考的是那条失败路径的脱敏与错误码形状。
//
// 两个字段都必须写真值,否则会被失效锁回收顺手删掉、锁根本占不住:
// `pid` 供 unix 侧判 owner 是否存活;`createdAt` 供年龄判定——
@@ -3794,6 +3794,60 @@ mod tests {
}
}
/// Hold `.agent/project.lock` for `hold_millis`, then release it from a
/// background thread. Both fields must carry real values or the stale-lock
/// reclaim in `acquire_project_write_lock` deletes the file on the first
/// contended attempt and nothing is actually held: `pid` is what the unix
/// liveness check reads, `createdAt` is what the age check reads (it is
/// preferred over the file mtime).
fn hold_project_lock_briefly(root: &Path, hold_millis: u64) -> std::thread::JoinHandle<()> {
let lock_path = root.join(".agent/project.lock");
let held = serde_json::json!({
"commandId": "test.hold",
"pid": std::process::id(),
"createdAt": unix_timestamp(),
"nonce": 0,
});
fs::write(
&lock_path,
serde_json::to_vec(&held).expect("serialize held lock"),
)
.expect("hold project lock");
std::thread::spawn(move || {
std::thread::sleep(std::time::Duration::from_millis(hold_millis));
fs::remove_file(&lock_path).expect("release project lock");
})
}
/// 审批按钮是一次性意图:点下去要么落盘,要么用户得重新找到卡片。批准放行后
/// runner 立刻续跑并开始写盘,所以决定命令和运行时会同时伸手去拿同一把项目锁。
/// 这里断言的是决定不能因为撞上这种瞬时争用而失败——线上症状是点了批准、审批
/// 其实成功了,卡里却弹出 `项目正在被其他写操作占用`。
#[test]
fn decision_rides_out_a_briefly_held_project_lock() {
let (root, context, input) = submit_fixture();
execute_plan_submit_gdd(&root, &context, &input).expect("submit v1");
let gdd = read_plan_gdd_chain(&root)
.expect("read submitted GDD")
.pop()
.expect("GDD exists");
create_plan_gdd_approval_pending_at(&root, &gdd).expect("create approval pending");
let decision_input = approval_input(
&gdd,
"approve",
"gdd-response-00000000-0000-4000-8000-000000000041",
None,
);
let holder = hold_project_lock_briefly(&root, 120);
let decision = decide_plan_gdd_at(&root, &decision_input)
.expect("决定必须等过瞬时锁争用,而不是把失败甩回按钮");
holder.join().expect("lock holder thread");
assert_eq!(decision.outcome, "committed");
cleanup_fixture(root);
}
#[test]
fn approval_pending_is_single_latest_unreceipted_projection() {
let (root, context, input) = submit_fixture();
+7
View File
@@ -746,7 +746,14 @@ export function App({
setPlanGddState(nextState);
}
} catch (error) {
// 项目写锁争用是瞬时的:后端已经等过一个短窗口,仍然没抢到只说明此刻
// 运行时正在写盘。这条 effect 每次监工状态变化都会再跑一次,下一拍就能
// 拿到,所以保留上一份状态、不动错误位——否则一次刚落盘成功的审批会在
// 卡里显示成失败(decide 成功后紧跟的这次 hydrate 正是最容易撞上的时刻)。
const transientContention =
String(error).includes('项目正在被其他写操作占用:');
if (
!transientContention &&
requestSequence === planGddHydrateSequenceRef.current &&
localProjectPathRef.current === targetProjectPath
) {