fix(context): 不压 assistant tool_call 参数,断 run_python 投毒空转
旧 assistant tool_call.arguments(>800 字符)被压成 {"_compacted":...} marker 发给
LLM,模型在长 doc/ppt 任务里反复看到后仿写它当真实参数 → run_python 拿不到
code/script_path 报错空转(DB 实测最近 60 个 task 命中 83 次,其中 61 次是模型仿写
marker)。把原本只给 task_progress 的豁免升级成通用规则:删 _compact_assistant_tool_calls
/ _compact_tool_call_arguments,只压 tool 结果 + skill,assistant 参数一律原样保留。
附诊断脚本 scripts/diag_run_python_empty.py / diag_run_python_trace.py;全量 120 tests OK。
bump 0.10.0 -> 0.10.1
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
ef611b0666
commit
81da2f6f55
|
|
@ -21,6 +21,13 @@
|
|||
|
||||
## 已完成关键能力
|
||||
|
||||
### 2026-06-12(傍晚)修上下文压缩投毒 → run_python 空转报错
|
||||
|
||||
- **根因(DB 实测,60 个 task 命中 83 次 `[Error] bad arguments to run_python: code or script_path must be provided`)**:`core/context.py` 把旧 assistant `tool_call.arguments`(>800 字符)压成 `{"_compacted":true,"original_chars":N,"note":...}` marker 发给 LLM。模型在长 doc/ppt 任务里看到几十次"过去的 run_python 长这样",就**照葫芦画瓢把 marker 当真实参数原样吐出来** → executor 拿不到 code/script_path → 报错空转。83 次里 **61 次是模型仿写 marker**(铁证:抓到 `{"_compacted":true,"original_chars":85}`——85<800 压缩器根本不会出手、且缺 `note` 字段,压缩器必带 → 只能是模型伪造),22 次是真·空 `{}`。这正是代码里早已为 `task_progress` 单独豁免、注释明写"会毒化模型"的同一个坑,只是 run_python 没豁免。
|
||||
- **修复(方案 A,把 task_progress 特例升级成通用规则)**:删掉 `_compact_assistant_tool_calls` / `_compact_tool_call_arguments`,`prepare_messages_with_stats` 不再压任何 assistant tool_call 参数(去掉 `old_tool_arg_chars` 形参与 `compacted_tool_call_arguments` 统计)。**只压 tool 结果 + skill(省 token 的大头)**,参数原样留 = 模型看到的范本永远是真实可执行调用,投毒向量连根拔。代价仅个别一次性大参数(如 12KB pptx 脚本)留在历史 1 条消息,不随轮数翻倍。
|
||||
- 诊断脚本落盘可复用:`scripts/diag_run_python_empty.py`(扫最近 task 的报错形态分桶)、`scripts/diag_run_python_trace.py`(回溯每条报错配对的 assistant 参数)。
|
||||
- 验证:`tests/test_context_compaction.py` 改 2 条旧"压参数"断言为"原样保留"+ 去除已删统计键;全量 120 tests OK。bump 0.10.0 → 0.10.1。
|
||||
|
||||
### 2026-06-12
|
||||
|
||||
- **admin 管理后台(角色鉴权 + 独立监控页,可扩展为管理动作总入口)**:此前只有共享口令 `ZCBOT_ADMIN_TOKEN`(仅用于发用户),无"管理员角色"概念,运维指标只打 stdout(`[stats]`)无界面。本次落地按角色的 admin 区:① **schema**:`users` 加 `role` 列(`user`/`admin`,`server_default='user'`,migration 0009 只加列不动现有数据);② **鉴权**:`make_require_admin(cfg)` 先验 JWT(同 `require_user`)再查 `users.role=='admin'`,否则 403——**role 走 DB 查不进 JWT**,改完下次请求即时生效、老 token 不重签;③ **端点**:`web/admin.py` 的 `register_admin_routes` 挂 `GET /v1/admin/overview`(整组 `Depends(require_admin)`),一次返回 runtime(active_runs/max_workers/sse_subs/rss_peak,读 app.state,与 `_stats_logger` 同源)/ tasks(按 status+run_status 计数)/ users(总数+近7d活跃)/ usage(全局总用量+近7d按天+按模型)/ storage(各用户 bytes/file_count+配额)五段,全 GROUP BY 无 N+1;另挂 `GET /v1/admin/usage/users?page=&page_size=` 分页返**各用户 token 用量**(全表 LEFT JOIN usage_events 含零用量用户,cost desc,稳定排序兜底 user_id;cost 全 kind、token/缓存命中仅 chat,与总用量同源)——前端独立翻页、不随 overview 轮询丢页码;④ **前端**:独立单页 `web/static/admin.html`+`js/admin.js`(复用 localStorage `zcbot.token` 与 format 工具,不挂主应用模块图),纯数字卡片+表格不画图、**阈值/热力色差**(active_runs 逼近 max_workers 变橙/红、磁盘按配额占比变色、cost 列相对热力底色)、**响应式**(窄屏竖排)、默 10s 轮询(切后台暂停);401/403 给明确提示+回控制台链接;⑤ **入口**:`/v1/me` 返 `{user_id, role}`,dev SPA `enterApp` 拉一次,admin 才显顶栏"管理"链接(`/static/admin.html`);⑥ **建用户带 role**:`POST /v1/auth/admin/create_user` + 登录页弹框加角色下拉,`main.py user add --role` / 新增 `main.py user role --email X --role admin` 改角色。**命名取舍**:先按 inspect/dashboard 摇摆,最终定 **admin**——这页会长出建用户/改角色/配置(磁盘配额等)管理动作,admin 既盖"看"又盖"管"、且与 `require_admin`/`role='admin'`/`/v1/auth/admin/*` 一脉相承;监控总览只是其第一个 tab,后续在 `web/admin.py` 续挂 `/v1/admin/users`、`/v1/admin/config`。已用 TestClient 验:admin→200、非 admin→403、无 token→401;五段聚合对真实数据跑通。
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
# zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。
|
||||
# 改版本只动这一行。
|
||||
__version__ = "0.9.0"
|
||||
__version__ = "0.10.1"
|
||||
|
|
|
|||
|
|
@ -1,7 +1,11 @@
|
|||
"""LLM 上下文准备。
|
||||
|
||||
不改 Session 持久化历史,只在发给模型前做低风险压缩。第一阶段只压旧 tool
|
||||
消息内容,保留 tool_call 协议字段,避免历史命令输出 / 检索结果反复占满 prompt。
|
||||
不改 Session 持久化历史,只在发给模型前做低风险压缩。只压旧 tool 消息**内容**,
|
||||
绝不动 assistant 的 `tool_call.arguments` —— arguments 是模型"该怎么调工具"的范本,
|
||||
把它改写成 `{"_compacted":...}` 这种"看着像合法调用"的标记会毒化模型:它在长任务里
|
||||
看到几十次"过去的 run_python/write 长这样",就照葫芦画瓢把 marker 当参数原样吐出来,
|
||||
executor 拿不到 code/path → 报错空转(2026-06-12 DB 实测 60 个 task 命中 83 次,
|
||||
其中 61 次是模型仿写 marker;详 PROGRESS)。故 arguments 一律原样保留。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
|
@ -45,84 +49,24 @@ def _message_chars(msg: dict[str, Any]) -> int:
|
|||
return len(str(msg))
|
||||
|
||||
|
||||
def _compact_tool_call_arguments(raw: Any, max_chars: int, tool_name: str = "") -> tuple[Any, bool]:
|
||||
# task_progress 参数本就很小(3-7 个短步骤),压缩省的 token 微乎其微,但把它换成
|
||||
# `{"_compacted":true,"step_id":...}` 这种"看起来像合法调用"的标记会:① 毒化模型,
|
||||
# 让它照葫芦画瓢生成残废的 update_step(丢了 step.status)入库;② 残废格式前端
|
||||
# applyProgressAction 读不到 args.step → 进度还原错乱。故 task_progress 一律不压缩参数。
|
||||
if tool_name == "task_progress":
|
||||
return raw, False
|
||||
if not isinstance(raw, str) or len(raw) <= max_chars:
|
||||
return raw, False
|
||||
marker: dict[str, Any] = {
|
||||
"_compacted": True,
|
||||
"original_chars": len(raw),
|
||||
"note": "old assistant tool_call arguments omitted from context",
|
||||
}
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
except Exception:
|
||||
parsed = None
|
||||
if isinstance(parsed, dict):
|
||||
for key in ("path", "script_path", "file_path", "name"):
|
||||
value = parsed.get(key)
|
||||
if isinstance(value, str) and value:
|
||||
marker[key] = value
|
||||
content = parsed.get("content")
|
||||
if isinstance(content, str):
|
||||
marker["content_chars"] = len(content)
|
||||
return json.dumps(marker, ensure_ascii=False), True
|
||||
|
||||
|
||||
def _compact_assistant_tool_calls(
|
||||
msg: dict[str, Any],
|
||||
*,
|
||||
max_arg_chars: int,
|
||||
) -> tuple[int, int]:
|
||||
tool_calls = msg.get("tool_calls")
|
||||
if not isinstance(tool_calls, list):
|
||||
return 0, 0
|
||||
compacted = 0
|
||||
saved = 0
|
||||
for tc in tool_calls:
|
||||
if not isinstance(tc, dict):
|
||||
continue
|
||||
fn = tc.get("function")
|
||||
if not isinstance(fn, dict):
|
||||
continue
|
||||
before = fn.get("arguments")
|
||||
tool_name = fn.get("name") if isinstance(fn.get("name"), str) else ""
|
||||
after, did_compact = _compact_tool_call_arguments(
|
||||
before,
|
||||
max_chars=max(0, max_arg_chars),
|
||||
tool_name=tool_name,
|
||||
)
|
||||
if did_compact:
|
||||
fn["arguments"] = after
|
||||
compacted += 1
|
||||
saved += len(before) - len(after)
|
||||
return compacted, max(0, saved)
|
||||
|
||||
|
||||
def prepare_messages_for_llm(
|
||||
messages: List[dict[str, Any]],
|
||||
*,
|
||||
keep_recent: int = 12,
|
||||
old_tool_chars: int = 2_000,
|
||||
old_tool_arg_chars: int = 800,
|
||||
compact_threshold_chars: int = 0,
|
||||
) -> List[dict[str, Any]]:
|
||||
"""返回发给 LLM 的 messages 副本。
|
||||
|
||||
- system 和最近 keep_recent 条消息原样保留。
|
||||
- 较旧且过长的 tool content 压缩为头尾摘要。
|
||||
- assistant 的 tool_call.arguments 一律原样保留(改写会毒化模型,见模块注释)。
|
||||
- role/tool_call_id/name 等协议字段不变。
|
||||
"""
|
||||
prepared, _ = prepare_messages_with_stats(
|
||||
messages,
|
||||
keep_recent=keep_recent,
|
||||
old_tool_chars=old_tool_chars,
|
||||
old_tool_arg_chars=old_tool_arg_chars,
|
||||
compact_threshold_chars=compact_threshold_chars,
|
||||
)
|
||||
return prepared
|
||||
|
|
@ -133,7 +77,6 @@ def prepare_messages_with_stats(
|
|||
*,
|
||||
keep_recent: int = 12,
|
||||
old_tool_chars: int = 2_000,
|
||||
old_tool_arg_chars: int = 800,
|
||||
compact_threshold_chars: int = 0,
|
||||
) -> tuple[List[dict[str, Any]], dict[str, int]]:
|
||||
"""返回发给 LLM 的 messages 副本和压缩统计。
|
||||
|
|
@ -155,7 +98,6 @@ def prepare_messages_with_stats(
|
|||
"saved_chars": 0,
|
||||
"compacted_tool_messages": 0,
|
||||
"compacted_skill_messages": 0,
|
||||
"compacted_tool_call_arguments": 0,
|
||||
"compaction_skipped": 1,
|
||||
}
|
||||
return prepared, stats
|
||||
|
|
@ -164,16 +106,10 @@ def prepare_messages_with_stats(
|
|||
prepared: List[dict[str, Any]] = []
|
||||
compacted_tool_messages = 0
|
||||
compacted_skill_messages = 0
|
||||
compacted_tool_call_arguments = 0
|
||||
for idx, msg in enumerate(messages):
|
||||
new_msg = deepcopy(msg)
|
||||
is_recent = idx >= recent_start
|
||||
if not is_recent and new_msg.get("role") == "assistant":
|
||||
n_args, _ = _compact_assistant_tool_calls(
|
||||
new_msg,
|
||||
max_arg_chars=old_tool_arg_chars,
|
||||
)
|
||||
compacted_tool_call_arguments += n_args
|
||||
# assistant 的 tool_call.arguments 一律原样保留 —— 压成 marker 会毒化模型(见模块注释)。
|
||||
if (
|
||||
not is_recent
|
||||
and new_msg.get("role") == "tool"
|
||||
|
|
@ -199,7 +135,6 @@ def prepare_messages_with_stats(
|
|||
"saved_chars": max(0, original_chars - sent_chars),
|
||||
"compacted_tool_messages": compacted_tool_messages,
|
||||
"compacted_skill_messages": compacted_skill_messages,
|
||||
"compacted_tool_call_arguments": compacted_tool_call_arguments,
|
||||
"compaction_skipped": 0,
|
||||
}
|
||||
return prepared, stats
|
||||
|
|
|
|||
|
|
@ -0,0 +1,79 @@
|
|||
"""扫最近的 task,定位「bad arguments to run_python: code or script_path must be
|
||||
provided」到底什么时候真正触发。
|
||||
|
||||
两条线:
|
||||
A. 直接在 tool-result 消息里搜这句错误 —— 这是运行时真的报了的铁证。
|
||||
B. 看产生它的那条 assistant run_python 调用,arguments 到底长啥样。
|
||||
排除 `_compacted`(那是入库后上下文压缩留下的历史占位,运行时是有 code 的,不算)。
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
env = Path(__file__).resolve().parent.parent / ".env"
|
||||
for line in env.read_text(encoding="utf-8").splitlines():
|
||||
if line.strip().startswith("ZCBOT_DB_URL="):
|
||||
os.environ["ZCBOT_DB_URL"] = line.split("=", 1)[1].strip()
|
||||
|
||||
from sqlalchemy import create_engine, text # noqa: E402
|
||||
|
||||
engine = create_engine(os.environ["ZCBOT_DB_URL"])
|
||||
ERR = "bad arguments to run_python: code or script_path must be provided"
|
||||
|
||||
with engine.connect() as conn:
|
||||
tasks = conn.execute(
|
||||
text("select task_id, created_at from tasks order by created_at desc limit 60")
|
||||
).fetchall()
|
||||
|
||||
per_task = Counter()
|
||||
shapes = Counter()
|
||||
samples = []
|
||||
for tid, created in tasks:
|
||||
msgs = conn.execute(
|
||||
text("select idx, payload from messages where task_id=:t order by idx"),
|
||||
{"t": tid},
|
||||
).fetchall()
|
||||
# 建 tool_call_id -> arguments 映射(看错误对应的调用 args)
|
||||
call_args = {}
|
||||
for idx, payload in msgs:
|
||||
if payload.get("role") == "assistant":
|
||||
for tc in payload.get("tool_calls") or []:
|
||||
call_args[tc.get("id")] = (tc.get("function") or {}).get("arguments")
|
||||
for idx, payload in msgs:
|
||||
if payload.get("role") != "tool":
|
||||
continue
|
||||
content = payload.get("content") or ""
|
||||
if isinstance(content, list):
|
||||
content = json.dumps(content, ensure_ascii=False)
|
||||
if ERR not in content:
|
||||
continue
|
||||
per_task[(str(tid)[:8], str(created)[:16])] += 1
|
||||
raw = call_args.get(payload.get("tool_call_id"))
|
||||
# 归类 args 形态
|
||||
try:
|
||||
args = json.loads(raw) if raw else {}
|
||||
except Exception:
|
||||
shape = "MANGLED(非法JSON)"
|
||||
else:
|
||||
if args == {}:
|
||||
shape = "空 {}"
|
||||
elif "_compacted" in args:
|
||||
shape = "_compacted(历史占位)"
|
||||
else:
|
||||
shape = "其他: " + repr(raw)[:80]
|
||||
shapes[shape] += 1
|
||||
if len(samples) < 25:
|
||||
samples.append((str(tid)[:8], idx, shape, repr(raw)[:140]))
|
||||
|
||||
print(f"扫了最近 {len(tasks)} 个 task")
|
||||
print(f"真正触发该错误的 tool-result 条数: {sum(per_task.values())}\n")
|
||||
print("=== 按 task 分布(task / 创建时间 / 次数)===")
|
||||
for (t, c), n in per_task.most_common():
|
||||
print(f" {t} {c} -> {n} 次")
|
||||
print("\n=== 触发时 run_python 的 arguments 形态 ===")
|
||||
for s, n in shapes.most_common():
|
||||
print(f" {n:>3}x {s}")
|
||||
print("\n=== 样本 ===")
|
||||
for t, idx, shape, raw in samples:
|
||||
print(f" [{t} #{idx}] {shape}: {raw}")
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
"""对某 task:列出每条 run_python 报错的 tool-result,并回溯它配对的 assistant
|
||||
tool_call 的 arguments(按 tool_call_id),判断报错那一刻 DB 里存的 args 是
|
||||
真实 code / 空{} / 还是 _compacted 占位。"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
env = Path(__file__).resolve().parent.parent / ".env"
|
||||
for line in env.read_text(encoding="utf-8").splitlines():
|
||||
if line.strip().startswith("ZCBOT_DB_URL="):
|
||||
os.environ["ZCBOT_DB_URL"] = line.split("=", 1)[1].strip()
|
||||
|
||||
from sqlalchemy import create_engine, text # noqa: E402
|
||||
|
||||
engine = create_engine(os.environ["ZCBOT_DB_URL"])
|
||||
prefix = sys.argv[1] if len(sys.argv) > 1 else "9956b139"
|
||||
ERR = "code or script_path must be provided"
|
||||
|
||||
with engine.connect() as conn:
|
||||
tid = conn.execute(
|
||||
text("select task_id from tasks where task_id::text like :p"),
|
||||
{"p": prefix + "%"},
|
||||
).fetchone()[0]
|
||||
msgs = conn.execute(
|
||||
text("select idx, payload from messages where task_id=:t order by idx"),
|
||||
{"t": tid},
|
||||
).fetchall()
|
||||
|
||||
# id -> (assist_idx, name, raw_args)
|
||||
by_id = {}
|
||||
for idx, payload in msgs:
|
||||
if payload.get("role") == "assistant":
|
||||
for tc in payload.get("tool_calls") or []:
|
||||
fn = tc.get("function") or {}
|
||||
by_id[tc.get("id")] = (idx, fn.get("name"), fn.get("arguments"))
|
||||
|
||||
print(f"task {tid}\n")
|
||||
n = 0
|
||||
for idx, payload in msgs:
|
||||
if payload.get("role") != "tool":
|
||||
continue
|
||||
content = payload.get("content") or ""
|
||||
if isinstance(content, list):
|
||||
content = json.dumps(content, ensure_ascii=False)
|
||||
if ERR not in content:
|
||||
continue
|
||||
n += 1
|
||||
tcid = payload.get("tool_call_id")
|
||||
src = by_id.get(tcid)
|
||||
if src is None:
|
||||
print(f"[err #{idx}] tcid={tcid} -> 找不到配对的 assistant 调用!")
|
||||
continue
|
||||
a_idx, name, raw = src
|
||||
print(f"[err #{idx}] <- assist #{a_idx} {name} : {repr(raw)[:110]}")
|
||||
print(f"\n共 {n} 条报错")
|
||||
|
|
@ -96,28 +96,9 @@ class ContextCompactionTests(unittest.TestCase):
|
|||
self.assertGreater(stats["saved_chars"], 0)
|
||||
self.assertEqual(len(prepared), len(messages))
|
||||
|
||||
def test_defaults_compact_medium_sized_old_write_arguments(self) -> None:
|
||||
args = json.dumps({"path": "slides/p01.py", "content": "A" * 1000})
|
||||
messages = [
|
||||
{"role": "system", "content": "rules"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [{
|
||||
"id": "tc1",
|
||||
"type": "function",
|
||||
"function": {"name": "write", "arguments": args},
|
||||
}],
|
||||
},
|
||||
] + [{"role": "user", "content": f"recent {i}"} for i in range(12)]
|
||||
|
||||
prepared, stats = prepare_messages_with_stats(messages)
|
||||
|
||||
compacted_args = json.loads(prepared[1]["tool_calls"][0]["function"]["arguments"])
|
||||
self.assertTrue(compacted_args["_compacted"])
|
||||
self.assertEqual(compacted_args["path"], "slides/p01.py")
|
||||
self.assertEqual(stats["compacted_tool_call_arguments"], 1)
|
||||
|
||||
def test_compacts_old_assistant_tool_call_arguments(self) -> None:
|
||||
def test_keeps_old_large_tool_call_arguments_verbatim(self) -> None:
|
||||
# 旧 assistant tool_call.arguments 一律原样保留,哪怕很大 —— 改写成 `{"_compacted":...}`
|
||||
# marker 会被模型仿写成参数(2026-06-12 DB 实测:run_python 因此空转报错 60+ 次)。
|
||||
args = json.dumps({"path": "slides/p01.py", "content": "A" * 5000})
|
||||
messages = [
|
||||
{"role": "system", "content": "rules"},
|
||||
|
|
@ -134,25 +115,16 @@ class ContextCompactionTests(unittest.TestCase):
|
|||
{"role": "user", "content": "next"},
|
||||
]
|
||||
|
||||
prepared, stats = prepare_messages_with_stats(
|
||||
messages,
|
||||
keep_recent=1,
|
||||
old_tool_arg_chars=200,
|
||||
)
|
||||
prepared, stats = prepare_messages_with_stats(messages, keep_recent=1)
|
||||
tc = prepared[1]["tool_calls"][0]
|
||||
compacted_args = json.loads(tc["function"]["arguments"])
|
||||
|
||||
self.assertEqual(tc["id"], "tc1")
|
||||
self.assertEqual(tc["type"], "function")
|
||||
self.assertEqual(tc["function"]["name"], "write")
|
||||
self.assertTrue(compacted_args["_compacted"])
|
||||
self.assertEqual(compacted_args["path"], "slides/p01.py")
|
||||
self.assertNotIn("A" * 100, tc["function"]["arguments"])
|
||||
self.assertEqual(stats["compacted_tool_call_arguments"], 1)
|
||||
# 协议字段 + 完整参数都原样保留,marker 永不出现。
|
||||
self.assertEqual(tc["function"]["arguments"], args)
|
||||
self.assertNotIn("_compacted", tc["function"]["arguments"])
|
||||
self.assertNotIn("compacted_tool_call_arguments", stats)
|
||||
|
||||
def test_keeps_old_task_progress_arguments_intact(self) -> None:
|
||||
# task_progress 参数本就很小,且压成 `{"_compacted":...,"step_id":...}` 这种"像合法调用"
|
||||
# 的标记会毒化模型 + 毁掉前端进度还原。故旧的 task_progress 调用参数必须原样保留。
|
||||
# task_progress 参数本就很小,压成 marker 还会毁掉前端进度还原。和所有工具一样原样保留。
|
||||
args = json.dumps({
|
||||
"action": "set_plan",
|
||||
"steps": [
|
||||
|
|
@ -179,7 +151,7 @@ class ContextCompactionTests(unittest.TestCase):
|
|||
self.assertNotIn("_compacted", kept_args)
|
||||
self.assertEqual(kept_args["action"], "set_plan")
|
||||
self.assertEqual(kept_args["steps"][0]["title"], "理解需求")
|
||||
self.assertEqual(stats["compacted_tool_call_arguments"], 0)
|
||||
self.assertNotIn("compacted_tool_call_arguments", stats)
|
||||
|
||||
def test_old_task_progress_tool_result_uses_tiny_marker(self) -> None:
|
||||
messages = [
|
||||
|
|
|
|||
Loading…
Reference in New Issue