Compare commits

..

35 Commits

Author SHA1 Message Date
caoqianming 5b052c464b fix(web): make run cancellation respond immediately 2026-09-03 16:54:02 +08:00
caoqianming ca73a4affb release: 发布 0.71.0 2026-09-03 16:42:19 +08:00
caoqianming 09cb7d327b feat(freecad): 接入受控建模 Windows Node 2026-09-03 16:39:02 +08:00
caoqianming 4af4816aa2 fix(agent): 修复工具链恢复与产物发布 2026-09-03 16:16:20 +08:00
caoqianming 2c6091db03 feat(observability): record structured health events 2026-09-03 10:06:37 +08:00
caoqianming dbaa9faa6e perf(admin): reduce tool health polling load 2026-09-03 09:32:10 +08:00
caoqianming 2ff648afe6 feat(admin): 重构总览与分组工作区 2026-09-03 08:56:14 +08:00
caoqianming e82b6842c7 merge: integrate neutral CAD capability 2026-09-03 08:47:11 +08:00
caoqianming 6b4c54a1fe feat(cad): add neutral geometry preparation 2026-09-03 08:21:21 +08:00
caoqianming 7b541a2256 feat(sandbox): raise per-user execution capacity 2026-09-02 16:21:33 +08:00
caoqianming 11108f31c2 feat(admin): manage provider API credentials 2026-09-02 16:18:35 +08:00
caoqianming aa8ccec738 feat(web): improve Mermaid diagram styling 2026-09-02 10:13:02 +08:00
caoqianming a1068f0ec1 feat(web): optimize admin operations overview 2026-09-02 09:56:52 +08:00
caoqianming f50bfc1bcf release: 发布 0.70.0 2026-09-02 08:48:54 +08:00
caoqianming 572da9a365 feat(web): 收敛管理页并分页加载更新日志 2026-09-02 08:48:30 +08:00
caoqianming 605e4eecfe feat(sandbox): 增加共享执行容量与依赖观测 2026-09-02 08:15:31 +08:00
caoqianming e47ff1f942 fix(web): stabilize conversation rail width 2026-09-01 10:16:19 +08:00
caoqianming 7917331e5a feat(web): consolidate conversation navigation 2026-09-01 10:04:00 +08:00
caoqianming 37aa83f8fb feat(web): improve conversation side rails 2026-09-01 08:54:03 +08:00
caoqianming 38deea7d1d feat(windows-node): complete WPF host migration 2026-08-31 17:22:11 +08:00
caoqianming 0d5cbb8407 fix(llm): stream DeepSeek reasoning immediately 2026-08-31 16:03:27 +08:00
caoqianming e05de152ad fix(models): restrict DeepSeek Pro to pro tier 2026-08-31 15:40:18 +08:00
caoqianming 08754be1a4 fix(pricing): correct domestic model cost accounting 2026-08-30 11:37:12 +08:00
caoqianming 7ffa9eb645 feat(admin): consolidate dashboard overview cards 2026-08-28 17:04:07 +08:00
caoqianming 3bed132c30 feat(admin): add operations summary cards 2026-08-28 16:08:00 +08:00
caoqianming 510033d06f style(admin): compact dashboard layout 2026-08-28 15:11:18 +08:00
caoqianming 7f439a91d5 fix(web): hide idle software job indicators 2026-08-28 14:00:05 +08:00
caoqianming 0bc6d695ef feat(admin): show active Windows Node jobs 2026-08-28 13:40:44 +08:00
caoqianming a7c3ee8b16 style(admin): polish dashboard layout and visuals 2026-08-28 13:40:02 +08:00
caoqianming b6d0e34b51 fix(web): 让 embed 模式可访问管理入口 2026-08-28 13:37:15 +08:00
caoqianming bff6125cd1 fix(storage): 删除文件后释放用户配额 2026-08-28 13:23:28 +08:00
caoqianming 9728fc3dd2 feat(models): add GLM-5.3 Flash multimodal support 2026-08-27 14:55:17 +08:00
caoqianming a475207781 release: 0.69.1 2026-08-27 13:26:38 +08:00
caoqianming b0a7a86aa8 feat(web): add software result rail 2026-08-27 13:25:13 +08:00
caoqianming a219f8a0c0 fix(windows-node): 显示专业软件任务标题 2026-08-27 09:14:00 +08:00
219 changed files with 18681 additions and 3868 deletions

View File

@ -6,6 +6,44 @@
> 开发中的用户文案可先写入 `## Unreleased`;该区不会被前端解析,正式发布时再替换为数字版本和日期。
> 工程口径的完整记录见 `PROGRESS.md` / git log。
## Unreleased
- 修复深度思考或事件流重连期间点击“停止”偶尔没有反应的问题;停止信号会在状态落库前立即送达,重复停止也会可靠补发。
## 0.71.0 — 2026-09-03
- 提高工具参数异常时的自动恢复率,产物发布和工具健康分类也更加稳定准确。
- 新增 FreeCAD 受控建模:可通过参数化 Recipe 组合基础体、布尔运算、变换和 STEP 导入,交付可继续调整的 FCStd 工程、STEP、模型清单与无显卡预览并支持版本续作和失败回滚。
- 管理后台会异步汇总工具健康大数,并避免短时间内重复请求;工具异常统一按结构化事件统计,代理主动保护与真正故障分开展示,多人同时查看或自动刷新时不再反复扫描历史消息。
- 单用户可同时运行的重型任务由 2 个提升到 3 个;任务等待执行容量时,对话会直接说明是当前用户、整机或宿主内存限制,获得槽位后自动继续。
- 管理员现在可在管理后台安全录入、测试和更换模型、媒体、检索及语音服务凭据并查看来源、脱敏尾号和可用状态数据库凭据可随时删除并回退原有环境配置。DeepSeek 余额低于 30 元、额度耗尽或认证失败时会主动提醒。
- 新增中性 CAD 几何准备:可生成或检查 STEP组合基础体、截面、布尔、圆角、阵列等受控建模步骤并交付几何清单和预览结果可继续用于 ANSYS、Abaqus 等分析任务。
- Mermaid 图表采用更清晰的科研配色与更精致的图框,新生成的流程图还会按数据、处理、判断、结果等角色使用协调的语义色,减少单调的灰白图。
- 管理后台改为常驻的紧凑总览与“运行、用量、用户、资源、集成、质量”分组工作区;总览卡可直接定位到同口径详情,执行容量仍可从侧栏快速查看,桌面端和移动端都能更快进入具体管理内容。
## 0.70.0 — 2026-09-02
- 容器长任务会在整机繁忙时安全排队,后台任务可查看或取消排队状态;管理员可在“容器状态”查看实时容量,并通过“容器依赖”分析用户临时安装的 Python 依赖及占用。
- 软件作业与轮次导航统一收在对话右侧,并增加醒目的数量徽章;既可悬停速览,也可固定展开,进行中或未读作业会直接通过数字提醒。
- 模型体验进一步升级:思考过程恢复实时显示,新版 GLM 向默认档位开放,并支持直接理解上传图片和延续长工具任务的分析状态。
- 专业软件节点启用完整的新管理界面节点状态、首次注册、软件环境、本机任务、ANSYS 验收和数据目录迁移均可直接管理;应用图标、导航、状态提示和窄窗口布局也更清晰,关闭窗口后仍会继续在托盘运行。
## 0.69.1 — 2026-08-27
- 专业软件任务的进度与结果改为在当前对话左侧集中堆叠,不再用多条结果消息打断正文;可直接打开 Blender 等软件的多张预览,手机端从“软件结果”入口统一查看。
- 修复专业软件已经出图后,对话偶尔仍持续显示“思考中”的问题。
## 0.69.0 — 2026-08-26
- Origin 新增受控科研分析,可进行数据体检、归一化、平滑、求导、积分和线性拟合,并交付结果表、诊断信息及可继续检查的 Origin 工程。

View File

@ -136,7 +136,7 @@ Eval 与生产 core 解耦,通过现有 `/v1` API 创建专用任务、监听
默认 `deepseek_v4.flash`;复杂 bug / 终稿升 pro + reasoning_effort=max;fallback 手动切 Claude。成本量级:修 bug flash ~$0.01 / 完整申报书 flash ~$0.30(pro-max ~$1.5,Opus ~$10+)。99% 任务 flash 够用。
模型思考参数由 profile 统一表达:`thinking_enabled` 只表示开关,`thinking_transport` 只表示已验证的传输协议,`reasoning_effort` 只表示开启后的推理强度,`reasoning_replay` 只表示历史 reasoning 的 provider 回传策略(`none` / `tool_calls` / `all``core/llm_params.py` 是请求参数构造唯一入口,`core/context.py` 是历史消息清洗唯一入口。原始 assistant 响应仍完整落库供展示与导出,发模型前才按 profile 裁剪且上下文统计、压缩与折叠都使用裁剪后的请求视图。DeepSeek、GLM、方舟当前共享 `extra_body` 协议DeepSeek V4 仅为带工具调用的 assistant 消息保留 reasoning未验证网关明确用 `none`、不猜参数协议,主循环不再按 family 分支。`/v1/models` 只返回语义明确的 `thinking_enabled`
模型思考参数由 profile 统一表达:`thinking_enabled` 只表示开关,`thinking_transport` 只表示已验证的传输协议,`reasoning_effort` 只表示开启后的推理强度,`thinking_clear` 表示 provider 是否清除历史思考,`reasoning_replay` 表示状态生命周期(`none` / `tool_turn` / `conversation` / `provider_managed``core/llm_params.py` 是请求参数构造唯一入口,`core/context.py` 是历史消息清洗唯一入口。原始 assistant 响应完整落库provider-bound 副本只向相同生产模型回放未改写的 reasoningDeepSeek V4 仅保留当前用户轮次的工具链状态GLM-5.3 Flash 在同模型会话内保留完整状态,未验证网关明确用 `none`,未来签名/加密 block 走 `provider_managed`。模型切换、上下文折叠和普通压缩都在同一入口应用隔离;上下文统计使用裁剪后的请求视图,原生图片 token 不反向污染 chars/token 校准
---
@ -174,7 +174,7 @@ Eval 与生产 core 解耦,通过现有 `/v1` API 创建专用任务、监听
**无感部署(蓝绿,0.41)**:生产双 systemd 实例(`ZCBOT_INSTANCE=blue/green`)+ nginx upstream 切流(对外仍 8765),取代单实例 restart 的 503 窗口。双实例并存的互踩由三件事消解:`tasks.run_owner`(0020)让 reaper 只收自己色、sandbox 容器名/label 带实例色各管各的、微信长轮询 PG advisory lock 选主;定时任务 claim 本就 SKIP LOCKED 天然安全。部署编排在 `deploy/update_bluegreen.sh`(流程/兜底见 RUN)。
**broker 外置(Redis pub/sub,✅ 0.42)**:0.41 落地时 event/cancel broker 留在进程内,代价是切换窗口内"刷新看不到旧实例 run 直播 / 停止送达不到"两个边缘。2026-07-06 重评(用户量已非个位数)决定实施 —— ①部署时总有 in-flight run,窗口边缘从偶发变常态;②单进程逼近天花板后,最近的扩容手段是稳态双实例同时接流量,broker 外置是它的硬前提(否则 POST 与 SSE 落不同实例直播全瞎)。选 Redis 不选 PG LISTEN/NOTIFY(token 级 delta 全过主库太吵:NOTIFY 全局队列 + 8KB payload 限制,把实时路径耦到 PG)、不选 nginx sticky hash(upstream 变更时 hash 重排,恰在部署窗口失效,治标)。实现(`web/broker.py`,LocalRunBroker/RedisRunBroker 同接口鸭子类型):`ZCBOT_REDIS_URL` env 开关,不设即进程内(dev 零影响);event 走 `PUBLISH zcbot:ev:<tid>` + 单条 async pubsub reader 路由到本地订阅 queue(Redis 只管跨进程一跳,fan-out 最后一公里仍在进程内);done = SETEX key(60s)+ publish 双通道,订阅先挂 channel 再查 key 不漏;cancel = SETEX/GET/DEL key,loop 在 chunk 间 poll(localhost ~0.1ms)。取消另以共享 PG 中已经提交的 `tasks.run_status='cancelling'` 作可靠兜底:运行线程每秒低频读取一次,未配置 Redis 的蓝绿窗口或 Redis 运行中故障时仍可跨实例停止PG 不承载 token 直播事件。容错纪律:启动 ping 不通 fail-fast(同 sandbox init),运行中失败 10s 节流 log + 降级丢帧,reader 1s 退避重连。Redis 不解决的:run 本体仍绑进程(进程死 run 死,drain/reaper 不变)、线程池上限(调 `ZCBOT_RUN_MAX_WORKERS` 的事)。
**broker 外置(Redis pub/sub,✅ 0.42)**:0.41 落地时 event/cancel broker 留在进程内,代价是切换窗口内"刷新看不到旧实例 run 直播 / 停止送达不到"两个边缘。2026-07-06 重评(用户量已非个位数)决定实施 —— ①部署时总有 in-flight run,窗口边缘从偶发变常态;②单进程逼近天花板后,最近的扩容手段是稳态双实例同时接流量,broker 外置是它的硬前提(否则 POST 与 SSE 落不同实例直播全瞎)。选 Redis 不选 PG LISTEN/NOTIFY(token 级 delta 全过主库太吵:NOTIFY 全局队列 + 8KB payload 限制,把实时路径耦到 PG)、不选 nginx sticky hash(upstream 变更时 hash 重排,恰在部署窗口失效,治标)。实现(`web/broker.py`,LocalRunBroker/RedisRunBroker 同接口鸭子类型):`ZCBOT_REDIS_URL` env 开关,不设即进程内(dev 零影响);event 走 `PUBLISH zcbot:ev:<tid>` + 单条 async pubsub reader 路由到本地订阅 queue(Redis 只管跨进程一跳,fan-out 最后一公里仍在进程内);done = SETEX key(60s)+ publish 双通道,订阅先挂 channel 再查 key 不漏;cancel 在鉴权后先置本实例 Event再 SETEX/GET/DEL key 跨实例传递loop 在 chunk 间优先 poll 本地信号(通常约 100ms 内退出,不等待 Redis 或 DB 往返)。取消另以共享 PG 中已经提交的 `tasks.run_status='cancelling'` 作可靠兜底:运行线程每秒低频读取一次,未配置 Redis 的蓝绿窗口或 Redis 运行中故障时仍可跨实例停止PG 不承载 token 直播事件。容错纪律:启动 ping 不通 fail-fast(同 sandbox init),运行中失败 10s 节流 log + 降级丢帧,reader 1s 退避重连。Redis 不解决的:run 本体仍绑进程(进程死 run 死,drain/reaper 不变)、线程池上限(调 `ZCBOT_RUN_MAX_WORKERS` 的事)。
### 7.1 心智模型:Task 一等公民 + Dir 文件副视图
@ -184,7 +184,7 @@ Eval 与生产 core 解耦,通过现有 `/v1` API 创建专用任务、监听
**对话产物与生命周期(0025/0028/0033)**:真实文件仍是内容事实源;`artifacts` 表记录已发布产物的稳定身份和生命周期,包含 user-root 相对当前路径、来源 task、复制来源、可空的 `software_job_id`、哈希/大小及 active/deleted、回收路径。`software_job_id` 不设外键,非空即表示该正式产物由对应专业软件 Job 生成复制品保留该来源Job 或 task 生命周期结束也不抹除来源标识。新 `messages.artifact_refs` 使用 `{version:2, artifact_id, scope:"working_dir", path:"reports/a.pdf", label?:"最终报告"}``path` 是兼容快照,预览/下载优先按 `artifact_id` 找当前路径因此移动或重命名后历史卡片仍有效。version 1 和 `NULL` 旧消息继续走原 task-scoped 兼容链。普通源码树、中间文件、`.meta/` 技术信息和配套资源不登记agent 仅用 `publish_artifacts` 显式提升少量最终文件。移动保持身份,复制为每个副本创建新身份并记录直接来源;删除将文件移入 `.zcbot_artifact_trash/` 并软删记录,普通文件仍物理删除。
**用户消息附件(0031)**:`messages.attachment_refs` 与助手产物分开表达输入附件,元素为 `{version:1, scope:"working_dir", path, label, kind, media_type, size_bytes}`;文件仍是事实源,不登记为已发布 artifact也不承诺独立生命周期。`payload.content` 只保存用户自然语言,模型上下文在内存中按附件类型补兼容提示,避免 UI 协议污染正文。`NULL` 表示旧客户端/旧历史,前端继续解析正文标记;`[]` 表示新消息明确无附件。独立 attachment 表暂缓,只有出现跨消息复用、稳定身份、版本快照或附件级审计需求时再抽象 `file_assets + message_attachments`
**用户消息附件(0031)**:`messages.attachment_refs` 与助手产物分开表达输入附件,元素为 `{version:1, scope:"working_dir", path, label, kind, media_type, size_bytes}`;文件仍是事实源,不登记为已发布 artifact也不承诺独立生命周期。`payload.content` 只保存用户自然语言,Session 内存补兼容路径提示并保留私有附件引用;发模型前按 `input_modalities` 决定纯文本退化或把图片物化为 OpenAI-compatible Base64 `image_url` blockBase64 永不落库、物化仍经过 working_dir/user_root 越界与大小校验。原生看图模型不注册 `look_at_image``read_document` 继续承担扫描 PDF 等文档摄取。`NULL` 表示旧客户端/旧历史,`[]` 表示新消息明确无附件;独立 attachment 表仍待跨消息复用、稳定身份、版本快照或附件级审计需求出现后再抽象
### 7.2 资源模型(/v1)
@ -208,7 +208,7 @@ Admin GET /v1/admin/*(require_admin;overview + usage/models|users + storage/
Export GET /v1/tasks/{id}/export(docx)
```
**SSE 事件**:`run_start / llm_start / text{delta} / reasoning{delta}(thinking 模型推理流,前端灰色折叠卡)/ progress_snapshot{run_id,steps,waiting}(当前 user message 即 run 边界,从 messages 投影恢复)/ tool_call / tool_result(预览,完整走 DB)/ llm_end / model_switch / warn{msg}(熔断·重复拦截·折叠失败等运行时提醒)/ context_fold{phase,...}(§8.8 Phase 2 折叠 start/done)/ cancelled / error / done`。`task_progress` 每次提交完整步骤快照,前端整体替换;旧 `set_plan/update_step` 仅在历史投影时兼容。进度 dock 是运行状态而非历史消息:活跃时紧凑显示,正常完成后隐藏,`ask_user` 等待确认/取消/异常时折叠保留。fan-out:每订阅独立 queue;迟到订阅先从 PG 恢复当前 run 最新进度,终态迟到订阅立收 done。普通直播事件不持久化(messages 走 PG)。
**SSE 事件**:`run_start / llm_start / text{delta} / reasoning{delta}(thinking 模型推理流,前端灰色折叠卡)/ reasoning_reset(传输层放弃当前响应并重试时清除已直播推理)/ progress_snapshot{run_id,steps,waiting}(当前 user message 即 run 边界,从 messages 投影恢复)/ tool_call / tool_result(预览,完整走 DB)/ llm_end / model_switch / warn{msg}(熔断·重复拦截·折叠失败等运行时提醒)/ context_fold{phase,...}(§8.8 Phase 2 折叠 start/done)/ cancelled / error / done`。`task_progress` 每次提交完整步骤快照,前端整体替换;旧 `set_plan/update_step` 仅在历史投影时兼容。进度 dock 是运行状态而非历史消息:活跃时紧凑显示,正常完成后隐藏,`ask_user` 等待确认/取消/异常时折叠保留。fan-out:每订阅独立 queue;迟到订阅先从 PG 恢复当前 run 最新进度,终态迟到订阅立收 done。普通直播事件不持久化(messages 走 PG)。
**版本化**:`/v1` minor 半年兼容,major 6 个月 deprecation。**CORS**:本地 `*`,部署收紧。
### 7.3 认证
@ -244,6 +244,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
- working_dir 存相对 ROOT posix 串,读写统一过 `core/paths.py`;入口 `validate_task_name` 拒空/`/\NUL`/`.` 起头。
- `auto_title_pending`(0023)只是一轮 UI 命名闸,不是 task 状态机;旧创建入口/存量行恒 false快速入口首发后消费人工改名优先清闸。
- **0004 简化**:runs 表只写不读、独立 run 实体在单活形态下冗余 → 合并 `run_status/run_error` 入 tasks需要前端关联本轮时直接复用该轮 user message UUID 为 `run_id`,不恢复 runs 表。**0006**:`tasks.model_profile` 为 source-of-truth(PATCH 切、下条 send 生效);usage_events 重建 v2 多态形态,统计 source-of-truth;tasks 三列保留作粗概览。run_status 终态:ok 收回 idle,error(出错)与 cancelled(用户停止)是持久终态 —— 前端 `renderPersistedRunTerminal` 据此在每次重渲后补持久卡(扛过收尾 loadMessages 整屏重建),刷新/切任务仍在;下次起新 run(post_message 写 running)覆盖清掉。
- **版本化模型计费**(2026-08-30):provider 响应的 usage token 是用量事实源,直连模型 YAML 中带生效时间/币种/来源的 `pricing.periods` 是价格事实源LiteLLM price map 仅给没有本地价目的网关模型兜底,避免第三方价目滞后覆盖已核对的国内厂商价格。`core/pricing.py` 按调用时间选择价格版本和 DeepSeek 峰谷时段,统一拆缓存命中/未命中/输出成本;主对话、标题、折叠、知识库摘要与 prompt 润色共用 `record_chat_usage`。每行 `usage_events.units` 快照 revision/tier/原币单价/汇率/分项成本,无 schema migration公开按量价是产品估算口径赠送额度、套餐和商务折扣以 provider 账单为最终财务口径。历史调价只通过默认 dry-run、显式 apply 且保存旧值的重算工具修正,不让新配置静默改写旧账。
- **0029 消息序号**:`tasks.next_message_idx` 在 task 行锁下统一分配 `messages.idx`Web、agent 与渠道追加不再各自维护序号或依赖冲突重试;分配时仍与 `max(idx)` 校准,允许蓝绿发布窗口内旧实例继续写入。清空消息与计数器在同一事务归零。
- **No-subtask**:同 user 下前缀互含即拒(归一 posix 后 Python 端比对);同 working_dir 允许。
- **文件面板先备料**:user_root 与普通目录都可显式新建直接子目录;创建成功后前端进入该目录,用户可先上传/选入资料,再把顶层空目录选作新对话 working_dir。目录 leaf 复用 `validate_task_name`,不允许借 UI 创建点目录或路径式名称。
@ -252,11 +253,11 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
### 7.5 沙盒:Per-user 容器 + Per-tool exec
选型:**每 user 长驻容器**(文件模型本就以 user root 为安全边界,per-task 会切碎共享工作区)+ **每 tool 一次 docker exec**(exec 级 timeout/cwd/统计)+ 空闲 5 分钟回收 + bind mount user root→`/workspace`。
选型:**每 user 长驻容器**(文件模型本就以 user root 为安全边界,per-task 会切碎共享工作区)+ **每 tool 一次 docker exec**(exec 级 timeout/cwd/统计)+ 无执行活动 10 分钟回收 + bind mount user root→`/workspace`。
**边界划分**:Control plane 留宿主(auth/DB/files 校验/SSE/LLM/受控 web 工具/配额审计),Execution plane 进容器(shell/run_python/任意生成代码)。目标不是"所有操作进容器",是"所有不可信执行不能在宿主"——否则凭据反被带进执行面。
**硬限制**:cgroup CPU/mem、pids-limit、exec timeout、并发数、read-only rootfs、tmpfs /tmp、no-new-privileges、drop ALL caps、非 root、`--shm-size`。**软配额**:按 user 计 DB(磁盘/LLM cost/wall time/流量/并发),超额 429。**网络**:默认 deny outbound,搜索抓取走宿主受控工具。
**硬限制**:单容器 4 GiB/2 CPU、pids 1024、`/dev/shm` 512 MiB、`/tmp` 1 GiB、exec timeout、read-only rootfs、no-new-privileges、drop ALL caps、非 root。整机重型执行由宿主共享文件账本 + advisory lock 统一准入:前后台合计 6、后台最多 4、单用户合计 3蓝绿/多 Web 实例共享同一账本,前台排队不计 timeout 且可取消,并通过 SSE 明示单用户、整机、内存压力或队列顺序等等待原因;宿主 MemAvailable 低于阈值时暂停新放行、不杀存量。轻量 fs 工具不占重型槽。普通容器显式跟踪 active execreaper 只回收无 active exec 且超 TTL 的容器。**软配额**:按 user 计 DB(磁盘/LLM cost/wall time/流量),超额 429。**网络**:默认 deny outbound,搜索抓取走宿主受控工具。
**落地清单(Stage C 硬协议,实施按此对账)**:
1. **网络 blocklist 硬编码段**(任一缺失=未完成):`169.254/16`(metadata)、内网三段、CGNAT `100.64/10`;**PG 实际 IP 单独再 block**(belt-and-suspenders)。**容器自身 loopback(`-o lo`)显式放行**——netns 隔离下容器内 127.0.0.1 到不了宿主,DROP 它无安全收益且误伤容器内 IPC(2026-07 实锤:puppeteer↔chromium DevTools 走 127.0.0.1,被 DROP 导致 mermaid 渲染 90 天 0 成功)。
@ -275,7 +276,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
| gVisor → Firecracker/e2b | 合规客户 / 单机 100+ user / 兼容墙 | 每 VM 100MB+ 不划算;e2b 与 storage_root 自持冲突 |
| docker exec → 容器内 tool-runner RPC | exec 开销 >30% 持续两周 / 长驻服务 / 单轮 >20 次调用 | 自管清理+观测损失 >> 200ms×N;美学统一 ≠ 理由 |
**Image 体积 / 多 user 资源 / 加包**(2026-05-28):① image 大 ≠ 运行时吃资源(layer 共享、不 exec 只是磁盘字节);② 瓶颈在并发 exec 不在 idle 容器,杠杆全在运行时限制;③ 新增依赖 = base 收敛 + **per-user venv**(`<user_root>/.venv/`,bind mount 回收不丢;不放共享 volume——install 脚本是任意代码,破坏隔离)+ 使用频次沉淀进 base
**Image 体积 / 多 user 资源 / 加包**(2026-09-01 修订):① image 大 ≠ 运行时吃资源(layer 共享、不 exec 只是磁盘字节);② 瓶颈在并发 exec 不在 idle 容器,杠杆在共享准入与 cgroup;③ requirements 保持全局 site-packages非 root + readonly rootfs + `HOME=/tmp` 使普通 pip 临时落到 `/tmp/.local`、cache 落 `/tmp/.cache`。不建 venv、不持久化用户包、不加额外包目录容器删除前由镜像内可信扫描器只解析 `.dist-info`,与构建期基础清单比较 added/override/reinstall空扫描不入库、失败不阻塞删除。Admin 只分析依赖会话,不自动改基础镜像
### 7.6 / 7.7 改造项与阶段
@ -298,7 +299,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
- **path-as-identity 而非 folder_id**:folder 真实存在于 FS,folder_id 是第二份 source of truth;rename 走 DB-aware 同事务 cascade。
- **files API 单一 mutation 入口**(2026-05-18):"顶层目录分支"从数据状态派生而非客户端意图,放服务端才有强制力;双命名空间(/folders vs /files)把分支搬给 client,失强制力且端点翻倍。
- **task 软删除(2026-06-17 推翻 hard cascade)**:公测后对话轨迹是训练/研究语料,`deleted_at` 置位 + restore,避免用户误删立即永久丢失。**当前实现仍无限期保留软删数据**,物理清理仅有管理员手段;后续生命周期已定为“软删除后保留 30 天再物理清理”(待容量信号实施,见 §8.5),届时恢复能力明确限于宽限期内。
- **文件留存**:普通用户文件仍是 FS 直接删除;已写入结构化 `messages.artifact_refs` 的已发布产物,在统一 files delete 入口删除时原子移动到用户根目录的平台隐藏区 `.zcbot_artifact_trash/`,原路径与历史卡片立即表现为已删除。递归目录只回收其中 artifact其他文件照常删除回收内容仍计入用户配额。上传入口默认以原子独占创建保留同名两份,只有用户从文件菜单明确选择并确认「替换文件」时才允许原子覆盖;替换保持原路径和 artifact 身份、内容更新。该机制仍不防 agent/shell 绕过 files API 或整盘损坏。完整地基仍采用 restic/borg 定时增量备份(与应用解耦,捕获删除+覆盖+所有写入口),后续容量需要时再补 `data_events` 用户意图事件和回收区清理/恢复管理。
- **文件留存**:普通用户文件仍是 FS 直接删除;已写入结构化 `messages.artifact_refs` 的已发布产物,在统一 files delete 入口删除时原子移动到用户根目录的平台隐藏区 `.zcbot_artifact_trash/`,原路径与历史卡片立即表现为已删除。递归目录只回收其中 artifact其他文件照常删除回收内容不占用户逻辑配额,但后台扫描在 `user_disk_usage` 中单列其字节数和文件数Admin 展示“配额占用 / 回收站 / 物理合计”,避免释放用户额度后失去服务器真实容量可见性。上传入口默认以原子独占创建保留同名两份,只有用户从文件菜单明确选择并确认「替换文件」时才允许原子覆盖;替换保持原路径和 artifact 身份、内容更新。该机制仍不防 agent/shell 绕过 files API 或整盘损坏。完整地基仍采用 restic/borg 定时增量备份(与应用解耦,捕获删除+覆盖+所有写入口),后续容量需要时再补 `data_events` 用户意图事件和回收区清理/恢复管理。
- **0004 删 runs/usage_events 旧表**:只写不读的死代码;代价是失历史 run 元数据,真要细粒度审计再补(届时是新需求非技术债)。
- **本地也用 PG 不用 SQLite**:dogfood ≡ 真实路径;Docker 已是必然依赖;双 adapter 维护税 > 一次性配置。
- **API-only,UI 由 platform 实现**(2026-05-15):本仓库再维护一套 UI 是双套浪费;SSE payload 从 HTML 切 JSON;沉淀的 sink/broker/路径安全全保留。**dev SPA 留一份**作 dogfood 主路径(SSE 调试 curl/Swagger 都覆盖不了)。
@ -316,7 +317,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
### 8.1 图像理解 + Seedream i2i(✅ 2026-06-16)
缺口:主模型纯文本;t2i 无法"改已生成图"或"读上传图"。**选 E+C 组合**:seedream 加 `reference_images` 走 i2i + 新增 `look_at_image`(Doubao Seed 2.0 Lite,一次读图 <¥0.01)让 DeepSeek 自决何时"借眼睛"。**不选 A**(主模型换多模态:V4 code/tool-calling 是核心,换=降能力+改 loop 引 multimodal,工程 5×);**不选 B**(每条消息隐式 vision 路由:烧 token+失 agentic 控制权)。关键实测:ARK 接受 base64 data URL → 内网无需对象存储。**升级到 A 的信号**:用户要"贴图直接对话读图"成高频——当前假设"图是工具调用对象"而非"对话内容"
历史基线是纯文本主模型配 `look_at_image` 借眼睛,生图改图独立走 `reference_images`。2026-08-27 随 GLM-5.3 Flash 原生多模态进入默认档位,升级信号已满足:附件输入改为 capability-driven原生图片模型直接收到用户上传图片纯文本模型继续使用路径提示与辅助视觉工具不会把所有文件或工具产物隐式送入模型避免无关 token、越界读取和缓存污染。生图/改图仍是显式外部动作,原生理解不替代图像生成工具;运行中产生的新图片若未作为后续用户附件加入,不自动注入下一轮
### 8.2 Token 优化与上下文治理(✅ 2026-06-04 起)
@ -330,7 +331,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
**优雅 drain 已是单实例上限**(SIGTERM 拒新 run + 等收尾);先撞的瓶颈是线程池(每活跃 run 占 1 线程)。落地排序:① 轻量监控(显式 executor + 60s `[stats]` 周期日志——要历史峰值不是快照);② 按数据决策扩容;③ `--reload` 缩 503 窗。**不做监控界面**:运维健康是少数标量,日志够;业务分析走 DB SQL。界面阶梯:日志 → /v1/stats → Grafana → 只读 dashboard,现停第一级。无感换版已由蓝绿落地(0.41)、broker 外置 Redis 已实施(0.42,均见 §7.0);扩容路径:调大线程池 → 稳态双实例分流(broker 已外置,纯 nginx 配置动作)。
**工具失败聚集的信号分层(2026-07-29)**:7 天窗口继续保留取证,但管理端默认判断对象是“近 24h 仍活跃且跨 ≥2 task”的系统性故障单 task 反复试错、按设计非零退出的质量门、近 24h 已归零的历史尾巴分别展示避免正常迭代挤占故障榜首。API 保留低阈值全量 `clusters`,只加 `category=failure|quality_gate`,分区属于前端读侧语义,不删除既有字段。空输出 shell 非零退出从相邻 assistant tool_call 提取稳定命令类别(如 search/no match、dependency probe只改善签名不持久化第二份命令事实。RepeatGuard 只把 `run_python` 的 traceback + 非零退出纳入同错 streak不泛化到 shell避免 grep 未命中和质量门复检被误拦。provider wire 健康另走只读派生端点,按 `model_profile+tool` 对已有 `tool_salvaged/tool_malformed` 事件计算 24h/窗口抢救率;它是失败数的分母与趋势解释,不混进 cluster不新增表/索引也不反向改变并行调用、salvage 或重试策略
**工具健康事件与信号分层(2026-09-03)**:`usage_events` 是工具健康的唯一事实源,不再从 `messages.payload` 回扫或补算;普通 tool result 在落消息后仅当分类为失败时写 `tool_failure`(质量门写 `quality_gate`并在写入时完成稳定签名、shell 命令类别和类别分类。切换前仅存在于 messages 的普通失败历史有意不迁移、不纳入新口径。畸形参数、抢救成功、run 终态错误沿用 `tool_malformed/tool_salvaged/run_error/empty_response`;主动停止、重复调用硬拦、上下文折叠失败分别记 `run_stopped/agent_guard/context_fold_failure`。这些控制事件统一 `category=agent_control`,与 `category=failure|quality_gate` 分栏管理页“工具健康”大数只计真实失败避免把保护机制触发误报为平台故障。7 天窗口继续保留取证,默认判断对象是近 24h 活跃信号;单 task、跨 task、质量门、控制事件和已安静历史分别展示。provider wire 健康仍按 `model_profile+tool``tool_salvaged/tool_malformed` 计算 24h/窗口抢救率,不混入失败数。读侧合并为一次带 `(kind, created_at)` 部分索引的事件查询Admin 首次进入即加载,之后 90 秒节流并由服务端短时单飞缓存兜底
### 8.5 定时任务(✅ 2026-06-18)
@ -406,15 +407,15 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
**决策**:把长进程从 zcbot 进程树上摘下来,OS 就是"任务组件",不引 Celery/RQ/队列。`shell`/`run_python` 加 `background=true`(**模型判断**:预计 >~1min 走后台;前台超时报错里提示改后台——判断错了纠错路径只有一步;用户显式指令永远优先)。状态协议**纯文件**:`<user_root>/.zcbot_procs/<task_id>/<proc_id>/{proc.json, output.log, exit_code}`——dotfile 用户不可见(同 `.zcbot_tmp` 惯例),`exit_code` 文件出现是唯一终态信号,状态判定全靠文件 + 现场探测(pid / 容器 running),**无常驻登记,天然扛重启**。查询/终止走配套 `check_process` 工具(host in-process,两种 backend 通吃)。
- **host backend**:detach 独立 wrapper(`core/proc_wrapper.py`,stdlib-only,sys.executable 直跑不依赖 PYTHONPATH):限时(默认 7200s / cap 86400s)、超时杀进程树记 124、日志截尾 10MB、最后写 exit_code。
- **docker backend**:**专用容器** `zcbot-proc-<id>`(pool.run_proc_container,同款硬化 + iptables init),`product=proc` + 无 instance label —— 与 sandbox 容器的 idle reaper / shutdown_all 生命周期**解耦**,dockerd 托管,蓝绿切换/实例重启不中断。不用 `docker exec -d` 进 sandbox 容器:idle 5min reaper + 启动 shutdown_all 会把长进程随容器带走
- **回收**:check_process 见终态顺手 rm 容器;web lifespan 每小时 `procs.sweep`(终态目录 7d TTL / exited 孤儿容器),幂等,蓝绿双实例同时跑无害
- **docker backend**:**专用容器** `zcbot-proc-<id>`(pool.run_proc_container,同款硬化 + iptables init),`product=proc` + 无 instance label —— 与普通 sandbox 的 idle reaper / shutdown_all 生命周期**解耦**,dockerd 托管,蓝绿切换/实例重启不中断。无容量时先写 `state=queued`,调度器获得共享前后台槽后才启动,故排队时间不计运行 timeout`check_process` 兼容 queued/running/finished/lost并可取消 queued
- **回收**:后台仍运行即有活动,不适用普通容器 idle TTLcheck_process 见终态顺手扫描临时包并 rm 容器web lifespan 约 30 秒 sweep/dispatch终态目录仍保留 7d幂等蓝绿双实例以 proc 目录锁避免重复启动。运行中容器由 Docker 托管,服务重启恢复 queued 并保留 running
- **通知/可视**:不做服务端推送 —— 前端轮询 `GET /v1/procs`(用户级,纯文件读取,仅有 running proc 时 5s 一拉):`[Background]` 工具结果卡本身活化(spinner+跳秒+停止按钮,与前台工具卡同体验,历史重渲同样恢复;`POST .../procs/<id>/kill`)、running→终态弹 toast(跨 task 也提醒,点击跳转)。proc 完成时刻往往没有活跃 run,SSE 通道根本不在,轮询是诚实的选型。
### 8.13 扫描件 PDF 直读:方舟文档理解,不接外部 OCR(✅ 2026-07-21)
缺口:markitdown 只抽 PDF 文本层,扫描件(老标准/检测报告/红头指南,建材院高频)转出为空=死路。**选复用 seed-2.0-lite 的方舟文档理解**(chat file 内容块,PDF 整本 base64 内联)新增 `read_document`:零新供应商(敏感文档不出已有豆包面)、零新基础设施、记账复用 vision 通道;实测 ~1300 输入 token/页(约 1 厘/页)、100 页全覆盖、17MB 内联可用。**不选专用解析 API**(MinerU/Textin:版面还原最好,但申报书/专利底稿要上传新第三方 + 免费额度政策不稳);**不选本地 OCR**(PaddleOCR 类:镜像塞推理依赖,需求未量化前过度投资);**不选 file_url/file_id 传址**(前者要给用户文件开免认证公网直链=新安全面、开发机 NAT 后还跑不通;后者要接 TOS 多落一份存储;base64 是零新增面的唯一形态,行业惯例 chat 端点也不收 multipart)。防上下文爆:多页 OCR 强制 `save_md` 落盘只返预览。**升级信号**:>100 页/>30MB 巨件成高频 → 接 TOS 走 file_id;要高保真版面/公式还原 → 再评 MinerU。probe/smoke 留仓(`scripts/probe_ark_doc.py` / `smoke_read_document.py`)。
- **对话锁(前端)**:bg proc 运行期间该 task 的 composer 锁定(发送→停止,Enter 拦截),观感与前台执行完全一致 —— 后台化的收益定位为「进程扛超时/服务重启」,**不改变"一个任务同时只做一件事"的对话心智**;完成的那次轮询解锁 + toast「可继续对话」。锁只在前端,服务端不 409:「停止」入口必须可达,且多设备/渠道绕过前端锁属可接受边缘(等的是同一个进程,发了消息也不冲突)。
- **防失控**:每用户并发 running 上限(`ZCBOT_MAX_BG_PROCS` 默 3);前台默认超时不放大(它是逼模型做前台/后台选择的杠杆)。
- **防失控**:前后台统一使用整机 6 / 后台 4 / 单用户 3 的共享准入;前台默认超时不放大(它是逼模型做前台/后台选择的杠杆)。
**边界(防滑坡)**:只覆盖「单个本地长进程」。①**外部异步作业**(seedance 等 submit/poll 形态)不进这里——工具内轮询 + `resume_task_id` 续查已够;②**job 链/依赖/自动重试**不做——那是 workflow 引擎,编排的唯一归属是 agent loop(模型 check 后自己决定下一步),同 §6 拒绝编排的理由;③**完成后自动续跑 run**不做——zcbot 的长任务产物多为终点交付物(与 Claude Code"build 是中间步骤"不同),自动续跑=无人在场烧 token,通知给人、下一步由人/下次对话决定。
@ -471,10 +472,14 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
云端控制面使用独立的 `software_node_enrollments``software_nodes`,不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵,数据库只保存 SHA-256 摘要;节点注册在行锁事务中校验有效期、预期名称和服务端支持的能力,成功后原子消费。注册码只建立节点身份,不要求管理员逐项授权专业软件能力。每个节点获得独立高熵 Token数据库只保存 bcrypt 强哈希,明文仅在注册响应出现一次。
专业软件采用“共享能力契约 + 本机 adapter”边界。仓库根目录 `software-contracts/*.json` 是语言无关的声明式源码事实源,描述 capability、请求 JSON Schema、输入额度、输出 manifest、feature 与最低 adapter 版本Core 按文件签名热加载契约,只负责身份、账本、调度、传输、摘要与最终发布,不包含 Origin 或其他软件的操作分支。热加载先完整校验新快照再原子切换写入中或非法版本继续使用上一份有效快照。Windows Node 是稳定的可信宿主,只负责持久化 job 目录、下载/上传、恢复、取消,以及从程序目录 `adapters/*/adapter.json` 发现 adapter发布过程把对应源码合同复制进各 adapter 目录Host 只读取 manifest 同目录指向的这一份合同并做通用 JSON Schema 校验,不再读取共享合同目录或采用本地优先 fallback。这样完整 Node 包与独立 adapter 包具有相同部署单元manifest、合同、Worker 和依赖必须整体更新避免旧合同静默覆盖新合同。manifest 声明 capability、实际 adapter 版本、运行类型、入口和契约文件,并可声明 11440 分钟的单任务超时;旧 manifest 缺省时保持 30 分钟,版本与时限不编译进 EXE。Node 在心跳中上报实际加载合同的 SHA-256 和 Workspace 协议版本,服务端只向明确兼容的节点调度 Workspace Job。Host 另提供本机运维层:按 `runtime_id` 合并软件卡,允许管理员自动检测或选择预定义应用位置,并只按内置软件清单及固定 requirements 单独构建受管 runtimeUI 不接受任意安装命令任务请求仍不能影响解释器、应用路径或安装过程。Host 将已验证的位置以固定环境变量注入对应 WorkerWorker 的 `--probe` 继续承担最终软件版本、SDK/COM 和执行门健康校验。Worker 可以是受管 Python 脚本或独立 EXE进程只接收 job 目录,并通过 `state.json`、`terminal.json` 与固定输出目录交接Python、COM 或某个 SDK 都不属于通用任务协议。当前 Origin、ANSYS 和 Blender adapter 使用 Python 只是实现选择。
专业软件采用“共享能力契约 + 本机 adapter”边界。仓库根目录 `software-contracts/*.json` 是语言无关的声明式源码事实源,描述 capability、请求 JSON Schema、输入额度、输出 manifest、feature 与最低 adapter 版本Core 按文件签名热加载契约,只负责身份、账本、调度、传输、摘要与最终发布,不包含 Origin 或其他软件的操作分支。契约可附带 `active/deprecated` 生命周期、`standard/diagnostic_only` 可见性和替代 capability普通能力枚举隐藏诊断入口显式诊断查询与历史 Job 读取仍保留。热加载先完整校验新快照再原子切换写入中或非法版本继续使用上一份有效快照。Windows Node 是稳定的可信宿主,只负责持久化 job 目录、下载/上传、恢复、取消,以及从程序目录 `adapters/*/adapter.json` 发现 adapter发布过程把对应源码合同复制进各 adapter 目录Host 只读取 manifest 同目录指向的这一份合同并做通用 JSON Schema 校验,不再读取共享合同目录或采用本地优先 fallback。这样完整 Node 包与独立 adapter 包具有相同部署单元manifest、合同、Worker 和依赖必须整体更新避免旧合同静默覆盖新合同。manifest 声明 capability、实际 adapter 版本、运行类型、入口和契约文件,并可声明 11440 分钟的单任务超时;旧 manifest 缺省时保持 30 分钟,版本与时限不编译进 EXE。Node 在心跳中上报实际加载合同的 SHA-256 和 Workspace 协议版本,服务端只向明确兼容的节点调度 Workspace Job。Host 另提供本机运维层:按 `runtime_id` 合并软件卡桌面软件允许管理员自动检测或选择预定义应用位置runtime-only 软件使用 `SoftwarePathKind.None` 且不接受路径;两类软件都只按内置清单及固定 requirements 单独构建受管 runtime。UI 不接受任意安装命令任务请求仍不能影响解释器、应用路径或安装过程。Host 将已验证的位置以固定环境变量注入对应 WorkerWorker 的 `--probe` 继续承担最终软件版本、SDK/COM 或内核 roundtrip 健康校验。Worker 可以是受管 Python 脚本或独立 EXE进程只接收 job 目录,并通过 `state.json`、`terminal.json` 与固定输出目录交接Python、COM 或某个 SDK 都不属于通用任务协议。当前 Origin、ANSYS、Blender 和中性 CAD adapter 使用 Python只是实现选择。
Core 在 Job offer 中附带由 capability 合同生成的 `request_summary`供本机任务列表统一显示标题Host 不解析各软件私有请求结构,旧落盘 Job 只做通用兼容读取。
扩展现有 capability 的 feature 时修改共享契约、对应 Worker 和测试,不修改 Core 调度、Job 生命周期或 Node EXE增加新专业软件时新增契约与一个 adapter 目录。Cloud 会在下一次契约查询时获得新校验和工具 schemaNode 在启动时从本地目录获得能力、版本和入口,并在首次连接及每次心跳中上报全部已发现 adapter。服务端用本次上报中自己具有共享契约的 capability 原子替换节点能力列表,未知能力不进入调度;管理端只读展示该结果,不保存人工能力选择。现有节点因此不需要清身份或重新注册,升级 Node 后重启一次即可在下一次心跳同步。当前选择明确的运维边界zcbot 契约可热更新,不要求重启;本机 adapter 更新时先退出托盘 Node整体替换 adapter 目录后重新启动,不重装、不替换 Node EXE也不建设自动更新平台。当前 Node 仍按整机单执行槽保守串行,未来只有真实并行软件需求出现时,才把 slot 账本升级为 per-capability 租约,而不改变 Job 协议。
桌面宿主采用 WPF `Application` 持有唯一 GUI 消息循环WPF 主窗口只通过 ViewModel/Command 消费应用服务与控制器状态WinForms 被限制在 `NotifyIcon`、文件/目录选择、确认框和剪贴板互操作,不再存在第二套配置窗口或 Host 生命周期。托盘宿主负责把后台状态经 Dispatcher 投递到 WPF、控制窗口显隐并复用应用控制器完成注册、重连、清身份、数据目录迁移与安全退出。CLI 参数在 WPF 初始化前分流,因此 `run --headless` 不创建窗口或托盘。节点概览、首次注册、登录自启动、专业软件/runtime、本机任务、ANSYS 固定验收和数据目录迁移均已进入 WPF`SoftwareManagementService`、`JobMonitorService`、`AnsysAcceptanceService`、`DataRootManagementService` 与 `DiagnosticService` 分别封装固定业务边界ViewModel 不解析 Job 落盘格式、不访问注册表、不创建 Worker也不直接持有应用控制器。
Node 通过 `Authorization: Bearer``X-Node-Id` 建立 `/v1/software-nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活,新连接关闭旧连接;`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态,再关闭现有连接;断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。
第二阶段已增加 `software_jobs`(专业软件任务)账本与 `origin.plot@v2` 的 offer/accept 骨架。用户只能在本人 task 下以幂等键提交固定 schema云端规范化请求并记录 SHA-256按当前进程真实在线、能力匹配、健康且有空闲 slot 的 Node 创建短期 offer。Node 再次校验 schema、图形类型和输出格式使用 write-through、flush 与原子 rename 先落本机任务目录,再回 `job_accept`;重复 job 只有 digest 一致才接受。过期或发送失败的 offer 回到队列lease、Node 和 digest 不匹配的响应被拒绝。Node 接收后云端进入 `dispatched` 而非 `running`,并将 slot 降为 0只有固定 Worker 真正启动后才进入软件无关的 `software_running`,具体软件和操作由 capability/request 表达。
@ -487,13 +492,23 @@ Origin 科研分析以独立 `origin.analysis@v1` capability 承载,不把数
Recipe 是专业软件的声明式目标状态,不是任意脚本或逐次鼠标命令。`origin.plot@v2` 以新增 `plot.type=recipe + recipe_version=1` 复用已验证的 `layout/panels/series/axis/legend` 组合结构;旧 `multi_panel` 与 Recipe 进入同一个受控执行器,单图和特殊统计图继续保留兼容入口。模型可以组合契约允许的原子图形能力,但不能提供 Python、LabTalk、解释器、模板名或文件路径。其他软件可采用同一通用 Job 外壳和各自的声明式 Recipe不建设跨软件万能 DSLAPI/SDK executor 优先,未来的 UI Automation 或 Computer Use 仅作为 adapter 内部执行后端,不改变云端 Recipe。
`cad.geometry.prepare@v1` 是独立、无状态、runtime-only 的中性几何入口,以固定 CadQuery 2.8.0 / OCP 7.9.3.1 生成 STEP它只负责检查、基础体和受控有序 Recipe不承载材料、网格、接触、边界条件或求解。Recipe 覆盖显式坐标闭合截面、拉伸/旋转/扫掠/放样、布尔、圆角/倒角/抽壳、变换、线性/圆周阵列、STEP import 与多实体组合,所有引用、轮廓平面性、形状有效性、实体上限和选择数量都机械校验。输出固定为 STEP、规范化 Recipe、确定性几何 manifest、无 GUI PNG 预览和 provenancemanifest 只保存单位、几何量、拓扑/表面摘要和已解析区域,不把 OCCT、ANSYS 或 Abaqus 的临时实体 ID 作为跨软件事实。区域选择器以几何语义描述 face/edge/solid其中 `extreme_face``planar_faces` 和 ANSYS 静力 v2 保持同构并由合同测试守护。输入首版只接受显式声明单位的 STEP/STP统一转换到请求必填的 `canonical_unit`,缺失或歧义单位直接失败。
首版不输出 GLB当前稳定事实源和跨求解器交换面是 STEP现有 GLB 链路依赖 Blender/装配导出,未在独立 CAD runtime 上形成同等稳定的 roundtrip 与打包验证;后续只有在 CAD runtime 内可独立验证时再作为新增可选产物。
`freecad.model.author@v1` 是面向已安装 FreeCAD 1.1 稳定版的有状态受控建模入口,只接受 `>=1.1.3,<1.2.0`,不接受 RC、weekly 或 experimental 构建。受管 `freecad` Python runtime 只负责发现 `FreeCADCmd.exe`、版本门禁、任务级配置隔离和 Pillow 软件渲染;固定 FreeCAD 侧 Worker 在应用自带 Python 中调用 `FreeCAD/Part`,不把 FreeCAD 安装进 venv也不依赖 FreeCADGui/GPU。每个 Job 固定启用 safe mode、独立 user/system cfg 与 user data/temp 目录,并清除不可信 Python 搜索路径请求不能影响脚本、程序路径、命令行、本机路径、URL、宏、插件或 Workbench。
FreeCAD v1 的 Recipe 是模型唯一事实源,每次 revision 都从空文档按完整目标状态重建,而不是继续编辑上一版 FCStd 中的隐式状态;轻量命名参数只在 adapter 内解析为有限数后写入 Shape不生成 Spreadsheet、alias 或原生 Expression。首版操作限制为 box/cylinder/cone/sphere/torus、STEP import、union/cut/intersect、translate/rotate/combine输入只允许零或一个已登记 STEP/STP外部 FCStd 与更高阶特征留待后续。固定输出 `model.FCStd`、`model.step`、无 GUI 等轴测预览、Recipe、manifest 和 provenance发布前必须完成 recompute、Shape 有效性、FCStd 重开与 STEP 重开并对实体数、包围盒和体积做容差比较。Workspace 继续只管理 current/rollback 两代,续作沿用通用 `software_job_revise`,不新增协议或数据库状态。
`ansys.geometry.inspect@v1` 继续保留原合同和 adapter专门验证 Mechanical 真实导入后的 GeoData、单位与 Named Selection它至少保留一个发布周期但生命周期标为 deprecated/diagnostic-only并指向中性 CAD 入口。普通几何准备不再推荐占用 ANSYS 许可证,显式诊断请求与历史 Job/旧节点协议不受影响。
Blender 入口为 `blender.scene.author@v3`,唯一场景类型是 `recipe`;它是 Blender 专属的声明式静态场景目标状态,不是跨软件万能 DSL也不是 Python、逐步按键或任意节点脚本。Recipe v1 组合带稳定 ID、父子层级和集合归属的 geometry/generator/text/empty 对象geometry 覆盖 box/cylinder/cone/sphere/torus/plane、二维轮廓拉伸、轴向旋转、路径扫掠和有壁厚直管受控 modifier 覆盖 boolean/bevel/mirror/array/solidify并统一声明材质、变换、灯光、世界背景和 18 个相机视图;`rotary_kiln` 只是 generator 组件,可与其他对象处于同一场景,不再占据顶层 capability。adapter 1.0.0 把合同校验后的 Recipe 编译为固定 `bpy` 调用,限制请求 256 KiB、200 个声明对象、展开后 1000 个 Blender 对象、100 万 mesh polygon、材质/灯光/视图和修改器数量Worker 还机械校验 ID 唯一性、引用、父子关系、布尔目标、管壁和请求预览的一致性。输出固定包含 `.blend`、首张 PNG、场景清单和溯源可再选择最多七张 PNG 与 GLB预览用 `preview_01``preview_08` 稳定槽位,视图业务名称记录在 manifest。
Scene Recipe 每次按完整目标状态从 factory-empty 场景确定性重建,不把上一版 `.blend` 中的隐式手工状态当作事实源Workspace 继续保存当前与 rollback 两份工程供结果留存和失败恢复,完整 Recipe 保存在 Job 请求中作为续作事实源。为避免旧的回转窑专用状态与新语义混用,本次按用户明确授权直接移除 v2 合同与调度,旧 v2 Workspace 文件不删除但不能作为 v3 source。目标执行版本仍为 Blender 5.2 LTS受管 Python 只负责发现并以 `--background --factory-startup --disable-autoexec --python-exit-code 1` 拉起已安装的 Blender实际 `bpy` 在 Blender 自带 Python 内执行请求不能提供脚本、命令行、本机路径、URL、插件、driver 或任意 Geometry Nodes。
专业软件的连续加工以独立 `software_workspaces` 为稳定边界Job 只记录一次操作。首个 Job 创建独立 `workspace_id`,两者不共用 ID服务端保存 user 归属、capability、`home_node_id`、`head_job_id`、容量与留存元数据Node 物理目录只使用 `<data_root>/workspaces/<workspace_id>/`,不再按 user 建上层目录。全新节点的 `data_root` 默认是当前专用账号的 `%LocalAppData%/Zcbot/WindowsNode`,以便无需提权即可从 UI 完成首次配置;升级节点若已有 `%ProgramData%/Zcbot/WindowsNode` 则继续原位使用,不做隐式迁移。专用运行账号可在 UI 中选择其他本机固定磁盘,机器级 `ZCBOT_WINDOWS_NODE_DATA_DIR` 拥有最高优先级并使 UI 只读;账号级选择保存在 HKCU避免把根目录定位信息放入其自身。迁移先停止调度并等待本机任务及导出收尾在目标同级临时目录复制并逐文件校验后才原子切换旧目录不自动删除。这样用户统计集中在数据库完成节点只管理实际落到本机的 workspace也避免多节点各自维护一套用户目录树。Workspace 一旦首次调度即粘在 home node后续 Job 必须以当前 head 为 source、串行回到同一节点节点离线时等待而不复制工程或静默改派。
Node 对每个 Workspace 只保留 `current``rollback` 两代。续作开始前把 `current` 原子切换为 `rollback`Worker 打开该工程并将新状态写入 Job 输出区;成功后输出区提升为新 `current`,失败或进程中断则恢复 `rollback`。因此连续加工不会为每个 Job 永久复制一份工程物理空间上限约为当前工程加一份回滚工程Job 账本仍完整保留参数、摘要和 local manifest。`software_job_revise(source_job_id, operation, outputs)` 只允许从 Workspace 当前 head 继续,复用已登记输入并重新经过当前契约校验。当前 Origin adapter 以 OPJU 为 workspace stateANSYS 静力 capability 仍是无状态一次性执行,契约中的 `workspace: null` 明确保持原发布流程。
Node 对每个 Workspace 只保留 `current``rollback` 两代。续作开始前把 `current` 原子切换为 `rollback`Worker 打开该工程并将新状态写入 Job 输出区;成功后输出区提升为新 `current`,失败或进程中断则恢复 `rollback`。因此连续加工不会为每个 Job 永久复制一份工程物理空间上限约为当前工程加一份回滚工程Job 账本仍完整保留参数、摘要和 local manifest。`software_job_revise(source_job_id, operation, outputs)` 只允许从 Workspace 当前 head 继续,复用已登记输入并重新经过当前契约校验。当前 Origin adapter 以 OPJU、Blender 以 BLEND、FreeCAD 以 FCStd 为 workspace stateANSYS 静力 capability 仍是无状态一次性执行,契约中的 `workspace: null` 明确保持原发布流程。
科研统计图继续以独立 feature 增量扩展同一契约:`box/histogram/violin` 的系列只绑定原始 Y 列,统计规则由 Origin 固定模板决定;`violin` 把跨输入的观测列复制进受控全 Y 展示表,通过与菜单等价的固定 `worksheet -p 206 Violin` 一次性提交完整选择,使 Origin 正确建立分类轴和每列一组的核密度分布。原始输入表仍保留,默认以分类标签替代图例,并固定预留底部空间显示分类标签与轴标题;首版拒绝逐系列样式,因为原生多 Y 小提琴在图层内是单个统计对象,不能稳定映射普通曲线样式。`bubble` 增加正数 `size` 数据角色,由受控 modifier column 驱动符号尺寸;`band` 要求同一输入的 `x/y/lower/upper`先画上下界并填充到下一曲线再叠加中心线Worker 在打开 Origin 前拒绝非有限尺寸、非正尺寸和倒置边界。`stacked_column/stacked_area/stacked_bar` 统一把两条以上 XY 系列复制进内部连续 XYY 工作表,严格校验横坐标相同后建立 plot group并只执行 Worker 内置的固定累计图层命令;请求不能提供命令、模板或工作表范围。材料谱图使用独立 `stacked_line` feature请求显式给出 `stack_gap_percent`Worker 保留原始输入表,并在内部展示表中把每条曲线的最小值对齐为自身基线,再按全体最大 Y 跨度及间隔百分比逐条纵向偏移;各系列可拥有不同 X 采样点,展示纵轴不支持对数尺度。默认在曲线右端直接标注系列名并隐藏易遮挡谱线的图例,用户显式提供 `legend` 时仍按请求显示;单图标题按实际画布宽度定位。该实现不依赖节点上的 `OFFSETSTACKY.OTP` 或隐式主题,避免模板版本差异,同时 OPJU 中仍可追溯未变换数据。`area/polar/pie` 继续使用固定 Origin 类型 ID。未经过目标 Origin 版本真机验证的统计属性不进入公共 schema避免暴露看似可配但不能稳定复现的参数。
@ -503,7 +518,7 @@ Workspace capability 的默认完成协议不发布正式产物。Node 只自动
第六阶段增加用户级 Job 中心与 Agent typed tools。`software_capability_list` 只暴露固定能力及当前在线空闲节点数,`software_job_submit/status/cancel` 在构造时绑定当前 user/task模型不能跨用户或跨对话指定归属。`software_job_submit` 的唯一入口为通用 `inputs[] + operation + outputs[]`;输入和输出使用任务内稳定 key具体 selector、type、format 和 options 由 capability 校验。`register_artifact` 是普通文件获得输入身份的唯一入口,并明确返回 UUID。右下角 Job 中心按用户聚合各对话任务,活动期短轮询、空闲期降频;终态变化通知用户,成功任务可回到原对话发起分析。取消采用协作协议:未派发任务直接终止,已派发任务先进入 `cancelling`,云端通过 WebSocket 发送并在心跳时重放 `job_cancel`Node 杀死固定 Worker 进程树后回报 `cancelled`;终态写入仍由云端账本裁决。
Job 完成后的对话闭环复用 `software_jobs` 账本,不增加通用事件表。提交时用 `completion_action=report|analyze` 区分“只报告产物”和“报告并分析”,未明确分析时默认 `report`成功终态将 `followup_status` 置为 `pending`分发器只在原 task 空闲时领取:`report` 直接写入带 artifact refs 的固定 assistant 消息,不产生模型费用;`analyze` 写入前端隐藏的内部完成事件,并复用 task 单活锁、持久消息和 SSE run 自动续跑 Agent。`pending/running/completed/failed` 状态使服务重启、对话繁忙和重复完成回执均不会造成重复回复Job 中心的手动深入分析通过同一入口重新排入分析,而不是模拟点击或只预填输入框。
Job 完成后的交互闭环复用 `software_jobs` 账本,不增加通用事件表,也不把每个软件结果复制为新的聊天消息。提交时用 `completion_action=report|analyze` 区分“只展示结果”和“结果完成后继续分析”,未明确分析时默认 `report`;终态将 `followup_status` 置为 `pending``report` 不依赖原 task 是否空闲,分发器只完成回调账本;当前 task 的左侧结果轨按 Job 独立堆叠运行状态和结果,成功预览可直接打开,移动端收敛为结果入口。右侧 Job 中心仍是跨 task 的完整管理面,主消息流只保留用户与 Agent 的叙事;历史 `software_job_report` 消息继续兼容读取,但不再新增。只有 `analyze` 需要等待原 task 空闲,随后写入前端隐藏的内部完成事件,并复用 task 单活锁、持久消息和 SSE run 自动续跑 Agent。`pending/running/completed/failed` 状态使服务重启、对话繁忙和重复完成回执均不会造成重复分析Job 中心的手动深入分析通过同一入口重新排入分析,而不是模拟点击或只预填输入框。
ANSYS 能力固定面向 Windows 上的 Mechanical 2024 R2revision 242并以共享 GeoData 层拆成检查和求解两个声明式入口。`ansys.geometry.inspect@v1` 只导入一份已登记 STEP/Parasolid/IGES枚举 assembly/part/body/face、几何单位、包围盒、面质心/法向/面积、已有 Named Selection并输出 JSON manifest、整体预览和溯源它不划分网格或求解。`ansys.mechanical.static_structural@v2` 接受显式单位、内置 Structural Steel、全局网格、固定支撑、力/压力和三类结果,边界区域可引用导入的 Named Selection也可用全局轴极值平面或带法向/偏置/容差的平面查询;查询只在 Mechanical 导入后的真实 GeoData 上解析,按实体 ID 创建项目内 Named Selection可用 `expected_count` 硬断言选择数量,并把最终面、来源和选区预览写入结果。两者都不接受 Mechanical 工程、Python、APDL、Journal、URL 或本机路径;静力输出固定为 MECHDAT、汇总/结果表、选区及应力/变形图、solver log 和 `.meta` 溯源。
@ -519,10 +534,20 @@ ANSYS 能力固定面向 Windows 上的 Mechanical 2024 R2revision 242
当前网关与 zcbot 共用 host但 HTML 响应同时用 CSP `sandbox` 和 iframe sandbox 强制 opaque origin且不授予 `allow-same-origin`;这一约束在签名 URL 被顶层打开时同样生效,项目代码不能读取主站 localStorage/JWT。`Referrer-Policy: no-referrer` 防能力 URL 随外链泄露,资源 URL 自带同一签名所以 ES module 不依赖认证 Cookie。将来需要 LocalStorage、Service Worker 或全栈服务器时,升级条件是独立 wildcard preview origin + 专用只挂载当前项目目录的容器 + HTTP/WebSocket/SSE 代理、健康检查和空闲回收,而不是放宽当前同源 sandbox。
### 8.18 Provider API 凭据动态控制面implementation2026-09-02
模型、媒体、平台数据与语音 Provider 采用代码内静态可信注册表Provider id、显示名、凭据字段组、固定测试协议、余额能力和巡检频率均由代码定义Admin 只能提交凭据值,不能编辑目标 URL、`api_base`、模型、协议或请求模板。这一控制面与可编辑的用户外部系统 definition 分离,避免把平台根凭据送往管理员可变目标。
`provider_credentials` 每个 Provider 只保存一套当前凭据。各字段以 `provider + field` 为 AES-GCM AAD 独立加密,未配置 `ZCBOT_CREDENTIAL_MASTER_KEY` 时拒绝写数据库,不存在明文降级。数据库覆盖优先;没有完整覆盖时沿用既有 env模型能力、价格、`model_id` 和 `api_base` 仍以 `config/models` / `config/media` 为事实源。
凭据生命周期采用请求级解析LLM 在每次构造新的 provider 请求时解析一次,媒体、检索和语音在每次新 HTTP/WebSocket 会话开始时解析;因此更新后下一次请求立即生效,已经发出的 HTTP 或流式响应保持启动时的 Key。候选凭据先在内存中测试认证成功含低余额告警后才以 revision 乐观并发原子替换;失败不改旧密文。
Provider 测试将 402/明确额度不足、认证失败、网络不可达和普通 429 分开分类。DeepSeek 读取官方 CNY 余额,`< ¥30` 为低余额30 分钟巡检;其他 Provider 没有官方余额字段时只展示认证/连通状态。业务调用的明确 402、额度耗尽或认证失败进入同一状态与提醒入口。后台一轮以 PostgreSQL advisory lock 在蓝绿实例间选主,首次异常立即邮件,持续异常最多每日一次,恢复后重置提醒状态。
---
## 附录:DeepSeek V4 关键事实(2026-04-24)
- V4-Pro:1.6T/49B 激活,1M context,SWE-Bench 80.6;V4-Flash:284B/13B 激活,1M context
- 推理:non-thinking / thinking / thinking-max;价格 in ~$0.145/M、out ~$1.74/M(约 Opus 1/6)
- 推理:non-thinking / thinking / thinking-max2026-08-16 起按工作日 UTC 峰谷计费,具体单价以官方价格页与版本化模型档案为准
- 旧 `deepseek-chat/reasoner` 已于 2026-07 下线,全库仅存 `deepseek-v4-flash/pro`

View File

@ -2,7 +2,7 @@
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`
最后更新:2026-08-26(Origin 科研分析与平台来源升级bump 0.69.0)
最后更新:2026-09-03Unreleased工具健康统一事件口径
---
@ -20,6 +20,38 @@
---
## 已完成关键能力
- **09-03 / 0.71.0 / 工具链可靠性收敛**tool arguments salvage 新增“至少两个完整且完全一致副本 + 尾部截断副本”的保守恢复,语义不一致仍拒绝。修复 artifacts 部分唯一索引谓词被参数化后 PostgreSQL 无法匹配 `ON CONFLICT` 的问题,并让输出中任意行首 `[GATE FAIL]` 均按质量门记录,避免 SVG/PPT 质检占用真实故障大数。无 schema/migration/API 变化,生产失败历史未改写。
- **09-03 / 0.71.0 / 工具健康统一写入 usage_events**:普通工具失败在 tool result 落消息后结构化写 `tool_failure`,并新增 `run_stopped`、`agent_guard`、`context_fold_failure`、`quality_gate` 四类任务异常事件;工具健康聚合彻底移除 messages JSONB 回扫,只用一次 usage_events 查询切换前的普通失败历史不迁移。Admin 大数只计真实 failure代理主动停止/重复保护/上下文异常改为独立控制区,质量门继续单列。新增 0040 migration为健康 kind 时间窗和普通失败 message 去重补部分索引。
- **09-03 / 0.71.0 / 工具健康监控请求收敛**Admin 首次进入时异步汇总工具健康大数,之后固定按 90 秒节流刷新并对进行中请求单飞,不再每 10 秒重复扫描近 7 天记录。服务端按查询参数增加 90 秒进程内单飞缓存多页面或多管理员并发时共享一次扫描日巡检继续直读不受缓存影响。本提交本身未新增表、字段、migration结构化事件口径由后续同日改动完成。
- **09-03 / 0.71.0 / 管理后台紧凑总览与分组工作区**Admin 由纵向长页改为顶部常驻的两级总览和“运行、用量、用户、资源、集成、质量”六个页签四项运行状态使用主卡、四项运营指标收进紧凑指标栏点击总览可切换页签并精确定位同口径详情任务、容量、用户活跃、Token/成本和存储均补齐详情指标。页签 hash 支持刷新与前进后退桌面端页签吸顶移动端顶栏操作收进菜单、运营指标与页签可横向滑动1440px/390px 浏览器渲染、六页签与深链交互检查通过,无 API、schema、migration 或运行方式变化。
- **09-02 / 0.71.0 / 单用户重型执行容量与排队提示**:共享执行容量保持整机 6、后台 4单用户上限由 2 提升到 3前台重型任务首次等待时通过 SSE 告知单用户、整机、内存压力或队列顺序原因,放行后恢复正常执行提示,排队仍不计入命令 timeout 且可取消。无 schema、migration 或 API 变化。
- **09-02 / 0.71.0 / Admin Provider API 凭据管理与余额提醒**:新增代码内静态可信 Provider 注册表和单行当前凭据控制面0039 以 AES-GCM 字段级 AAD 密文保存Admin 可查看 DB/env 来源与尾号、先测后原子替换、手动测试和删除覆盖LLM/媒体/平台来源/语音均按新请求热解析。DeepSeek 每 30 分钟检查官方 CNY 余额,低于 ¥30 及明确 402/认证失败进入统一状态与邮件提醒,蓝绿实例以 PostgreSQL advisory lock 单轮选主;持续异常每日最多一次、恢复重置。未连接生产数据库、未调用真实第三方 API。
- **09-03 / 0.71.0 / FreeCAD Windows Node 接入**:新增 `freecad.model.author@v1` / adapter 1.0.0 与独立 `freecad` runtime目标仅为正式版 FreeCAD `>=1.1.3,<1.2.0`。两层 Worker 由受管 Python 负责 `FreeCADCmd.exe` 发现、版本门禁、safe mode/独立配置与 Pillow 等轴测渲染,由 FreeCAD 自带 Python 固定调用 PartRecipe 每次从空文档完整重建,参数引用先解析为数值,不写 Spreadsheet/Expression。首版覆盖五类基础体、STEP import、三类布尔、平移/旋转/组合,固定交付 FCStd、STEP、Recipe、manifest、预览和 provenance并执行 recompute、Shape 有效性、FCStd/STEP 重开及实体数/包围盒/体积容差校验。Windows Node 增加 FreeCAD 软件卡、UI/环境变量/PATH/卸载注册表/标准目录检测与独立/完整包集成;纯 Recipe、假版本 probe、合同、源码和打包自动测试已落地FreeCAD 1.1.3 真机 acceptance 待目标机执行,未伪造通过。
- **09-02 / 0.71.0 / 中性 CAD 几何准备**:新增无状态 `cad.geometry.prepare@v1`,以独立 `cad` runtime 和固定 CadQuery 2.8.0/OCP 7.9.3.1 生成供 ANSYS/Abaqus 复用的 STEP首版含 inspect、八类基础体和受控 Recipe覆盖显式二维截面、拉伸/旋转/扫掠/放样、布尔、圆角/倒角/抽壳、变换、阵列、STEP import 与多实体组合。固定交付 STEP、规范化 Recipe、几何 manifest、无 GUI 预览和 provenance加入单位、请求/输入/步骤/点/实体/区域/超时与有限数防线,区域选择与 ANSYS v2 语义一致且支持 `expected_count`。Windows Node 新增 runtime-only 软件卡和基本体/STEP roundtrip probe通用合同生命周期让旧 `ansys.geometry.inspect@v1` 停止普通推荐、保留显式诊断与历史兼容。本地数值 oracle、STEP 重开、确定性 manifest、区域与固定打包已验证ANSYS/Abaqus 真实导入仍待目标机验收,未伪造通过。
- **09-02 / 0.71.0 / 管理后台运行总览分层**Admin 八张摘要卡按当前运行与运营资源重排,主指标统一为近期或实时口径,累计与低频明细降级;执行容量新增可自动刷新的右侧详情抽屉,集中展示前后台队列、容器生命周期、单用户占用与宿主资源,并移除物理占用与逻辑配额口径不一致的存储进度条。无 API、schema、migration 或运行方式变化。
- **09-02 / 0.70.0 / 管理与更新日志界面收敛**Admin 顶部“容器状态”直接承载前后台执行、排队、回收、宿主资源与单用户占用,移除重复的独立容量区块,并统一采用“容器依赖”“专业软件节点”用户文案;用户版更新日志只保留可感知变化,公开接口新增向后兼容的 offset 分页元数据,前端每页加载 5 个版本并按需继续加载。
- **09-01 / 0.70.0 / Sandbox 执行容量与临时依赖观测**:默认单容器资源提升到 4 GiB/2 CPU、pids 1024、shm 512 MiB、tmp 1 GiB普通容器无活动 10 分钟回收;新增跨蓝绿/多 Web 实例共享的整机 6、后台 4、单用户 2 重型槽位,前台可取消排队且排队不计 timeout活动计数防长命令被 reaper 误删内存压力只暂停新放行。Docker 后台 proc 无槽时持久化 queued、约 30 秒恢复调度,运行中仍由 Docker 托管,终态目录继续保留 7 天。镜像构建基础 Python 包清单,容器删除前可信扫描 `/tmp/.local``.dist-info` 并以单会话表幂等记录非空差异Admin 顶部“容器状态”集中展示实时容量,“容器依赖”提供 7/30 天/全部依赖会话统计。新增 0038 migration未连接或迁移生产数据库、未启动真实专业软件。
- **08-31 / 0.70.0 / Windows Node 完整 WPF 管理界面**本机任务、ANSYS 固定验收、诊断复制和数据目录迁移全部进入 WPF任务页每秒刷新本地持久化记录并保持选择验收支持进度、停止、通过报告校验与机器级执行门数据迁移继续经过活动任务检查、停连、复制、SHA-256 校验、失败回滚和重启。新增 `JobMonitorService`、`AnsysAcceptanceService`、`DataRootManagementService` 与 `DiagnosticService`,删除经典 `ConfigurationForm` 和旧 `TrayApplicationContext`WinForms 只保留托盘及受控系统对话框互操作117 项 .NET 行为测试、28 项 Windows Node 源码专项、格式与 diff 检查通过,未启动专业软件、连接生产服务或读写真实节点数据。
- **08-31 / 0.70.0 / Windows Node Host/WPF 重构阶段 06 第二切片**WPF 在节点概览、首次注册和运行设置基础上新增专业软件页Origin、ANSYS、Blender 卡片通过 `SoftwareManagementService` 统一获取位置、runtime 和 probe 快照,可选择或恢复自动检测位置、安装/更新独立 Python 3.12 环境并取消安装。选择对话框仍限制在 Tray WinForms 互操作层ViewModel 不访问注册表或启动进程;全机执行门会禁用安装,`ManagedRuntimeInstaller` 入口继续二次拒绝活动任务并保持临时验证、原子替换和失败 rollback。同步完成 WPF 视觉打磨:主题 token、圆角控件模板、导航选中态、状态色带/徽标、软件可用性标识、忙碌进度和窄窗口操作换行统一落地。ANSYS 验收、本机任务和数据目录迁移仍从经典配置进入。73 项 .NET 行为测试、191 项 Origin/ANSYS/Blender/合同/Node/UI 专项、格式与 diff 检查、Release 发布包及隔离 GUI/headless 烟测通过,未安装 runtime、启动专业软件、连接生产服务或读写真实节点数据。
- **08-30 / 0.70.0 / 国内模型版本化计费与历史矫正工具**:新增本地价格目录解析,直连 DeepSeek、豆包、智谱不再被 LiteLLM 滞后价目覆盖DeepSeek 按 2026-08-16 官方新价及工作日 UTC 峰谷窗口计费豆包使用人民币固定价GLM 绑定国内 BigModel 标准按量价。主对话、Prompt 润色、自动标题、上下文折叠和知识库摘要统一提取缓存 usage 并保存价格 revision/tier/币种/汇率/分项快照;新增默认 dry-run、显式确认 apply/rollback 的历史重算脚本,可针对调价后 DeepSeek 事件审计并矫正成本。无 schema/migration未连接或写入生产数据库。
- **08-27 / 0.70.0 / GLM-5.3 Flash + 通用 reasoning/多模态能力**:下架 GLM-5.1/5.2 的可选入口并以隐藏别名将存量 `glm.pro/pro52` 统一解析到 `glm.flash53`Flash53 加入默认与专业档位;模型能力新增原生输入模态和 `none/tool_turn/conversation/provider_managed` reasoning 生命周期历史状态仅向同一生产模型回放DeepSeek 限当前工具轮、GLM 保留同模型会话并发送 `clear_thinking=false`。结构化图片附件仍以文件引用为事实源,只在 provider 请求边界安全物化为 Base64 `image_url`GLM 不再注册 `look_at_image`,文档读取与生图/视频工具保持独立;无 schema/migration/依赖变化,未调用真实模型或连接生产数据库。
- **08-27 / 0.69.1 / 专业软件结果轨与终态收敛**:当前对话的专业软件 Job 按独立状态点堆叠在聊天左侧,悬停展示名称与进度,成功的 Workspace 任务可直接打开多预览画廊,移动端收敛为“软件结果”入口;主消息流只保留用户与 Agent 叙事,默认 `report` 完成后只更新 Job 账本和视图,显式 `analyze` 才重新进入 Agent 对话。Web SSE 终态统一延后到任务状态落库之后发送,结果刷新会清理不再活动的 live card避免软件已出图但“思考中”被重新点亮。既有历史软件报告消息继续兼容显示无 schema、migration 或 HTTP API 变化。
- **08-26 / 0.69.0 / Origin 科研分析 MVP**:新增非默认的 `origin.analysis@v1` / adapter 0.1.0,与稳定的 `origin.plot@v2` 分离;首批提供数据体检、最大值/面积/参考点归一化、固定局部多项式平滑、一/二阶导数、区间/累计梯形积分和带 95% 置信区间的线性拟合,固定输出可编辑 OPJU、CSV、诊断、分析规格与溯源可选 XLSX线性拟合工程同时保留 Origin 原生报告。六个合成真值用例在 Origin 2024 / originpro 1.1.15 / numpy 2.2.6 上完成生产 Worker、数值 oracle、OPJU 重开和进程释放门;前三例来自组合执行的逐例通过记录,后三例分别通过,目标机 COM 完整退出约需 2 分钟。Origin/合同/Job/Node 专项 159 项 unittest、Release .NET build、完整 Node publish 与独立 adapter ZIP 校验通过,未连接或写入数据库。
- **08-26 / 0.69.0 / platform_sources bounded context**:将 paper_server、内部材料库和 Materials Project 收敛为独立顶层包,以显式可信 Provider 列表统一可用性与生命周期,单来源配置/装配失败不阻断其他来源;`core/tool_registry.py` 只保留一个构建入口,与用户连接 `external_systems` 完全独立。模型工具统一改为来源明确的 `paper_server_*`、`materials_library_*`、`materials_project_*`保留材料库批量检索、MP 专用 SDK/CIF/entries 语义及 paper_server 安全下载边界;部署环境变量仍是事实源,无数据库迁移、无版本提升。
@ -124,6 +156,7 @@
- **08-12 / 0.64.1 / PDF/PPT 连续预览滚轮与触摸修复**:外层预览的防滚动穿透逻辑不再拦截来自 PDF 内层 viewport 的普通 wheel 事件,保留图片 Ctrl+滚轮缩放和其他预览的边界隔离PDF viewport 同步声明触摸平移与 WebView/iOS 惯性滚动使鼠标滚轮、触控板和手机上下滑动都直接驱动连续页列表。Node 前端预览 11 项、JavaScript 语法及 diff 检查通过;无 schema、migration、HTTP API、依赖或运行方式变化未连接生产 DB。
- **08-12 / 0.64.0 / artifact 稳定身份 + 隐藏回收**:新增 0028 `artifacts` 生命周期表并回填存量结构化引用;新发布消息写带 `artifact_id` 的 v2 引用,移动/重命名保持身份,复制创建独立身份并记录来源,历史卡片按身份解析最新路径。删除已发布产物时移动到用户隐藏目录 `.zcbot_artifact_trash/` 并软删记录普通文件仍物理删除Python 35 项(测试库门控 1 skip、Node 前端 11 项、mypy、Alembic 单 head、编译、Ruff 致命规则及 diff 检查通过,未连接或写入生产 DB。
- **08-28 / 0.70.0 / 回收站与用户配额分离**:0037 为 `user_disk_usage` 增加回收站字节与文件数快照;后台扫描从用户配额中排除 `.zcbot_artifact_trash`同时保留其物理占用统计Admin 分列展示配额占用、回收站和物理合计。相关 pytest 48 项与 41 个子测试、Python 编译、JavaScript 语法、Alembic 单 head、Ruff 致命规则及 diff 检查通过;本机 Docker daemon 未运行DB 集成测试按安全门控跳过,未连接或写入生产 DB。
- **08-12 / 0.63.11 / PDF/PPT 连续预览滚动布局修复**:补齐 PDF 预览根容器的纵向 Flex 布局与可收缩高度约束,使内部连续页 viewport 获得实际可用高度和独立滚动区域,不再被外层 `overflow:hidden` 裁断回归测试同步锁定根容器、viewport 与页列表三层布局契约。Node 前端预览 11 项、JavaScript 语法及 diff 检查通过;无 schema、migration、HTTP API、依赖或运行方式变化未连接生产 DB。
@ -428,7 +461,9 @@ core/llm_transport.py 438 ← wire 层健壮性:畸形/吐空检测+留
core/tool_registry.py 264 ← 声明式工具注册表((组名,gate,factory);secret/host 工具按实际能力 gate)
core/context.py 95 ← LLM 调用前压缩旧 tool / load_skill 消息(带压力门槛),保 tool_call 协议字段
core/external_systems/*.py ← 外部系统目录/用户授权/凭据加密 + 通用 OpenAPI/MCP connector
core/provider_credentials/*.py ← 静态平台 Provider 目录、DB/env resolver、候选测试、状态提醒与蓝绿巡检
core/software_nodes.py ← Windows Node 注册码、身份认证与运行状态
core/software_contracts.py ← 专业软件合同热加载、生命周期/可见性与 feature 兼容门
core/sinks.py 101
core/paths.py 50 ← task_dir db form 归一
core/probe.py 243
@ -445,10 +480,10 @@ core/ark_client.py 105 ← 火山方舟 HTTP 客户端
core/asr_xfyun.py 170 ← 讯飞语音听写 IAT wss 客户端(整段 PCM→文本;web 语音输入用,diag: scripts/diag_asr.py)
core/asr_lfasr.py 250 ← 讯飞录音文件转写 LFASR 客户端(异步订单 + 说话人分离;transcribe_audio 工具底座,diag: scripts/diag_lfasr.py)
core/agent_builder.py 649 ← 装配 lib:build_agent/system prompt(工具注册块已迁 tool_registry)
core/executor.py / sandbox/{network,pool}.py / executor_docker.py ← Executor ABC + Docker per-user 容器池
core/executor.py / sandbox/{network,pool,capacity,package_scans}.py / executor_docker.py / procs.py ← Executor ABC + Docker per-user 容器池 + 宿主共享执行槽 + bg proc 文件队列 + 临时依赖扫描
tools/{base,output,fs,shell,run_python,skill_tool,skill_authoring,media_common,seedream,seedance,gpt_image,look_at_image,read_document,image_ref,web_search,web_fetch,documents,materials_project,transcribe_audio,office_to_pdf,external_systems}.py ← media_common=媒体五工具共享原语;external_systems=host-side 外部系统元工具
main.py ~210 ← 入口:web / db / probe / user / sandbox check
db/migrations/versions/ 0001-0030
db/migrations/versions/ 0001-0039
web/app.py ~210 ← 工厂 + lifespan 编排(07-23 拆分;路由在 routers/,协程在 background 等)
web/routers/*.py ← 含 external_systems 用户连接与 software_nodes 节点路由
web/{background,scheduler_runner,wechat_runner}.py ← lifespan 后台协程按域析出
@ -456,7 +491,8 @@ web/{runs,common,schemas,model_gate,userfiles}.py ← BG worker/共享 helper/
web/auth.py ~190 ← 邮箱密码 + platform_key → JWT
web/broker.py / sinks.py / pptx_render.py / admin.py
web/static/dev.html + admin.html + ui.css + js/*.js ← dev SPA 与管理后台零构建前端
windows-node/ ← .NET 10 WinForms 托盘节点、注册与登录后自启动
software-contracts/ ← 语言无关专业软件合同(含 FreeCAD、中性 CAD 与生命周期)
windows-node/ ← .NET 10 WPF/托盘节点、固定 adapter、FreeCAD/CAD runtime 与发布脚本
web/static/vendor/ ~1 MB ← jszip / docx-preview / xlsx
tests/ 360 项 ← golden/rendering 基线 + 路由两套(nodb/db)+ DB 套件(ZCBOT_TEST_DB_URL 门控)
─────────────────────────────────

92
RUN.md
View File

@ -74,13 +74,13 @@
# SMTP_PASSWORD=<授权码/应用专用密码,非登录密码>
# SMTP_FROM=you@qq.com # 可选,默认取 SMTP_USER
# SMTP_FROM_NAME=总院科研辅助智能体 # 可选,发件人显示名,默"总院科研辅助智能体"(不暴露内部代号)
# 工具失败聚集巡检(0.58.11,反 9dcae061"挂 90 天无人知"):web 进程每天扫近 7 天
# tool 错误消息 + 被丢弃的畸形 tool_call 参数(0.58.19,kind=malformed),同签名 >=5 次
# 工具失败聚集巡检(0.58.11,2026-09-03 改为只扫 usage_events):web 进程每天扫近 7 天
# 结构化工具/运行异常事件(含畸形 tool_call 参数),同签名 >=5 次
# 且跨 >=2 task 判聚集 → 发下面邮箱(复用上面 SMTP_*;未配邮箱或 SMTP 则只打日志
# [toolfail] 行)。**只发近 24h 仍活跃的聚集**(0.58.19):去重集是内存态、每次部署清零,
# 不加活跃过滤的话高频部署期每次重启都会把 7 天窗口内早已安静的存量聚集重发一遍。
# admin 页仍取低阈值全量,分区展示:近24h跨任务系统性故障 / 单任务反复失败 /
# 按设计拦截的质量门 / 已安静历史;修复后看对应项进入“已安静”。同区的“工具
# 代理控制与上下文异常 / 按设计拦截的质量门 / 已安静历史;修复后看对应项进入“已安静”。同区的“工具
# 调用链路健康”按模型档+工具展示 tool_salvaged/tool_malformed 的24h/7d抢救率。
# ZCBOT_DEVELOPER_EMAIL=dev@example.com
# ZCBOT_TOOLFAIL_SCAN_INTERVAL=86400 # 秒,默 86400;<=0 关掉巡检
@ -93,9 +93,8 @@
# 单次 LLM 请求超时(秒),默 600(与 litellm 默认一致,显式化 + 可调)。长思考模型
# 被掐("600s 无字节 → run 标 error")再调大;流式正常出 chunk 不触发。
# ZCBOT_LLM_TIMEOUT_S=600
# 后台进程(bg proc,DESIGN §8.12;shell/run_python background=true):每用户并发上限,默 3
# 后台进程与前台重型执行统一受 sandbox.max_* / ZCBOT_MAX_* 共享容量控制
# 状态锚 <user_root>/.zcbot_procs/,后台默认限时 7200s(工具 timeout 参数可调,cap 86400)。
# ZCBOT_MAX_BG_PROCS=3
# 静态多文件 Web 项目预览会话 TTL(秒),默认 7 天,平台限制 5 分钟30 天。
# 改后重启 web 生效;只影响新建/再次发布的预览。
# ZCBOT_WEB_PREVIEW_TTL_SECONDS=604800
@ -141,7 +140,8 @@
# ZCBOT_CREDENTIAL_MASTER_KEY=<至少 32 字符随机串>
```
> litellm 在 import 时副作用加载 .env;入口走 `main.py`,`.env` 自动生效。直跑 `python -c "from core.storage import ..."` 不经 litellm 链路时记得自己 `import litellm` 触发,或手动 `export ZCBOT_DB_URL=...`
- **平台托管来源配置**:`PAPER_SERVER_*`、`DOCUMENT_SEARCH_*`、`MP_API_KEY` 由宿主部署环境提供,修改后重启 web 生效不进数据库、用户连接、prompt、tool result、`run_python` 或 Docker sandbox。若需求升级为用户级凭据/per-user grant接入 `external_systems`;若需要在线动态 definition、revision/reverify 或 OAuth先设计独立 `platform_sources` 控制面,不直接扩展当前静态 env registry。
- **平台 Provider 凭据**`DEEPSEEK_API_KEY`、`ZHIPUAI_API_KEY`、`ARK_API_KEY`、`UNIFYLLM_API_KEY`、`LOCAL_LLM_API_KEY`、`BOCHA_API_KEY`、`PAPER_SERVER_API_KEY`、`DOCUMENT_SEARCH_API_KEY`、`MP_API_KEY` 和两组 `XFYUN_*` 可继续由宿主 env 提供,也可在 0039 migration 后由 Admin「API 凭据」加密覆盖。数据库覆盖在下一次新外部请求生效;删除覆盖立即回退当前 env。URL、模型和协议继续由代码/YAML 管理Admin 不可编辑。凭据不进入 prompt、tool result、`run_python` 或 Docker sandbox。
- **Provider 凭据部署**:先配置至少 32 字符的 `ZCBOT_CREDENTIAL_MASTER_KEY`(轮换沿用 `ZCBOT_CREDENTIAL_KEY_ID` / `ZCBOT_CREDENTIAL_PREVIOUS_KEYS`),再执行 `.venv/Scripts/python.exe main.py db upgrade head` 创建 `provider_credentials`。蓝绿部署须先迁移、后启动新代码。未配置 master key 时 env 调用保持可用,但 Admin 保存会明确拒绝master key 丢失或 AAD 不匹配时已有 DB 覆盖不会静默回退 env应恢复正确 keyring 或在可解密后删除覆盖。
- **依赖**:`pip install -r requirements.txt`(已在 `.venv` 里;含 `bcrypt`、`segno`、`cryptography`)。
- **微信接入(ClawBot,§8.7)**:① `main.py db upgrade head` 带上 migration `0012`;② `.env``ZCBOT_WECHAT_BOT_ENABLED=1` + `ZCBOT_WECHAT_SECRET_KEY=<串>`;③ 用户登录后点**左栏 rail「微信」按钮**(`/static/wechat_bind.html` 仍保留作独立/嵌入入口)扫码绑定(需个人微信 8.0.70+ 且灰度到 ClawBot 插件)。绑定后在微信「微信 ClawBot」对话即走 zcbot;**主动推送需用户近 24h 在微信开口过一次**(冷启动/超期推不出,退邮件兜底)。**支持语音消息**(voice_item SILK v3 → pilk 解码 → 讯飞 IAT 转写进对话,回执「🎤 已识别:…」;需 `XFYUN_*` 三件套 + ffmpeg + pilk,pilk 随 requirements 装)。
- **企业微信(渠道 B,纯推送,§8.7)**:① 管理员建自建应用 → 填 `WECOM_CORPID/AGENTID/SECRET`(+ 可见范围含目标用户);② `main.py db upgrade head`。**绑定两条路,任选**:
@ -156,7 +156,7 @@
- **channel 长会话上下文(微信/企业微信通用,0019)**:常驻会话不再无限膨胀。① **自动分段**——入站时距上次消息超过 `config.json``channel.session_gap_hours`(默 **6** 小时,设 `<=0` 关闭)→ 软重置:只把「最后一条 user 消息起」喂模型(保留上一轮做续聊锚点),之前的历史仍全留 DB,网页端照旧翻完整记录;② **手动新话题**——用户在微信/企业微信里直接发「新话题 / 新会话 / `/new` / 清空上下文」→ 硬重置,彻底从零(回执提示已归档)。两者都**不删任何消息**,只移动「喂给模型的窗口起点」`tasks.context_base_idx`。网页端「清空对话」(`POST /v1/tasks/{id}/clear`)仍整清并把 base 归 0。需 `main.py db upgrade head` 带上 `0019`
- **PG**:`ZCBOT_DB_URL` 必填。本地 docker compose / 远端 dev / 生产任选;未设置时启动清晰报错,不引导 docker(§7.4)。
- **OpenAPI / MCP 外部系统**:① `.env` 配置独立的 `ZCBOT_CREDENTIAL_MASTER_KEY`,可选 `ZCBOT_CREDENTIAL_KEY_ID` 标识当前密钥;轮换时把旧 key 以 JSON 对象放入 `ZCBOT_CREDENTIAL_PREVIOUS_KEYS`,待用户凭据完成重写后再移除。② 执行 `main.py db upgrade head`。③ admin 进入管理后台「外部系统」,选择通用 OpenAPI 或通用 MCP具体 MES/ERP/LIMS 都作为数据库 definition 配置,不新增专用 provider。MCP 填写与登录 Base URL 同源的 Streamable HTTP URL可选填写期望 Server 名称;连接后以 `tools/list` 为事实源。④ 普通用户点击左栏 **「外部」**,页面按 definition 动态显示用户名密码、API Key 或 Bearer Token目标、Server 身份、登录、认证绑定或 TLS 变化后保留密文并暂停调用重新测试成功后恢复。OpenAPI spec 和 MCP tool catalog 只在进程内按连接身份有界缓存,登录与业务响应均限长,普通用户和模型不能传任意 URL。
- **Artifact 生命周期(0028)**:部署本版本必须先执行 `.venv/Scripts/python.exe main.py db upgrade head`。migration 会从存量 `messages.artifact_refs` 回填 active artifact 身份;删除后的已发布产物保存在用户根目录隐藏区 `.zcbot_artifact_trash/`,默认不自动清理且继续计入磁盘配额。普通文件删除语义不变。
- **Artifact 生命周期(0028/0037)**:部署本版本必须先执行 `.venv/Scripts/python.exe main.py db upgrade head`。migration 会从存量 `messages.artifact_refs` 回填 active artifact 身份;删除后的已发布产物保存在用户根目录隐藏区 `.zcbot_artifact_trash/`,默认不自动清理、不计用户配额,但其物理占用由后台扫描单列并展示在 Admin。普通文件删除语义不变。
- **用户消息结构化附件(0031)**:部署新版前执行 `.venv/Scripts/python.exe main.py db upgrade head`,为 `messages` 增加 nullable `attachment_refs`。migration 不回写或改动存量正文;旧客户端仍可只传 `content`,新版会发送结构化 `attachments`
- **旧 `factory_mes` definition 一次性转换**:新版代码不再识别 `factory_mes`;部署时保持旧服务进程运行,先从新代码目录执行数据脚本,转换成功后再重启到新版。脚本不加载 `.env`、不读取 `ZCBOT_DB_URL`,只认显式的 `ZCBOT_MIGRATION_DB_URL`;默认 dry-run检查同名冲突与配置合法性。确认输出后加 `--apply`,脚本把 definition 转为 `generic_openapi + query`、物化 JWT/提示/只读 POST 配置并同步 active connection revision不解密或改写用户凭据。
```powershell
@ -177,7 +177,7 @@
容器留着复用即可(alpine 版很轻);schema 升级后重跑一次 `db upgrade head`
- **Auth env**:`PLATFORM_KEY` + `JWT_SECRET` 任一缺失 web 启动 fail-fast。生成随机串:`python -c "import secrets; print(secrets.token_urlsafe(48))"`。
- **用户管理**(`users.email/password_hash/role`,0005 UNIQUE(email)、0009 role):dev SPA 登录后端。发用户两条路径任选:CLI `main.py user add`(下方),或在登录页右下角"+ 管理员添加用户"链接(需先设 `ZCBOT_ADMIN_TOKEN` env,弹窗输入 email/密码/管理员口令/角色)。撤用户 `DELETE FROM users WHERE email=...`(先 DELETE 该 user 的 tasks)。**用户自助改密**:登录后顶栏「改密码」按钮(走 `POST /v1/auth/change_password`,需知道旧密码);改邮箱 / 用户忘了旧密码无法自助 → 手动 SQL(见故障兜底)。
- **角色与管理后台**(`users.role` ∈ `user`/`admin`):admin 才显顶栏"管理"入口 → `/static/admin.html`(非 admin 403)。页面:左侧目录(点击滚到对应区)+ 运行态/任务/用户用量/按模型/各用户用量/存储;「按模型」「各用户用量」支持时间筛选(全部/近7天/近30天)+ 排序(按成本/按用量),「各用户用量」「存储」分页;顶栏「导出 PDF」走浏览器打印(在打印对话框选"另存为 PDF",列表取前 10)。提管理员 `main.py user role --email X --role admin`(改完即时生效,role 走 DB 查不进 JWT)。`ZCBOT_ADMIN_TOKEN` 是另一回事(发用户共享口令),与 role 互不相干。
- **角色与管理后台**(`users.role` ∈ `user`/`admin`):admin 才显顶栏"管理"入口 → `/static/admin.html`(非 admin 403)。页面顶部常驻紧凑总览,下方按“运行/用量/用户/资源/集成/质量”页签切换;点击总览卡会进入对应页签并定位同口径详情。执行容量也可从详情侧栏查看;「按模型」「各用户用量」支持时间筛选(全部/近7天/近30天)+ 排序(按成本/按用量),「各用户用量」「存储」分页;顶栏「导出 PDF」走浏览器打印(在打印对话框选"另存为 PDF",列表取前 10)。提管理员 `main.py user role --email X --role admin`(改完即时生效,role 走 DB 查不进 JWT)。`ZCBOT_ADMIN_TOKEN` 是另一回事(发用户共享口令),与 role 互不相干。
---
@ -215,7 +215,7 @@ python -m venv .venv
```bash
# 模型能力对账(费 token)
.venv/Scripts/python.exe main.py probe --model deepseek_v4.flash
.venv/Scripts/python.exe main.py probe --model glm.pro # 智谱 GLM-5.1(走 litellm zai provider + 国内站 bigmodel.cn)
.venv/Scripts/python.exe main.py probe --model glm.flash53 # 智谱 GLM-5.3 Flash(原生多模态,国内站 bigmodel.cn)
.venv/Scripts/python.exe main.py probe --model local.r1 # 内网 DeepSeek-R1(满血,128K),涉密任务用;需 .env 设 LOCAL_LLM_API_KEY
.venv/Scripts/python.exe main.py probe --model local.qwen3 # 内网 Qwen3-30B-A3B(MoE,原生 32K);共享 LOCAL_LLM_API_KEY
@ -361,6 +361,10 @@ $env:ZCBOT_EVAL_TOKEN = "<dedicated-eval-user-jwt>"
| `GET /v1/skills/{name}` | 返某 skill 完整 SKILL.md 正文(前端「技能」modal 点开查看);同名按 user wins | 必填 |
| `DELETE /v1/skills/{name}` | 删当前 user 私有 skill(`.skills/<name>/` 整目录);只删 user 源,内置不可删 → 404;`.skills` 文件面板隐藏,这是 UI 上删自己 skill 的唯一入口 | 必填 |
| `GET /v1/external-system-providers` | 列管理员已启用的外部系统目录和安全的动态凭据字段声明;不返回完整配置或密钥 | 必填 |
| `GET /v1/admin/provider-credentials` | Admin 查看静态 Provider 目录、来源、脱敏尾号、测试/余额/提醒状态;不返回密文或明文 | admin |
| `PUT /v1/admin/provider-credentials/{provider_id}` | Admin 提交完整凭据组与 `expected_revision`;候选先测试,成功后原子替换 | admin |
| `POST /v1/admin/provider-credentials/{provider_id}/test` | Admin 测试当前 DB/env 凭据;标记 billable 的 Provider 前端会二次确认 | admin |
| `DELETE /v1/admin/provider-credentials/{provider_id}` | Admin 按 revision 删除数据库覆盖并回退 env | admin |
| `GET/POST /v1/external-systems` | 列当前用户连接 / 新建并在线验证连接;创建 body `{definition_id,name,credentials}`,旧 Factory `{username,password}` 请求继续兼容 | 必填 |
| `PUT /v1/external-systems/{id}/credentials` | 用 `{credentials}` 重新提交并在线验证当前用户连接;凭据不提供读取接口,旧用户名/密码格式继续兼容 | 必填 |
| `POST /v1/external-systems/{id}/test` | 用已保存密文凭据测试登录和 Swagger 可读性,并更新连接状态 | 必填 |
@ -392,7 +396,7 @@ $env:ZCBOT_EVAL_TOKEN = "<dedicated-eval-user-jwt>"
| `GET /v1/models` | 列 chat LLM 模型清单(扫 `config/models/*.yaml`),前端顶栏切换 / 新建对话框下拉用 | 必填 |
| `GET /v1/image_models` | 列图像生成 variant 清单(扫 `config/media/doubao.yaml` image 段),前端"生图"下拉用;yaml 无 image variant → 空列表 → UI 隐藏下拉 | 必填 |
**SSE 事件**(每帧 `event: <type>` + `data: <JSON>`):建连时若当前 run 已发布计划,先补 `progress_snapshot{run_id,steps,waiting}``run_start{}``llm_start{}``text{delta}` / `tool_call{name,args,args_preview}` / `tool_result{name,preview,truncated}` → `llm_end{prompt_tokens,completion_tokens}``done{}`;cancel 走 `cancelled{}` 后随 `done{}` 收流;异常走 `error{msg}`。`task_progress` 新协议每次携带完整 `steps`,客户端整体替换;消息分页响应也附加 `progress_snapshot`,刷新不依赖当前 30 条窗口。`waiting=true` 表示本轮已调用 `ask_user` 等待确认;正常完成回看时隐藏进度,等待/取消/异常则折叠保留。30s 无 event 服务端发 `: ping` 心跳。nginx 反代记得关 buffering(响应头已带 `X-Accel-Buffering: no` 默认起效)。
**SSE 事件**(每帧 `event: <type>` + `data: <JSON>`):建连时若当前 run 已发布计划,先补 `progress_snapshot{run_id,steps,waiting}``run_start{}``llm_start{}``text{delta}` / `tool_call{name,args,args_preview}` / `execution_queue{state,reason,...}` / `tool_result{name,preview,truncated}` → `llm_end{prompt_tokens,completion_tokens}``done{}``execution_queue` 仅在前台重型工具确实等待共享容量时发送 `waiting`,放行后发送 `admitted`cancel 走 `cancelled{}` 后随 `done{}` 收流;异常走 `error{msg}`。`task_progress` 新协议每次携带完整 `steps`,客户端整体替换;消息分页响应也附加 `progress_snapshot`,刷新不依赖当前 30 条窗口。`waiting=true` 表示本轮已调用 `ask_user` 等待确认;正常完成回看时隐藏进度,等待/取消/异常则折叠保留。30s 无 event 服务端发 `: ping` 心跳。nginx 反代记得关 buffering(响应头已带 `X-Accel-Buffering: no` 默认起效)。
**SSE 客户端注意**:浏览器原生 `EventSource` 不支持自定义 header,无法塞 Bearer token。要么 `fetch + ReadableStream` 自解 SSE 帧(dev.html 走的就是这条),要么后端日后加 `?token=...` query(目前不支持,避免 token 进 access log)。
@ -678,12 +682,18 @@ sudo -u zcbot docker network create zcbot-sandbox-net
# 容器 runtime(切 gVisor 用 runsc,Firecracker 用 kata;默 runc)
# ZCBOT_SANDBOX_RUNTIME=
# 空闲多少秒回收(默 300)
# ZCBOT_SANDBOX_IDLE_TTL=300
# ZCBOT_SANDBOX_IDLE_TTL=600
# 资源限制(优先级 env > yaml `sandbox.*` > 默);改后重启 web 新起容器生效
# ZCBOT_SANDBOX_MEMORY=2g
# ZCBOT_SANDBOX_CPUS=1.0
# ZCBOT_SANDBOX_PIDS_LIMIT=256
# ZCBOT_SANDBOX_MEMORY=4g
# ZCBOT_SANDBOX_CPUS=2.0
# ZCBOT_SANDBOX_PIDS_LIMIT=1024
# ZCBOT_SANDBOX_SHM_SIZE=512m # chromium/mmdc 渲 mermaid 的 /dev/shm(docker 默 64MB 不够会挂超时)
# ZCBOT_SANDBOX_TMP_SIZE=1g
# ZCBOT_MAX_ACTIVE_EXECS=6
# ZCBOT_MAX_BACKGROUND_EXECS=4
# ZCBOT_MAX_ACTIVE_EXECS_PER_USER=3
# 三个并发 env 只用于向下收紧,代码硬上限固定为 6 / 4 / 3。
# ZCBOT_MIN_MEM_AVAILABLE=1g # 低于阈值暂停新放行,不杀正在执行的任务
# PG 实际 IP,逗号分隔。defense-in-depth ── 即便落内网三段(§7.5 #1),
# init.sh 再加一遍 DROP 规则。生产部署必填。
ZCBOT_PG_IPS=10.1.2.3,10.1.2.4
@ -705,7 +715,7 @@ ZCBOT_SANDBOX_BACKEND=docker .venv/bin/python main.py web
# 触发任一 shell / run_python 消息后,容器应已起
sudo -u zcbot docker ps --filter label=zcbot.product=sandbox
# 应看到 zcbot-sandbox-<your-uid>,STATUS = Up ...
# 5 分钟无新消息后 reaper 自动 rm
# 10 分钟无执行活动后 reaper 自动 rm
```
也可直接起一个测试容器单验 hardening(不依赖 web 进程):
@ -717,10 +727,10 @@ sudo -u zcbot docker run -d \
--label zcbot.product=sandbox \
--label zcbot.user_id=$USER_ID \
--network zcbot-sandbox-net \
--read-only --tmpfs /tmp:exec,size=512m,mode=1777 \
--read-only --tmpfs /tmp:exec,size=1g,mode=1777 --shm-size=512m \
--cap-drop=ALL --cap-add=NET_ADMIN \
--security-opt=no-new-privileges \
--pids-limit=256 --memory=2g --cpus=1.0 \
--pids-limit=1024 --memory=4g --cpus=2.0 \
-v /opt/zcbot/workspace/users/$USER_ID:/workspace \
-e ZCBOT_PG_IPS=10.1.2.3 \
zcbot-sandbox:latest
@ -823,23 +833,29 @@ docker rm -f zcbot-sandbox-<user-uuid>
docker rm -f $(docker ps -aq -f label=zcbot.product=sandbox) 2>/dev/null
```
磁盘配额查看(0008 表):
磁盘配额与回收站查看(0008/0037 表):
```bash
# 当前所有 user 用量
# 当前所有 user 的配额占用、回收站与文件物理合计
psql "$ZCBOT_DB_URL" -c "
SELECT user_id, bytes_used/1024/1024 AS mb, file_count, scanned_at
SELECT user_id,
bytes_used/1024/1024 AS quota_mb,
trash_bytes/1024/1024 AS trash_mb,
(bytes_used+trash_bytes)/1024/1024 AS physical_mb,
file_count, trash_file_count, scanned_at
FROM user_disk_usage ORDER BY bytes_used DESC;"
# 强制扫描某 user(改 yaml interval 太久不耐烦时)
.venv/bin/python -c "
from pathlib import Path
from uuid import UUID
from core.storage.disk_quota import scan_user_dir, upsert_user_usage
from core.storage.disk_quota import scan_user_dir, scan_user_trash, upsert_user_usage
uid = UUID('<your-user-uuid>')
b, c = scan_user_dir(Path('/home/lighthouse/zcbot/workspace/users') / str(uid))
upsert_user_usage(uid, b, c)
print(f'{b/1024/1024:.1f} MB / {c} files')"
root = Path('/home/lighthouse/zcbot/workspace/users') / str(uid)
b, c = scan_user_dir(root)
tb, tc = scan_user_trash(root)
upsert_user_usage(uid, b, c, tb, tc)
print(f'quota={b/1024/1024:.1f} MB, trash={tb/1024/1024:.1f} MB')"
```
DNS / resolv.conf 文件位置(host 侧):
@ -979,7 +995,7 @@ sudo xfs_quota -x -c "limit -p bhard=10g zcbot_<user_uuid>" /opt
| prod 想把 workspace 落独立数据盘 | **别用 env / 别指 ROOT 外绝对路径**(workspace 锚定 ROOT,ROOT 外会让文件面板 / agent / 新建 task 三家分叉)。用 **bind mount**`/data/...` 接到 `ROOT/workspace`,逻辑路径不变,DB 不用改。详「workspace 落独立数据盘」段 |
| 文件面板"目录尚未创建"但文件确实在 / agent 写的文件面板看不到 | workspace 被指到了 ROOT 外(旧 `ZCBOT_WORKSPACE_DIR` 绝对路径残留)→ 文件面板走 `resolve_workspace` 看一处、agent 走 DB `from_db_path`(锚 ROOT)看另一处。删掉 env、改用 bind mount(见上段),三家归一 |
| `docker run zcbot-sandbox:latest``Unable to find image` | 镜像没 build。`sudo -u zcbot docker build -f deploy/sandbox/Dockerfile --build-arg HOST_UID=$(id -u zcbot) --build-arg HOST_GID=$(id -g zcbot) -t zcbot-sandbox:latest .` |
| 后台进程(bg proc)疑似残留 / 想手工排查 | docker 模式:`docker ps --filter label=zcbot.product=proc` 列在跑的 proc 容器,`docker rm -f zcbot-proc-<id>` 手杀;host 模式:状态锚在 `<user_root>/.zcbot_procs/<task_id>/<proc_id>/`(proc.json 有 pid,exit_code 文件在 = 已结束)。web 进程每小时 sweep 自动回收(终态目录 7 天 TTL);正常终止走前端停止按钮 / `check_process(action="kill")` |
| 后台进程(bg proc)疑似残留 / 想手工排查 | docker 模式:`docker ps --filter label=zcbot.product=proc` 列在跑的 proc 容器;状态锚在 `<user_root>/.zcbot_procs/<task_id>/<proc_id>/``proc.json` 的 `state=queued` 表示等待共享容量,`exit_code` 存在表示已结束。web 约每 30 秒调度 queued 并回收终态容器,终态目录仍保留 7 天;正常终止/取消排队走前端停止按钮或 `check_process(action="kill")`。若手工 `docker rm -f`,状态会收敛为 lost。 |
| 后台进程状态显示 lost | 进程没留退出码就没了 —— 宿主重启 / OOM killer / docker daemon 重启把它带走(bg proc 扛 zcbot 重启和蓝绿,但不扛宿主级重启)。output.log 保留到中断为止,需重新 background=true 发起 |
| 镜像 build pip 报 `THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE`(本仓 requirements 未钉 hash) | **不是被篡改、也不是 require-hashes**:镜像 index 声明的 wheel hash 与它实际吐出的文件字节不符 = 该镜像存的文件损坏 / 截断(2026-06-03 腾讯源就这么坏过 litellm-1.87.0)。换源重 build:`PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple/ sudo -E bash deploy/update.sh`。验真伪:`https://pypi.org/pypi/<pkg>/<ver>/json` 看官方 sha256 是哪边对。与下面"版本滞后(Could not find)"是两回事 |
| 镜像 build pip 报 `ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', ...)` | 境内访问 PyPI 抖动。加 `--build-arg PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple/`(清华,现默认)或腾讯 / 阿里源,详 RUN.md「镜像构建」段。Dockerfile 已把 pip timeout 拉到 60s,主因仍是源不通而非超时 |
@ -1015,6 +1031,8 @@ sudo xfs_quota -x -c "limit -p bhard=10g zcbot_<user_uuid>" /opt
| `.pptx` 预览首次慢几秒 | 正常 —— soffice 冷启 + 转换 ~2-4s,转完缓存到源同目录 `.preview/<stem>.<hash>.pdf`,再点即时。源文件一改(mtime/size 变)hash 变、自动重转 |
| `materials_project_*` tool 没出现在对话里 | `.env` 没设 `MP_API_KEY`,build_agent 跳过注册。设了重启 web 即可;Materials Project 联网查询走 host-side tool,离线 pymatgen 不受影响。 |
| 豆包调价了 | 改 `config/media/doubao.yaml``price_cny_per_image` 一行 → 重启 web。**历史 usage_events 不受影响**(units jsonb 里有当时单价 snapshot,聚合查仍按旧价);新写入按新价。涨价瞬间到改 YAML 中间这段记账偏低,开发期接受 |
| 文本模型调价了 | 在 `config/models/<family>.yaml``pricing.periods` 追加带 `revision/effective_from/source_url` 的新价格版本并重启 web直连模型以本地版本化价格为准LiteLLM 只兜底无本地价格的网关。新事件会在 `usage_events.units` 快照版本、时段、币种、汇率和分项成本,历史记录不会被配置静默改写。 |
| DeepSeek 调价后的历史成本不准 | 先显式设置目标库 `ZCBOT_DB_URL`,运行 `.venv/Scripts/python.exe scripts/reprice_usage.py --from 2026-08-16T16:00:00Z` 查看 dry-run核对脱敏 DB 目标、行数和差额后,才追加 `--apply --confirm APPLY_DEEPSEEK_REPRICE`。脚本单事务、按 revision 幂等,并保存 `previous_cost_cny`;回退用相同参数加 `--rollback --confirm APPLY_DEEPSEEK_REPRICE`。缺缓存明细的旧辅助调用按未命中估算并标记。 |
| `kill -HUP <pid>``/openapi.json` 没新接口 | uvicorn **不响应 SIGHUP**(没装 handler,落 Python 默认终止;Windows 上信号本身无效)。Ubuntu 上用 `systemctl restart zcbot`,或 unit 加 `--reload` 让 uvicorn 监听文件自动重起(见"部署"段)。验证:`curl -s http://127.0.0.1:8765/openapi.json \| python3 -c 'import sys,json;print([p for p in json.load(sys.stdin)["paths"] if "auth" in p])'` |
| `systemctl restart zcbot` 要等几十秒才退 | 正常 —— 优雅 drain 在等在跑的 run 收尾(`shutdown.drain_timeout` 默 30s),没在跑 run 时秒退。journal 出现 `[shutdown] draining N in-flight run(s)` 即正常。真急(不在乎杀掉在跑 run):`systemctl kill -s KILL zcbot` |
| 部署后在跑的对话被标 `error: server restarted before run finished` | 该 run 在 drain 期内没收尾、cancel 也没在 `cancel_grace` 内退,被 SIGKILL 后下次启动 reaper 标的。多半是 run 卡在不 poll cancel 的长动作(如单次超长 docker exec)或 `TimeoutStopSec` 配得比 drain 预算还小被提前 SIGKILL。先核对 unit `TimeoutStopSec > drain_timeout + cancel_grace`;真有超长 run 把 `drain_timeout` 调大 |
@ -1101,21 +1119,29 @@ Zcbot.WindowsNode.exe
Windows Node 不需要安装脚本。解压发布包后,使用实际运行 Node 的专用 Windows 账号直接启动 `Zcbot.WindowsNode.exe`:在主窗口注册节点,按需勾选“登录 Windows 后自动启动节点”,再到“专业软件”区域逐项配置并安装本机实际拥有的软件。开机启动保存为当前账号的 Windows `Run` 项,不安装 Windows Service也不要求管理员创建计划任务。
专业软件 runtime 安装要求系统能通过 `py -3.12` 或 PATH 中的 `python.exe` 找到 Python 3.12。依赖默认显式使用清华 PyPI 镜像;如需切换官方源或院内镜像,在启动 Node 前设置 `ZCBOT_PIP_INDEX_URL`。解释器位于 `<data_root>\runtimes\origin|ansys|blender\Scripts\python.exe`;如需由集中运维提供既有受管解释器,机器级 `ZCBOT_ADAPTER_ORIGIN_PYTHON`、`ZCBOT_ADAPTER_ANSYS_PYTHON``ZCBOT_ADAPTER_BLENDER_PYTHON` 继续兼容,旧名 `ZCBOT_ORIGIN_PYTHON` 暂时兼容。`node.json`、可恢复任务、workspace 和 runtime 集中保存在同一数据根目录,不会因替换程序目录而丢失。任务请求无权选择解释器、脚本、安装依赖或应用路径。
专业软件 runtime 安装要求系统能通过 `py -3.12` 或 PATH 中的 `python.exe` 找到 Python 3.12。既有桌面软件依赖默认显式使用清华 PyPI 镜像;中性 CAD 因二进制 wheel 完整性要求默认使用官方 PyPI。统一覆盖源或切换院内镜像时,在启动 Node 前设置 `ZCBOT_PIP_INDEX_URL`,但不得绕过 pip 哈希校验。解释器位于 `<data_root>\runtimes\origin|ansys|blender|freecad|cad\Scripts\python.exe`;如需由集中运维提供既有受管解释器,机器级 `ZCBOT_ADAPTER_ORIGIN_PYTHON`、`ZCBOT_ADAPTER_ANSYS_PYTHON`、`ZCBOT_ADAPTER_BLENDER_PYTHON`、`ZCBOT_ADAPTER_FREECAD_PYTHON` 或 `ZCBOT_ADAPTER_CAD_PYTHON` 可指定固定解释器,旧名 `ZCBOT_ORIGIN_PYTHON` 暂时兼容。`node.json`、可恢复任务、workspace 和 runtime 集中保存在同一数据根目录,不会因替换程序目录而丢失。任务请求无权选择解释器、脚本、安装依赖或应用路径。
数据根目录可在 Node 主窗口“运行设置”中迁移到其他本机固定磁盘。已有未完成任务时 UI 会拒绝迁移;开始后先停止接收新任务,把整个根目录复制到目标同级临时目录并逐文件 SHA-256 校验,成功后才保存账号级路径并重启。旧目录不会自动删除,确认新目录正常后再由管理员手工清理。机器级 `ZCBOT_WINDOWS_NODE_DATA_DIR` 的优先级高于 UI 配置,存在时 UI 只读;设置或修改该环境变量后必须重启 Node并由管理员自行保证完整数据已经迁入。软件卡安装的 runtime 始终跟随当前数据根目录。
Workspace 目录固定为 `<data_root>\workspaces\<workspace-id>\`,不使用 user ID 上层目录。`current` 是当前可续作工程,`rollback` 只保留上一版用于失败恢复;每次成功续作替换 `current`,不会按 Job 永久堆积工程副本。不要手工移动 workspace 到其他节点:服务端会把后续 Job 固定调度到其 home node节点离线时任务保持等待。备份或迁移节点时必须把数据库中的 workspace 归属和该目录作为一个整体处理。
主窗口“专业软件”按 Origin、ANSYS Mechanical 2024 R2 和 Blender 分别管理应用位置与隔离 runtime。每张软件卡都可单独执行“安装/更新运行环境”,只会读取安装包内固定 requirements并先在临时目录完成后再替换该软件的 `runtimes/<runtime_id>`;有对应活动任务时拒绝替换。安装要求本机存在 Python 3.12,可通过 `py -3.12` 或 PATH 中的 `python.exe` 找到。三种软件均可用“选择位置”保存当前专用账号的配置,点“自动检测”会清除手工值并重新探测;Origin 按 `ZCBOT_ORIGIN_EXE`、COM 注册、Windows 卸载注册表和 OriginLab 标准目录查找 `Origin*.exe`ANSYS 与 Blender 继续按各自环境变量、注册表、PATH 或标准目录查找。环境变量 `ZCBOT_ORIGIN_EXE`、`ZCBOT_BLENDER_EXE` 与 `AWP_ROOT242` 继续兼容。选择 Origin 程序只确定安装位置和版本读取来源,实际执行仍要求 Origin COM probe 通过。配置或安装完成后不必重新注册节点,后续 probe/心跳会刷新能力状态。
主窗口“专业软件”按 Origin、ANSYS Mechanical 2024 R2、Blender、FreeCAD 和中性 CAD 分别管理隔离 runtime前四者同时管理桌面应用位置中性 CAD 是 runtime-only 软件卡,不显示也不接受“选择位置/自动检测”。每张软件卡都可单独执行“安装/更新运行环境”,只会读取安装包内固定 requirements并先在临时目录完成后再替换该软件的 `runtimes/<runtime_id>`;有对应活动任务时拒绝替换。安装要求本机存在 Python 3.12,可通过 `py -3.12` 或 PATH 中的 `python.exe` 找到。Origin 按 `ZCBOT_ORIGIN_EXE`、COM 注册、Windows 卸载注册表和 OriginLab 标准目录查找 `Origin*.exe`ANSYS、Blender 与 FreeCAD 继续按各自环境变量、注册表、PATH 或标准目录查找。环境变量 `ZCBOT_ORIGIN_EXE`、`ZCBOT_BLENDER_EXE`、`ZCBOT_FREECAD_CMD``AWP_ROOT242` 继续兼容FreeCAD 位置必须是 `FreeCADCmd.exe`。选择 Origin 程序只确定安装位置和版本读取来源,实际执行仍要求 Origin COM probe 通过CAD probe 会验证固定依赖、基本体和 STEP roundtrip。配置或安装完成后不必重新注册节点,后续 probe/心跳会刷新能力状态。
Blender 能力为 `blender.scene.author@v3`,目标节点版本为 Blender 5.2 LTS。请求固定使用 `scene.type=recipe`、`recipe_version=1`,通过 `objects[]` 组合 geometry/generator/text/emptygeometry 支持 box、cylinder、cone、sphere、torus、plane、extrude、revolve、sweep 和 pipegenerator 当前提供可与其他对象组合的 `rotary_kiln`;对象可声明父级、集合、材质、变换及 boolean/bevel/mirror/array/solidify 修改器。`render.views` 提供 18 个透视或正交相机,每个视图绑定唯一的 `preview_01``preview_08`,必须包含 `preview_01`;首张预览固定生成,其余视图必须在 `outputs` 请求同名 PNGGLB 仍按需请求。任务同时生成 `scene.blend`、`scene-manifest.json`、`provenance.json` 和可选 `scene.glb`
Recipe 是完整目标状态,每次从空场景重建;修改已有 v3 结果时,从 `software_job_status``editable_request` 形成完整下一版 `operation/outputs` 后调用 `software_job_revise`。v2 回转窑合同已破坏性移除,旧 Workspace 不能作为 v3 source节点磁盘上的旧 `.blend` 不会自动删除;确需保留时应在部署前从 `<data_root>\workspaces\<workspace-id>\current\scene.blend` 备份。Node 会优先使用 UI 中指定的 `blender.exe`,否则从 `ZCBOT_BLENDER_EXE`、Windows 卸载注册表、PATH 或 `%ProgramFiles%\Blender Foundation\Blender *\blender.exe` 探测 Blender。启动器固定使用 Blender 后台、factory startup、禁止自动脚本和 Python 异常退出码,渲染优先使用 5.2 的 Eevee Next 标识并向旧 Eevee 标识回退;不读取用户启动文件,也不接受请求传入 Python、插件、命令行、URL、本机路径、driver 或任意节点脚本。
ANSYS 能力为 `ansys.geometry.inspect@v1``ansys.mechanical.static_structural@v2`,只面向 Mechanical 2024 R2revision 242。adapter 0.3.0 使用 PyMechanical 0.11.3前者导入几何并输出面清单、单位、包围盒、Named Selection 和预览,后者可按已有 Named Selection、全局轴极值面或平面法向/偏置/容差建立支撑与载荷选区再执行真实静力求解。v2 直接替换旧 v1旧请求不会在新版节点调度。Node 会从 `adapters/` 自动发现并在每次心跳上报全部能力,现有节点不需要清身份或重新注册,但必须整体替换发布目录并重启。默认 probe 仍返回不可用;在目标机器完成真实许可证与基准算例验收前,不得设置 `ZCBOT_ANSYS_242_VALIDATED=1`
FreeCAD 能力为 `freecad.model.author@v1`,只支持正式版 `>=1.1.3,<1.2.0`RC、weekly、experimental、旧版和 1.2+ 都会在 probe 阶段保持不可用。请求固定使用 `model.type=recipe`、`recipe_version=1`、`unit=mm`,可声明最多 64 个有限数参数,并在步骤数值位置用 `{"parameter":"name"}` 引用adapter 会先解析成具体数值FCStd 不绑定 Spreadsheet 或 Expression。首版步骤为 box、cylinder、cone、sphere、torus、import_step、union、cut、intersect、translate、rotate、combine只能绑定零或一个已登记 STEP/STP不能提交 FCStd。固定输出 `model.FCStd`、`model.step`、`model-preview.png`、`model-recipe.json`、`model-manifest.json` 和隐藏 `.meta/provenance.json`;预览自动回传,其余按 Workspace/导出机制处理。续作从 `editable_request` 形成完整下一版 Recipe 后调用 `software_job_revise`,每版重新构建并由通用 current/rollback 管理
普通几何先提交 `ansys.geometry.inspect@v1``outputs` 传空数组;成功后以 `geometry-manifest.json` 的单位、包围盒、面质心/法向/面积和 `geometry-preview.png` 判断边界区域。规则板件可在静力 v2 中使用 `{"type":"extreme_face","axis":"x","side":"min","expected_count":1}`;已知平面可使用 `{"type":"planar_faces","normal":[1,0,0],"offset":200,"tolerance":0.01,"expected_count":1}`,其中偏置和容差沿用 `analysis.unit_system` 的长度单位;已有 CAD Named Selection 则使用 `{"type":"named_selection","name":"fixed_mount"}`。每个 boundary/load 还必须提供唯一的 ASCII `name`,用于 Mechanical 树、反力结果和溯源。`expected_count` 可省略,但自动化生产请求应尽量提供;数量不符、没有平面、查询为空或名称冲突都会在网格和求解前失败。复杂装配或歧义区域不得只凭预览猜测,应回到 CAD 建立 Named Selection 后再提交。
FreeCAD 启动器只接受 UI 配置或 `ZCBOT_FREECAD_CMD` 指向的 `FreeCADCmd.exe`,否则按 PATH、卸载注册表与 `%ProgramFiles%\FreeCAD*` 标准目录探测。每个 Job 固定使用 `--safe-mode`、独立 `--user-cfg/--system-cfg``FREECAD_USER_HOME/FREECAD_USER_DATA/FREECAD_USER_TEMP`,并清理外部 Python 搜索路径请求不能指定脚本、程序、命令行、URL、本机路径、宏、插件或 Workbench。FreeCAD 自带 Python 完成 recompute、Shape 校验、FCStd 保存重开和 STEP 导出重开,受管 Python/Pillow 根据 tessellation 生成无 GUI/GPU 的等轴测预览。
ANSYS 能力为 `ansys.geometry.inspect@v1``ansys.mechanical.static_structural@v2`,只面向 Mechanical 2024 R2revision 242。adapter 0.3.0 使用 PyMechanical 0.11.3:前者保留为真实 Mechanical 导入专项诊断,已标记 deprecated/diagnostic-only普通能力枚举会指向 `cad.geometry.prepare@v1`其外部合同、adapter、历史 Job 和明确诊断调用仍兼容至少一个发布周期。后者可按已有 Named Selection、全局轴极值面或平面法向/偏置/容差建立支撑与载荷选区再执行真实静力求解。Node 会从 `adapters/` 自动发现并在每次心跳上报全部能力,现有节点不需要清身份或重新注册,但必须整体替换发布目录并重启。默认 probe 仍返回不可用;在目标机器完成真实许可证与基准算例验收前,不得设置 `ZCBOT_ANSYS_242_VALIDATED=1`
普通几何先提交 `cad.geometry.prepare@v1``outputs` 传空数组;`inspect` 必须绑定一个已登记 STEP/STP`primitive` 不带输入,`recipe` 可选绑定一个供 `import_step` 引用的输入。三种 feature 都必须显式给 `canonical_unit=mm|m`,不会猜测缺失或歧义 STEP 单位;固定输出为 `geometry.step`、`geometry-recipe.json`、`geometry-manifest.json`、`geometry-preview.png` 和 `.meta/provenance.json`。成功后的 `geometry.step` 会登记为普通 Artifact可直接作为后续 `ansys.mechanical.static_structural@v2` 或 Abaqus capability 的 `geometry` 输入,不新增数据库关系。
CAD primitive 覆盖 box、cylinder、tube、cone/frustum、sphere、wedge、torus。Recipe 只接受合同列出的有序步骤和稳定 ASCII ID基础体、显式三维坐标且共面的二维闭合轮廓、extrude/revolve/sweep/loft、union/cut/intersect、fillet/chamfer/shell、translate/rotate/mirror、linear/circular pattern、STEP import 和 combine。请求上限为 256 KiB、一个 256 MiB STEP、128 步、64 轮廓、每轮廓 512 点、64 区域和 256 个结果实体adapter 超时 15 分钟;数值必须有限且坐标/尺寸受合同范围约束。任何未知引用、退化轮廓、空选区、`expected_count` 不符、无效布尔结果或 STEP roundtrip 数值偏差都会让 Job 清晰失败,不静默跳过。
规则板件可在 CAD 区域和静力 v2 中一致使用 `{"type":"extreme_face","axis":"x","side":"min","expected_count":1}`;已知平面可使用 `{"type":"planar_faces","normal":[1,0,0],"offset":200,"tolerance":0.01,"expected_count":1}`。CAD 还支持按 surface type 选 face、按 curve type 选 edge 和选择全部 solidmanifest 记录的是选择器、数量及几何摘要,不记录 OCCT/Mechanical 内部实体 ID。静力请求中的偏置和容差仍沿用 `analysis.unit_system` 长度单位;已有 CAD Named Selection 可直接使用 `{"type":"named_selection","name":"fixed_mount"}`。复杂装配或歧义区域不得只凭预览猜测,应在明确几何语义或 Named Selection 后再提交。
验收使用安装包自带的 NIST FTC-11 AP203 几何adapter 通过与生产请求相同的 `planar_faces` 法向、偏置、容差及 `expected_count=1` 建立 `fixed_face`、`load_face`;不再走验收专用选面旁路。先退出托盘 Node、整体替换新版发布目录并启动在 ANSYS 软件卡中安装/更新运行环境确认能力仍显示待验收。随后在“ANSYS Mechanical 真机验收”卡片点击“运行内置基准验收”只需选择报告父目录UI 会使用固定参数执行一次完整进程树取消和连续 20 次求解,不能通过界面改变脚本、次数、试件或执行门条件。验收成功后先在许可证管理端确认席位均已归还,再点击“开启 ANSYS 执行门”;该按钮只会在本次 `acceptance-report.json` 明确通过后启用,写入机器级环境变量需要管理员权限。随后从托盘完全退出并重新启动 Node。内置 STEP 来源于 NIST MBE PMI Validation and Conformance Testing Project可不受限制地使用文件头保留来源说明。
@ -1152,12 +1178,16 @@ Web 用户登录后,文件栏 Job 中心会聚合本人最近任务。活动
常用二维扩展单图 type 包括 `area`、`polar`、`pie`,均使用 `input/x/y`;其中 polar 的 X 是角度、Y 是半径pie 的 X 是扇区标签、Y 是数值,两者使用 Origin 原生坐标/标签系统且不接受笛卡尔 `x_axis/y_axis/z_axis` 参数。`stacked_area`、`stacked_bar` 与 `stacked_column` 一样至少提供两条 `input/x/y`。堆叠系列可以来自不同输入,但行数和逐行 X 值必须完全一致,否则 Worker 在启动绘图前以 `STACKED_PLOT_X_VALUES_MISMATCH` 拒绝,避免错位累计。统计图与这些扩展图暂不混入 `multi_panel.series[].kind`
专业软件契约的源码事实源位于 `software-contracts/*.json`。zcbot 服务端会在文件变化后校验并热加载完整契约快照正常修改契约不需要重启服务非法或尚未写完的文件不会替换上一份有效快照。Node 发布时把对应事实源复制进 `adapters/<capability>/`,运行时只读取 manifest 同目录指向的合同,不存在共享目录 fallback因此完整 Node 包与独立 adapter 包具有相同结构,更新时必须整体替换 adapter 目录并重启 Node。Node 心跳上报实际合同 SHA-256 和 Workspace 协议版本,服务端不会把 Workspace Job 派给未明确声明支持的节点。manifest 可用 `worker_timeout_minutes` 为每种软件声明 11440 分钟的执行上限;旧 manifest 未声明时继续使用 30 分钟。Worker 可以是受管 Python 脚本或独立 EXE。
专业软件契约的源码事实源位于 `software-contracts/*.json`。zcbot 服务端会在文件变化后校验并热加载完整契约快照,正常修改契约不需要重启服务;非法或尚未写完的文件不会替换上一份有效快照。可选 `lifecycle` 声明状态、普通/诊断可见性及替代 capability缺失时按 active/standard 解释,兼容既有合同。Node 发布时把对应事实源复制进 `adapters/<capability>/`,运行时只读取 manifest 同目录指向的合同,不存在共享目录 fallback因此完整 Node 包与独立 adapter 包具有相同结构,更新时必须整体替换 adapter 目录并重启 Node。Node 心跳上报实际合同 SHA-256 和 Workspace 协议版本,服务端不会把 Workspace Job 派给未明确声明支持的节点。manifest 可用 `worker_timeout_minutes` 为每种软件声明 11440 分钟的执行上限;旧 manifest 未声明时继续使用 30 分钟。Worker 可以是受管 Python 脚本或独立 EXE。
两个 JSON 的职责不同:`adapter.json` 只描述本机如何启动实际版本、runtime、入口和契约文件名`origin.plot.v2.json` 描述云端与本机共同遵守的业务请求/输出契约。前者不能替代后者;独立打包脚本只是把根目录的同一份业务契约复制进 adapter 交付目录,不维护第二份源码。
只更新 Origin adapter 时,不需要编译、重装或替换 Node EXE运行 `windows-node\package-origin-adapter.bat` 可直接把 manifest、Worker、验收脚本、依赖清单和契约打成 `dist\origin.plot@v2-adapter.zip`。在 Node 机器上先从托盘退出 zcbot Windows Node备份并整体替换 EXE 同级的 `adapters\origin.plot@v2\`,再启动 Node。不要在任务执行中覆盖目录。若 `requirements.txt` 发生变化,在对应软件卡中“安装/更新运行环境”;仅 `adapter.json`、契约、`worker.py` 或 `acceptance.py` 变化时无需更新 runtime。Node 启动后会用 Worker 的 `--probe` 核对 manifest 与 Worker 版本,版本不一致时该能力保持不可用。
中性 CAD adapter 可用 `windows-node\package-cad-adapter.bat` 独立打成 `dist\cad.geometry.prepare@v1-adapter.zip`。退出 Node 后整体替换 `adapters\cad.geometry.prepare@v1\`,再从“中性 CAD”卡安装/更新固定 `cad` runtimeprobe 必须显示 CadQuery 2.8.0 / OCP 7.9.3.1 且基本体与 STEP roundtrip 可用。若默认官方 PyPI 下载失败,可通过 `ZCBOT_PIP_INDEX_URL` 指向已同步并校验 wheel 的院内源;出现 `THESE PACKAGES DO NOT MATCH THE HASHES` 时应修复镜像缓存,不能添加忽略哈希参数。
FreeCAD adapter 可用 `windows-node\package-freecad-adapter.bat` 独立打成 `dist\freecad.model.author@v1-adapter.zip`;退出 Node 后整体替换 `adapters\freecad.model.author@v1\`在“FreeCAD”卡确认应用位置并安装/更新固定 `freecad` runtime。目标机用 `<data_root>\runtimes\freecad\Scripts\python.exe <node_program>\adapters\freecad.model.author@v1\acceptance.py --work-root <new_empty_parent>\freecad-acceptance-YYYYMMDD --repeat 3` 执行固定验收;目录必须不存在。报告 `acceptance-report.json` 覆盖基础体、参数、布尔、变换、STEP import、FCStd/STEP roundtrip、模拟 Workspace revise/rollback、取消、空格/中文路径、连续运行及 FreeCAD 进程释放;报告通过且人工检查预览后才能记录目标机通过。
Origin adapter 1.1.0 起随包提供固定真机 QA harness但暂不接入 Node UI。在专用测试节点确认没有活动 Job 后,从 PowerShell 使用 Origin 卡片已安装的受管解释器运行:`<data_root>\runtimes\origin\Scripts\python.exe <node_program>\adapters\origin.plot@v2\acceptance.py --work-root <new_empty_parent>\origin-acceptance-YYYYMMDD`。`--work-root` 必须指向尚不存在的新目录adapter 1.3.0 默认依次执行 annotations、recipe_2x2、heatmap、surface_3d、stacked、stacked_line、violin、band也可重复传入 `--case <name>` 只跑指定用例。每个用例先通过生产 Worker 子进程生成 OPJU/PNG/SVG/PDF再由独立 Origin 会话重开工程,核对图层、系列、工作表/矩阵和数值 oracle复导三种图件并等待 Origin 进程退出;必要时可用 `--release-wait <seconds>` 调整释放等待时间。最终报告位于 `<work-root>\acceptance-report.json`。输入均为脚本生成的合成数据,但 PNG/SVG/PDF 仍需人工完成视觉 gate 后才能判定目标机器通过。
`origin.analysis@v1` 是与绘图分离的无状态科研分析入口,首批 operation 为 `data_check`、`normalize`、`smooth`、`differentiate`、`integrate`、`linear_fit`。请求只绑定一个 CSV/XLSX/JSON artifact`data_check` 可只检查整表外,其余操作必须显式提供 `input/x/y`。数据体检通过 `parameters.fail_on` 指定需要阻断后续流程的问题,命中时 Job 仍成功并在 `diagnostics.json` 返回 `passed=false`;调用方应读取诊断后再决定是否提交分析。归一化的 `max_abs` 使用最大绝对值,`area` 使用有符号梯形积分,`reference` 在数据范围内按 X 线性插值;平滑固定为完整奇数窗口的局部多项式,求导支持一/二阶且接受非等间距 X积分输出累计值及可选 `from/to` 区间值线性拟合输出参数、95% t 置信区间、R²、RMSE、拟合值和残差并在 OPJU 内保留 Origin 原生线性拟合报告。所有数值操作拒绝缺失、非有限、重复或非严格递增的 X/Y 数据,不做隐式排序或删行。
@ -1170,7 +1200,7 @@ Blender adapter 可独立运行 `windows-node\package-blender-adapter.bat` 打
注册配置写入 `<data_root>\node.json`Token 使用 DPAPI `LocalMachine` 加密ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号配置、注册并运行 Node。当前 MVP 可由 UI 写入该账号的登录启动项,不安装 Windows Service。
直接双击 EXE 启动托盘 UI红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开配置窗。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless`
直接双击 EXE 启动托盘 UI红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开 WPF 主窗口,可完成首次注册、重连、清除身份、复制诊断信息和登录自启动设置。“专业软件”页可配置 Origin、ANSYS、Blender、FreeCAD 位置及五套独立 runtime中性 CAD 无应用位置配置。该页还可运行或停止 ANSYS 固定验收、在报告通过后开启机器级执行门;“本机任务”页每秒刷新本地持久化记录;“运行设置”页可打开或安全迁移完整数据目录。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless`
若执行中的 Job 遇到 Node 重连,客户端会先等待已接收的本地执行管线收尾,再建立新连接,避免同一 Worker 被新旧连接同时恢复。若云端已将 Job 判为失败或取消、但本地 Worker 随后仍生成了结果,本机任务会显示“云端已终止”并保留工作区文件,不再永久停在 90% 重传;这些本地结果不会反向覆盖云端终态。

View File

@ -12,7 +12,7 @@ models_dir: config/models
model_tiers:
default: # 基线:所有未分配档位的用户(= 公测期默认可用)
- deepseek_v4.flash
- deepseek_v4.pro
- glm.flash53
- local.r1 # 内网模型(涉密任务)
- local.qwen3
- seedream_5 # 图(config/media/doubao.yaml image 段)
@ -26,8 +26,7 @@ model_tiers:
- doubao.turbo
- doubao.pro
- doubao.evolving
- glm.pro
- glm.pro52
- glm.flash53
- unifyllm.fable5 # unifyllm 网关(海外旗舰,走代理出口)
- unifyllm.opus48
- unifyllm.sonnet46
@ -67,14 +66,20 @@ shutdown:
cancel_grace_seconds: 15 # 超时转 cancel 后再给的退场宽限
# Sandbox 容器资源限制(docker run flag,env 可 override);改后重启 web 生效,
# 新起的容器用新值,已 running 的不变(idle 5min 回收后下次起)。
# 新起的容器用新值,已 running 的不变(idle 10min 回收后下次起)。
sandbox:
memory: 2g # --memory (env: ZCBOT_SANDBOX_MEMORY)
cpus: 1.0 # --cpus (env: ZCBOT_SANDBOX_CPUS)
memory: 4g # --memory (env: ZCBOT_SANDBOX_MEMORY)
cpus: 2.0 # --cpus (env: ZCBOT_SANDBOX_CPUS)
pids_limit: 1024 # --pids-limit (env: ZCBOT_SANDBOX_PIDS_LIMIT);线程也计数,
# chromium headless 一次 ~150-200 线程 + 超时残留进程,256 会被
# 打满致 mmdc 渲 mermaid 必崩(pthread_create EAGAIN)
shm_size: 512m # --shm-size (env: ZCBOT_SANDBOX_SHM_SIZE);chromium/mmdc 渲 mermaid 的 /dev/shm,默 64MB 不够会挂
tmp_size: 1g # /tmp tmpfs (env: ZCBOT_SANDBOX_TMP_SIZE)
idle_ttl_seconds: 600 # 普通容器无 exec 活动 10 分钟回收
max_active_execs: 6 # 宿主硬上限;前后台统一计数
max_background_execs: 4
max_active_execs_per_user: 3
min_mem_available: 1g # 低于阈值暂停新放行,不杀已运行任务
# 容器 DNS server 显式配置(docker run --dns,容器 /etc/resolv.conf 直接写,
# 绕过 docker daemon 上游 DNS 探测路径;腾讯云轻量 / 部分云上 daemon 探测
# systemd-resolved 上游会失败,导致 embedded DNS 127.0.0.11 forward 出去也跪)。

View File

@ -16,17 +16,39 @@ variants:
thinking_transport: extra_body
reasoning_effort_levels: [low, high, max]
default_reasoning_effort: high
reasoning_replay: tool_calls # 普通跨轮剥离;工具调用 reasoning 按官方协议完整回传
reasoning_replay: tool_turn # 只在当前用户轮次的工具链内原样回传 reasoning
code_quality: good
enable_run_python: true
max_iterations: 120 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
optimal_temperature: 0.3
prompt_caching: false
extended_thinking: false
# Flash-0731 官方美元价按 1 USD ~= 7.2 CNY 折算;峰谷价正式生效后再同步。
input_cny_per_mtoken: 1.0
output_cny_per_mtoken: 2.0
cache_hit_cny_per_mtoken: 0.02 # 官方 $0.0028 / M tokens
# 扁平价保留给旧调用方,取当前谷时折算值;正式记账使用下面的版本化 pricing。
input_cny_per_mtoken: 1.584
output_cny_per_mtoken: 4.752
cache_hit_cny_per_mtoken: 0.0504
pricing:
periods:
- revision: deepseek-v4-20260816
currency: USD
fx_to_cny: 7.2
source_url: https://api-docs.deepseek.com/quick_start/pricing/
checked_at: 2026-08-30
effective_from: 2026-08-16T16:00:00Z
default: # 谷时(官方峰时之外)
input_per_mtoken: 0.22
cache_hit_per_mtoken: 0.007
output_per_mtoken: 0.66
time_tiers:
- name: peak
timezone: UTC
weekdays: [mon, tue, wed, thu, fri]
windows:
- ["01:00", "04:00"]
- ["06:00", "10:00"]
input_per_mtoken: 0.44
cache_hit_per_mtoken: 0.014
output_per_mtoken: 1.32
pro:
display_name: DeepSeek V4 Pro
@ -42,13 +64,35 @@ variants:
thinking_transport: extra_body
reasoning_effort_levels: [low, medium, high, max]
default_reasoning_effort: medium
reasoning_replay: tool_calls
reasoning_replay: tool_turn
code_quality: excellent
enable_run_python: true
max_iterations: 150 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
optimal_temperature: 0.2
prompt_caching: false
extended_thinking: false
input_cny_per_mtoken: 2.0
output_cny_per_mtoken: 20.0
cache_hit_cny_per_mtoken: 0.2 # DeepSeek 前缀缓存命中价(input 的 ~0.1x)
input_cny_per_mtoken: 4.752
output_cny_per_mtoken: 14.256
cache_hit_cny_per_mtoken: 0.1584
pricing:
periods:
- revision: deepseek-v4-20260816
currency: USD
fx_to_cny: 7.2
source_url: https://api-docs.deepseek.com/quick_start/pricing/
checked_at: 2026-08-30
effective_from: 2026-08-16T16:00:00Z
default:
input_per_mtoken: 0.66
cache_hit_per_mtoken: 0.022
output_per_mtoken: 1.98
time_tiers:
- name: peak
timezone: UTC
weekdays: [mon, tue, wed, thu, fri]
windows:
- ["01:00", "04:00"]
- ["06:00", "10:00"]
input_per_mtoken: 1.32
cache_hit_per_mtoken: 0.044
output_per_mtoken: 3.96

View File

@ -32,6 +32,17 @@ variants:
input_cny_per_mtoken: 3.0
output_cny_per_mtoken: 15.0
cache_hit_cny_per_mtoken: 0.6
pricing:
periods:
- revision: doubao-seed21-20260628
currency: CNY
source_url: https://www.volcengine.com/product/ark
checked_at: 2026-08-30
effective_from: 2026-06-28T00:00:00Z
default:
input_per_mtoken: 3.0
cache_hit_per_mtoken: 0.6
output_per_mtoken: 15.0
pro:
display_name: 豆包 Seed 2.1 Pro
@ -56,6 +67,17 @@ variants:
input_cny_per_mtoken: 6.0
output_cny_per_mtoken: 30.0
cache_hit_cny_per_mtoken: 1.2
pricing:
periods:
- revision: doubao-seed21-20260628
currency: CNY
source_url: https://www.volcengine.com/product/ark
checked_at: 2026-08-30
effective_from: 2026-06-28T00:00:00Z
default:
input_per_mtoken: 6.0
cache_hit_per_mtoken: 1.2
output_per_mtoken: 30.0
evolving:
# 自进化版:统一 model_id `doubao-seed-evolving`,每周至少迭代一次,始终指向最新版。
@ -83,3 +105,14 @@ variants:
input_cny_per_mtoken: 6.0
output_cny_per_mtoken: 30.0
cache_hit_cny_per_mtoken: 1.2
pricing:
periods:
- revision: doubao-seed-evolving-202607
currency: CNY
source_url: https://www.volcengine.com/product/ark
checked_at: 2026-08-30
effective_from: 2026-07-01T00:00:00Z
default:
input_per_mtoken: 6.0
cache_hit_per_mtoken: 1.2
output_per_mtoken: 30.0

View File

@ -1,55 +1,55 @@
# 智谱 GLM 模型档案
# 走 litellm 原生 zai provider(1.83+ 内置)。litellm 默认 api_base 是国际站 api.z.ai;
# 国内站 bigmodel.cn 通过 yaml 的 api_base 字段覆盖。两站 API key 不通用,env 也分开命名。
# thinking 走统一 extra_body 协议,由本档 `thinking_enabled` 决定开关。当前均 false=禁用 ——
# 因网关侧默认开 thinking,重任务上会把输出预算烧在 reasoning_content 上撞满 65536 上限
# 被截断→空响应(task 35744bea 案);禁用后线上探针实测 reasoning_content 归零、正文照常。
# 走 LiteLLM 原生 zai provider。国内站 bigmodel.cn 与国际站 API key 不通用。
# GLM-5.3-Flash 强制 thinkingzcbot 保留同模型完整 reasoning 以维持长工具链连续性。
family: glm
variants:
pro:
display_name: GLM 5.1
model_id: zai/glm-5.1
flash53:
display_name: GLM-5.3 Flash
model_id: zai/glm-5.3-flash
api_base: https://open.bigmodel.cn/api/paas/v4
api_key_env: ZHIPUAI_API_KEY
max_context: 200000
reliable_context: 100000
max_output: 8192
parallel_tools: false
tool_calling_quality: good
thinking_enabled: false
thinking_transport: extra_body
reasoning_effort_levels: []
default_reasoning_effort: ""
code_quality: good
enable_run_python: true
max_iterations: 50
optimal_temperature: 0.3
prompt_caching: false
extended_thinking: false
# GLM 5.2:与 5.1 并存(新增 variant,不动 glm.pro,线上 task 仍引 5.1 不受影响)。
# 旗舰基座,真正可用的 1M 上下文,适合大仓库/长链路工程任务。thinking 同 pro 留 false(协议同 5.1)。
pro52:
display_name: GLM 5.2
model_id: zai/glm-5.2
api_base: https://open.bigmodel.cn/api/paas/v4
api_key_env: ZHIPUAI_API_KEY
max_context: 1000000 # 真 1M
max_context: 1000000
reliable_context: 262144
max_output: 8192
max_output: 131072
input_modalities: [text, image]
parallel_tools: false
tool_calling_quality: good
thinking_enabled: false
tool_calling_quality: excellent
thinking_enabled: true
thinking_transport: extra_body
reasoning_effort_levels: []
default_reasoning_effort: ""
thinking_clear: false
reasoning_effort_levels: [low, high, max]
default_reasoning_effort: high
reasoning_replay: conversation
code_quality: excellent
enable_run_python: true
max_iterations: 50
optimal_temperature: 0.3
max_iterations: 120
optimal_temperature: 1.0
prompt_caching: false
extended_thinking: false
input_cny_per_mtoken: 8.0
output_cny_per_mtoken: 28.0
cache_hit_cny_per_mtoken: 2.0
# 当前 endpoint 是国内 BigModel使用国内按量标准价限时活动/套餐抵扣不进入
# 长期公开价估算,实际财务扣费仍以账号费用中心账单为准。
input_cny_per_mtoken: 0.8
output_cny_per_mtoken: 2.8
cache_hit_cny_per_mtoken: 0.23
pricing:
periods:
- revision: bigmodel-glm53-flash-20260826
currency: CNY
source_url: https://bigmodel.cn/pricing
checked_at: 2026-08-30
effective_from: 2026-08-26T00:00:00Z
default:
input_per_mtoken: 0.8
cache_hit_per_mtoken: 0.23
output_per_mtoken: 2.8
# 隐藏兼容别名:真实调用、能力和记账身份全部归 glm.flash53。
# 待一个兼容版本确认无旧 profile 活跃后删除。
pro:
hidden: true
alias_of: flash53
pro52:
hidden: true
alias_of: flash53

View File

@ -1,3 +1,3 @@
# zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。
# 改版本只动这一行。
__version__ = "0.69.0"
__version__ = "0.71.0"

View File

@ -56,10 +56,12 @@ _MEDIA_LOOK_SEG = """\
- **何时调**:用户消息里出现 `[用户上传的参考图] <路径>` 且需要据图内容回答("这图里写了啥 / 是什么 / 表格数据多少");或要基于 task 内某张图(`figures/xxx.png`)**实际内容**做事(不是改图,改图走 seedream) `image=<路径>`并且**必须在 `question` 里一次写清完成当前任务所需的具体信息**别分多次看
- **问题范围**:只问完成用户任务所需内容例如问仪表读数就只读数问某列数据就只提取该列仅当用户明确要求"全文 OCR / 转文字 / 完整识别整张图"才要求逐字识别全部文字不要无条件同时要求画面描述全文 OCR 和图表解析
- **何时不调**:用户只是要改图( seedream i2i)/ 只要文件名不关心内容 / 图是你自己刚生成的且 prompt 已知(无需再读)/ 图的内容对当前任务可有可无**绝不对同一张图反复看**(每看一次都是几十秒等待)"""
_MEDIA_NATIVE_LOOK_SEG = """\
- **原生看图**:你可以直接理解用户随消息上传的图片图片已作为多模态内容随请求送入直接据图分析**不要调用辅助看图工具**消息中的 `[用户上传的参考图]` 路径仅用于定位同一附件或交给改图工具"""
_MEDIA_READDOC_SEG = """\
- `read_document` PDF(豆包 Seed 2.0 Lite 文档理解),**专治扫描件**:markitdown 对某 PDF 转出** / 近空**(纯图页无文本层) 用它逐页 OCR markdown每页约 1-2 厘钱,单次上限 100 (更长先拆分卷)
- **多页整本 OCR 必传 `save_md`**( `save_md='source/xxx.md'`):全文落文件只返回预览,后续直接 read 那个 md;只问局部内容("第3章指标是什么") `question` 即可不落盘
- **何时不调**:PDF 有文本层且 markitdown 已转出正文(直接用那份 md,别重复花钱);DOCX/PPTX/XLSX( markitdown);单张图片( look_at_image)"""
- **何时不调**:PDF 有文本层且 markitdown 已转出正文(直接用那份 md,别重复花钱);DOCX/PPTX/XLSX( markitdown);单张图片由主模型原生读取或走本轮实际提供的看图能力"""
# PDF 摄取是跨 skill 的基础入口,始终注入,不能依赖 agent 是否正确加载 paper / proposal。
# read_document 只在配了 ARK 时挂载;没挂时扫描件明确停在“需 OCR”,不诱导现场装包或
@ -87,14 +89,19 @@ _MEDIA_SEEDANCE_SEG = """\
- 兜底硬约束:用户没主动要视频就别装饰性生成(比生图更严重的红线);同一目的不满意**绝不连发**(1 次错 = ¥4+60s,连发 2 = ¥8+2min);固定开场图传 `image`,多张主体/产品/场景参考图传 `reference_images`当前不支持 video-to-video"""
def _media_tools_block(has_ark: bool, image_tool: str) -> str:
def _media_tools_block(
has_ark: bool, image_tool: str, native_image_input: bool = False
) -> str:
"""按本 run 实际挂载的媒体工具拼指引段;一个都没挂 → 返 ""(prompt 不出现媒体段)。
image_tool: "seedream" / "gpt_image" / ""(无图像工具)look_at_image seedance
只跟豆包(ARK_API_KEY);图像工具二选一,prompt 只描述实际存在的那个
"""
segs: list[str] = []
if native_image_input:
segs.append(_MEDIA_NATIVE_LOOK_SEG)
if has_ark:
if not native_image_input:
segs.append(_MEDIA_LOOK_SEG)
segs.append(_MEDIA_READDOC_SEG)
if image_tool == "seedream":
@ -107,7 +114,7 @@ def _media_tools_block(has_ark: bool, image_tool: str) -> str:
segs.append(_MEDIA_SEEDANCE_SEG)
if not segs:
return ""
return "## 媒体工具(图像生成 / 视频生成 / 看图)\n" + "\n".join(segs)
return "## 媒体能力(图像生成 / 视频生成 / 看图)\n" + "\n".join(segs)
def _choose_image_variant(
@ -154,6 +161,14 @@ _HOST_ENV_BLOCK = """\
## 运行环境(本地 host)
`shell` 走的是 **Windows cmd.exe**( bash):避免 unix-only flag,`mkdir -p` 不识别 `run_python` `os.makedirs(..., exist_ok=True)` 建目录;复杂管道/重定向用 run_python 更稳"""
_MERMAID_STYLE_BLOCK = """\
## Mermaid 视觉规范
生成 Mermaid 图时默认采用**克制的语义配色**,不要输出全灰单色图,也不要随机彩虹色:
- 流程图 / 架构图用 `classDef` 按角色分组,通常 35 色足够:输入/数据用蓝色,处理/模块用青绿色,判断/风险用橙色,结果/成功用绿色,异常用红色;同类节点保持同色
- 使用浅色填充较深描边与高对比深色文字,线条颜色统一;分组框比节点更淡,避免大面积高饱和背景
- 颜色服务于信息层级;若用户指定品牌色黑白打印或已有 `classDef` / `style` 规则,优先遵循用户设置,不要覆盖"""
def load_config() -> dict:
return yaml.safe_load((ROOT / "config" / "agent.yaml").read_text(encoding="utf-8")) or {}
@ -356,6 +371,7 @@ def _build_system_prompt(
is_docker = os.getenv("ZCBOT_SANDBOX_BACKEND", "host").lower() == "docker"
# 运行环境段紧跟模板(平台/网络是基础事实,放前面);general_v1 的「平台」段指向这里。
prompt += _CONTAINER_ENV_BLOCK if is_docker else _HOST_ENV_BLOCK
prompt += _MERMAID_STYLE_BLOCK
if skills.skills:
prompt += f"\n\n## 可用 skill (用 load_skill 加载完整指引)\n{skills.discovery_block()}"
# .memory/ 在 agent 视角下的可写路径:docker 给容器路径,host 给宿主绝对路径。
@ -525,6 +541,7 @@ def build_agent(
model = cfg["default_model"]
caps = ModelCapabilities.load(model, ROOT / cfg["models_dir"])
model = caps.profile
llm = LLM(caps)
workspace_dir = resolve_workspace(workspace, cfg)
@ -603,7 +620,9 @@ def build_agent(
system_prompt = _build_system_prompt(
cfg, skills, workspace_dir, tool_base, working_dir_path, uid,
task_id, task_state.name, task_state.skill,
media_block=_media_tools_block(ark_cfg is not None, image_tool_name),
media_block=_media_tools_block(
ark_cfg is not None, image_tool_name, caps.native_image_input
),
allow_working_dir_rename=not scheduled_run,
office_to_pdf_available=office_pdf_available,
)
@ -671,6 +690,7 @@ def build_agent(
if current_profile.split(".", 1)[0] == "unifyllm":
return None
new_caps = ModelCapabilities.load(profile, ROOT / cfg["models_dir"])
profile = new_caps.profile
new_llm = LLM(new_caps) # 缺 api key 会 raise → loop 捕获转 warn,不切换
from sqlalchemy import update as _update
from core.storage import session_scope as _scope

View File

@ -5,7 +5,6 @@ litellm 不覆盖豆包的图像/视频生成端点,这里自己用 httpx 直调
"""
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Optional
@ -35,6 +34,11 @@ class ArkConfig:
api_key: str
base_url: str
raw: dict # 完整 yaml 内容(便于 caller 按 image/video 子键再取)
api_key_env: str = ""
def request_api_key(self) -> str:
from core.provider_credentials.runtime import resolve_env_secret
return resolve_env_secret(self.api_key_env) if self.api_key_env else self.api_key
@classmethod
def load(cls, path: Optional[Path] = None) -> Optional["ArkConfig"]:
@ -50,7 +54,8 @@ class ArkConfig:
return None
data = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
env = data.get("api_key_env") or data.get("ark_api_key_env") or "ARK_API_KEY"
key = os.environ.get(env, "").strip()
from core.provider_credentials.runtime import resolve_env_secret
key = resolve_env_secret(env)
if not key:
return None
base = (
@ -58,7 +63,7 @@ class ArkConfig:
or data.get("ark_base_url")
or "https://ark.cn-beijing.volces.com/api/v3"
)
return cls(api_key=key, base_url=str(base).rstrip("/"), raw=data)
return cls(api_key=key, base_url=str(base).rstrip("/"), raw=data, api_key_env=env)
class ArkClient:
@ -70,10 +75,11 @@ class ArkClient:
def __init__(self, cfg: ArkConfig, timeout_s: float = 60.0) -> None:
self.cfg = cfg
self.timeout_s = timeout_s
self._api_key = cfg.request_api_key()
self._client = httpx.Client(
base_url=cfg.base_url,
headers={
"Authorization": f"Bearer {cfg.api_key}",
"Authorization": f"Bearer {self._api_key}",
},
timeout=timeout_s,
)
@ -118,17 +124,32 @@ class ArkClient:
raise ArkTimeoutError(f"network error calling GET {path}: {e}") from e
return self._parse(resp, f"GET {path}")
@staticmethod
def _parse(resp: httpx.Response, label: str) -> dict:
def _parse(self, resp: httpx.Response, label: str) -> dict:
if resp.status_code >= 400:
try:
from core.provider_credentials.registry import BY_ENV
from core.provider_credentials.service import record_business_failure
binding = BY_ENV.get(self.cfg.api_key_env)
if binding:
record_business_failure(
binding[0], status_code=resp.status_code, detail=resp.text[:300]
)
except Exception:
pass
# ark 错误 body 一般是 {"error": {"code": ..., "message": ...}};能解就解
try:
err = resp.json().get("error") or {}
msg = err.get("message") or resp.text[:300]
key = self._api_key
msg = str(msg).replace(key, "***") if key else str(msg)
code = err.get("code") or resp.status_code
raise ArkError(f"{label} → HTTP {resp.status_code} ({code}): {msg}")
except ValueError:
raise ArkError(f"{label} → HTTP {resp.status_code}: {resp.text[:300]}")
detail = resp.text[:300]
key = self._api_key
if key:
detail = detail.replace(key, "***")
raise ArkError(f"{label} → HTTP {resp.status_code}: {detail}")
try:
return resp.json()
except ValueError as e:

View File

@ -9,7 +9,7 @@ from datetime import datetime, timezone
from pathlib import Path
from uuid import UUID, uuid4
from sqlalchemy import or_, select
from sqlalchemy import or_, select, text
from sqlalchemy.dialects.postgresql import insert as pg_insert
from .artifacts import ARTIFACT_TRASH_DIR, ArtifactRef
@ -85,7 +85,10 @@ def register_published_artifacts(
content_sha256=content_sha256,
).on_conflict_do_update(
index_elements=[Artifact.user_id, Artifact.current_path],
index_where=Artifact.status == "active",
# 必须与 0028 的部分唯一索引谓词按 SQL 字面量一致。若用
# ``Artifact.status == "active"``SQLAlchemy 会编译成绑定参数,
# PostgreSQL 无法据此推断 ON CONFLICT 对应哪个部分索引。
index_where=text("status = 'active'"),
set_=update_values,
).returning(Artifact.artifact_id)
artifact_id = session.execute(statement).scalar_one()

View File

@ -18,7 +18,6 @@ import base64
import hashlib
import hmac
import json
import os
import time
from typing import Callable, Optional
@ -57,15 +56,15 @@ class LfasrCancelled(LfasrError):
def is_configured() -> bool:
"""LFASR 凭据是否齐 —— agent_builder 据此决定挂不挂 transcribe_audio tool。"""
return all(
(os.getenv(k) or "").strip()
for k in ("XFYUN_APPID", "XFYUN_LFASR_SECRET_KEY")
)
from core.provider_credentials.runtime import provider_available
return provider_available("xfyun_lfasr")
def _load_credentials() -> tuple[str, str]:
appid = (os.getenv("XFYUN_APPID") or "").strip()
secret = (os.getenv("XFYUN_LFASR_SECRET_KEY") or "").strip()
from core.provider_credentials.runtime import resolve_credentials
values = resolve_credentials("xfyun_lfasr").values
appid = values.get("appid", "")
secret = values.get("secret_key", "")
if not (appid and secret):
raise LfasrNotConfigured(
"录音转写未配置:需在 .env 设 XFYUN_APPID / XFYUN_LFASR_SECRET_KEY"
@ -99,6 +98,15 @@ def _post(path: str, params: dict, content: Optional[bytes] = None,
headers={"Content-Type": "application/octet-stream"} if content else None,
timeout=timeout_s,
)
if resp.status_code >= 400:
try:
from core.provider_credentials.service import record_business_failure
record_business_failure(
"xfyun_lfasr", status_code=resp.status_code,
detail=resp.text[:200],
)
except Exception:
pass
resp.raise_for_status()
body = resp.json()
except LfasrError:
@ -108,6 +116,14 @@ def _post(path: str, params: dict, content: Optional[bytes] = None,
code = str(body.get("code") or "")
if code != "000000":
desc = body.get("descInfo") or "未知错误"
if code in {"10105", "10106", "10107", "11200"}:
try:
from core.provider_credentials.service import record_business_failure
record_business_failure(
"xfyun_lfasr", status_code=401, detail="讯飞认证失败"
)
except Exception:
pass
raise LfasrError(f"讯飞录音转写失败({code}):{desc}", code=code)
return body.get("content") or {}

View File

@ -18,7 +18,6 @@ import base64
import hashlib
import hmac
import json
import os
import time
from contextlib import suppress
from typing import Any
@ -84,9 +83,11 @@ def build_auth_url(api_key: str, api_secret: str) -> str:
def _load_credentials() -> tuple[str, str, str]:
appid = (os.getenv("XFYUN_APPID") or "").strip()
api_key = (os.getenv("XFYUN_API_KEY") or "").strip()
api_secret = (os.getenv("XFYUN_API_SECRET") or "").strip()
from core.provider_credentials.runtime import resolve_credentials
values = resolve_credentials("xfyun_iat").values
appid = values.get("appid", "")
api_key = values.get("api_key", "")
api_secret = values.get("api_secret", "")
if not (appid and api_key and api_secret):
raise XfyunASRNotConfigured(
"语音识别未配置:需在 .env 设 XFYUN_APPID / XFYUN_API_KEY / XFYUN_API_SECRET"
@ -94,6 +95,16 @@ def _load_credentials() -> tuple[str, str, str]:
return appid, api_key, api_secret
def _report_auth_error(code: object) -> None:
if str(code) not in {"10105", "10106", "10107", "11200"}:
return
try:
from core.provider_credentials.service import record_business_failure
record_business_failure("xfyun_iat", status_code=401, detail="讯飞认证失败")
except Exception:
pass
class XfyunStream:
"""流式会话:实时喂 PCM 分片,partial 全文经 on_text 异步回调推出(wpgs 已合并)。
@ -151,6 +162,7 @@ class XfyunStream:
msg = json.loads(await self._ws.recv())
code = msg.get("code")
if code:
_report_auth_error(code)
hint = _ERR_HINTS.get(code, msg.get("message") or "未知错误")
raise XfyunASRError(f"讯飞识别失败({code}):{hint}", code=code)
data = msg.get("data") or {}
@ -272,6 +284,7 @@ async def transcribe(pcm: bytes, *, language: str = "zh_cn") -> str:
msg = json.loads(raw)
code = msg.get("code")
if code:
_report_auth_error(code)
hint = _ERR_HINTS.get(code, msg.get("message") or "未知错误")
raise XfyunASRError(f"讯飞识别失败({code}):{hint}", code=code)
data = msg.get("data") or {}

View File

@ -1,7 +1,9 @@
"""Structured user-message attachments and model-context compatibility rendering."""
from __future__ import annotations
import base64
import mimetypes
from copy import deepcopy
from collections.abc import Iterable
from pathlib import Path
from typing import Any
@ -10,6 +12,14 @@ from .artifacts import ArtifactPathError, resolve_artifact_path
ATTACHMENT_REF_VERSION = 1
MAX_ATTACHMENTS_PER_MESSAGE = 10
MAX_NATIVE_IMAGE_BYTES = 10 * 1024 * 1024
_NATIVE_IMAGE_MIME = {
".gif": "image/gif",
".jpeg": "image/jpeg",
".jpg": "image/jpeg",
".png": "image/png",
".webp": "image/webp",
}
_IMAGE_EXTENSIONS = {
".avif", ".bmp", ".gif", ".heic", ".heif", ".jpeg", ".jpg",
".png", ".svg", ".tif", ".tiff", ".webp",
@ -74,3 +84,60 @@ def content_for_model(content: str, refs: Iterable[dict], working_dir_name: str)
return text
suffix = "\n".join(lines)
return f"{text}\n\n{suffix}" if text else suffix
def materialize_native_images(
messages: Iterable[dict],
*,
enabled: bool,
working_dir: Path,
user_root: Path,
) -> tuple[list[dict], int]:
"""把内存附件引用按需物化为 OpenAI-compatible image_url blocks。
Base64 只存在于本次 provider-bound 副本不写数据库历史文件已删除越界
格式不受支持或超过上限时保留原有文字路径提示并跳过图片块不能让旧附件阻断续聊
所有 ``_`` 内部字段在返回前剥离
"""
output: list[dict] = []
image_count = 0
for message in messages:
new_msg = deepcopy(message)
refs = list(new_msg.pop("_attachment_refs", []) or [])
new_msg.pop("_model_profile", None)
if not enabled or new_msg.get("role") != "user" or not refs:
output.append(new_msg)
continue
blocks: list[dict] = []
text = new_msg.get("content")
if isinstance(text, str) and text:
blocks.append({"type": "text", "text": text})
message_image_count = 0
for ref in refs:
if not isinstance(ref, dict) or ref.get("kind") != "image":
continue
raw_path = str(ref.get("path") or "")
try:
path, _ = resolve_artifact_path(
raw_path,
working_dir=working_dir,
user_root=user_root,
require_file=True,
)
mime = _NATIVE_IMAGE_MIME.get(path.suffix.lower())
if mime is None or path.stat().st_size > MAX_NATIVE_IMAGE_BYTES:
continue
data = base64.b64encode(path.read_bytes()).decode("ascii")
except (ArtifactPathError, OSError):
continue
blocks.append({
"type": "image_url",
"image_url": {"url": f"data:{mime};base64,{data}"},
})
image_count += 1
message_image_count += 1
if message_image_count:
new_msg["content"] = blocks
output.append(new_msg)
return output, image_count

View File

@ -1,7 +1,6 @@
"""博查 (Bocha AI) Web Search API 客户端,共享给 web_search tool。"""
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
@ -22,6 +21,11 @@ class BochaError(RuntimeError):
class BochaConfig:
api_key: str
base_url: str
api_key_env: str = ""
def request_api_key(self) -> str:
from core.provider_credentials.runtime import resolve_env_secret
return resolve_env_secret(self.api_key_env) if self.api_key_env else self.api_key
@classmethod
def load(cls, path: Optional[Path] = None) -> Optional["BochaConfig"]:
@ -35,12 +39,14 @@ class BochaConfig:
return None
data = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
env = data.get("bocha_api_key_env") or "BOCHA_API_KEY"
key = os.environ.get(env, "").strip()
from core.provider_credentials.runtime import resolve_env_secret
key = resolve_env_secret(env)
if not key:
return None
return cls(
api_key=key,
base_url=str(data.get("bocha_base_url") or "https://api.bochaai.com/v1").rstrip("/"),
api_key_env=env,
)
@ -49,10 +55,11 @@ class BochaClient:
def __init__(self, cfg: BochaConfig, timeout_s: float = 15.0) -> None:
self.cfg = cfg
self._api_key = cfg.request_api_key()
self._client = httpx.Client(
base_url=cfg.base_url,
headers={
"Authorization": f"Bearer {cfg.api_key}",
"Authorization": f"Bearer {self._api_key}",
"Content-Type": "application/json",
},
timeout=timeout_s,
@ -73,13 +80,21 @@ class BochaClient:
raise BochaError(f"博查网络错误: {e}") from e
return self._parse(resp)
@staticmethod
def _parse(resp: httpx.Response) -> dict:
def _parse(self, resp: httpx.Response) -> dict:
if resp.status_code >= 400:
try:
from core.provider_credentials.service import record_business_failure
record_business_failure(
"bocha", status_code=resp.status_code, detail=resp.text[:300]
)
except Exception:
pass
try:
msg = resp.json().get("message", resp.text[:300])
except ValueError:
msg = resp.text[:300]
key = self._api_key
msg = str(msg).replace(key, "***") if key else str(msg)
raise BochaError(f"博查 API → HTTP {resp.status_code}: {msg}")
try:
return resp.json()

View File

@ -3,13 +3,26 @@ from __future__ import annotations
from dataclasses import dataclass, field, fields
from pathlib import Path
from typing import List
from typing import Any, Dict, List, Optional
import yaml
from .llm_params import THINKING_TRANSPORTS
REASONING_REPLAY_POLICIES = {"none", "tool_calls", "all"}
REASONING_REPLAY_POLICIES = {
"none",
"tool_turn",
"conversation",
"provider_managed",
}
def model_profile_of(caps: object) -> str:
"""返回能力对象的 canonical profile兼容测试和旧调用方的鸭子类型。"""
profile = getattr(caps, "profile", "")
if profile:
return str(profile)
return f"{getattr(caps, 'family', '')}.{getattr(caps, 'variant', '')}".strip(".")
@dataclass
@ -34,10 +47,16 @@ class ModelCapabilities:
thinking_transport: str = "none"
reasoning_effort_levels: List[str] = field(default_factory=list)
default_reasoning_effort: str = ""
# 是否清除历史 thinking。None=不发送该 provider 可选字段False=保留历史推理。
thinking_clear: Optional[bool] = None
# 历史 reasoning 发回模型的策略。持久化原始响应与 provider-bound 输入分离:
# none=全部剥离tool_calls=仅工具调用 assistant 保留all=完整原样回传。
# none=剥离tool_turn=仅当前工具链conversation=同模型会话;
# provider_managed=同模型不透明签名/block。
reasoning_replay: str = "none"
# 原生输入模态。附件仍以文件引用落库,发模型前才物化成 provider content blocks。
input_modalities: List[str] = field(default_factory=lambda: ["text"])
# 代码 / 沙盒
code_quality: str = "good"
enable_run_python: bool = False
@ -56,6 +75,9 @@ class ModelCapabilities:
# 前缀缓存命中价(DeepSeek 等自动缓存 prompt 前缀,命中部分按此价,通常 ~0.1x input)。
# 0 = 不区分,缓存命中按 input 全价记(安全兜底,不会少记)。
cache_hit_cny_per_mtoken: float = 0.0
# 版本化价格配置。存在时由 core.pricing 作为本地事实源;上面三个扁平字段仅作
# 旧档案/旧调用方兼容与无版本价格兜底。
pricing: Dict[str, Any] = field(default_factory=dict)
# API 接入
api_base: str = ""
@ -81,6 +103,16 @@ class ModelCapabilities:
)
var = dict(variants[variant])
# 已下架 profile 可保留为一版隐藏别名,确保存量 task / scheduled job 能续跑;
# 真正能力和记账身份统一归目标 variant不维持旧模型双轨。
seen_aliases = {variant}
while var.get("alias_of"):
target = str(var["alias_of"]).strip()
if not target or target in seen_aliases or target not in variants:
raise ValueError(f"档案 {path} 的 variant={variant} alias_of 无效")
seen_aliases.add(target)
variant = target
var = dict(variants[target])
valid_keys = {f.name for f in fields(cls)}
kwargs = {k: v for k, v in var.items() if k in valid_keys}
kwargs["family"] = data.get("family", family)
@ -109,3 +141,11 @@ class ModelCapabilities:
f"{caps.default_reasoning_effort!r} 不在 reasoning_effort_levels 中"
)
return caps
@property
def profile(self) -> str:
return f"{self.family}.{self.variant}"
@property
def native_image_input(self) -> bool:
return "image" in self.input_modalities

View File

@ -14,6 +14,8 @@ from typing import Any, List
import json
import re
from .capabilities import REASONING_REPLAY_POLICIES
# chars↔tokens 粗折算(CJK+代码+json 混合保守按 ~2.5 char/token)与压缩触发比例。
# loop(50% 压缩门槛)与 context_fold(85% 折叠门槛)共用,单一事实源;压缩/折叠是
# 成本/安全优化、非正确性关键,估算粗糙无妨。
@ -40,30 +42,42 @@ _REASONING_FIELDS = {
def filter_reasoning_for_replay(
messages: List[dict[str, Any]],
policy: str = "none",
target_model_profile: str = "",
) -> tuple[List[dict[str, Any]], dict[str, int]]:
"""构造 provider-bound 消息副本,按能力档案裁剪历史 reasoning。
原始 Session/DB payload 不动继续供 UI导出和排障使用工具调用需要 reasoning
连续性的 provider 可用 ``tool_calls``未声明能力的 provider 默认 ``none``避免
跨模型发送私有状态保留时把 LiteLLM 可能放在 provider_specific_fields 里的
reasoning_content 提升到 DeepSeek/OpenAI-compatible 所需的顶层字段
``tool_turn`` 只保留当前用户轮次内工具链的状态``conversation`` 保留同模型
会话状态``provider_managed`` 保留同模型的不透明签名/block未声明能力默认
``none``只要历史消息的生产模型与目标模型不同就无条件剥离私有状态
"""
if policy not in {"none", "tool_calls", "all"}:
if policy not in REASONING_REPLAY_POLICIES:
raise ValueError(f"unsupported reasoning replay policy: {policy!r}")
last_user = max(
(i for i, msg in enumerate(messages) if msg.get("role") == "user"),
default=-1,
)
filtered: List[dict[str, Any]] = []
stripped_messages = 0
stripped_chars = 0
for message in messages:
for idx, message in enumerate(messages):
new_msg = deepcopy(message)
if policy == "all":
filtered.append(new_msg)
continue
source_profile = str(new_msg.pop("_model_profile", "") or "")
compatible = not target_model_profile or source_profile == target_model_profile
preserve = (
policy == "tool_calls"
compatible
and new_msg.get("role") == "assistant"
and (
policy in {"conversation", "provider_managed"}
or (
policy == "tool_turn"
and idx > last_user
and bool(new_msg.get("tool_calls"))
)
)
)
if preserve:
provider_fields = new_msg.get("provider_specific_fields")
if (
@ -242,6 +256,7 @@ def prepare_messages_for_llm(
old_tool_chars: int = 2_000,
compact_threshold_chars: int = 0,
reasoning_replay: str = "none",
target_model_profile: str = "",
) -> List[dict[str, Any]]:
"""返回发给 LLM 的 messages 副本。
@ -257,6 +272,7 @@ def prepare_messages_for_llm(
old_tool_chars=old_tool_chars,
compact_threshold_chars=compact_threshold_chars,
reasoning_replay=reasoning_replay,
target_model_profile=target_model_profile,
)
return prepared
@ -268,6 +284,7 @@ def prepare_messages_with_stats(
old_tool_chars: int = 2_000,
compact_threshold_chars: int = 0,
reasoning_replay: str = "none",
target_model_profile: str = "",
) -> tuple[List[dict[str, Any]], dict[str, int]]:
"""返回发给 LLM 的 messages 副本和压缩统计。
@ -279,7 +296,9 @@ def prepare_messages_with_stats(
"""
if keep_recent < 0:
keep_recent = 0
messages, reasoning_stats = filter_reasoning_for_replay(messages, reasoning_replay)
messages, reasoning_stats = filter_reasoning_for_replay(
messages, reasoning_replay, target_model_profile
)
# 先补齐被中断 run 留下的悬空 tool_calls(否则原样发给模型会被拒,见函数注释)。
messages, repaired_tool_calls = _repair_dangling_tool_calls(messages)
original_chars = sum(_message_chars(m) for m in messages)

View File

@ -25,6 +25,7 @@ from uuid import UUID
from sqlalchemy import update
from .capabilities import model_profile_of
from .context import (
CHARS_PER_TOKEN,
COMPACT_CONTEXT_RATIO,
@ -33,7 +34,8 @@ from .context import (
filter_reasoning_for_replay,
prepare_messages_with_stats,
)
from .storage import session_scope
from .llm_transport import extract_usage_details
from .storage import record_context_fold_failure, session_scope
from .storage.models import Task
from .storage.usage import record_chat_usage
@ -144,7 +146,10 @@ def maybe_fold(
"""
msgs = session.messages
reasoning_replay = getattr(caps, "reasoning_replay", "none")
provider_msgs, _ = filter_reasoning_for_replay(msgs, reasoning_replay)
target_profile = model_profile_of(caps)
provider_msgs, _ = filter_reasoning_for_replay(
msgs, reasoning_replay, target_profile
)
before_chars = sum(_message_chars(m) for m in provider_msgs)
est_tokens = estimate_window_tokens(provider_msgs, session.last_measured_usage())
if est_tokens < caps.reliable_context * FOLD_TRIGGER_RATIO:
@ -164,6 +169,7 @@ def maybe_fold(
msgs,
compact_threshold_chars=compact_threshold,
reasoning_replay=reasoning_replay,
target_model_profile=target_profile,
)
p_cut = _kth_user_index(prepared, _user_ordinal(msgs, cutoff))
if p_cut is None: # 理论不可达,防御
@ -174,20 +180,36 @@ def maybe_fold(
choices = getattr(response, "choices", None) or []
summary = ((choices[0].message.content if choices else "") or "").strip()
if not summary:
try:
record_context_fold_failure(
task_id=session.task_id,
user_id=user_id,
model_profile=target_profile,
reason="empty_summary",
)
except Exception:
pass
emit({"type": "warn", "msg": "context fold: 摘要调用返回空,本轮跳过折叠"})
return None
usage = getattr(response, "usage", None)
usage = extract_usage_details(getattr(response, "usage", None))
try:
record_chat_usage(
task_id=session.task_id,
user_id=user_id,
message_id=None, # 摘要不入 messages,event 挂 task 维度
model_profile=f"{caps.family}.{caps.variant}",
prompt_tokens=getattr(usage, "prompt_tokens", 0) or 0,
completion_tokens=getattr(usage, "completion_tokens", 0) or 0,
prompt_tokens=usage["tokens_in"],
completion_tokens=usage["tokens_out"],
input_cny_per_mtoken=caps.input_cny_per_mtoken,
output_cny_per_mtoken=caps.output_cny_per_mtoken,
cache_hit_tokens=usage["cache_hit_tokens"],
cache_hit_cny_per_mtoken=getattr(caps, "cache_hit_cny_per_mtoken", 0.0),
pricing=getattr(caps, "pricing", {}) or {},
extra_units={
k: v for k, v in usage.items()
if k not in ("tokens_in", "tokens_out") and v
},
response=response,
# 独立 kind:按 kind 对账"折叠烧多少钱",且折叠次数 = count(该 kind)
# (任务详情 context_folds 字段的数据源,免加计数列)
@ -200,7 +222,7 @@ def maybe_fold(
persist_fold(session.task_id, new_base, summary)
session.apply_fold(cutoff, summary)
after_provider_msgs, _ = filter_reasoning_for_replay(
session.messages, reasoning_replay
session.messages, reasoning_replay, target_profile
)
after_chars = sum(_message_chars(m) for m in after_provider_msgs)
stats = {

View File

@ -35,6 +35,7 @@ class ExecCtx:
task_id: UUID
working_dir: Path
cancel_check: Optional[Callable[[], bool]] = None
event_emit: Optional[Callable[[dict], None]] = None
@dataclass

View File

@ -25,7 +25,7 @@ run_python tmp .py 落 host 侧 `<user_root>/.zcbot_tmp/<task_id>/<rand>.py`(bin
Cancel limitation(第一版接受):
- docker exec 客户端断开后,容器内 server 端进程**不会**因此终止 这是 docker 设计
- 第一版只杀 docker CLI(Popen.kill());容器内残留进程靠 idle 5min reaper / 下次
- 第一版只杀 docker CLI(Popen.kill());容器内残留进程靠 idle 10min reaper / 下次
ensure rm -f 兜底
- 升级触发(§7.5 #3 PGID 协议):用户反馈"取消了但还在烧 CPU" / 多次 cancel 后
容器内进程堆积 启用ZCBOT_EXEC_ID env + PGID 写文件 + 二次 exec kill协议
@ -145,8 +145,12 @@ class DockerExecutor(Executor):
return ToolResult(content=f"[Error] unknown tool: {name}", exit_code=2)
try:
if name == "shell":
if not args.get("background"):
return self._call_heavy_foreground(name, args, ctx)
return self._exec_shell(args, ctx)
if name == "run_python":
if not args.get("background"):
return self._call_heavy_foreground(name, args, ctx)
return self._exec_python(args, ctx)
if name in FS_TOOLS:
return self._exec_fs_tool(name, args, ctx)
@ -157,6 +161,23 @@ class DockerExecutor(Executor):
)
return ToolResult(content=f"[Error] unhandled container tool: {name}", exit_code=2)
def _call_heavy_foreground(self, name: str, args: Dict[str, Any], ctx: ExecCtx) -> ToolResult:
"""前台重型工具先排共享槽;排队不进入命令 timeout。"""
def notify(payload: dict) -> None:
if ctx.event_emit is not None:
ctx.event_emit({"type": "execution_queue", **payload})
with self.pool.capacity.foreground(
str(self.user_id), ctx.cancel_check, notify
) as admitted:
if not admitted:
return ToolResult(content="[Error] command cancelled by user while queued", exit_code=130)
self.pool.exec_started(self.user_id)
try:
return self._exec_shell(args, ctx) if name == "shell" else self._exec_python(args, ctx)
finally:
self.pool.exec_finished(self.user_id)
# ── shell ────────────────────────────────────────────────
def _exec_shell(self, args: Dict[str, Any], ctx: ExecCtx) -> ToolResult:
@ -248,7 +269,7 @@ class DockerExecutor(Executor):
def _exec_background(self, name: str, args: Dict[str, Any], ctx: ExecCtx) -> ToolResult:
"""background=true:专用容器跑长进程,立即返回 proc_id。
为什么不是 `docker exec -d` sandbox 容器:sandbox idle 5min reaper +
为什么不是 `docker exec -d` sandbox 容器:sandbox idle 10min reaper +
启动时 shutdown_all,长进程会随容器陪葬专用容器 product=proc 与这两条
生命周期解耦(pool.run_proc_container),状态协议同 host 模式(core/procs.py):
runner.sh 结束时写 exit_code,check_process/sweep 负责回收容器
@ -256,15 +277,6 @@ class DockerExecutor(Executor):
from core import procs
anchor = self.user_root
if procs.count_running(anchor) >= procs.MAX_RUNNING_PER_USER:
return ToolResult(
content=(
f"[Error] 已有 {procs.MAX_RUNNING_PER_USER} 个后台进程在跑(上限)。"
f"用 check_process 查看,等待完成或 kill 掉不需要的再启动。"
),
exit_code=2,
)
raw_timeout = args.get("timeout")
fg_default = 60 if name == "shell" else 120
try:
@ -334,29 +346,16 @@ class DockerExecutor(Executor):
"timeout_s": timeout_s,
"created_at": time.strftime("%Y-%m-%dT%H:%M:%S"),
"created_ts": time.time(),
"user_id": str(self.user_id),
"exec_user": self.exec_user,
"state": "queued",
}
procs.write_meta(d, meta)
try:
container = self.pool.run_proc_container(self.user_id, proc_id, str(d))
except Exception as e:
return ToolResult(content=f"[Error] 后台容器启动失败: {e}", exit_code=1)
meta["container"] = container
procs.write_meta(d, meta)
argv = self._docker_exec_argv(
container, extra_env=_sandbox_env({"PYTHONIOENCODING": "utf-8"}), detach=True
) + ["bash", f"{cdir}/runner.sh"]
r = subprocess.run(argv, capture_output=True, text=True, timeout=60)
if r.returncode != 0:
subprocess.run(["docker", "rm", "-f", container], capture_output=True)
return ToolResult(
content=f"[Error] 后台进程启动失败: {(r.stderr or '').strip()[:300]}",
exit_code=1,
)
started = procs.start_queued_docker(meta, d, self.pool)
return ToolResult(
content=(
f"[Background] 已启动后台进程 proc_id={proc_id}({display[:150]}),"
f"[Background] 已{'启动' if started else '排队'}后台进程 proc_id={proc_id}({display[:150]}),"
f"最长运行 {timeout_s}s。\n"
f"用 check_process(proc_id=\"{proc_id}\") 查进度和日志。进程独立于本轮对话运行,"
f"服务重启也不中断。现在可以继续其他工作;若无事可做,结束回合并告知用户稍后询问进度。"

View File

@ -30,6 +30,7 @@ from core.kb import format_index_line, kb_dir, parse_index, pending_sources
from core.file_store import atomic_write_text
from core.kb_lock import KbBusyError, kb_is_locked, kb_mutation_lock
from core.llm import LLM
from core.llm_transport import extract_usage_details
from core.storage.usage import record_chat_usage, record_vision_usage
# 摘要模型固定走最便宜档(与 web FALLBACK_MODEL_PROFILE 同值;单次几千 token,不随任务模型)
@ -184,20 +185,30 @@ def _summarize(
except Exception as e:
print(f"[kb_ingest] summary llm failed ({filename}): {type(e).__name__}: {e}", flush=True)
return fallback
usage = getattr(response, "usage", None)
usage = extract_usage_details(getattr(response, "usage", None))
try:
record_chat_usage(
task_id=None,
user_id=user_id,
message_id=None,
model_profile=f"{caps.family}.{caps.variant}",
prompt_tokens=getattr(usage, "prompt_tokens", 0) or 0,
completion_tokens=getattr(usage, "completion_tokens", 0) or 0,
prompt_tokens=usage["tokens_in"],
completion_tokens=usage["tokens_out"],
input_cny_per_mtoken=caps.input_cny_per_mtoken,
output_cny_per_mtoken=caps.output_cny_per_mtoken,
cache_hit_tokens=usage["cache_hit_tokens"],
cache_hit_cny_per_mtoken=getattr(caps, "cache_hit_cny_per_mtoken", 0.0),
pricing=getattr(caps, "pricing", {}) or {},
response=response,
kind="kb_ingest",
extra_units={"kb": kb_name, "source": filename},
extra_units={
"kb": kb_name,
"source": filename,
**{
k: v for k, v in usage.items()
if k not in ("tokens_in", "tokens_out") and v
},
},
)
except Exception:
pass

View File

@ -37,16 +37,39 @@ _REQUEST_TIMEOUT_S = int(os.getenv("ZCBOT_LLM_TIMEOUT_S", "600"))
class LLM:
def __init__(self, capabilities: ModelCapabilities) -> None:
def __init__(
self,
capabilities: ModelCapabilities,
*,
credential_resolver: Callable[[str], str] | None = None,
) -> None:
self.caps = capabilities
env_name = capabilities.api_key_env or "DEEPSEEK_API_KEY"
self.api_key = os.environ.get(env_name)
self.env_name = capabilities.api_key_env or "DEEPSEEK_API_KEY"
if credential_resolver is None:
from .provider_credentials.runtime import resolve_env_secret
credential_resolver = resolve_env_secret
self._credential_resolver = credential_resolver
self.api_key = self._credential_resolver(self.env_name)
self.api_base = capabilities.api_base or None
if not self.api_key:
raise RuntimeError(
f"环境变量 {env_name} 未设置,无法调用 {capabilities.model_id}"
f"Provider 凭据 {self.env_name} 未设置,无法调用 {capabilities.model_id}"
)
def _request_api_key(self) -> str:
key = self._credential_resolver(self.env_name)
if not key:
# 非注册表内的测试/私有模型仍保持构造期 env 快照兼容;正式受管
# Provider 不回退快照,删除 DB 覆盖后应立即回到当前 env 或报未配置。
from .provider_credentials.registry import BY_ENV
if self.env_name not in BY_ENV:
key = self.api_key
if not key:
raise RuntimeError(
f"Provider 凭据 {self.env_name} 未设置,无法调用 {self.caps.model_id}"
)
return key
def _build_kwargs(
self,
messages: List[dict],
@ -54,11 +77,19 @@ class LLM:
parallel_tool_calls: Optional[bool],
reasoning_effort: Optional[str],
) -> dict:
# Session 私有元数据永不进入 provider 请求。原生图片由主 loop 在此之前
# 物化;折叠/标题等旁路则安全退化为纯文本路径提示。
provider_messages = [
{k: v for k, v in message.items() if not str(k).startswith("_")}
for message in messages
]
kwargs: dict = {
"model": self.caps.model_id,
"messages": messages,
"messages": provider_messages,
"temperature": self.caps.optimal_temperature,
"api_key": self.api_key,
# 请求级解析:已经创建的 LLM 在下一次请求热切新 Key本次 kwargs
# 构建后保持局部快照,流式迭代期间不会中途换 Key。
"api_key": self._request_api_key(),
"timeout": _REQUEST_TIMEOUT_S,
}
if self.api_base:
@ -72,12 +103,29 @@ class LLM:
enabled=self.caps.thinking_enabled,
transport=self.caps.thinking_transport,
reasoning_effort=reasoning_effort,
clear_thinking=self.caps.thinking_clear,
)
)
if self.caps.prompt_caching:
kwargs["extra_headers"] = {"anthropic-beta": "prompt-caching-2024-07-31"}
return kwargs
def _report_provider_error(self, error: Exception) -> None:
try:
from .provider_credentials.registry import BY_ENV
from .provider_credentials.service import record_business_failure
binding = BY_ENV.get(self.env_name)
if binding is None:
return
raw_status = getattr(error, "status_code", 0)
if hasattr(raw_status, "value"):
raw_status = raw_status.value
record_business_failure(
binding[0], status_code=int(raw_status or 0), detail=str(error)
)
except Exception:
pass
def chat(
self,
messages: List[dict],
@ -94,6 +142,7 @@ class LLM:
return response
except (RateLimitError, APIConnectionError, ServiceUnavailableError, Timeout, APIError) as e:
last_err = e
self._report_provider_error(e)
if attempt == max_retries - 1:
break
time.sleep(2 ** attempt)
@ -130,8 +179,7 @@ class LLM:
yield from self._chat_stream_direct(kwargs, max_retries=max_retries)
@staticmethod
def _chat_stream_direct(kwargs: dict, *, max_retries: int) -> Iterator[Any]:
def _chat_stream_direct(self, kwargs: dict, *, max_retries: int) -> Iterator[Any]:
"""无取消调用方的直连路径probe 等保持原有同步语义)。"""
last_err: Optional[Exception] = None
@ -141,6 +189,7 @@ class LLM:
break
except (RateLimitError, APIConnectionError, ServiceUnavailableError, Timeout, APIError) as e:
last_err = e
self._report_provider_error(e)
if attempt == max_retries - 1:
raise
time.sleep(2 ** attempt)
@ -159,8 +208,8 @@ class LLM:
except Exception:
pass
@staticmethod
def _chat_stream_interruptible(
self,
kwargs: dict,
*,
max_retries: int,
@ -199,6 +248,7 @@ class LLM:
APIError,
) as e:
last_err = e
self._report_provider_error(e)
if attempt == max_retries - 1:
raise
if stop.wait(2 ** attempt):

View File

@ -7,7 +7,11 @@ THINKING_TRANSPORTS = {"none", "extra_body"}
def build_thinking_kwargs(
*, enabled: bool, transport: str, reasoning_effort: str | None
*,
enabled: bool,
transport: str,
reasoning_effort: str | None,
clear_thinking: bool | None = None,
) -> dict[str, Any]:
"""把统一的 thinking 配置转换为 LiteLLM 调用参数。
@ -22,9 +26,12 @@ def build_thinking_kwargs(
f"不支持的 thinking_transport={transport!r};可选: {sorted(THINKING_TRANSPORTS)}"
)
body: dict[str, Any] = {
"thinking": {"type": "enabled" if enabled else "disabled"}
thinking: dict[str, Any] = {
"type": "enabled" if enabled else "disabled"
}
if enabled and clear_thinking is not None:
thinking["clear_thinking"] = clear_thinking
body: dict[str, Any] = {"thinking": thinking}
if enabled and reasoning_effort:
body["reasoning_effort"] = reasoning_effort
return {"extra_body": body}

View File

@ -22,7 +22,8 @@ import litellm
from . import pptx_guard
from .artifacts import MAX_ARTIFACTS_PER_MESSAGE
from .capabilities import ModelCapabilities
from .attachments import materialize_native_images
from .capabilities import ModelCapabilities, model_profile_of
from .context import (
CHARS_PER_TOKEN,
COMPACT_CONTEXT_RATIO,
@ -45,8 +46,13 @@ from .llm_transport import (
from .salvage import salvage_tool_arguments
from .session import Session
from .storage import (
record_agent_guard,
record_chat_usage,
record_context_fold_failure,
record_quality_gate,
record_run_stopped,
record_salvaged_tool_call,
record_tool_failure,
)
from .task_actions import DeferredTaskActions
@ -312,20 +318,21 @@ class AgentLoop:
assert response is not None
msg = response.choices[0].message
tool_calls = getattr(msg, "tool_calls", None) or []
model_profile = model_profile_of(self.caps)
asst_msg_id = self.session.append(
msg,
artifact_refs=(list(self._pending_artifact_refs) if not tool_calls else None),
source_model_profile=model_profile,
)
usage_details = extract_usage_details(getattr(response, "usage", None))
pt, ct = usage_details["tokens_in"], usage_details["tokens_out"]
# 用本轮实报 prompt_tokens 刷新 chars/token 校准比值(下一轮门槛/占用环即用)。
if pt > 0 and self._last_sent_chars > 0:
if pt > 0 and self._last_sent_chars > 0 and not self._last_had_native_images:
self._ctx_chars_per_token = clamp_ratio(self._last_sent_chars / pt)
# 记账(0006):一行 usage_event + 回填 messages.tokens_in/out + model_profile。
# 任何失败都吞掉(litellm cost map miss / DB 异常),不阻塞主 loop;
# message 仍在 session/DB 里,后续重启不影响。
model_profile = f"{self.caps.family}.{self.caps.variant}"
try:
record_chat_usage(
task_id=self.session.task_id,
@ -338,6 +345,7 @@ class AgentLoop:
output_cny_per_mtoken=self.caps.output_cny_per_mtoken,
cache_hit_tokens=usage_details["cache_hit_tokens"],
cache_hit_cny_per_mtoken=self.caps.cache_hit_cny_per_mtoken,
pricing=self.caps.pricing,
extra_units={
k: v for k, v in usage_details.items()
if k not in ("tokens_in", "tokens_out") and v
@ -381,7 +389,7 @@ class AgentLoop:
result, productive, artifacts = self._execute_tool_call(tc)
self._remember_artifacts(artifacts)
step_productive = step_productive or productive
self.session.append(
message_id = self.session.append(
{
"role": "tool",
"tool_call_id": tc.id,
@ -389,6 +397,20 @@ class AgentLoop:
"content": result,
}
)
if message_id is not None:
try:
record_tool_failure(
task_id=self.session.task_id,
user_id=self.user_id,
message_id=message_id,
model_profile=model_profile_of(self.caps),
tool=tc.function.name,
content=result,
arguments=tc.function.arguments,
)
except Exception:
# 可观测性留痕失败不能打断主对话。
pass
# ask_user:本步调用了人工选择工具 → 提前结束本轮,等用户点选项 / 文字讨论,
# 不回灌 LLM。选项已随该 tool_call 的 arguments 流给前端渲染成选项卡;tool 结果
@ -404,6 +426,15 @@ class AgentLoop:
else:
self._stall += 1
if self._stall >= self._STALL_LIMIT:
try:
record_run_stopped(
task_id=self.session.task_id,
user_id=self.user_id,
model_profile=model_profile_of(self.caps),
reason="no_progress",
)
except Exception:
pass
self._emit({
"type": "warn",
"msg": (
@ -415,6 +446,15 @@ class AgentLoop:
return "[stopped: no progress]"
# 跑满 backstop:不是出错,是单轮自主步数到顶。明确提示可续跑,别静默停。
try:
record_run_stopped(
task_id=self.session.task_id,
user_id=self.user_id,
model_profile=model_profile_of(self.caps),
reason="max_iterations",
)
except Exception:
pass
self._emit({
"type": "warn",
"msg": (
@ -456,6 +496,7 @@ class AgentLoop:
# 无实测时的回退(详 context.py 常数注释)。
_ctx_chars_per_token: Optional[float] = None
_last_sent_chars: int = 0
_last_had_native_images: bool = False
def _context_ratio(self) -> float:
"""当前窗口的 chars/token 换算比值(校准态)。
@ -470,6 +511,7 @@ class AgentLoop:
provider_messages, _ = filter_reasoning_for_replay(
self.session.messages,
getattr(self.caps, "reasoning_replay", "none"),
model_profile_of(self.caps),
)
self._ctx_chars_per_token = calibrated_chars_per_token(
provider_messages, self.session.last_measured_usage()
@ -492,6 +534,16 @@ class AgentLoop:
user_id=self.user_id, emit=self._emit,
)
except Exception as e:
try:
record_context_fold_failure(
task_id=self.session.task_id,
user_id=self.user_id,
model_profile=model_profile_of(self.caps),
reason="exception",
detail=f"{type(e).__name__}: {e}",
)
except Exception:
pass
self._emit({
"type": "warn",
"msg": f"context fold failed: {type(e).__name__}: {e};本轮跳过折叠",
@ -514,7 +566,18 @@ class AgentLoop:
self.session.messages,
compact_threshold_chars=compact_threshold,
reasoning_replay=getattr(self.caps, "reasoning_replay", "none"),
target_model_profile=model_profile_of(self.caps),
)
native_images = 0
if self.user_root is not None:
llm_messages, native_images = materialize_native_images(
llm_messages,
enabled=self.caps.native_image_input,
working_dir=self.working_dir,
user_root=self.user_root,
)
self._last_had_native_images = native_images > 0
context_stats["native_images"] = native_images
self._last_sent_chars = context_stats.get("sent_chars", 0)
llm_start_event = {
"type": "llm_start",
@ -623,9 +686,8 @@ class AgentLoop:
cancel_check=self._is_cancelled,
)
cancelled = False
pending_events: List[dict] = []
may_reroute = self.caps.family == "deepseek_v4"
output_route_known = not may_reroute
reasoning_emitted = False
try:
for chunk in stream:
if self._is_cancelled():
@ -637,7 +699,10 @@ class AgentLoop:
may_reroute
and any(name in self._DEEPSEEK_NONSTREAM_TOOLS for name in tool_names)
):
# 推理 delta 先暂存,避免切换后非流式完整 reasoning 再发一次造成重复。
# 当前流会被放弃并以非流式完整重发。已实时展示的推理属于被放弃
# 的响应,先通知前端清掉,避免随后展示最终响应时重复或与历史不一致。
if reasoning_emitted:
self._emit({"type": "reasoning_reset"})
raise PreferNonstreamToolCall(next(
name for name in tool_names
if name in self._DEEPSEEK_NONSTREAM_TOOLS
@ -647,20 +712,13 @@ class AgentLoop:
# _execute_tool_call 时机发更直观)。
delta_text = extract_delta_content(chunk)
if delta_text:
pending_events.append({"type": "text", "delta": delta_text})
self._emit({"type": "text", "delta": delta_text})
# thinking 模型的推理 delta 也实时流出(reasoning 事件):深度推理可达
# 分钟级,不发的话前端全程静止"思考中",用户以为卡死。
delta_reasoning = extract_delta_reasoning(chunk)
if delta_reasoning:
pending_events.append({"type": "reasoning", "delta": delta_reasoning})
# 首个正文或任意工具名已确定本轮不会再切换;释放之前暂存的推理片段,
# 后续 chunk 也继续即时释放。只有尚未看到输出类型时才短暂缓冲。
if delta_text or tool_names:
output_route_known = True
if output_route_known and pending_events:
for event in pending_events:
self._emit(event)
pending_events.clear()
self._emit({"type": "reasoning", "delta": delta_reasoning})
reasoning_emitted = True
# interruptible stream 会在无新 chunk 的等待期直接因 cancel 结束迭代;
# 循环体没有机会执行上面的检查,故在正常耗尽处再判一次。
if self._is_cancelled():
@ -674,10 +732,6 @@ class AgentLoop:
if cancelled:
return None, True
# 极少数 provider 只有 reasoning/空收尾、始终没有正文或工具名,不能吞掉已收内容。
for event in pending_events:
self._emit(event)
# 用 litellm 官方 helper 拼回完整 response(包括 tool_calls 拼接 + usage)。
# messages 参数仅用于失败时回填 prompt token 估算,正常路径 stream_options.include_usage
# 已让最后一个 chunk 带准确 usage。
@ -765,6 +819,7 @@ class AgentLoop:
task_id=self.session.task_id,
working_dir=self.working_dir,
cancel_check=self.cancel_check,
event_emit=self._emit,
)
tool_started_at = time.time()
tool_result = self.executor.call_tool(name, args, ctx)
@ -834,6 +889,17 @@ class AgentLoop:
"""
if self._repeat_guard.should_block(name, args):
n, _blocked = self._repeat_guard.register_block(name, args)
try:
record_agent_guard(
task_id=self.session.task_id,
user_id=self.user_id,
model_profile=model_profile_of(self.caps),
tool=name,
guard="same_arguments",
count=n,
)
except Exception:
pass
result = (
f"[已拦截重复调用] {name} 用完全相同的参数已调用 {n} 次且结果始终未变,本次未执行。"
"这通常意味着思路卡死:① 换不同的参数或方法;② 读一下相关文件/报错重新定位;"
@ -845,6 +911,18 @@ class AgentLoop:
if self._repeat_guard.should_block_err(name):
cnt, esig = self._repeat_guard.register_err_block(name)
try:
record_agent_guard(
task_id=self.session.task_id,
user_id=self.user_id,
model_profile=model_profile_of(self.caps),
tool=name,
guard="same_error",
count=cnt,
signature=esig,
)
except Exception:
pass
result = (
f"[已拦截重复调用] {name} 已连续 {cnt} 次撞同一个错误「{esig}」(每次只微调了参数)。"
"再这么试下去不会有新结果。换个做法:① 先 read 目标文件/用 grep 看确切内容"
@ -937,6 +1015,16 @@ class AgentLoop:
except Exception:
guard_msg = None # 机检自身故障绝不拖垮工具链路
if guard_msg:
try:
record_quality_gate(
task_id=self.session.task_id,
user_id=self.user_id,
model_profile=model_profile_of(self.caps),
tool=name,
gate="pptx_full_page_image",
)
except Exception:
pass
result += "\n\n" + guard_msg
self._emit({
"type": "warn",

View File

@ -6,7 +6,7 @@
- 某档成员里出现 `"*"` 该档全开(含未来新增模型),给内部档用
模型 id 约定( list 端点 / resolve 校验一致):
- 文本模型 = `family.variant`(config/models/<family>.yaml), `doubao.pro``glm.pro52`
- 文本模型 = `family.variant`(config/models/<family>.yaml), `doubao.pro``glm.flash53`
- / 视频模型 = variant key(config/media/doubao.yaml), `seedream_5``seedance_2_fast`
两者命名不冲突(文本带点媒体 variant 不带点),同一档集合里混放即可

209
core/pricing.py Normal file
View File

@ -0,0 +1,209 @@
"""版本化 chat 价格解析与成本计算。
模型 API 通常只返回 token usage不返回实际金额这里以模型 YAML 中经过核对的
provider 价格为事实源并把命中的价格版本时段和汇率快照交给 usage_events 留痕
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, time, timezone
from decimal import Decimal
from typing import Any, Mapping, Optional
_MTOKEN = Decimal("1000000")
_WEEKDAYS = {
"mon": 0,
"tue": 1,
"wed": 2,
"thu": 3,
"fri": 4,
"sat": 5,
"sun": 6,
}
def _decimal(value: Any, default: str = "0") -> Decimal:
if value is None or value == "":
return Decimal(default)
return Decimal(str(value))
def _parse_datetime(value: Any) -> Optional[datetime]:
if not value:
return None
if isinstance(value, datetime):
dt = value
else:
raw = str(value).strip().replace("Z", "+00:00")
dt = datetime.fromisoformat(raw)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone.utc)
def _parse_time(value: str) -> time:
hour, minute = (int(part) for part in value.split(":", 1))
return time(hour=hour, minute=minute)
def _in_window(now: time, start: time, end: time) -> bool:
if start <= end:
return start <= now < end
return now >= start or now < end
@dataclass(frozen=True)
class PriceQuote:
revision: str
source_url: str
currency: str
fx_to_cny: Decimal
tier: str
input_per_mtoken: Decimal
output_per_mtoken: Decimal
cache_hit_per_mtoken: Decimal
@dataclass(frozen=True)
class CostBreakdown:
total_cny: Decimal
input_cny: Decimal
cache_hit_cny: Decimal
output_cny: Decimal
cache_hit_tokens: int
cache_miss_tokens: int
def _periods(pricing: Mapping[str, Any]) -> list[Mapping[str, Any]]:
periods = pricing.get("periods")
if isinstance(periods, list):
return [p for p in periods if isinstance(p, Mapping)]
# 兼容一个价格版本直接写在 pricing 顶层的简写。
return [pricing]
def _select_period(
pricing: Mapping[str, Any], occurred_at: datetime
) -> Optional[Mapping[str, Any]]:
candidates: list[tuple[datetime, Mapping[str, Any]]] = []
for period in _periods(pricing):
start = _parse_datetime(period.get("effective_from")) or datetime.min.replace(
tzinfo=timezone.utc
)
end = _parse_datetime(period.get("effective_to"))
if start <= occurred_at and (end is None or occurred_at < end):
candidates.append((start, period))
if not candidates:
return None
return max(candidates, key=lambda item: item[0])[1]
def _tier_matches(tier: Mapping[str, Any], occurred_at: datetime) -> bool:
tz_name = str(tier.get("timezone") or "UTC").upper()
if tz_name != "UTC":
raise ValueError(f"pricing time tier 暂只支持 UTC收到 {tz_name!r}")
now = occurred_at.astimezone(timezone.utc)
weekdays = tier.get("weekdays") or []
if weekdays:
allowed = {_WEEKDAYS[str(day).lower()] for day in weekdays}
if now.weekday() not in allowed:
return False
windows = tier.get("windows") or []
if not windows:
return True
current = now.time().replace(tzinfo=None)
return any(
_in_window(current, _parse_time(window[0]), _parse_time(window[1]))
for window in windows
if isinstance(window, (list, tuple)) and len(window) == 2
)
def resolve_chat_price(
pricing: Mapping[str, Any] | None,
*,
occurred_at: Optional[datetime] = None,
) -> Optional[PriceQuote]:
"""解析调用时刻适用的价格;无配置或尚未生效时返回 ``None``。"""
if not pricing:
return None
when = occurred_at or datetime.now(timezone.utc)
if when.tzinfo is None:
when = when.replace(tzinfo=timezone.utc)
when = when.astimezone(timezone.utc)
period = _select_period(pricing, when)
if period is None:
return None
rates = period.get("default") or {}
tier_name = "default"
for tier in period.get("time_tiers") or []:
if isinstance(tier, Mapping) and _tier_matches(tier, when):
rates = tier
tier_name = str(tier.get("name") or "time_tier")
break
currency = str(period.get("currency") or "CNY").upper()
if currency not in {"CNY", "USD"}:
raise ValueError(f"不支持的 pricing currency: {currency!r}")
fx = _decimal(period.get("fx_to_cny"), "1") if currency == "USD" else Decimal("1")
return PriceQuote(
revision=str(period.get("revision") or "unversioned"),
source_url=str(period.get("source_url") or ""),
currency=currency,
fx_to_cny=fx,
tier=tier_name,
input_per_mtoken=_decimal(rates.get("input_per_mtoken")),
output_per_mtoken=_decimal(rates.get("output_per_mtoken")),
cache_hit_per_mtoken=_decimal(
rates.get("cache_hit_per_mtoken"),
str(rates.get("input_per_mtoken") or 0),
),
)
def calculate_chat_cost(
quote: PriceQuote,
*,
prompt_tokens: int,
completion_tokens: int,
cache_hit_tokens: int = 0,
) -> CostBreakdown:
"""按价格快照拆分缓存命中、未命中和输出成本,返回人民币金额。"""
tokens_in = max(0, int(prompt_tokens))
hit = max(0, min(int(cache_hit_tokens), tokens_in))
miss = tokens_in - hit
tokens_out = max(0, int(completion_tokens))
input_cost = Decimal(miss) * quote.input_per_mtoken * quote.fx_to_cny / _MTOKEN
hit_cost = Decimal(hit) * quote.cache_hit_per_mtoken * quote.fx_to_cny / _MTOKEN
output_cost = Decimal(tokens_out) * quote.output_per_mtoken * quote.fx_to_cny / _MTOKEN
quant = Decimal("0.000001")
return CostBreakdown(
total_cny=(input_cost + hit_cost + output_cost).quantize(quant),
input_cny=input_cost.quantize(quant),
cache_hit_cny=hit_cost.quantize(quant),
output_cny=output_cost.quantize(quant),
cache_hit_tokens=hit,
cache_miss_tokens=miss,
)
def pricing_snapshot(quote: PriceQuote, breakdown: CostBreakdown) -> dict[str, Any]:
"""生成可直接合并进 usage_events.units 的价格快照。"""
return {
"pricing_revision": quote.revision,
"pricing_source": "local_catalog",
"pricing_source_url": quote.source_url,
"pricing_currency": quote.currency,
"price_tier": quote.tier,
"fx_to_cny": float(quote.fx_to_cny),
"input_price_per_mtoken": float(quote.input_per_mtoken),
"cache_hit_price_per_mtoken": float(quote.cache_hit_per_mtoken),
"output_price_per_mtoken": float(quote.output_per_mtoken),
"cache_hit_tokens": breakdown.cache_hit_tokens,
"cache_miss_tokens": breakdown.cache_miss_tokens,
"input_cost_cny": float(breakdown.input_cny),
"cache_hit_cost_cny": float(breakdown.cache_hit_cny),
"output_cost_cny": float(breakdown.output_cny),
}

View File

@ -45,9 +45,6 @@ PROCS_SUBDIR = ".zcbot_procs"
DEFAULT_TIMEOUT_S = 7200
MAX_TIMEOUT_S = 86400
# 每用户并发 bg 进程上限(跨 task 合计);防模型失控起一堆
MAX_RUNNING_PER_USER = int(os.getenv("ZCBOT_MAX_BG_PROCS", "3"))
# 终态 proc 目录保留时长,sweep 超期删除
FINISHED_TTL_S = 7 * 86400
@ -151,12 +148,14 @@ def _container_running(name: str) -> bool:
def status_of(meta: Dict[str, Any], d: Path) -> Tuple[str, Optional[int]]:
"""("running"|"finished"|"lost", exit_code|None)。exit_code 文件是唯一终态锚。"""
"""queued/running/finished/lost。exit_code 文件是唯一终态锚。"""
try:
raw = (d / "exit_code").read_text(encoding="utf-8").strip()
return "finished", int(raw)
except (OSError, ValueError):
pass
if meta.get("state") == "queued":
return "queued", None
if meta.get("backend") == "docker":
if _container_running(str(meta.get("container") or "")):
return "running", None
@ -304,10 +303,24 @@ def kill_proc(meta: Dict[str, Any], d: Path) -> str:
st, _ = status_of(meta, d)
if st == "finished":
return "进程已结束,无需终止"
if st == "queued":
meta["state"] = "finished"
meta["killed"] = True
write_meta(d, meta)
(d / "exit_code").write_text("137", encoding="utf-8")
return "已取消排队"
if meta.get("backend") == "docker":
name = str(meta.get("container") or "")
if name:
_scan_proc_container(meta, d, name)
subprocess.run(["docker", "rm", "-f", name], capture_output=True, timeout=30)
try:
from core.sandbox import get_pool
pool = get_pool()
if pool is not None:
pool.capacity.release(meta.get("capacity_lease_id"))
except Exception:
pass
else:
wrapper_pid = int(meta.get("wrapper_pid") or 0)
child_pid = int(meta.get("child_pid") or 0)
@ -338,6 +351,69 @@ def kill_proc(meta: Dict[str, Any], d: Path) -> str:
return "已终止"
def start_queued_docker(meta: Dict[str, Any], d: Path, pool) -> bool:
"""尝试为 queued proc 获取共享槽并启动;无槽返回 False保持排队。"""
from core.file_store import interprocess_file_lock
with interprocess_file_lock(d / "dispatch.lock", timeout_seconds=None):
meta = read_meta(d) or meta
if status_of(meta, d)[0] != "queued":
return False
uid = str(meta.get("user_id") or "")
lease_id = f"bg-{meta.get('proc_id')}"
lease = pool.capacity.try_acquire(uid, "background", lease_id=lease_id,
proc_id=str(meta.get("proc_id")))
if not lease:
return False
try:
from uuid import UUID
container = pool.run_proc_container(UUID(uid), str(meta["proc_id"]), str(d))
cdir = f"/workspace/{PROCS_SUBDIR}/{meta['task_id']}/{meta['proc_id']}"
argv = ["docker", "exec", "--user", str(meta.get("exec_user") or "zcbot"),
"--workdir", str(meta.get("cwd") or "/workspace"), "-d",
"-e", "PYTHONPATH=/sandbox:/workspace", "-e", "HOME=/tmp",
"-e", "PYTHONIOENCODING=utf-8", container, "bash", f"{cdir}/runner.sh"]
r = subprocess.run(argv, capture_output=True, text=True, timeout=60)
if r.returncode != 0:
subprocess.run(["docker", "rm", "-f", container], capture_output=True)
raise RuntimeError((r.stderr or "").strip()[:300])
meta.update({"container": container, "capacity_lease_id": lease,
"state": "running", "started_ts": time.time()})
write_meta(d, meta)
return True
except Exception as exc:
pool.capacity.release(lease)
meta["last_start_error"] = f"{type(exc).__name__}: {exc}"
write_meta(d, meta)
return False
def dispatch_queued(user_root_base: Path, pool) -> int:
started = 0
base = Path(user_root_base)
if not base.is_dir():
return 0
candidates = []
running_ids: set[str] = set()
for uroot in base.iterdir():
root = uroot / PROCS_SUBDIR
if not root.is_dir():
continue
for meta_file in root.glob("*/*/proc.json"):
d = meta_file.parent
meta = read_meta(d)
if meta:
status = status_of(meta, d)[0]
if status == "queued":
candidates.append((float(meta.get("created_ts") or 0), meta, d))
elif status == "running":
running_ids.add(str(meta.get("proc_id") or d.name))
pool.capacity.reconcile_background(running_ids)
for _, meta, d in sorted(candidates, key=lambda x: x[0]):
if start_queued_docker(meta, d, pool):
started += 1
return started
# ───────────── 清扫(web lifespan 周期调用) ─────────────
def sweep(user_root_base: Path, ttl_s: int = FINISHED_TTL_S) -> Dict[str, int]:
@ -365,16 +441,27 @@ def sweep(user_root_base: Path, ttl_s: int = FINISHED_TTL_S) -> Dict[str, int]:
if not meta:
continue
st, _ = status_of(meta, d)
if st != "running" and meta.get("backend") == "docker":
if st in {"finished", "lost"} and meta.get("capacity_lease_id"):
try:
from core.sandbox import get_pool
pool = get_pool()
if pool is not None:
pool.capacity.release(meta.get("capacity_lease_id"))
meta.pop("capacity_lease_id", None)
write_meta(d, meta)
except Exception:
pass
if st not in {"running", "queued"} and meta.get("backend") == "docker":
name = str(meta.get("container") or "")
if name and _container_exists(name):
_scan_proc_container(meta, d, name)
subprocess.run(
["docker", "rm", "-f", name],
capture_output=True, timeout=30,
)
reaped_containers += 1
age = now - float(meta.get("created_ts") or now)
if st != "running" and age > ttl_s:
if st not in {"running", "queued"} and age > ttl_s:
shutil.rmtree(d, ignore_errors=True)
removed_dirs += 1
try:
@ -410,6 +497,16 @@ def _container_exists(name: str) -> bool:
return False
def _scan_proc_container(meta: Dict[str, Any], d: Path, name: str) -> None:
try:
from uuid import UUID
from core.sandbox.package_scans import scan_and_persist
uid = UUID(str(meta.get("user_id")))
scan_and_persist(name, uid, "background", str(meta.get("proc_id") or d.name))
except Exception as exc:
print(f"[sandbox-package-scan] proc finalizer failed container={name}: {type(exc).__name__}: {exc}")
# ───────────── LLM 面向的描述格式化(check_process / 启动返回共用) ─────────────
def format_proc_line(meta: Dict[str, Any]) -> str:
@ -418,7 +515,7 @@ def format_proc_line(meta: Dict[str, Any]) -> str:
elapsed = ""
ts = meta.get("created_ts")
if ts:
elapsed = f" · 已运行 {_fmt_elapsed(time.time() - float(ts))}" if st == "running" else ""
elapsed = f" · 已运行 {_fmt_elapsed(time.time() - float(meta.get('started_ts') or ts))}" if st == "running" else ""
tail = f"(exit {ec})" if st == "finished" else ""
return (
f"- {meta.get('proc_id')} [{st}{tail}] {meta.get('kind')}: "

View File

@ -0,0 +1,5 @@
"""Admin 管理的平台 Provider 凭据控制面。"""
from .runtime import resolve_credentials, resolve_secret
__all__ = ["resolve_credentials", "resolve_secret"]

View File

@ -0,0 +1,50 @@
"""蓝绿单轮选主的 Provider 巡检与开发者邮件。"""
from __future__ import annotations
import os
from core.storage import get_engine
from .registry import get_provider
from .service import due_provider_ids, test_current_credentials
from .testing import TestResult
_LOCK_SQL = "SELECT pg_try_advisory_lock(31331, 2)"
_UNLOCK_SQL = "SELECT pg_advisory_unlock(31331, 2)"
def send_notification(provider_id: str, event: str, result: TestResult) -> None:
provider = get_provider(provider_id)
label = "恢复正常" if event == "recovered" else result.detail
print(f"[provider] {provider_id} {event}: {label}")
email = os.getenv("ZCBOT_DEVELOPER_EMAIL", "").strip()
if not email:
return
try:
from tools.send_email import send_email_smtp, smtp_configured
if smtp_configured():
send_email_smtp(
email, f"[zcbot] {provider.display_name} {label}",
f"Provider: {provider.display_name}\n状态: {event}\n详情: {result.detail}",
)
except Exception as exc: # noqa: BLE001 - 告警失败不得影响业务状态
print(f"[provider] notification failed: {type(exc).__name__}")
def run_due_checks() -> int:
engine = get_engine()
with engine.connect() as connection:
claimed = bool(connection.exec_driver_sql(_LOCK_SQL).scalar())
if not claimed:
return 0
try:
count = 0
for provider_id in due_provider_ids():
try:
test_current_credentials(provider_id, notify=send_notification)
count += 1
except Exception as exc: # noqa: BLE001 - 单 Provider 隔离
print(f"[provider] {provider_id} check failed: {type(exc).__name__}")
return count
finally:
connection.exec_driver_sql(_UNLOCK_SQL)

View File

@ -0,0 +1,83 @@
"""静态可信 Provider 注册表;网络目标和协议不可由 Admin 修改。"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class CredentialField:
name: str
label: str
env: str
secret: bool = True
@dataclass(frozen=True)
class ProviderDefinition:
provider_id: str
display_name: str
category: str
fields: tuple[CredentialField, ...]
test_kind: str
test_url: str = ""
test_url_env: str = ""
billable: bool = False
balance_supported: bool = False
check_interval_seconds: int = 86400
low_balance_threshold: float | None = None
def _key(env: str, label: str = "API Key") -> CredentialField:
return CredentialField("api_key", label, env)
PROVIDERS: tuple[ProviderDefinition, ...] = (
ProviderDefinition("deepseek", "DeepSeek", "模型", (_key("DEEPSEEK_API_KEY"),),
"deepseek_balance", "https://api.deepseek.com/user/balance",
balance_supported=True, check_interval_seconds=1800,
low_balance_threshold=30.0),
ProviderDefinition("zhipuai", "智谱开放平台", "模型", (_key("ZHIPUAI_API_KEY"),),
"bearer_get", "https://open.bigmodel.cn/api/paas/v4/models"),
ProviderDefinition("ark", "火山方舟", "模型与媒体", (_key("ARK_API_KEY"),),
"bearer_get", "https://ark.cn-beijing.volces.com/api/v3/models"),
ProviderDefinition("unifyllm", "国际旗舰模型网关", "模型与媒体", (_key("UNIFYLLM_API_KEY"),),
"bearer_get", "https://unifyllm.ai/v1/models"),
ProviderDefinition("local_llm", "内网本地模型", "模型", (_key("LOCAL_LLM_API_KEY"),),
"bearer_get", "http://182.54.21.126:9000/v1/models"),
ProviderDefinition("bocha", "博查搜索", "搜索与平台数据", (_key("BOCHA_API_KEY"),),
"bocha_search", "https://api.bochaai.com/v1/web-search", billable=True),
ProviderDefinition("document_search", "内部材料库", "搜索与平台数据", (_key("DOCUMENT_SEARCH_API_KEY"),),
"bearer_get", "https://ai.ctc-zc.com:8100/api/document_search/list_knowledge_bases",
test_url_env="DOCUMENT_SEARCH_URL"),
ProviderDefinition("paper_server", "论文服务", "搜索与平台数据", (_key("PAPER_SERVER_API_KEY"),),
"query_get", "http://paper.xxhhcty.xyz:8080/api/resm/paper/",
test_url_env="PAPER_SERVER_URL"),
ProviderDefinition("materials_project", "Materials Project", "搜索与平台数据", (_key("MP_API_KEY"),),
"mp_get", "https://api.materialsproject.org/materials/summary/?_limit=1"),
ProviderDefinition(
"xfyun_iat", "讯飞语音听写 IAT", "语音",
(CredentialField("appid", "APPID", "XFYUN_APPID"),
CredentialField("api_key", "API Key", "XFYUN_API_KEY"),
CredentialField("api_secret", "API Secret", "XFYUN_API_SECRET")),
"xfyun_iat", billable=True,
),
ProviderDefinition(
"xfyun_lfasr", "讯飞录音转写 LFASR", "语音",
(CredentialField("appid", "APPID", "XFYUN_APPID"),
CredentialField("secret_key", "Secret Key", "XFYUN_LFASR_SECRET_KEY")),
"xfyun_lfasr", "https://raasr.xfyun.cn/v2/api/getResult", billable=False,
),
)
BY_ID = {provider.provider_id: provider for provider in PROVIDERS}
BY_ENV = {
field.env: (provider.provider_id, field.name)
for provider in PROVIDERS for field in provider.fields
}
def get_provider(provider_id: str) -> ProviderDefinition:
try:
return BY_ID[provider_id]
except KeyError as exc:
raise ValueError("unknown provider") from exc

View File

@ -0,0 +1,73 @@
"""每个新外部请求调用一次的数据库优先/env fallback 凭据解析器。"""
from __future__ import annotations
import os
from dataclasses import dataclass
from sqlalchemy import select
from sqlalchemy.exc import SQLAlchemyError
from core.external_systems.crypto import decrypt_secret
from .registry import BY_ENV, get_provider
@dataclass(frozen=True)
class ResolvedCredentials:
provider_id: str
source: str
values: dict[str, str]
def _env_values(provider_id: str) -> dict[str, str]:
provider = get_provider(provider_id)
return {
field.name: value
for field in provider.fields
if (value := (os.getenv(field.env) or "").strip())
}
def resolve_credentials(provider_id: str) -> ResolvedCredentials:
provider = get_provider(provider_id)
row = None
try:
from core.storage import session_scope
from core.storage.models import ProviderCredential
with session_scope() as session:
row = session.execute(
select(ProviderCredential).where(
ProviderCredential.provider_id == provider_id
)
).scalar_one_or_none()
except (RuntimeError, SQLAlchemyError):
# CLI、migration 前部署或 DB 短暂不可用时保持历史 env 行为。
pass
if row is not None and all(field.name in row.credentials for field in provider.fields):
# 已存在完整 DB 覆盖时master key/AAD 错误必须显式失败,不能静默绕回 env。
values = {
field.name: decrypt_secret(
row.credentials[field.name], aad=f"provider:{provider_id}:{field.name}"
)
for field in provider.fields
}
return ResolvedCredentials(provider_id, "database", values)
values = _env_values(provider_id)
source = "env" if len(values) == len(provider.fields) else "missing"
return ResolvedCredentials(provider_id, source, values)
def resolve_secret(provider_id: str, field: str = "api_key") -> str:
return resolve_credentials(provider_id).values.get(field, "")
def resolve_env_secret(env_name: str) -> str:
binding = BY_ENV.get(env_name)
if binding is None:
return (os.getenv(env_name) or "").strip()
return resolve_secret(*binding)
def provider_available(provider_id: str) -> bool:
provider = get_provider(provider_id)
return len(resolve_credentials(provider_id).values) == len(provider.fields)

View File

@ -0,0 +1,263 @@
"""Provider 凭据 CRUD、乐观替换、状态持久化与定时检查。"""
from __future__ import annotations
from collections.abc import Callable
from datetime import datetime, timedelta, timezone
from uuid import UUID
from sqlalchemy import delete, select, update
from sqlalchemy.exc import IntegrityError
from core.external_systems.crypto import configured as crypto_configured
from core.external_systems.crypto import encrypt_secret
from core.storage import session_scope
from core.storage.models import ProviderCredential
from .registry import PROVIDERS, get_provider
from .runtime import _env_values, resolve_credentials
from .testing import TestResult, classify_response, test_provider
_BAD = {"low_balance", "exhausted", "auth_error", "unreachable"}
_NOTIFY_COOLDOWN = timedelta(hours=24)
class ProviderCredentialError(RuntimeError):
pass
class RevisionConflict(ProviderCredentialError):
pass
def _hint(value: str) -> str:
value = str(value or "")
return f"***{value[-4:]}" if len(value) >= 4 else "***"
def _row_payload(provider, row: ProviderCredential | None) -> dict:
database = bool(row and all(field.name in row.credentials for field in provider.fields))
env_values = _env_values(provider.provider_id)
env_source = "env" if len(env_values) == len(provider.fields) else "missing"
source = "database" if database else env_source
hints = row.credential_hint if database else {
field.name: _hint(env_values.get(field.name, ""))
for field in provider.fields if env_values.get(field.name)
}
return {
"provider_id": provider.provider_id,
"display_name": provider.display_name,
"category": provider.category,
"fields": [
{"name": field.name, "label": field.label, "required": True,
"hint": hints.get(field.name, "")}
for field in provider.fields
],
"configured": source != "missing",
"source": source,
"revision": row.revision if row else 0,
"test_status": row.test_status if row else "untested",
"test_detail": row.test_detail if row else None,
"last_tested_at": row.last_tested_at.isoformat() if row and row.last_tested_at else None,
"balance": (
{"amount": str(row.balance_amount), "currency": row.balance_currency}
if row and row.balance_amount is not None else None
),
"balance_supported": provider.balance_supported,
"billable_test": provider.billable,
"alerting": bool(row and row.test_status in _BAD),
}
def list_providers() -> list[dict]:
with session_scope() as session:
rows = {
row.provider_id: row
for row in session.execute(select(ProviderCredential)).scalars()
}
return [_row_payload(provider, rows.get(provider.provider_id)) for provider in PROVIDERS]
def _validate_values(provider_id: str, values: dict[str, str]) -> dict[str, str]:
provider = get_provider(provider_id)
expected = {field.name for field in provider.fields}
cleaned = {str(k): str(v).strip() for k, v in values.items()}
if set(cleaned) != expected or any(not value for value in cleaned.values()):
raise ProviderCredentialError("凭据字段不完整或包含未知字段")
return cleaned
def _notification_transition(
old_status: str, old_notified_at: datetime | None, result: TestResult, now: datetime
) -> str | None:
if result.status == "normal" and old_status in _BAD:
return "recovered"
if result.status not in _BAD:
return None
if old_status not in _BAD or old_status != result.status:
return result.status
if old_notified_at is None or now - old_notified_at >= _NOTIFY_COOLDOWN:
return result.status
return None
def _apply_result(
provider_id: str,
result: TestResult,
*,
notify: Callable[[str, str, TestResult], None] | None = None,
) -> None:
now = datetime.now(timezone.utc)
event: str | None = None
with session_scope() as session:
row = session.execute(
select(ProviderCredential).where(ProviderCredential.provider_id == provider_id)
).scalar_one_or_none()
if row is None:
row = ProviderCredential(
provider_id=provider_id, credentials={}, credential_hint={}, revision=1
)
session.add(row)
session.flush()
old_status = row.test_status
event = _notification_transition(old_status, row.last_notified_at, result, now)
row.test_status = result.status
row.test_detail = result.detail[:500]
row.balance_amount = result.balance_amount
row.balance_currency = result.balance_currency
row.last_tested_at = now
if event == "recovered":
row.last_notified_at = None
row.last_notified_status = None
elif event:
row.last_notified_at = now
row.last_notified_status = result.status
if event and notify:
notify(provider_id, event, result)
def replace_credentials(
provider_id: str,
values: dict[str, str],
*,
expected_revision: int,
updated_by: UUID,
request=None,
notify: Callable[[str, str, TestResult], None] | None = None,
) -> dict:
if not crypto_configured():
raise ProviderCredentialError(
"未配置 ZCBOT_CREDENTIAL_MASTER_KEY禁止保存数据库凭据"
)
provider = get_provider(provider_id)
cleaned = _validate_values(provider_id, values)
kwargs = {"request": request} if request is not None else {}
result = test_provider(provider_id, cleaned, **kwargs)
if not result.accepted:
raise ProviderCredentialError(f"候选凭据测试失败:{result.detail}")
encrypted = {
field.name: encrypt_secret(
cleaned[field.name], aad=f"provider:{provider_id}:{field.name}"
)
for field in provider.fields
}
hints = {field.name: _hint(cleaned[field.name]) for field in provider.fields}
now = datetime.now(timezone.utc)
try:
with session_scope() as session:
if expected_revision == 0:
session.add(ProviderCredential(
provider_id=provider_id, credentials=encrypted,
credential_hint=hints, revision=1, test_status=result.status,
test_detail=result.detail, balance_amount=result.balance_amount,
balance_currency=result.balance_currency, last_tested_at=now,
updated_by=updated_by,
))
revision = 1
else:
changed = session.execute(
update(ProviderCredential)
.where(ProviderCredential.provider_id == provider_id)
.where(ProviderCredential.revision == expected_revision)
.values(
credentials=encrypted, credential_hint=hints,
revision=expected_revision + 1, test_status=result.status,
test_detail=result.detail, balance_amount=result.balance_amount,
balance_currency=result.balance_currency, last_tested_at=now,
updated_by=updated_by, updated_at=now,
)
)
if int(changed.rowcount or 0) != 1:
raise RevisionConflict("凭据已被其他管理员更新,请刷新后重试")
revision = expected_revision + 1
except IntegrityError as exc:
raise RevisionConflict("凭据已被其他管理员更新,请刷新后重试") from exc
if result.status == "low_balance" and notify:
notify(provider_id, result.status, result)
with session_scope() as session:
session.execute(
update(ProviderCredential)
.where(ProviderCredential.provider_id == provider_id)
.where(ProviderCredential.revision == revision)
.values(last_notified_at=now, last_notified_status=result.status)
)
return {"provider_id": provider_id, "revision": revision,
"test_status": result.status, "test_detail": result.detail}
def test_current_credentials(
provider_id: str, *, request=None,
notify: Callable[[str, str, TestResult], None] | None = None,
) -> TestResult:
resolved = resolve_credentials(provider_id)
provider = get_provider(provider_id)
if len(resolved.values) != len(provider.fields):
raise ProviderCredentialError("Provider 凭据未完整配置")
kwargs = {"request": request} if request is not None else {}
result = test_provider(provider_id, resolved.values, **kwargs)
_apply_result(provider_id, result, notify=notify)
return result
def delete_override(provider_id: str, *, expected_revision: int) -> None:
get_provider(provider_id)
with session_scope() as session:
changed = session.execute(
delete(ProviderCredential)
.where(ProviderCredential.provider_id == provider_id)
.where(ProviderCredential.revision == expected_revision)
)
if int(changed.rowcount or 0) != 1:
raise RevisionConflict("凭据已被更新或不存在,请刷新后重试")
def record_business_failure(
provider_id: str, *, status_code: int = 0, detail: str = "",
notify: Callable[[str, str, TestResult], None] | None = None,
) -> str | None:
result = classify_response(status_code, detail)
if result.status not in {"auth_error", "exhausted"}:
return None
if notify is None:
from .monitor import send_notification
notify = send_notification
_apply_result(provider_id, result, notify=notify)
return result.status
def due_provider_ids(now: datetime | None = None) -> list[str]:
now = now or datetime.now(timezone.utc)
with session_scope() as session:
rows = {
row.provider_id: row for row in session.execute(select(ProviderCredential)).scalars()
}
due = []
for provider in PROVIDERS:
resolved = resolve_credentials(provider.provider_id)
if len(resolved.values) != len(provider.fields):
continue
row = rows.get(provider.provider_id)
if row is None or row.last_tested_at is None or (
now - row.last_tested_at >= timedelta(seconds=provider.check_interval_seconds)
):
due.append(provider.provider_id)
return due

View File

@ -0,0 +1,172 @@
"""Provider 候选凭据测试与统一、保守的错误分类。"""
from __future__ import annotations
import base64
import hashlib
import hmac
import json
import os
import re
import time
from collections.abc import Callable
from dataclasses import dataclass
from decimal import Decimal, InvalidOperation
from typing import Any
import httpx
from .registry import ProviderDefinition, get_provider
_EXHAUSTED_RE = re.compile(
r"insufficient[ _-]?(?:balance|quota|credit)|balance[ _-]?not[ _-]?enough|余额不足|额度不足|quota exhausted",
re.IGNORECASE,
)
_AUTH_RE = re.compile(
r"invalid.*(?:key|token)|authentication|unauthori[sz]ed|鉴权|认证失败",
re.IGNORECASE,
)
@dataclass(frozen=True)
class TestResult:
status: str
detail: str
balance_amount: Decimal | None = None
balance_currency: str | None = None
@property
def accepted(self) -> bool:
return self.status in {"normal", "low_balance"}
def classify_response(status_code: int, text: str = "") -> TestResult:
safe = re.sub(r"\s+", " ", text or "").strip()[:300]
if status_code in {401, 403} or _AUTH_RE.search(safe):
return TestResult("auth_error", f"认证失败HTTP {status_code}")
if status_code == 402 or _EXHAUSTED_RE.search(safe):
return TestResult("exhausted", f"余额或额度已耗尽HTTP {status_code}")
if status_code == 429:
return TestResult("unreachable", "服务限流HTTP 429未判定为余额耗尽")
if status_code >= 400:
return TestResult("unreachable", f"服务返回 HTTP {status_code}")
return TestResult("normal", "认证与连通性正常")
def _deepseek(response: httpx.Response, provider: ProviderDefinition) -> TestResult:
base = classify_response(response.status_code, response.text)
if base.status != "normal":
return base
try:
body = response.json()
infos = body.get("balance_infos") or []
cny = next(x for x in infos if str(x.get("currency", "")).upper() == "CNY")
amount = Decimal(str(cny.get("total_balance")))
except (ValueError, KeyError, StopIteration, InvalidOperation, TypeError):
return TestResult("unreachable", "余额响应格式异常")
if body.get("is_available") is False or amount <= 0:
return TestResult("exhausted", "余额已耗尽", amount, "CNY")
threshold = Decimal(str(provider.low_balance_threshold or 0))
status = "low_balance" if amount < threshold else "normal"
detail = f"CNY 可用余额 ¥{amount:.2f}"
return TestResult(status, detail, amount, "CNY")
def _xfyun_sign(credentials: dict[str, str]) -> dict[str, str]:
ts = str(int(time.time()))
md5hex = hashlib.md5(
(credentials["appid"] + ts).encode(), usedforsecurity=False
).hexdigest()
signa = base64.b64encode(
hmac.new(credentials["secret_key"].encode(), md5hex.encode(), hashlib.sha1).digest()
).decode()
return {"appId": credentials["appid"], "ts": ts, "signa": signa,
"orderId": "zcbot-credential-check", "resultType": "transfer"}
def test_provider(
provider_id: str,
credentials: dict[str, str],
*,
request: Callable[..., httpx.Response] = httpx.request,
) -> TestResult:
provider = get_provider(provider_id)
expected = {field.name for field in provider.fields}
if set(credentials) != expected or any(not str(v).strip() for v in credentials.values()):
return TestResult("auth_error", "凭据字段不完整")
headers: dict[str, str] = {}
params: dict[str, str] = {}
json_body: dict[str, Any] | None = None
method = "GET"
test_url = provider.test_url
if provider.test_url_env:
base = (os.getenv(provider.test_url_env) or "").strip().rstrip("/")
if base and provider_id == "document_search":
test_url = f"{base}/document_search/list_knowledge_bases"
elif base and provider_id == "paper_server":
test_url = f"{base}/api/resm/paper/"
if provider.test_kind in {"deepseek_balance", "bearer_get", "bocha_search"}:
headers["Authorization"] = f"Bearer {credentials['api_key']}"
if provider.test_kind == "query_get":
params["api_key"] = credentials["api_key"]
params["page_size"] = "1"
elif provider.test_kind == "mp_get":
headers["X-API-KEY"] = credentials["api_key"]
elif provider.test_kind == "bocha_search":
method = "POST"
json_body = {"query": "水泥", "count": 1, "freshness": "noLimit"}
elif provider.test_kind == "xfyun_lfasr":
method = "POST"
params.update(_xfyun_sign(credentials))
elif provider.test_kind == "xfyun_iat":
# IAT 鉴权只存在于 WebSocket upgrade复用官方签名函数并允许测试替换
# requester。200/101 均视为握手成功,真实 handler 不发送音频、不计费。
from core.asr_xfyun import build_auth_url
url = build_auth_url(credentials["api_key"], credentials["api_secret"])
if request is httpx.request:
try:
from websockets.sync.client import connect
with connect(url, open_timeout=8) as websocket:
websocket.send(json.dumps({
"common": {"app_id": credentials["appid"]},
"business": {"language": "zh_cn", "domain": "iat", "accent": "mandarin"},
"data": {"status": 2, "format": "audio/L16;rate=16000",
"encoding": "raw", "audio": ""},
}))
body = json.loads(websocket.recv(timeout=8))
if int(body.get("code") or 0) == 0:
return TestResult("normal", "WebSocket 凭据组鉴权正常")
return TestResult("auth_error", "WebSocket 凭据组认证失败")
except Exception as exc: # noqa: BLE001 - WebSocket 库异常族随版本变化
text = str(exc)
status = int(getattr(exc, "status_code", 0) or 0)
return classify_response(status, text) if status else TestResult(
"unreachable", "WebSocket 连接失败"
)
response = request("GET", url, headers={"X-Appid": credentials["appid"]}, timeout=8)
return classify_response(response.status_code, response.text)
try:
response = request(
method, test_url, headers=headers, params=params,
json=json_body, timeout=8,
)
except httpx.HTTPError:
return TestResult("unreachable", "网络连接失败")
if provider.test_kind == "deepseek_balance":
return _deepseek(response, provider)
if provider.test_kind == "xfyun_lfasr" and response.status_code < 400:
try:
body = response.json()
code = str(body.get("code") or "")
description = str(body.get("descInfo") or "")
if code in {"10105", "10106", "10107"}:
return TestResult("auth_error", "认证失败")
# 虚拟订单的“订单不存在”说明签名已通过。
if code and re.search(
r"order|订单.*(?:不存在|无效)", description, re.IGNORECASE
):
return TestResult("normal", "认证正常(未创建计费订单)")
if code:
return TestResult("unreachable", "服务未确认凭据有效")
except (ValueError, AttributeError):
pass
return classify_response(response.status_code, response.text)

View File

@ -22,22 +22,45 @@ def salvage_tool_arguments(raw: str, allowed_keys: set) -> Optional[dict]:
(a) parse-to-end 成功(json.loads 要求整个后缀是合法 JSON,尾部有残渣就失败);
(b) 结果是非空 dict;
(c) 顶层 key 该工具 schema 的参数名
双护栏缺一不可:(a) 挡尾部被截断的半个 JSON(截断则拒绝回落既有重试,不冒险执行
残缺写入);(c) 挡垃圾前缀里恰好自洽的旁支 JSON(如模型正文里引用的 `{...}` 片段)
双护栏缺一不可:(a) 挡尾部被截断的半个 JSON唯一例外是 wire 已给出至少两个
完全一致的完整参数对象随后又粘了截断副本此时一致副本可作为相互校验
(c) 挡垃圾前缀里恰好自洽的旁支 JSON(如模型正文里引用的 `{...}` 片段)
都不满足返回 None纯函数无副作用,便于单测(线上真前缀样本当夹具)
"""
# 只可能出现在 '{' 处;非 '{' 直接跳过,省下大部分 json.loads 尝试。
start = 0
decoder = json.JSONDecoder()
repeated_candidates: list[dict] = []
while True:
i = raw.find("{", start)
if i < 0:
return None
break
try:
obj = json.loads(raw[i:])
except (json.JSONDecodeError, ValueError):
# 某些 wire 抖动会形成 ``{完整参数}{完整参数}{截断副本``。
# parse-to-end 无法处理,但 raw_decode 仍能取出前面的完整对象。
# 这里只收集候选,最终必须至少两个且完全一致才可执行。
try:
partial_obj, _end = decoder.raw_decode(raw, i)
except (json.JSONDecodeError, ValueError):
start = i + 1
continue
if (
isinstance(partial_obj, dict)
and partial_obj
and set(partial_obj.keys()) <= allowed_keys
):
repeated_candidates.append(partial_obj)
start = i + 1
continue
if isinstance(obj, dict) and obj and set(obj.keys()) <= allowed_keys:
return obj
# 解析成功但不是想要的 dict(如前缀里一段自洽 JSON):继续往后找,真 JSON 可能在更后面。
start = i + 1
if (
len(repeated_candidates) >= 2
and all(obj == repeated_candidates[0] for obj in repeated_candidates[1:])
):
return repeated_candidates[0]
return None

268
core/sandbox/capacity.py Normal file
View File

@ -0,0 +1,268 @@
"""宿主级 Sandbox 重型执行容量。
状态保存在 workspace/.sandbox 下并由 advisory file lock 串行化因此蓝绿和多
Web 进程共享同一组槽位业务 DB 不承载实时状态后台排队事实源仍是
``.zcbot_procs``这里只记录已经获得槽位的租约和短暂的前台候选
"""
from __future__ import annotations
import json
import os
import time
import uuid
from contextlib import contextmanager, suppress
from pathlib import Path
from typing import Callable, Dict, Iterator, Optional
from core.file_store import atomic_write_text, interprocess_file_lock
DEFAULT_MAX_ACTIVE_EXECS = 6
DEFAULT_MAX_BACKGROUND_EXECS = 4
DEFAULT_MAX_ACTIVE_EXECS_PER_USER = 3
DEFAULT_MIN_MEM_AVAILABLE_BYTES = 1024 ** 3
def _parse_bytes(value: object, default: int) -> int:
text = str(value or "").strip().lower()
if not text:
return default
units = {"k": 1024, "kb": 1024, "m": 1024**2, "mb": 1024**2,
"g": 1024**3, "gb": 1024**3}
for suffix, factor in sorted(units.items(), key=lambda x: -len(x[0])):
if text.endswith(suffix):
return int(float(text[:-len(suffix)]) * factor)
return int(text)
def mem_available_bytes() -> Optional[int]:
try:
for line in Path("/proc/meminfo").read_text(encoding="ascii").splitlines():
if line.startswith("MemAvailable:"):
return int(line.split()[1]) * 1024
except (OSError, ValueError, IndexError):
return None
return None
class ExecCapacity:
def __init__(self, state_dir: Path, cfg: Optional[dict] = None) -> None:
cfg = cfg or {}
self.state_dir = Path(state_dir)
self.state_path = self.state_dir / "exec-capacity.json"
self.lock_path = self.state_dir / "exec-capacity.lock"
self.max_active = max(1, min(DEFAULT_MAX_ACTIVE_EXECS, int(os.getenv("ZCBOT_MAX_ACTIVE_EXECS") or cfg.get("max_active_execs") or DEFAULT_MAX_ACTIVE_EXECS)))
self.max_background = max(1, min(DEFAULT_MAX_BACKGROUND_EXECS, int(os.getenv("ZCBOT_MAX_BACKGROUND_EXECS") or cfg.get("max_background_execs") or DEFAULT_MAX_BACKGROUND_EXECS)))
self.max_per_user = max(1, min(DEFAULT_MAX_ACTIVE_EXECS_PER_USER, int(os.getenv("ZCBOT_MAX_ACTIVE_EXECS_PER_USER") or cfg.get("max_active_execs_per_user") or DEFAULT_MAX_ACTIVE_EXECS_PER_USER)))
self.min_mem_available = _parse_bytes(
os.getenv("ZCBOT_MIN_MEM_AVAILABLE") or cfg.get("min_mem_available"),
DEFAULT_MIN_MEM_AVAILABLE_BYTES,
)
def _read(self) -> dict:
try:
data = json.loads(self.state_path.read_text(encoding="utf-8"))
if isinstance(data, dict):
data.setdefault("leases", {})
data.setdefault("foreground_queue", [])
data.setdefault("containers", {})
return data
except (OSError, ValueError):
pass
return {"leases": {}, "foreground_queue": [], "containers": {}}
def _write(self, state: dict) -> None:
atomic_write_text(self.state_path, json.dumps(state, ensure_ascii=False, sort_keys=True))
def _prune(self, state: dict) -> None:
leases = state["leases"]
for key, lease in list(leases.items()):
if lease.get("kind") == "foreground":
pid = int(lease.get("owner_pid") or 0)
if pid and not self._pid_alive(pid):
leases.pop(key, None)
state["foreground_queue"] = [
q for q in state["foreground_queue"]
if self._pid_alive(int(q.get("owner_pid") or 0))
]
@staticmethod
def _pid_alive(pid: int) -> bool:
try:
os.kill(pid, 0)
return True
except PermissionError:
return True
except OSError:
return False
def _can_admit(self, state: dict, user_id: str, kind: str) -> bool:
leases = list(state["leases"].values())
if len(leases) >= self.max_active:
return False
if sum(1 for x in leases if x.get("user_id") == user_id) >= self.max_per_user:
return False
if kind == "background" and sum(1 for x in leases if x.get("kind") == "background") >= self.max_background:
return False
available = mem_available_bytes()
return available is None or available >= self.min_mem_available
def try_acquire(self, user_id: str, kind: str, *, lease_id: Optional[str] = None, proc_id: Optional[str] = None) -> Optional[str]:
lease_id = lease_id or uuid.uuid4().hex
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read()
self._prune(state)
if lease_id in state["leases"]:
return None
if not self._can_admit(state, str(user_id), kind):
self._write(state)
return None
state["leases"][lease_id] = {
"lease_id": lease_id, "user_id": str(user_id), "kind": kind,
"proc_id": proc_id, "owner_pid": os.getpid(), "started_ts": time.time(),
}
self._write(state)
return lease_id
def acquire_foreground(
self,
user_id: str,
cancel_check: Optional[Callable[[], bool]] = None,
wait_notify: Optional[Callable[[dict], None]] = None,
) -> Optional[str]:
ticket = uuid.uuid4().hex
wait_notified = False
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read(); self._prune(state)
state["foreground_queue"].append({"ticket": ticket, "user_id": str(user_id), "created_ts": time.time(), "owner_pid": os.getpid()})
self._write(state)
while True:
admitted = False
notification = None
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read(); self._prune(state)
queue = state["foreground_queue"]
eligible = next((q for q in queue if q.get("user_id") == str(user_id)), None)
if eligible and eligible.get("ticket") == ticket and self._can_admit(state, str(user_id), "foreground"):
queue[:] = [q for q in queue if q.get("ticket") != ticket]
state["leases"][ticket] = {"lease_id": ticket, "user_id": str(user_id), "kind": "foreground", "owner_pid": os.getpid(), "started_ts": time.time()}
self._write(state)
admitted = True
else:
self._write(state)
if not admitted and not wait_notified and wait_notify is not None:
leases = list(state["leases"].values())
user_running = sum(
1 for lease in leases
if lease.get("user_id") == str(user_id)
)
available = mem_available_bytes()
if user_running >= self.max_per_user:
reason = "per_user_limit"
elif len(leases) >= self.max_active:
reason = "global_limit"
elif available is not None and available < self.min_mem_available:
reason = "memory_pressure"
else:
reason = "queue_order"
notification = {
"state": "waiting",
"reason": reason,
"user_running": user_running,
"user_limit": self.max_per_user,
"global_running": len(leases),
"global_limit": self.max_active,
}
wait_notified = True
if admitted:
if wait_notified and wait_notify is not None:
with suppress(Exception):
wait_notify({"state": "admitted"})
return ticket
if notification is not None and wait_notify is not None:
with suppress(Exception):
wait_notify(notification)
if cancel_check is not None and cancel_check():
self.cancel_waiter(ticket)
return None
time.sleep(0.1)
def cancel_waiter(self, ticket: str) -> None:
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read()
state["foreground_queue"] = [q for q in state["foreground_queue"] if q.get("ticket") != ticket]
self._write(state)
def release(self, lease_id: Optional[str]) -> None:
if not lease_id:
return
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read(); state["leases"].pop(lease_id, None); self._write(state)
def reconcile_background(self, running_proc_ids: set[str]) -> None:
"""服务重启后保留 Docker 仍在跑的租约,清启动窗口遗留的孤儿租约。"""
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read()
for key, lease in list(state["leases"].items()):
if (lease.get("kind") == "background"
and str(lease.get("proc_id") or "") not in running_proc_ids
and not self._pid_alive(int(lease.get("owner_pid") or 0))):
state["leases"].pop(key, None)
self._write(state)
def touch_container(self, name: str, *, active_delta: int = 0) -> None:
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read()
row = state["containers"].setdefault(name, {"active_execs": 0})
row["active_execs"] = max(0, int(row.get("active_execs") or 0) + active_delta)
row["last_active_ts"] = time.time()
self._write(state)
def remove_container(self, name: str) -> None:
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read(); state["containers"].pop(name, None); self._write(state)
def reconcile_containers(self, running_names: set[str]) -> None:
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read()
for name in list(state["containers"]):
if name not in running_names:
state["containers"].pop(name, None)
self._write(state)
@contextmanager
def foreground(
self,
user_id: str,
cancel_check: Optional[Callable[[], bool]] = None,
wait_notify: Optional[Callable[[dict], None]] = None,
) -> Iterator[bool]:
lease = self.acquire_foreground(user_id, cancel_check, wait_notify)
try:
yield lease is not None
finally:
self.release(lease)
def snapshot(self) -> dict:
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
state = self._read(); self._prune(state); self._write(state)
leases = list(state["leases"].values())
fg = sum(x.get("kind") == "foreground" for x in leases)
bg = sum(x.get("kind") == "background" for x in leases)
by_user: Dict[str, int] = {}
for x in leases:
by_user[x["user_id"]] = by_user.get(x["user_id"], 0) + 1
available_mem = mem_available_bytes()
memory_paused = available_mem is not None and available_mem < self.min_mem_available
containers = state.get("containers", {})
now = time.time()
return {
"limits": {"active": self.max_active, "background": self.max_background, "per_user": self.max_per_user},
"foreground_running": fg, "foreground_queued": len(state["foreground_queue"]),
"background_running": bg, "per_user": by_user,
"admit_available": 0 if memory_paused else max(0, self.max_active - len(leases)),
"memory_paused": memory_paused,
"mem_available_bytes": available_mem,
"cpu_load": list(os.getloadavg()) if hasattr(os, "getloadavg") else None,
"active_sandbox_containers": len(containers),
"idle_reap_candidates": sum(int(x.get("active_execs") or 0) == 0 and float(x.get("last_active_ts") or now) < now - getattr(self, "idle_ttl_seconds", 600) for x in containers.values()),
}

View File

@ -0,0 +1,57 @@
"""Sandbox 临时 Python 包扫描与 best-effort 持久化。"""
from __future__ import annotations
import json
import subprocess
from datetime import datetime, timezone
from typing import Optional
from uuid import UUID
def scan_container(container: str) -> Optional[dict]:
try:
scan = subprocess.run(["docker", "exec", "--user", "zcbot", "--workdir", "/sandbox", container,
"python", "-I", "-S", "/sandbox/package_scan.py"], capture_output=True,
text=True, timeout=30)
if scan.returncode != 0:
raise RuntimeError(scan.stderr.strip())
packages = json.loads(scan.stdout)
if not isinstance(packages, list):
raise ValueError("scanner output is not a package list")
inspect = subprocess.run(["docker", "inspect", "--format={{.Image}}", container],
capture_output=True, text=True, timeout=15)
py = subprocess.run(["docker", "exec", "--workdir", "/sandbox", container, "python", "-I", "-S", "-c",
"import sys;print('.'.join(map(str,sys.version_info[:3])))"],
capture_output=True, text=True, timeout=15)
if inspect.returncode != 0 or py.returncode != 0:
raise RuntimeError("failed to inspect image or Python version")
return {"packages": packages, "image_digest": inspect.stdout.strip(),
"python_version": py.stdout.strip(),
"total_installed_bytes": sum(int(x.get("installed_bytes") or 0) for x in packages)}
except Exception as exc:
print(f"[sandbox-package-scan] failed container={container}: {type(exc).__name__}: {exc}")
return None
def persist_scan(*, container_session_id: str, user_id: UUID, execution_kind: str,
result: Optional[dict]) -> None:
if not result or not result.get("packages"):
return
try:
from core.storage import session_scope
from core.storage.models import SandboxPackageScan
with session_scope() as s:
s.merge(SandboxPackageScan(
container_session_id=container_session_id, user_id=user_id,
execution_kind=execution_kind, image_digest=result["image_digest"],
python_version=result["python_version"], packages=result["packages"],
package_count=len(result["packages"]),
total_installed_bytes=result["total_installed_bytes"],
finished_at=datetime.now(timezone.utc),
))
except Exception as exc:
print(f"[sandbox-package-scan] persist failed session={container_session_id}: {type(exc).__name__}: {exc}")
def scan_and_persist(container: str, user_id: UUID, kind: str, session_id: str) -> None:
persist_scan(container_session_id=session_id, user_id=user_id,
execution_kind=kind, result=scan_container(container))

View File

@ -56,11 +56,11 @@ LABEL_INSTANCE_KEY = "zcbot.instance"
INSTANCE = os.getenv("ZCBOT_INSTANCE", "").strip()
DEFAULT_IMAGE = "zcbot-sandbox:latest"
DEFAULT_IDLE_TTL_SECONDS = 300
DEFAULT_IDLE_TTL_SECONDS = 600
# 容器资源限制默认值(可被 yaml `sandbox.*` / env override,详 SandboxPool ctor)
DEFAULT_MEMORY = "2g"
DEFAULT_CPUS = "1.0"
DEFAULT_MEMORY = "4g"
DEFAULT_CPUS = "2.0"
# pids-limit 把**线程**也计数:chromium headless 一次启动(browser+GPU+network+renderer
# 多进程)就 ~150-200 线程,256 本来就贴边;shell 超时又只杀 host 侧 docker CLI,容器内
# mmdc+chromium 树留着(见 executor_docker.py 头注 Cancel limitation),残留几棵就把配额
@ -70,8 +70,9 @@ DEFAULT_PIDS_LIMIT = 1024
# chromium(mmdc 渲 mermaid / puppeteer)默认走 /dev/shm,docker 不传 --shm-size 时
# 只给 64MB,起不来就一直挂到 timeout。镜像备的 puppeteer-config 有 --disable-dev-shm-usage,
# 但模型不一定用那份;这里从根上把 /dev/shm 撑到够用,任何 chromium 路径都不再挂。
# 从 --memory(默 2g)里切,512m 是上限非占用(tmpfs 按需用)。
# 从 --memory(默 4g)里切,512m 是上限非占用(tmpfs 按需用)。
DEFAULT_SHM_SIZE = "512m"
DEFAULT_TMP_SIZE = "1g"
def container_name(user_id: UUID) -> str:
@ -102,6 +103,11 @@ def _container_running(name: str) -> bool:
return r.returncode == 0 and r.stdout.strip() == "true"
def _container_session_id(name: str) -> str:
r = subprocess.run(["docker", "inspect", "--format={{.Id}}", name], capture_output=True, text=True)
return r.stdout.strip() if r.returncode == 0 else name
class SandboxPool:
def __init__(
self,
@ -115,7 +121,9 @@ class SandboxPool:
cpus: Optional[str] = None,
pids_limit: Optional[int] = None,
shm_size: Optional[str] = None,
tmp_size: Optional[str] = None,
dns: Optional[List[str]] = None,
capacity_cfg: Optional[Dict[str, object]] = None,
) -> None:
"""
user_root_base: per-user 子树父目录,典型 `<workspace>/users`bind mount
@ -134,9 +142,9 @@ class SandboxPool:
pg_ips: 逗号分隔的 PG IP ,塞容器 `ZCBOT_PG_IPS` env,init.sh DROP 规则
(env `ZCBOT_PG_IPS`)defense-in-depth 即便落内网三段
memory/cpus/pids_limit/shm_size:
容器资源限制, 2g/1.0/256/512m;env(`ZCBOT_SANDBOX_MEMORY` )
容器资源限制, 4g/2.0/1024/512m;env(`ZCBOT_SANDBOX_MEMORY` )
override caller 参数 override 默认改后重启 web 生效,新起的
容器用新值; running 不变(idle 5min 回收后下次起按新值)
容器用新值; running 不变(idle 10min 回收后下次起按新值)
shm_size chromium /dev/shm( 64MB 不够,mmdc 渲图会挂)
"""
self.user_root_base = user_root_base
@ -153,6 +161,7 @@ class SandboxPool:
self.memory = os.getenv("ZCBOT_SANDBOX_MEMORY") or memory or DEFAULT_MEMORY
self.cpus = os.getenv("ZCBOT_SANDBOX_CPUS") or cpus or DEFAULT_CPUS
self.shm_size = os.getenv("ZCBOT_SANDBOX_SHM_SIZE") or shm_size or DEFAULT_SHM_SIZE
self.tmp_size = os.getenv("ZCBOT_SANDBOX_TMP_SIZE") or tmp_size or DEFAULT_TMP_SIZE
self.pids_limit = int(
os.getenv("ZCBOT_SANDBOX_PIDS_LIMIT")
or (pids_limit if pids_limit is not None else DEFAULT_PIDS_LIMIT)
@ -166,6 +175,10 @@ class SandboxPool:
self._dict_lock = threading.Lock() # 保护 _locks / _last_active 的字典级 race
self._locks: Dict[UUID, threading.Lock] = {}
self._last_active: Dict[UUID, int] = {}
self._active_execs: Dict[UUID, int] = {}
from .capacity import ExecCapacity
self.capacity = ExecCapacity(self.user_root_base.parent / ".sandbox", capacity_cfg)
self.capacity.idle_ttl_seconds = self.idle_ttl
def _lock_for(self, user_id: UUID) -> threading.Lock:
with self._dict_lock:
@ -179,15 +192,18 @@ class SandboxPool:
name = container_name(user_id)
if _container_running(name):
self._last_active[user_id] = _now()
self.capacity.touch_container(name)
return name
if _container_exists(name):
# stopped / crashed ── rm 重起。iptables 规则随容器生命周期重新 apply。
self._scan_before_remove(name, user_id, "foreground")
subprocess.run(
["docker", "rm", "-f", name],
capture_output=True, check=False,
)
self._docker_run(user_id, name)
self._last_active[user_id] = _now()
self.capacity.touch_container(name)
return name
def _ensure_resolv_conf_file(self) -> Optional[Path]:
@ -255,7 +271,7 @@ class SandboxPool:
"--network", NETWORK_NAME,
# §7.5 硬限制(任一缺失视为 hardening 未完成)
"--read-only", # rootfs read-only
"--tmpfs", "/tmp:exec,size=512m,mode=1777", # 可写临时区,exec 允许 (run_python 写脚本)
"--tmpfs", f"/tmp:exec,size={self.tmp_size},mode=1777",
f"--shm-size={self.shm_size}", # chromium/mmdc 的 /dev/shm,默 64MB 不够会挂(DEFAULT_SHM_SIZE)
"--cap-drop=ALL", # 默全丢
"--cap-add=NET_ADMIN", # init.sh 配 iptables 需要;exec 进来的 uid 1000 拿不到
@ -305,20 +321,48 @@ class SandboxPool:
def mark_active(self, user_id: UUID) -> None:
"""每次 `docker exec` 完调一次,刷新 idle 计时。"""
self._last_active[user_id] = _now()
self.capacity.touch_container(container_name(user_id))
def exec_started(self, user_id: UUID) -> None:
"""普通容器内 exec 开始;显式计数防止 idle reaper 误删长命令。"""
with self._dict_lock:
self._active_execs[user_id] = self._active_execs.get(user_id, 0) + 1
self._last_active[user_id] = _now()
self.capacity.touch_container(container_name(user_id), active_delta=1)
def exec_finished(self, user_id: UUID) -> None:
with self._dict_lock:
n = max(0, self._active_execs.get(user_id, 0) - 1)
if n:
self._active_execs[user_id] = n
else:
self._active_execs.pop(user_id, None)
self._last_active[user_id] = _now()
self.capacity.touch_container(container_name(user_id), active_delta=-1)
def _scan_before_remove(self, name: str, user_id: UUID, kind: str, session_id: Optional[str] = None) -> None:
"""扫描/落库均 best-effort调用方无论如何继续删除容器。"""
try:
from .package_scans import scan_and_persist
scan_and_persist(name, user_id, kind, session_id or _container_session_id(name))
except Exception as exc:
print(f"[sandbox-package-scan] finalizer failed container={name}: {type(exc).__name__}: {exc}")
def reap_idle(self) -> List[str]:
"""杀超过 idle_ttl 没活跃的容器。返回已杀容器名列表(供日志 / 审计)。"""
removed: List[str] = []
cutoff = _now() - self.idle_ttl
for uid, ts in list(self._last_active.items()):
if ts < cutoff:
if ts < cutoff and self._active_execs.get(uid, 0) == 0:
name = container_name(uid)
self._scan_before_remove(name, uid, "foreground")
r = subprocess.run(
["docker", "rm", "-f", name],
capture_output=True, text=True,
)
if r.returncode == 0:
removed.append(name)
self.capacity.remove_container(name)
# 无论 rm 成功与否,从 dict 移除 ── 失败则下次启动靠 shutdown_all 兜底
del self._last_active[uid]
return removed
@ -341,6 +385,21 @@ class SandboxPool:
if list_r.returncode != 0 or not list_r.stdout.strip():
return []
ids = list_r.stdout.strip().splitlines()
for cid in ids:
info = subprocess.run(
["docker", "inspect", "--format={{index .Config.Labels \"zcbot.user_id\"}}", cid],
capture_output=True, text=True,
)
try:
uid = UUID(info.stdout.strip())
except (ValueError, AttributeError):
continue
self._scan_before_remove(cid, uid, "foreground", cid)
try:
name_r = subprocess.run(["docker", "inspect", "--format={{.Name}}", cid], capture_output=True, text=True)
self.capacity.remove_container(name_r.stdout.strip().lstrip("/") or cid)
except Exception:
pass
subprocess.run(
["docker", "rm", "-f", *ids],
capture_output=True, text=True,
@ -349,6 +408,31 @@ class SandboxPool:
self._last_active.clear()
return ids
def runtime_snapshot(self) -> dict:
try:
listed = subprocess.run(
["docker", "ps", "--format={{.Names}}", "--filter", f"label={LABEL_PRODUCT_KEY}={LABEL_PRODUCT_VALUE}"],
capture_output=True, text=True, timeout=15,
)
if listed.returncode == 0:
self.capacity.reconcile_containers(set(listed.stdout.split()))
except (OSError, subprocess.TimeoutExpired):
pass
snap = self.capacity.snapshot()
now = _now()
snap["idle_ttl_seconds"] = self.idle_ttl
try:
from core import procs
items = []
for uroot in self.user_root_base.iterdir():
items.extend(procs.list_all_procs(uroot))
snap["background_queued"] = sum(x.get("_status") == "queued" for x in items)
snap["running_proc_containers"] = sum(x.get("_status") == "running" and x.get("backend") == "docker" for x in items)
except OSError:
snap["background_queued"] = 0
snap["running_proc_containers"] = 0
return snap
def setup_pool(
user_root_base: Path,
@ -376,12 +460,16 @@ def setup_pool(
cpus = cfg.get("cpus")
pids_limit = cfg.get("pids_limit")
shm_size = cfg.get("shm_size")
tmp_size = cfg.get("tmp_size")
return SandboxPool(
user_root_base=user_root_base,
repo_root=repo_root,
idle_ttl=int(str(cfg.get("idle_ttl_seconds"))) if cfg.get("idle_ttl_seconds") is not None else None,
memory=memory if isinstance(memory, str) else None,
cpus=str(cpus) if cpus is not None else None,
pids_limit=int(str(pids_limit)) if pids_limit is not None else None,
shm_size=shm_size if isinstance(shm_size, str) else None,
tmp_size=tmp_size if isinstance(tmp_size, str) else None,
dns=[str(x) for x in dns_cfg],
capacity_cfg=cfg,
)

View File

@ -65,6 +65,7 @@ class Session:
msg: Any,
*,
artifact_refs: Optional[list[dict]] = None,
source_model_profile: str = "",
) -> Optional[UUID]:
"""追加消息;非 system 落 DB,system 仅内存。返回新落库行的 message_id。
@ -91,7 +92,10 @@ class Session:
f"[markdown:fence-warning] task={self.task_id} unclosed=1",
flush=True,
)
self.messages.append(msg_dict)
memory_msg = dict(msg_dict)
if source_model_profile and memory_msg.get("role") == "assistant":
memory_msg["_model_profile"] = source_model_profile
self.messages.append(memory_msg)
if msg_dict.get("role") == "system":
return None
@ -236,6 +240,9 @@ class Session:
attachment_refs,
wd_name,
)
payload["_attachment_refs"] = list(attachment_refs)
if payload.get("role") == "assistant" and row.model_profile:
payload["_model_profile"] = row.model_profile
if payload.get("role") == "assistant" and isinstance(payload.get("content"), str):
# 历史行不回写生产库;只在重建 LLM 上下文时应用同一窄修复,
# 与 Web 展示层保持一致,避免旧坏围栏继续污染后续轮次。

View File

@ -7,6 +7,7 @@ Loop 不直接 print,改 emit({type, ...})。Sink 决定怎么呈现。
llm_end {type, prompt_tokens, completion_tokens, elapsed}
text {type, content} assistant 文字段(整段,非流式)
tool_call {type, name, args, args_preview}
execution_queue {type, state, reason, ...} 前台重型工具等待/放行
tool_result {type, name, result, preview, truncated}
done {type} 一次 run 全部结束

View File

@ -4,6 +4,7 @@ from __future__ import annotations
import json
import logging
import math
import re
import threading
from dataclasses import dataclass
@ -38,6 +39,14 @@ class WorkspaceSpec:
required_adapter_version: str
@dataclass(frozen=True)
class CapabilityLifecycle:
status: str = "active"
visibility: str = "standard"
replacement: str | None = None
detail: str = ""
@dataclass(frozen=True)
class CapabilityContract:
capability: str
@ -52,8 +61,20 @@ class CapabilityContract:
summary: dict[str, Any]
legacy_runtime: dict[str, Any] | None
workspace: WorkspaceSpec | None
lifecycle: CapabilityLifecycle
@property
def ordinarily_visible(self) -> bool:
return (
self.lifecycle.status == "active"
and self.lifecycle.visibility == "standard"
)
def normalize_request(self, request: object) -> tuple[dict[str, Any], str]:
if not _all_numbers_finite(request):
raise SoftwareContractError(
f"invalid {self.capability} request: non-finite numbers are forbidden"
)
errors = sorted(
Draft202012Validator(
self.request_schema, format_checker=FormatChecker()
@ -65,7 +86,10 @@ class CapabilityContract:
location = ".".join(str(item) for item in first.absolute_path)
prefix = f"{location}: " if location else ""
raise SoftwareContractError(f"invalid {self.capability} request: {prefix}{first.message}")
encoded = json.dumps(request, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
encoded = json.dumps(
request, ensure_ascii=False, sort_keys=True, separators=(",", ":"),
allow_nan=False,
)
max_bytes = int(self.request_schema.get("x-maxBytes") or 256 * 1024)
if len(encoded.encode("utf-8")) > max_bytes:
raise SoftwareContractError(f"{self.capability} request is too large")
@ -145,6 +169,20 @@ def _value_at(value: object, path: list[str]) -> object:
return current
def _all_numbers_finite(value: object) -> bool:
if isinstance(value, bool) or value is None or isinstance(value, str):
return True
if isinstance(value, float):
return math.isfinite(value)
if isinstance(value, int):
return True
if isinstance(value, dict):
return all(_all_numbers_finite(item) for item in value.values())
if isinstance(value, (list, tuple)):
return all(_all_numbers_finite(item) for item in value)
return True
def _resolve_local_refs(value: Any, definitions: dict[str, Any]) -> Any:
if isinstance(value, list):
return [_resolve_local_refs(item, definitions) for item in value]
@ -171,7 +209,8 @@ def _load_contract(path: Path) -> CapabilityContract:
"input_policy", "outputs", "feature_path", "features", "summary", "legacy_runtime",
"workspace",
}
if not isinstance(raw, dict) or set(raw) != required:
optional = {"lifecycle"}
if not isinstance(raw, dict) or not required.issubset(raw) or set(raw) - required - optional:
raise RuntimeError(f"invalid software contract fields: {path.name}")
capability = raw["capability"]
namespace = raw["output_namespace"]
@ -234,6 +273,32 @@ def _load_contract(path: Path) -> CapabilityContract:
workspace = WorkspaceSpec(
state_output, tuple(preview_outputs), required_adapter_version
)
lifecycle_raw = raw.get("lifecycle") or {}
if not isinstance(lifecycle_raw, dict) or set(lifecycle_raw) - {
"status", "visibility", "replacement", "detail"
}:
raise RuntimeError(f"invalid lifecycle spec: {path.name}")
lifecycle = CapabilityLifecycle(
status=lifecycle_raw.get("status", "active"),
visibility=lifecycle_raw.get("visibility", "standard"),
replacement=lifecycle_raw.get("replacement"),
detail=lifecycle_raw.get("detail", ""),
)
if (
lifecycle.status not in {"active", "deprecated"}
or lifecycle.visibility not in {"standard", "diagnostic_only"}
or (
lifecycle.replacement is not None
and (
not isinstance(lifecycle.replacement, str)
or not re.fullmatch(
r"[a-z][a-z0-9_.-]+@v[1-9][0-9]*", lifecycle.replacement
)
)
)
or not isinstance(lifecycle.detail, str)
):
raise RuntimeError(f"unsafe lifecycle spec: {path.name}")
Draft202012Validator.check_schema(raw["request_schema"])
return CapabilityContract(
capability=capability,
@ -248,6 +313,7 @@ def _load_contract(path: Path) -> CapabilityContract:
summary=raw["summary"],
legacy_runtime=raw["legacy_runtime"],
workspace=workspace,
lifecycle=lifecycle,
)
@ -283,6 +349,15 @@ class _ContractRegistry:
raise RuntimeError(
"software contracts are missing or contain duplicate capabilities"
)
for item in loaded:
replacement = item.lifecycle.replacement
if replacement is not None and (
replacement == item.capability or replacement not in contracts
):
raise RuntimeError(
f"software contract has invalid lifecycle replacement: "
f"{item.capability}"
)
except Exception:
if initial:
raise

View File

@ -841,6 +841,9 @@ def offer_next_job(node_ids: set[UUID]) -> dict | None:
"capability": job.capability,
"request_digest": job.request_digest,
"request": job.request,
# Node 的本机任务列表只消费合同生成的通用摘要,不解析各软件
# 私有 request 结构。旧 Node 会忽略这个新增字段。
"request_summary": _request_summary(job),
"workspace": (
{
"workspace_id": str(job.workspace_id),

View File

@ -8,13 +8,19 @@ from hashlib import sha256
from uuid import UUID, uuid4
import bcrypt
from sqlalchemy import select, update
from sqlalchemy import func, select, update
from core.software_contracts import supported_capabilities
from core.storage.engine import session_scope
from core.storage.models import SoftwareNode, SoftwareNodeEnrollment, SoftwareWorkspace
from core.storage.models import (
SoftwareJob,
SoftwareNode,
SoftwareNodeEnrollment,
SoftwareWorkspace,
)
MAX_ENROLLMENT_FAILURES = 5
ACTIVE_JOB_STATUSES = {"offered", "dispatched", "running", "disconnected", "cancelling"}
class SoftwareNodeError(Exception):
@ -208,6 +214,16 @@ def list_nodes() -> list[dict]:
.scalars()
.all()
)
active_job_counts = dict(
session.execute(
select(SoftwareJob.node_id, func.count(SoftwareJob.job_id))
.where(
SoftwareJob.node_id.is_not(None),
SoftwareJob.status.in_(ACTIVE_JOB_STATUSES),
)
.group_by(SoftwareJob.node_id)
).all()
)
return [
{
"node_id": str(row.node_id),
@ -217,6 +233,7 @@ def list_nodes() -> list[dict]:
"node_version": row.node_version,
"os_version": row.os_version,
"runtime": row.runtime,
"active_job_count": int(active_job_counts.get(row.node_id, 0)),
"last_seen_at": row.last_seen_at.isoformat()
if row.last_seen_at
else None,

View File

@ -14,9 +14,14 @@ from .engine import (
)
from .telemetry import (
record_empty_response,
record_agent_guard,
record_context_fold_failure,
record_malformed_tool_call,
record_quality_gate,
record_run_error,
record_run_stopped,
record_salvaged_tool_call,
record_tool_failure,
)
from .usage import record_chat_usage
from .utils import (
@ -35,10 +40,15 @@ __all__ = [
"get_engine",
"get_task",
"record_chat_usage",
"record_agent_guard",
"record_context_fold_failure",
"record_empty_response",
"record_malformed_tool_call",
"record_quality_gate",
"record_run_error",
"record_run_stopped",
"record_salvaged_tool_call",
"record_tool_failure",
"session_scope",
"update_task",
"upsert_task",

View File

@ -1,8 +1,9 @@
"""Per-user 工作目录配额(§7.5 #4 软配额,应用层 gate)。
调用入口:
- `scan_user_dir(user_root) -> (bytes, count)` os.walk 累加, dotfile / 损坏 stat
- `upsert_user_usage(user_id, bytes, count)` user_disk_usage
- `scan_user_dir(user_root) -> (bytes, count)` 累加计入用户配额的文件
- `scan_user_trash(user_root) -> (bytes, count)` 单独统计隐藏回收站物理占用
- `upsert_user_usage(...)` 同时落配额占用与回收站快照
- `check_disk_quota(user_id, limit_bytes) -> Optional[str]` 写前查, None=放行 /
str=拒绝原因`limit_bytes <= 0` 短路放行(不限)
- `scan_all_users(user_root_base, limit_bytes)` lifespan 后台 task 周期跑,
@ -64,15 +65,18 @@ def parse_bytes(value) -> Optional[int]:
_SKIP_TOPLEVEL = frozenset({
".zcbot_cache",
".zcbot_external_results",
".zcbot_artifact_trash",
".zcbot_tmp",
".memory",
})
_ARTIFACT_TRASH_DIR = ".zcbot_artifact_trash"
def scan_user_dir(user_root: Path) -> Tuple[int, int]:
"""os.walk 累加 user_root 下所有文件大小,返 (bytes, count)。
跳过顶层平台缓存/临时区与 .memory(均有独立生命周期,不算入产品配额);
跳过顶层平台缓存/临时区隐藏回收站与 .memory(均有独立生命周期,
不算入产品配额);
follow_symlinks=False symlink 循环爆
"""
if not user_root.exists() or not user_root.is_dir():
@ -102,6 +106,14 @@ def scan_user_dir(user_root: Path) -> Tuple[int, int]:
return total_bytes, total_count
def scan_user_trash(user_root: Path) -> Tuple[int, int]:
"""统计隐藏 artifact 回收站的物理占用,不计入用户配额。"""
trash = user_root / _ARTIFACT_TRASH_DIR
if not trash.is_dir():
return 0, 0
return _walk_dir(trash)
def _walk_dir(d: Path) -> Tuple[int, int]:
total_b, total_c = 0, 0
for root, dirs, files in os.walk(d, followlinks=False, onerror=lambda _e: None):
@ -115,19 +127,29 @@ def _walk_dir(d: Path) -> Tuple[int, int]:
return total_b, total_c
def upsert_user_usage(user_id: UUID, bytes_used: int, file_count: int) -> None:
"""落 user_disk_usage 单行;首次 INSERT,后续 UPDATE。"""
def upsert_user_usage(
user_id: UUID,
bytes_used: int,
file_count: int,
trash_bytes: int = 0,
trash_file_count: int = 0,
) -> None:
"""落 user_disk_usage 单行;首次 INSERT后续 UPDATE。"""
from sqlalchemy import func
with session_scope() as s:
stmt = pg_insert(UserDiskUsage).values(
user_id=user_id,
bytes_used=bytes_used,
file_count=file_count,
trash_bytes=trash_bytes,
trash_file_count=trash_file_count,
).on_conflict_do_update(
index_elements=["user_id"],
set_={
"bytes_used": bytes_used,
"file_count": file_count,
"trash_bytes": trash_bytes,
"trash_file_count": trash_file_count,
"scanned_at": func.now(),
},
)
@ -202,8 +224,10 @@ def scan_all_users(user_root_base: Path) -> int:
count = 0
for uid in list_user_ids_with_root(user_root_base):
try:
b, c = scan_user_dir(user_root_base / str(uid))
upsert_user_usage(uid, b, c)
user_root = user_root_base / str(uid)
b, c = scan_user_dir(user_root)
trash_b, trash_c = scan_user_trash(user_root)
upsert_user_usage(uid, b, c, trash_b, trash_c)
count += 1
except Exception:
# 单 user 扫挂不阻塞其他 user;下次周期重试。日志靠 caller 注入。

View File

@ -258,6 +258,48 @@ class WebPreview(Base):
)
class ProviderCredential(Base):
"""平台级可信 Provider 当前凭据与健康状态credentials 只保存字段密文。"""
__tablename__ = "provider_credentials"
provider_id: Mapped[str] = mapped_column(Text, primary_key=True)
credentials: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
credential_hint: Mapped[dict[str, Any]] = mapped_column(
JSONB, nullable=False, default=dict
)
revision: Mapped[int] = mapped_column(
Integer, nullable=False, default=1, server_default="1"
)
test_status: Mapped[str] = mapped_column(
Text, nullable=False, default="untested", server_default="untested"
)
test_detail: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
balance_amount: Mapped[Optional[Decimal]] = mapped_column(
Numeric(18, 6), nullable=True
)
balance_currency: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
last_tested_at: Mapped[Optional[datetime]] = mapped_column(
DateTime(timezone=True), nullable=True
)
last_notified_at: Mapped[Optional[datetime]] = mapped_column(
DateTime(timezone=True), nullable=True
)
last_notified_status: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
updated_by: Mapped[Optional[UUID]] = mapped_column(
PG_UUID(as_uuid=True),
ForeignKey("users.user_id", ondelete="SET NULL"),
nullable=True,
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(),
onupdate=func.now(), nullable=False,
)
class Artifact(Base):
"""Stable identity and lifecycle metadata for a published workspace file."""
@ -324,6 +366,22 @@ class UsageEvent(Base):
__tablename__ = "usage_events"
__table_args__ = (
Index("ix_usage_created_brin", "created_at", postgresql_using="brin"),
Index(
"ix_usage_health_kind_created",
"kind",
"created_at",
postgresql_where=text(
"kind IN ('tool_failure', 'tool_malformed', 'tool_salvaged', "
"'run_error', 'empty_response', 'run_stopped', 'agent_guard', "
"'context_fold_failure', 'quality_gate')"
),
),
Index(
"ux_usage_tool_failure_message",
"message_id",
unique=True,
postgresql_where=text("kind IN ('tool_failure', 'quality_gate')"),
),
)
event_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
@ -355,8 +413,9 @@ class UserDiskUsage(Base):
"""per-user 工作目录字节使用快照(0008,§7.5 #4 软配额表)。
每个 user_id 单行 upsert,lifespan 后台 task 周期( 15min)扫描 user_root 落库;
write gate(DockerExecutor / /v1/files/upload)查这表对比 yaml `quotas.disk_bytes_per_user`,
超额返 [Error] 硬阻
write gate(DockerExecutor / /v1/files/upload) bytes_used 对比 yaml
`quotas.disk_bytes_per_user`,超额返 [Error] 硬阻trash_* 只记录隐藏回收站
的物理占用不参与用户配额判断
扫描间隙写入会突破上限一点(race-tolerant, image/video 配额一致接受);外部用户
开放前 OS xfs prjquota 兜底真上限 DESIGN §7.5 #4 / PROGRESS。
@ -371,6 +430,8 @@ class UserDiskUsage(Base):
)
bytes_used: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
file_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
trash_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
trash_file_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
scanned_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
@ -801,3 +862,27 @@ class ExternalSystemAudit(Base):
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
class SandboxPackageScan(Base):
"""一个发现临时 Python 包的容器会话一行(空扫描不入库)。"""
__tablename__ = "sandbox_package_scans"
__table_args__ = (
Index("ix_sandbox_package_scans_finished_at", "finished_at"),
Index("ix_sandbox_package_scans_user_finished", "user_id", "finished_at"),
)
container_session_id: Mapped[str] = mapped_column(Text, primary_key=True)
user_id: Mapped[UUID] = mapped_column(
PG_UUID(as_uuid=True), ForeignKey("users.user_id", ondelete="CASCADE"), nullable=False
)
execution_kind: Mapped[str] = mapped_column(Text, nullable=False)
image_digest: Mapped[str] = mapped_column(Text, nullable=False)
python_version: Mapped[str] = mapped_column(Text, nullable=False)
packages: Mapped[list[Any]] = mapped_column(JSONB, nullable=False)
package_count: Mapped[int] = mapped_column(Integer, nullable=False)
total_installed_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False)
finished_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)

View File

@ -1,6 +1,6 @@
"""失败埋点(telemetry)—— 与计费(usage.py)分家的另一类 usage_events 写入。
judged by kind:这里的四类 cost 0tokens 不入 kind=chat 汇总,语义是**可观测性
这里的异常类 kind cost 0tokens 不入 kind=chat 汇总,语义是**可观测性
留痕**而非用户花销 它们是 admin工具失败聚集面板与巡检邮件(core/toolfail.py)
的唯一持久数据源此前与真计费(chat/image/video/vision)混住 usage.py,污染
usage=计费的心智模型(架构审查数据层 Top4),2026-07-23 拆出
@ -11,6 +11,7 @@ kind 常量是 loop/llm_transport(写)与 toolfail(读 SQL)之间的契约单一
from __future__ import annotations
from decimal import Decimal
from typing import Any, Optional
from uuid import UUID
from .engine import session_scope
@ -19,8 +20,136 @@ from .models import UsageEvent
# ── kind 契约常量(写侧本模块 / 读侧 core/toolfail.py 共用)──
KIND_TOOL_MALFORMED = "tool_malformed"
KIND_TOOL_SALVAGED = "tool_salvaged"
KIND_TOOL_FAILURE = "tool_failure"
KIND_EMPTY_RESPONSE = "empty_response"
KIND_RUN_ERROR = "run_error"
KIND_RUN_STOPPED = "run_stopped"
KIND_AGENT_GUARD = "agent_guard"
KIND_CONTEXT_FOLD_FAILURE = "context_fold_failure"
KIND_QUALITY_GATE = "quality_gate"
def _record_health_event(
*,
kind: str,
task_id: UUID,
user_id: UUID,
model_profile: str,
units: dict[str, Any],
message_id: Optional[UUID] = None,
) -> None:
with session_scope() as s:
s.add(UsageEvent(
user_id=user_id,
task_id=task_id,
message_id=message_id,
kind=kind,
model_profile=model_profile,
units=units,
cost_cny=Decimal("0"),
))
def record_tool_failure(
*,
task_id: UUID,
user_id: UUID,
message_id: UUID,
model_profile: str,
tool: str,
content: str,
arguments=None,
) -> bool:
"""结构化记录普通工具失败;正常结果不写入并返回 False。"""
from core.tool_failure import structured_failure
units = structured_failure(tool, content, arguments=arguments)
if units is None:
return False
event_kind = KIND_QUALITY_GATE if units["category"] == "quality_gate" else KIND_TOOL_FAILURE
_record_health_event(
kind=event_kind,
task_id=task_id,
user_id=user_id,
message_id=message_id,
model_profile=model_profile,
units=units,
)
return True
def record_run_stopped(
*, task_id: UUID, user_id: UUID, model_profile: str, reason: str,
) -> None:
_record_health_event(
kind=KIND_RUN_STOPPED,
task_id=task_id,
user_id=user_id,
model_profile=model_profile,
units={
"tool": "(run)", "failure_kind": "stopped", "signature": reason,
"category": "agent_control", "sample": reason,
},
)
def record_agent_guard(
*,
task_id: UUID,
user_id: UUID,
model_profile: str,
tool: str,
guard: str,
count: int,
signature: str = "",
) -> None:
from core.tool_failure import normalize_failure_signature
stable = normalize_failure_signature(f"{guard}: {signature}" if signature else guard)
_record_health_event(
kind=KIND_AGENT_GUARD,
task_id=task_id,
user_id=user_id,
model_profile=model_profile,
units={
"tool": tool, "failure_kind": "guard", "signature": stable,
"category": "agent_control", "sample": f"{guard}, count={count}",
"count": int(count),
},
)
def record_context_fold_failure(
*, task_id: UUID, user_id: UUID, model_profile: str, reason: str, detail: str = "",
) -> None:
from core.tool_failure import normalize_failure_signature
_record_health_event(
kind=KIND_CONTEXT_FOLD_FAILURE,
task_id=task_id,
user_id=user_id,
model_profile=model_profile,
units={
"tool": "(context)", "failure_kind": "context", "signature": reason,
"category": "agent_control",
"sample": normalize_failure_signature(detail) if detail else reason,
},
)
def record_quality_gate(
*, task_id: UUID, user_id: UUID, model_profile: str, tool: str, gate: str,
) -> None:
_record_health_event(
kind=KIND_QUALITY_GATE,
task_id=task_id,
user_id=user_id,
model_profile=model_profile,
units={
"tool": tool, "failure_kind": "gate", "signature": gate,
"category": "quality_gate", "sample": gate,
},
)
def record_malformed_tool_call(

View File

@ -3,19 +3,20 @@
chat 类型的入口由 loop.py assistant message 入库后调用;媒体工具(image/video/audio)
tool execute 完后由 tool 直接调用对应入口(record_image_usage )
币种(0007):全表统一 CNY(`cost_cny` )chat 路径走 litellm USD cost_map 内部
×USD_TO_CNY 折算落库;媒体路径价格本身就是 CNY,直接落units jsonb snapshot 当时
的关键价格参数(chat 没有,media price_cny_per_image ),便于跨调价对账
币种(0007):全表统一 CNY(`cost_cny` )有版本化本地价格的直连模型以该配置为
事实源未配置的网关模型才走 LiteLLM USD cost map再退到扁平人民币价格
units jsonb 快照当时价格版本时段币种汇率和分项成本便于跨调价对账
"""
from __future__ import annotations
from datetime import datetime
from datetime import datetime, timezone
from decimal import Decimal
from typing import Any, Mapping, Optional
from uuid import UUID
from sqlalchemy import func, select, update
from ..pricing import calculate_chat_cost, pricing_snapshot, resolve_chat_price
from .engine import session_scope
from .models import Message, UsageEvent
@ -88,6 +89,8 @@ def record_chat_usage(
output_cny_per_mtoken: float = 0.0,
cache_hit_tokens: int = 0,
cache_hit_cny_per_mtoken: float = 0.0,
pricing: Optional[Mapping[str, Any]] = None,
occurred_at: Optional[datetime] = None,
extra_units: Optional[Mapping[str, Any]] = None,
response: Any = None,
kind: str = "chat",
@ -103,9 +106,32 @@ def record_chat_usage(
可挂,溯源靠 kind + extra_units(kb_ingest {"kb": 库名, "source": 原件名})
返回算出的 cost_cny(已落库),调用方可用作 SSE 显示
"""
# 直连 provider 的版本化本地价格是事实源,避免 LiteLLM 内置价目滞后覆盖已核对
# 的厂商价格。无本地适用价格的旧档案/网关模型才沿用 LiteLLM → 扁平价兜底。
when = occurred_at
if when is None:
created = getattr(response, "created", None)
try:
when = datetime.fromtimestamp(float(created), tz=timezone.utc) if created else None
except (TypeError, ValueError, OSError):
when = None
when = when or datetime.now(timezone.utc)
quote = resolve_chat_price(pricing, occurred_at=when)
price_units: dict[str, Any] = {}
if quote is not None:
breakdown = calculate_chat_cost(
quote,
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
cache_hit_tokens=cache_hit_tokens,
)
cost_cny = breakdown.total_cny
price_units = pricing_snapshot(quote, breakdown)
price_units["pricing_at"] = when.astimezone(timezone.utc).isoformat()
else:
cost_usd = _safe_chat_cost_usd(response)
cost_cny = (cost_usd * USD_TO_CNY).quantize(Decimal("0.000001"))
if cost_cny == 0 and (input_cny_per_mtoken or output_cny_per_mtoken):
if quote is None and cost_cny == 0 and (input_cny_per_mtoken or output_cny_per_mtoken):
cost_cny = _fallback_chat_cost_cny(
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
@ -114,19 +140,23 @@ def record_chat_usage(
cache_hit_tokens=cache_hit_tokens,
cache_hit_cny_per_mtoken=cache_hit_cny_per_mtoken,
)
units = {
units: dict[str, Any] = {
"tokens_in": int(prompt_tokens),
"tokens_out": int(completion_tokens),
# snapshot 折算系数,便于历史对账(汇率/价格涨跌后仍能还原当时折算逻辑)
"usd_to_cny": float(USD_TO_CNY),
"cache_hit_tokens": max(0, min(int(cache_hit_tokens), int(prompt_tokens))),
}
if quote is None:
# LiteLLM cost map 是 USD快照折算系数供旧路径历史对账。
units["usd_to_cny"] = float(USD_TO_CNY)
if input_cny_per_mtoken or output_cny_per_mtoken:
units.update({
"input_cny_per_mtoken": float(input_cny_per_mtoken or 0),
"output_cny_per_mtoken": float(output_cny_per_mtoken or 0),
"cache_hit_cny_per_mtoken": float(cache_hit_cny_per_mtoken or 0),
})
if extra_units:
units.update(extra_units)
units.update(price_units)
with session_scope() as s:
s.add(UsageEvent(

View File

@ -15,6 +15,7 @@ from sqlalchemy import select, update
from .agent_builder import ROOT, load_config
from .capabilities import ModelCapabilities
from .llm import LLM
from .llm_transport import extract_usage_details
from .storage import session_scope
from .storage.models import Task
from .storage.usage import record_chat_usage
@ -127,17 +128,24 @@ def generate_task_title(
)
applied = bool(getattr(result, "rowcount", 0))
if response is not None and caps is not None:
usage = getattr(response, "usage", None)
usage = extract_usage_details(getattr(response, "usage", None))
try:
record_chat_usage(
task_id=task_id,
user_id=user_id,
message_id=None,
model_profile=f"{caps.family}.{caps.variant}",
prompt_tokens=getattr(usage, "prompt_tokens", 0) or 0,
completion_tokens=getattr(usage, "completion_tokens", 0) or 0,
prompt_tokens=usage["tokens_in"],
completion_tokens=usage["tokens_out"],
input_cny_per_mtoken=caps.input_cny_per_mtoken,
output_cny_per_mtoken=caps.output_cny_per_mtoken,
cache_hit_tokens=usage["cache_hit_tokens"],
cache_hit_cny_per_mtoken=getattr(caps, "cache_hit_cny_per_mtoken", 0.0),
pricing=getattr(caps, "pricing", {}) or {},
extra_units={
k: v for k, v in usage.items()
if k not in ("tokens_in", "tokens_out") and v
},
response=response,
kind="task_title",
)

139
core/tool_failure.py Normal file
View File

@ -0,0 +1,139 @@
"""工具结果失败分类的单一事实源。"""
from __future__ import annotations
import json
import re
from typing import Any, Optional, Tuple
_RE_UUID = re.compile(
r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-"
r"[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"
)
_RE_HEX = re.compile(r"0x[0-9a-fA-F]+")
_RE_PATH = re.compile(r"(?:[A-Za-z]:)?(?:[/\\][\w.\-一-鿿*]+){2,}")
_RE_NUM = re.compile(r"\d+")
_RE_WS = re.compile(r"\s+")
_EXIT_TAIL = re.compile(r"\[exit (\d+)\]\s*$")
_STREAM_MARKS = ("[stdout]", "[stderr]")
_BARE_EXIT_RE = re.compile(r"^exit \d+$")
def normalize_failure_signature(value: str) -> str:
value = _RE_UUID.sub("<id>", value)
value = _RE_HEX.sub("<hex>", value)
value = _RE_PATH.sub("<path>", value)
value = _RE_NUM.sub("N", value)
return _RE_WS.sub(" ", value).strip()[:120]
def classify_failure(content: str) -> Optional[Tuple[str, str]]:
"""返回 ``(kind, 原始签名行)``;正常工具结果返回 None。"""
head = content.lstrip()
if "command timed out" in content:
return "timeout", "command timed out"
if head.startswith("[Error"):
return "error", head.splitlines()[0]
match = _EXIT_TAIL.search(content)
if match and match.group(1) != "0":
lines = [
line.strip()
for line in content.splitlines()[:-1]
if line.strip() and line.strip() not in _STREAM_MARKS
]
return "exit", (lines[-1] if lines else f"exit {match.group(1)}")
return None
def is_bare_exit_signature(value: str) -> bool:
return _BARE_EXIT_RE.match(value) is not None
def _command_hint(command: str) -> str:
command = command.strip().lower()
if not command:
return ""
if re.search(r"\b(?:apt|dpkg|pip)\b.*\b(?:list|show)\b", command):
return "dependency probe"
if re.search(r"(?:^|[;&|]\s*)(?:which|whereis|command\s+-v)\b", command):
return "dependency probe"
if re.search(r"(?:^|[;&|]\s*)(?:grep|rg)\b", command):
return "search/no match"
match = re.search(r"(?:^|&&|;|\|)\s*([a-z0-9_.-]+)", command)
return match.group(1) if match else ""
def shell_command_hint(prior_payload: Any, tool_call_id: str) -> str:
"""从历史 assistant tool_call 中提取稳定的 shell 命令类别。"""
if not isinstance(prior_payload, dict):
return ""
calls = prior_payload.get("tool_calls") or []
candidates = []
for call in calls:
if not isinstance(call, dict):
continue
function = call.get("function") or {}
if function.get("name") != "shell":
continue
if tool_call_id and call.get("id") == tool_call_id:
candidates = [call]
break
candidates.append(call)
if len(candidates) != 1:
return ""
return shell_command_hint_from_arguments(
(candidates[0].get("function") or {}).get("arguments")
)
def shell_command_hint_from_arguments(arguments: Any) -> str:
try:
args = json.loads(arguments) if isinstance(arguments, str) else arguments
except (TypeError, ValueError):
return ""
if not isinstance(args, dict):
return ""
return _command_hint(str(args.get("command") or ""))
def failure_category(kind: str, signature_line: str, sample: str) -> str:
if kind != "exit":
return "failure"
# 组合导出脚本会在 gate 行之后继续打印修复提示/命令,因此 gate 不一定是
# classify_failure 选中的最后一条签名行。按完整输出中的行首标记识别。
if signature_line.startswith("[GATE FAIL]") or re.search(
r"(?m)^\[GATE FAIL\]", sample
):
return "quality_gate"
if (
("[篇幅核算]" in sample or "[字数核算]" in sample)
and re.search(r"\[WARN\]\s*\d+\s*项超出\s*/\s*\d+\s*项不足", sample)
):
return "quality_gate"
if "[质量检查]" in sample and "[WARN] 共发现" in sample:
return "quality_gate"
return "failure"
def structured_failure(
tool: str,
content: str,
*,
arguments: Any = None,
) -> Optional[dict[str, str]]:
"""把工具结果转成可直接持久化的稳定失败事件。"""
hit = classify_failure(content)
if hit is None:
return None
kind, signature_line = hit
if tool == "shell" and kind == "exit" and _BARE_EXIT_RE.match(signature_line):
hint = shell_command_hint_from_arguments(arguments)
if hint:
signature_line = f"{signature_line} ({hint})"
return {
"tool": tool or "?",
"failure_kind": kind,
"signature": normalize_failure_signature(signature_line),
"category": failure_category(kind, signature_line, content),
"sample": content[:300],
}

View File

@ -269,17 +269,24 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
cancel_check=ctx.cancel_check, daily_limit=videos_per_day, **base,
)]
def _vision() -> list:
def _vision_kwargs() -> Optional[dict]:
# 看图 + 文档理解共用 vision variant(读图不分档,固定第一个)。
key, v = _pick_variant(ctx.ark_cfg.raw.get("vision") or {})
if v is None:
return []
kw = dict(
return None
return dict(
ark_cfg=ctx.ark_cfg, vision_variant_cfg=v, variant_key=key,
working_dir=ctx.working_dir_path, task_id=ctx.task_id, user_id=ctx.uid,
**base,
)
return [LookAtImageTool(**kw), ReadDocumentTool(**kw)]
def _look_at_image() -> list:
kw = _vision_kwargs()
return [LookAtImageTool(**kw)] if kw is not None else []
def _read_document() -> list:
kw = _vision_kwargs()
return [ReadDocumentTool(**kw)] if kw is not None else []
def _transcribe() -> list:
# 录音文件转写(讯飞 LFASR,与 IAT 听写是两个服务两套 key)。
@ -316,7 +323,13 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
("office_to_pdf", lambda: ctx.office_to_pdf_available, _office_to_pdf),
("image", lambda: ctx.img_cfg is not None, _image),
("video", lambda: ctx.ark_cfg is not None, _video),
("vision", lambda: ctx.ark_cfg is not None, _vision),
(
"look_at_image",
lambda: ctx.ark_cfg is not None
and not getattr(ctx.caps, "native_image_input", False),
_look_at_image,
),
("read_document", lambda: ctx.ark_cfg is not None, _read_document),
("transcribe_audio", lfasr_configured, _transcribe),
("web_search", lambda: BochaConfig.load() is not None, _web_search),
]

View File

@ -2,11 +2,12 @@
背景(2026-07,task 9dcae061 终案的结构性教训):mermaid 渲染在生产挂了 90
0 成功(67 次超时 + 26 launch fail烧掉数十万 token),没有任何机制发现,
靠人工扫 DB 才挖出来本模块把失败聚集变成信号:messages role=tool
的错误结果, (工具名 + 归一化错误签名) 聚合,超阈值即算聚集
靠人工扫 DB 才挖出来本模块把失败聚集变成信号:统一扫描 usage_events
中的结构化异常事件, (工具名 + 归一化错误签名) 聚合,超阈值即算聚集
纯只读查询无新表无状态;告警通道由调用方决定(web/app.py 的巡检 loop
开发者邮箱,admin API 直接返给前端表格)
普通工具失败从 2026-09-03 起在 tool result messages 后同步写 tool_failure
切换前只存在于 messages 的历史故障不再回扫纯只读查询无派生状态告警通道
由调用方决定(web/app.py 的巡检 loop 发开发者邮箱,admin API 直接返给前端表格)
第二数据源(0.58.19):被丢弃的畸形 tool_call 参数(kind=malformed)这类失败
整轮不入 messages(防投毒级联),loop usage_events(kind=tool_malformed),
@ -31,9 +32,10 @@ tool_use 漏成正文后丢空)。loop 落 usage_events(kind=empty_response),在
"""
from __future__ import annotations
import json
import math
import re
import threading
import time
from datetime import datetime, timedelta, timezone
from typing import Any, Dict, List, Optional, Tuple
@ -41,117 +43,42 @@ from sqlalchemy import text
from core.storage import session_scope
from core.storage.telemetry import (
KIND_AGENT_GUARD,
KIND_CONTEXT_FOLD_FAILURE,
KIND_EMPTY_RESPONSE,
KIND_QUALITY_GATE,
KIND_RUN_ERROR,
KIND_RUN_STOPPED,
KIND_TOOL_FAILURE,
KIND_TOOL_MALFORMED,
KIND_TOOL_SALVAGED,
)
# 签名归一:同一类错误在不同 task/参数下的差异(路径/数字/uuid/十六进制)抹平,
# 让 "figures/a.png doesn't exist" 和 "figures/b.png doesn't exist" 聚成一条。
_RE_UUID = re.compile(r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}")
_RE_HEX = re.compile(r"0x[0-9a-fA-F]+")
_RE_PATH = re.compile(r"(?:[A-Za-z]:)?(?:[/\\][\w.\-一-鿿*]+){2,}")
_RE_NUM = re.compile(r"\d+")
_RE_WS = re.compile(r"\s+")
_EXIT_TAIL = re.compile(r"\[exit (\d+)\]\s*$")
_STREAM_MARKS = ("[stdout]", "[stderr]")
_BARE_EXIT_RE = re.compile(r"^exit \d+$")
from core.tool_failure import (
failure_category as _failure_category,
normalize_failure_signature as _normalize,
)
def _normalize(s: str) -> str:
s = _RE_UUID.sub("<id>", s)
s = _RE_HEX.sub("<hex>", s)
s = _RE_PATH.sub("<path>", s)
s = _RE_NUM.sub("N", s)
s = _RE_WS.sub(" ", s).strip()
return s[:120]
_CACHE_TTL_SECONDS = 90.0
_failure_cache_lock = threading.Lock()
_wire_cache_lock = threading.Lock()
_failure_cache: dict[tuple[float, int, int], tuple[float, List[Dict[str, Any]]]] = {}
_wire_cache: dict[int, tuple[float, Dict[str, Any]]] = {}
def _classify(content: str) -> Optional[Tuple[str, str]]:
"""返回 (kind, 原始签名行) 或 None(不算失败)。"""
head = content.lstrip()
# 超时判定在前:超时结果形如 "[Error] command timed out after 30s",
# 让它归 timeout 而不是被 [Error 前缀截走(kind 对排查方向有指示意义)
if "command timed out" in content:
return "timeout", "command timed out"
if head.startswith("[Error"):
return "error", head.splitlines()[0]
m = _EXIT_TAIL.search(content)
if m and m.group(1) != "0":
# 签名取 [exit N] 前最后一行有实际内容的输出(通常是真正的报错行)
lines = [
ln.strip() for ln in content.splitlines()[:-1]
if ln.strip() and ln.strip() not in _STREAM_MARKS
]
return "exit", (lines[-1] if lines else f"exit {m.group(1)}")
return None
def _shell_command_hint(prior_payload: Any, tool_call_id: str) -> str:
"""从相邻 assistant tool_call 提取空输出 shell 的命令类别。
shell 仅返回 ``[exit 1]`` 原签名会把 grep 未命中依赖探测和真正的命令失败
全揉成 ``exit N``messages 已保存 tool_call_id可关联最近 assistant 精确找到
arguments这里只返回稳定类别不把完整命令或路径带进聚集签名
"""
if not isinstance(prior_payload, dict):
return ""
calls = prior_payload.get("tool_calls") or []
candidates = []
for call in calls:
if not isinstance(call, dict):
continue
fn = call.get("function") or {}
if fn.get("name") != "shell":
continue
if tool_call_id and call.get("id") == tool_call_id:
candidates = [call]
break
candidates.append(call)
if len(candidates) != 1:
return ""
raw = (candidates[0].get("function") or {}).get("arguments") or {}
try:
args = json.loads(raw) if isinstance(raw, str) else raw
except (TypeError, ValueError):
return ""
command = str((args or {}).get("command") or "").strip().lower()
if not command:
return ""
if re.search(r"\b(?:apt|dpkg|pip)\b.*\b(?:list|show)\b", command):
return "dependency probe"
if re.search(r"(?:^|[;&|]\s*)(?:which|whereis|command\s+-v)\b", command):
return "dependency probe"
if re.search(r"(?:^|[;&|]\s*)(?:grep|rg)\b", command):
return "search/no match"
m = re.search(r"(?:^|&&|;|\|)\s*([a-z0-9_.-]+)", command)
return m.group(1) if m else ""
def _failure_category(kind: str, sig_line: str, sample: str) -> str:
"""区分产品/内容质量门与平台工具故障;返回值作为 API 加法字段。"""
if kind != "exit":
return "failure"
if sig_line.startswith("[GATE FAIL]"):
return "quality_gate"
if (
("[篇幅核算]" in sample or "[字数核算]" in sample)
and re.search(r"\[WARN\]\s*\d+\s*项超出\s*/\s*\d+\s*项不足", sample)
):
return "quality_gate"
if "[质量检查]" in sample and "[WARN] 共发现" in sample:
return "quality_gate"
return "failure"
def _clear_tool_health_cache() -> None:
"""测试/运维进程内失效入口;不触碰数据库。"""
with _failure_cache_lock:
_failure_cache.clear()
with _wire_cache_lock:
_wire_cache.clear()
def scan_tool_failures(
days: float = 7,
min_count: int = 5,
min_tasks: int = 2,
) -> List[Dict[str, Any]]:
"""扫近 `days` 天的 tool 错误消息,返回超阈值的聚集。
"""扫近 `days` 天的结构化异常事件,返回超阈值的聚集。
阈值语义:同签名 >= min_count >= min_tasks task task
模型试错几次就自愈的正常噪音不触发; task 复现的才是平台性问题
@ -166,62 +93,21 @@ def scan_tool_failures(
cutoff = now - timedelta(days=days)
n_buckets = max(1, math.ceil(days))
with session_scope() as s:
rows = s.execute(
event_rows = s.execute(
text(
"select m.task_id, t.user_id, m.created_at, "
" m.payload->>'name' as tool_name, "
" m.payload->>'content' as content, "
" m.payload->>'tool_call_id' as tool_call_id, "
" p.payload as prior_payload "
"from messages m join tasks t on t.task_id = m.task_id "
"left join lateral ("
" select pm.payload from messages pm "
" where pm.task_id = m.task_id and pm.idx < m.idx "
" and pm.payload->>'role' = 'assistant' "
" order by pm.idx desc limit 1"
") p on true "
"where m.created_at >= :cutoff "
" and m.payload->>'role' = 'tool' "
" and (m.payload->>'content' like '[Error%' "
" or m.payload->>'content' like '%command timed out%' "
" or m.payload->>'content' like '%[exit %')"
),
{"cutoff": cutoff},
).fetchall()
# 第二段:被丢弃的畸形 tool_call 参数(kind=tool_malformed)。这类失败整轮
# 不入 messages(防投毒),llm_transport.log_malformed_args 落在 usage_events,
# 是它们进面板/巡检邮件的唯一路径。
mrows = s.execute(
text(
"select task_id, user_id, created_at, "
" units->>'tool' as tool_name, "
" units->>'err' as err, "
" units->>'head' as head, "
" units->>'tail' as tail "
"select kind, task_id, user_id, created_at, model_profile, "
" units->>'tool' as tool_name, units->>'err' as err, "
" units->>'head' as head, units->>'tail' as tail, "
" units->>'failure_kind' as failure_kind, "
" units->>'signature' as signature, "
" units->>'category' as category, "
" units->>'sample' as sample "
"from usage_events "
f"where kind = '{KIND_TOOL_MALFORMED}' and created_at >= :cutoff"
),
{"cutoff": cutoff},
).fetchall()
# 第三段:run 级终态错误(kind=run)。LLM 层抛异常时整轮无 tool 消息,
# tasks.run_error 只留最后一次,usage_events(kind=run_error)才是完整留痕。
rrows = s.execute(
text(
"select task_id, user_id, created_at, "
" units->>'err' as err "
"from usage_events "
f"where kind = '{KIND_RUN_ERROR}' and created_at >= :cutoff"
),
{"cutoff": cutoff},
).fetchall()
# 第四段:provider 吐空(kind=empty_response)。assistant 轮既无 tool_calls 又无正文,
# 会被 run loop 当正常收尾静默 done;loop 落 usage_events(kind=empty_response)是唯一
# 留痕。tool 名固定 "(empty)",签名固定,sample=model_profile —— 看是哪个网关档在吐空。
erows = s.execute(
text(
"select task_id, user_id, created_at, model_profile "
"from usage_events "
f"where kind = '{KIND_EMPTY_RESPONSE}' and created_at >= :cutoff"
f"where kind in ('{KIND_TOOL_FAILURE}', '{KIND_TOOL_MALFORMED}', "
f"'{KIND_RUN_ERROR}', '{KIND_EMPTY_RESPONSE}', '{KIND_RUN_STOPPED}', "
f"'{KIND_AGENT_GUARD}', '{KIND_CONTEXT_FOLD_FAILURE}', "
f"'{KIND_QUALITY_GATE}') "
" and created_at >= :cutoff"
),
{"cutoff": cutoff},
).fetchall()
@ -231,10 +117,15 @@ def scan_tool_failures(
def _add(
tool_name: str, kind: str, sig_line: str, sample: str,
task_id: Any, user_id: Any, created_at: datetime,
*, category: Optional[str] = None, signature_normalized: bool = False,
) -> None:
# DB 列若是 naive timestamp(存 UTC),补 tzinfo 才能和 now 做减法
ts = created_at if created_at.tzinfo else created_at.replace(tzinfo=timezone.utc)
key = (tool_name or "?", _normalize(sig_line))
key = (
tool_name or "?",
sig_line if signature_normalized else _normalize(sig_line),
)
resolved_category = category or _failure_category(kind, sig_line, sample)
c = agg.get(key)
if c is None:
c = agg[key] = {
@ -242,7 +133,7 @@ def scan_tool_failures(
"signature": key[1],
"kind": kind,
# 分类必须看未截断的完整结果sample 只保留 300 字给前端悬浮。
"category": _failure_category(kind, sig_line, sample),
"category": resolved_category,
"count": 0,
"tasks": set(),
"users": set(),
@ -251,7 +142,7 @@ def scan_tool_failures(
"sample": sample[:300],
"daily": [0] * n_buckets,
}
elif _failure_category(kind, sig_line, sample) == "quality_gate":
elif resolved_category == "quality_gate":
c["category"] = "quality_gate"
c["count"] += 1
c["tasks"].add(task_id)
@ -265,34 +156,30 @@ def scan_tool_failures(
c["last_at"] = ts
c["sample"] = sample[:300]
for row in rows:
# 兼容旧测试夹具的 5 列消息行;生产查询额外带 tool_call_id/prior_payload。
task_id, user_id, created_at, tool_name, content = row[:5]
tool_call_id = row[5] if len(row) > 5 else ""
prior_payload = row[6] if len(row) > 6 else None
if not content:
continue
hit = _classify(content)
if hit is None:
continue
kind, sig_line = hit
if tool_name == "shell" and kind == "exit" and _BARE_EXIT_RE.match(sig_line):
hint = _shell_command_hint(prior_payload, tool_call_id or "")
if hint:
sig_line = f"{sig_line} ({hint})"
_add(tool_name, kind, sig_line, content, task_id, user_id, created_at)
for task_id, user_id, created_at, tool_name, err, head, tail in mrows:
for (
event_kind, task_id, user_id, created_at, model_profile, tool_name,
err, head, tail, failure_kind, signature, category, sample,
) in event_rows:
if event_kind in {
KIND_TOOL_FAILURE,
KIND_RUN_STOPPED,
KIND_AGENT_GUARD,
KIND_CONTEXT_FOLD_FAILURE,
KIND_QUALITY_GATE,
}:
_add(
tool_name, "malformed", err or "?",
f"{head or ''}{tail or ''}",
tool_name, failure_kind or "failure", signature or "?", sample or "",
task_id, user_id, created_at,
category=category or "failure", signature_normalized=True,
)
elif event_kind == KIND_TOOL_MALFORMED:
_add(
tool_name, "malformed", err or "?", f"{head or ''}{tail or ''}",
task_id, user_id, created_at,
)
for task_id, user_id, created_at, err in rrows:
elif event_kind == KIND_RUN_ERROR:
_add("(run)", "run", err or "?", err or "", task_id, user_id, created_at)
for task_id, user_id, created_at, model_profile in erows:
elif event_kind == KIND_EMPTY_RESPONSE:
_add(
"(empty)", "empty", "provider returned empty response",
model_profile or "?", task_id, user_id, created_at,
@ -321,6 +208,30 @@ def scan_tool_failures(
return out
def scan_tool_failures_cached(
days: float = 7,
min_count: int = 5,
min_tasks: int = 2,
) -> List[Dict[str, Any]]:
"""管理页短时缓存;锁覆盖扫描,使同参数并发请求只扫库一次。"""
key = (float(days), int(min_count), int(min_tasks))
now = time.monotonic()
with _failure_cache_lock:
cached = _failure_cache.get(key)
if cached and cached[0] > now:
return cached[1]
for stale_key, (expires_at, _value) in list(_failure_cache.items()):
if expires_at <= now:
_failure_cache.pop(stale_key, None)
value = scan_tool_failures(
days=days,
min_count=min_count,
min_tasks=min_tasks,
)
_failure_cache[key] = (time.monotonic() + _CACHE_TTL_SECONDS, value)
return value
def scan_tool_wire_health(days: int = 7) -> Dict[str, Any]:
"""聚合 provider 工具参数损坏的抢救/残余比例;纯只读、无派生状态。
@ -407,6 +318,22 @@ def scan_tool_wire_health(days: int = 7) -> Dict[str, Any]:
return {"days": days, "rows": out, "total": totals}
def scan_tool_wire_health_cached(days: int = 7) -> Dict[str, Any]:
"""管理页链路健康短时缓存;同窗口并发请求共享一次聚合。"""
key = min(90, max(1, int(days)))
now = time.monotonic()
with _wire_cache_lock:
cached = _wire_cache.get(key)
if cached and cached[0] > now:
return cached[1]
for stale_key, (expires_at, _value) in list(_wire_cache.items()):
if expires_at <= now:
_wire_cache.pop(stale_key, None)
value = scan_tool_wire_health(days=key)
_wire_cache[key] = (time.monotonic() + _CACHE_TTL_SECONDS, value)
return value
# ── provider 级致命错误即时告警 ──
# 命中判据:错误文案含余额/配额/认证类关键词 —— 这类错误不是单任务偶发,而是该
# provider 上所有 run 全挂(如 Zai 余额不足),等日巡检的 5 次/2 task 阈值太慢。

View File

@ -0,0 +1,31 @@
"""Track artifact trash separately from quota usage.
Revision ID: 0037
Revises: 0036
Create Date: 2026-08-28
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
revision: str = "0037"
down_revision: str | None = "0036"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.add_column(
"user_disk_usage",
sa.Column("trash_bytes", sa.BigInteger(), nullable=False, server_default="0"),
)
op.add_column(
"user_disk_usage",
sa.Column("trash_file_count", sa.Integer(), nullable=False, server_default="0"),
)
def downgrade() -> None:
op.drop_column("user_disk_usage", "trash_file_count")
op.drop_column("user_disk_usage", "trash_bytes")

View File

@ -0,0 +1,37 @@
"""Add sandbox package scan sessions.
Revision ID: 0038
Revises: 0037
Create Date: 2026-09-01
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0038"
down_revision: str | None = "0037"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.create_table(
"sandbox_package_scans",
sa.Column("container_session_id", sa.Text(), primary_key=True),
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.user_id", ondelete="CASCADE"), nullable=False),
sa.Column("execution_kind", sa.Text(), nullable=False),
sa.Column("image_digest", sa.Text(), nullable=False),
sa.Column("python_version", sa.Text(), nullable=False),
sa.Column("packages", postgresql.JSONB(), nullable=False),
sa.Column("package_count", sa.Integer(), nullable=False),
sa.Column("total_installed_bytes", sa.BigInteger(), nullable=False),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_index("ix_sandbox_package_scans_finished_at", "sandbox_package_scans", ["finished_at"])
op.create_index("ix_sandbox_package_scans_user_finished", "sandbox_package_scans", ["user_id", "finished_at"])
def downgrade() -> None:
op.drop_table("sandbox_package_scans")

View File

@ -0,0 +1,46 @@
"""Add encrypted provider credentials control plane.
Revision ID: 0039
Revises: 0038
Create Date: 2026-09-02
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0039"
down_revision: str | None = "0038"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.create_table(
"provider_credentials",
sa.Column("provider_id", sa.Text(), primary_key=True),
sa.Column("credentials", postgresql.JSONB(), nullable=False),
sa.Column("credential_hint", postgresql.JSONB(), nullable=False),
sa.Column("revision", sa.Integer(), server_default="1", nullable=False),
sa.Column("test_status", sa.Text(), server_default="untested", nullable=False),
sa.Column("test_detail", sa.Text(), nullable=True),
sa.Column("balance_amount", sa.Numeric(18, 6), nullable=True),
sa.Column("balance_currency", sa.Text(), nullable=True),
sa.Column("last_tested_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_notified_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_notified_status", sa.Text(), nullable=True),
sa.Column(
"updated_by",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("users.user_id", ondelete="SET NULL"),
nullable=True,
),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.CheckConstraint("revision > 0", name="ck_provider_credentials_revision_positive"),
)
def downgrade() -> None:
op.drop_table("provider_credentials")

View File

@ -0,0 +1,41 @@
"""Add indexes for structured tool health events.
Revision ID: 0040
Revises: 0039
Create Date: 2026-09-03
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0040"
down_revision: Union[str, None] = "0039"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_index(
"ix_usage_health_kind_created",
"usage_events",
["kind", "created_at"],
postgresql_where=sa.text(
"kind IN ('tool_failure', 'tool_malformed', 'tool_salvaged', "
"'run_error', 'empty_response', 'run_stopped', 'agent_guard', "
"'context_fold_failure', 'quality_gate')"
),
)
op.create_index(
"ux_usage_tool_failure_message",
"usage_events",
["message_id"],
unique=True,
postgresql_where=sa.text("kind IN ('tool_failure', 'quality_gate')"),
)
def downgrade() -> None:
op.drop_index("ux_usage_tool_failure_message", table_name="usage_events")
op.drop_index("ix_usage_health_kind_created", table_name="usage_events")

View File

@ -64,6 +64,9 @@ RUN grep -v '\[host-only\]' /tmp/requirements.txt > /tmp/requirements.sandbox.tx
-r /tmp/requirements.sandbox.txt \
&& rm /tmp/requirements.txt /tmp/requirements.sandbox.txt
# 构建期基础包清单;运行时扫描只与它比较,不 import/执行用户包。
RUN mkdir -p /sandbox && python -c "import importlib.metadata as m,json; c=lambda s:s.lower().replace('_','-').replace('.','-'); print(json.dumps({c(d.metadata['Name']):d.version for d in m.distributions() if d.metadata.get('Name')},sort_keys=True))" > /sandbox/base-python-packages.json
# 持久化 pip 源到 /etc/pip.conf ── 让运行时模型用 `pip install foo` 也走 mirror,
# 不只 build 时。zcbot user / root 都吃这个 global 配置。
RUN printf '[global]\nindex-url = %s\ntimeout = 60\n%s\n' \
@ -176,6 +179,7 @@ COPY core/__init__.py core/file_store.py core/kb_lock.py /sandbox/core/
COPY core/sandbox/tool_runner.py /sandbox/tool_runner.py
COPY deploy/sandbox/init.sh /init.sh
COPY deploy/sandbox/package_scan.py /sandbox/package_scan.py
RUN chmod +x /init.sh
# 默认 cwd /workspace ── 但每次 `docker exec --workdir /workspace/<wd>` 会覆盖

View File

@ -0,0 +1,84 @@
"""可信临时包扫描器;只解析 dist-info 元数据,不 import 用户代码。"""
from __future__ import annotations
import csv
import email.parser
import json
from pathlib import Path
BASE = Path("/sandbox/base-python-packages.json")
ROOT = Path("/tmp/.local/lib")
def safe_file(root: Path, path: Path) -> bool:
try:
path.relative_to(root)
except ValueError:
return False
current = root
for part in path.relative_to(root).parts:
current = current / part
if current.is_symlink():
return False
return path.is_file()
def safe_dir(root: Path, path: Path) -> bool:
try:
rel = path.relative_to(root)
except ValueError:
return False
current = root
for part in rel.parts:
current = current / part
if current.is_symlink():
return False
return path.is_dir()
def canonical(name: str) -> str:
return name.lower().replace("_", "-").replace(".", "-")
def scan_packages(root: Path, base: dict) -> list[dict]:
out = []
if root.is_dir():
for site in sorted(root.glob("python*/site-packages")):
if not safe_dir(root, site):
continue
for dist in sorted(site.glob("*.dist-info")):
if len(out) >= 5000:
return out
meta_path = dist / "METADATA"
if not safe_file(site, meta_path) or meta_path.stat().st_size > 1024 * 1024:
continue
msg = email.parser.Parser().parsestr(meta_path.read_text(encoding="utf-8", errors="replace"), headersonly=True)
name, version = msg.get("Name", "").strip(), msg.get("Version", "").strip()
if not name:
continue
total = 0
record = dist / "RECORD"
if safe_file(site, record) and record.stat().st_size <= 32 * 1024 * 1024:
for index, row in enumerate(csv.reader(record.read_text(encoding="utf-8", errors="replace").splitlines())):
if index >= 200_000:
break
if not row:
continue
candidate = site / row[0]
if safe_file(site, candidate):
try: total += candidate.stat().st_size
except OSError: pass
key = canonical(name)
old = base.get(key)
change = "added" if old is None else ("reinstall" if old == version else "override")
out.append({"name": name, "version": version, "base_version": old,
"change": change, "direct_requested": safe_file(site, dist / "REQUESTED") or safe_file(site, dist / "direct_url.json"),
"installed_bytes": total})
return out
def main() -> None:
base = json.loads(BASE.read_text(encoding="utf-8"))
print(json.dumps(scan_packages(ROOT, base), ensure_ascii=False))
if __name__ == "__main__":
main()

View File

@ -39,7 +39,8 @@ _LIST_FIELDS = (
def _api_key() -> str:
key = os.environ.get("DOCUMENT_SEARCH_API_KEY", "").strip()
from core.provider_credentials.runtime import resolve_secret
key = resolve_secret("document_search")
if not key:
raise RuntimeError(
"DOCUMENT_SEARCH_API_KEY env 未设置 —— 配置后再使用 literature skill 的内部材料库来源"
@ -54,6 +55,19 @@ def _auth_headers(extra: Optional[dict] = None) -> dict:
return h
def _check_response(response: httpx.Response) -> None:
if response.status_code >= 400:
try:
from core.provider_credentials.service import record_business_failure
record_business_failure(
"document_search", status_code=response.status_code,
detail=response.text[:300],
)
except Exception:
pass
response.raise_for_status()
def _safe_name(name: str) -> str:
# 防目录穿越;保留扩展名
return name.replace("/", "_").replace("\\", "_").replace("..", "_")
@ -66,7 +80,7 @@ def list_kb() -> list[dict]:
/ create_time / file_count只返回 ID 映射里有效的(分类 1-7)
"""
r = httpx.get(f"{_API}/list_knowledge_bases", headers=_auth_headers(), timeout=_TIMEOUT)
r.raise_for_status()
_check_response(r)
payload = r.json()
data = payload.get("data") or {}
return list(data.get("knowledge_bases") or [])
@ -104,7 +118,7 @@ def search(
json=body,
timeout=_TIMEOUT,
)
r.raise_for_status()
_check_response(r)
payload = r.json()
data = payload.get("data") or {}
docs = data.get("documents") or []
@ -144,7 +158,7 @@ def download(
params=params,
timeout=_DOWNLOAD_TIMEOUT,
) as resp:
resp.raise_for_status()
_check_response(resp)
with open(dest, "wb") as f:
for chunk in resp.iter_bytes(chunk_size=64 * 1024):
f.write(chunk)

View File

@ -30,7 +30,8 @@ _DEFAULT_SUMMARY_FIELDS = [
def _mp_key() -> str:
key = os.environ.get("MP_API_KEY", "").strip()
from core.provider_credentials.runtime import resolve_secret
key = resolve_secret("materials_project")
if not key:
raise RuntimeError("MP_API_KEY env 未设置,无法查询 Materials Project")
return key
@ -61,6 +62,17 @@ def _mpr():
return MPRester(_mp_key())
def _report_provider_error(error: Exception) -> None:
try:
from core.provider_credentials.service import record_business_failure
status = int(getattr(error, "status_code", 0) or 0)
record_business_failure(
"materials_project", status_code=status, detail=str(error)
)
except Exception:
pass
class MaterialsProjectSearchTool(Tool):
name = "materials_project_search"
description = (
@ -128,6 +140,7 @@ class MaterialsProjectSearchTool(Tool):
num_chunks=1, chunk_size=limit, **kwargs
)
except Exception as e:
_report_provider_error(e)
detail = safe_error_text(e, ("MP_API_KEY",))
return f"[Error] materials_project_search failed: {type(e).__name__}: {detail}"
plain = [_to_plain(d) for d in list(docs)[:limit]]
@ -152,6 +165,7 @@ class MaterialsProjectSearchTool(Tool):
try:
session = _mpr()
except Exception as e:
_report_provider_error(e)
detail = safe_error_text(e, ("MP_API_KEY",))
return f"[Error] materials_project_search batch failed: {type(e).__name__}: {detail}"
agg: list[dict[str, Any]] = []
@ -169,6 +183,7 @@ class MaterialsProjectSearchTool(Tool):
n_ok += 1
agg.append({"formula": f, "n": len(results), "results": results})
except Exception as e:
_report_provider_error(e)
detail = safe_error_text(e, ("MP_API_KEY",))
agg.append({"formula": f, "error": f"{type(e).__name__}: {detail}"})
return self._render_batch(agg, fields, n_ok)
@ -251,6 +266,7 @@ class MaterialsProjectGetStructureTool(Tool):
dest.parent.mkdir(parents=True, exist_ok=True)
struct.to(filename=str(dest))
except Exception as e:
_report_provider_error(e)
detail = safe_error_text(e, ("MP_API_KEY",))
return f"[Error] materials_project_get_structure failed: {type(e).__name__}: {detail}"
return f"saved: {self._display(dest)}"

View File

@ -47,7 +47,8 @@ _LIST_FIELDS = (
def _config() -> tuple[str, str, str]:
base_url = os.environ.get("PAPER_SERVER_URL", _DEFAULT_BASE_URL).strip().rstrip("/")
api_key = os.environ.get("PAPER_SERVER_API_KEY", "").strip()
from core.provider_credentials.runtime import resolve_secret
api_key = resolve_secret("paper_server")
if not api_key:
raise RuntimeError("PAPER_SERVER_API_KEY env 未设置,无法查询 paper_server")
return base_url, f"{base_url}/api/resm/paper", api_key
@ -70,6 +71,14 @@ def _raise_response_error(response: httpx.Response) -> None:
except Exception:
pass
if response.status_code in (401, 403) or err_code in _AUTH_ERR_CODES:
try:
from core.provider_credentials.service import record_business_failure
record_business_failure(
"paper_server", status_code=response.status_code or 401,
detail=err_code,
)
except Exception:
pass
raise RuntimeError(
f"paper_server auth failed (HTTP {response.status_code}, "
f"{err_code or 'no err_code'}):请管理员检查平台 PAPER_SERVER_API_KEY"

View File

@ -54,7 +54,8 @@ class PlatformSourceProvider(Protocol):
def _env_set(name: str) -> bool:
return bool(os.environ.get(name, "").strip())
from core.provider_credentials.runtime import resolve_env_secret
return bool(resolve_env_secret(name))
def _require_valid_http_url(name: str, default: str) -> None:

225
scripts/reprice_usage.py Normal file
View File

@ -0,0 +1,225 @@
"""按版本化模型价格重算历史 usage_events默认只输出 dry-run 汇总。
示例
.venv/Scripts/python.exe scripts/reprice_usage.py --from 2026-08-16T16:00:00Z
.venv/Scripts/python.exe scripts/reprice_usage.py --from 2026-08-16T16:00:00Z \
--apply --confirm APPLY_DEEPSEEK_REPRICE
脚本不加载 .env调用方必须显式提供 ZCBOT_DB_URL生产写入前先核对 dry-run 输出和
打印的脱敏目标缺少 cache_hit_tokens 的旧辅助调用按全部未命中估算并留标记
"""
from __future__ import annotations
import argparse
import os
from collections import defaultdict
from datetime import datetime, timezone
from decimal import Decimal
from pathlib import Path
import sys
from typing import Any
from sqlalchemy import select
from sqlalchemy.engine import make_url
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from core.capabilities import ModelCapabilities # noqa: E402
from core.pricing import calculate_chat_cost, pricing_snapshot, resolve_chat_price # noqa: E402
from core.storage import session_scope # noqa: E402
from core.storage.models import UsageEvent # noqa: E402
DEFAULT_PROFILES = ("deepseek_v4.flash", "deepseek_v4.pro")
COST_KINDS = {"chat", "prompt_optimize", "context_fold", "task_title", "kb_ingest"}
CONFIRM_TEXT = "APPLY_DEEPSEEK_REPRICE"
PRICE_SNAPSHOT_KEYS = {
"pricing_revision",
"pricing_source",
"pricing_source_url",
"pricing_currency",
"pricing_at",
"price_tier",
"fx_to_cny",
"input_price_per_mtoken",
"cache_hit_price_per_mtoken",
"output_price_per_mtoken",
"input_cost_cny",
"cache_hit_cost_cny",
"output_cost_cny",
}
def _parse_datetime(value: str) -> datetime:
dt = datetime.fromisoformat(value.strip().replace("Z", "+00:00"))
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone.utc)
def _args() -> argparse.Namespace:
p = argparse.ArgumentParser(description="重算历史模型成本(默认 dry-run")
p.add_argument("--from", dest="from_at", required=True, type=_parse_datetime)
p.add_argument("--profile", action="append", dest="profiles")
p.add_argument("--apply", action="store_true", help="事务内写回;默认只预览")
p.add_argument("--rollback", action="store_true", help="恢复本 revision 保存的旧成本")
p.add_argument("--confirm", default="", help=f"写入确认串:{CONFIRM_TEXT}")
return p.parse_args()
def _target() -> str:
raw = os.environ.get("ZCBOT_DB_URL", "").strip()
if not raw:
raise RuntimeError("ZCBOT_DB_URL is not set")
return make_url(raw).render_as_string(hide_password=True)
def main() -> int:
args = _args()
if args.apply and args.rollback:
raise SystemExit("[ERR] --apply 与 --rollback 不能同时使用")
if (args.apply or args.rollback) and args.confirm != CONFIRM_TEXT:
raise SystemExit(f"[ERR] 写入必须传 --confirm {CONFIRM_TEXT}")
profiles = tuple(args.profiles or DEFAULT_PROFILES)
caps_by_profile = {
profile: ModelCapabilities.load(profile, ROOT / "config" / "models")
for profile in profiles
}
revisions = {
period.get("revision")
for caps in caps_by_profile.values()
for period in caps.pricing.get("periods", [])
if period.get("revision")
}
target = _target()
mode = "ROLLBACK" if args.rollback else ("APPLY" if args.apply else "DRY-RUN")
print(f"[INFO] mode={mode} target={target} from={args.from_at.isoformat()}")
print(f"[INFO] profiles={','.join(profiles)} revisions={','.join(sorted(revisions))}")
summary: dict[tuple[str, str], dict[str, Any]] = defaultdict(
lambda: {
"rows": 0,
"estimated": 0,
"old": Decimal("0"),
"new": Decimal("0"),
"estimated_old": Decimal("0"),
"estimated_new": Decimal("0"),
}
)
estimated_by_kind: dict[tuple[str, str], dict[str, Any]] = defaultdict(
lambda: {"rows": 0, "old": Decimal("0"), "new": Decimal("0")}
)
changed = 0
with session_scope() as s:
events = s.execute(
select(UsageEvent).where(
UsageEvent.model_profile.in_(profiles),
UsageEvent.kind.in_(COST_KINDS),
UsageEvent.created_at >= args.from_at,
).order_by(UsageEvent.created_at, UsageEvent.event_id)
).scalars().all()
for event in events:
units = dict(event.units or {})
caps = caps_by_profile[event.model_profile]
if args.rollback:
if units.get("repricing_revision") not in revisions:
continue
previous = units.get("previous_cost_cny")
if previous is None:
continue
old = Decimal(str(event.cost_cny or 0))
new = Decimal(str(previous))
previous_pricing = units.get("previous_pricing_units") or {}
for price_key in PRICE_SNAPSHOT_KEYS:
units.pop(price_key, None)
if isinstance(previous_pricing, dict):
units.update(previous_pricing)
for key in list(units):
if (
key.startswith("repricing_")
or key in {"previous_cost_cny", "previous_pricing_units"}
):
units.pop(key, None)
key = (event.model_profile, "rollback")
else:
quote = resolve_chat_price(caps.pricing, occurred_at=event.created_at)
if quote is None or units.get("repricing_revision") == quote.revision:
continue
tokens_in = int(units.get("tokens_in") or 0)
tokens_out = int(units.get("tokens_out") or 0)
# 旧主循环一直提取 DeepSeek cache usage但 record_chat_usage 的
# extra_units 会省略值为 0 的字段,所以 chat 缺 key 可判定为零命中;
# 旧辅助调用没有提取缓存明细,缺 key 才是真正的不确定。
has_cache = "cache_hit_tokens" in units or event.kind == "chat"
cache_hit = int(units.get("cache_hit_tokens") or 0)
breakdown = calculate_chat_cost(
quote,
prompt_tokens=tokens_in,
completion_tokens=tokens_out,
cache_hit_tokens=cache_hit,
)
old = Decimal(str(event.cost_cny or 0))
new = breakdown.total_cny
previous_pricing = {
key: units[key] for key in PRICE_SNAPSHOT_KEYS if key in units
}
units.update(pricing_snapshot(quote, breakdown))
units.update({
"repricing_revision": quote.revision,
"previous_cost_cny": float(old),
"repriced_at": datetime.now(timezone.utc).isoformat(),
"repricing_time_basis": "usage_event.created_at",
"repricing_estimated": not has_cache,
"previous_pricing_units": previous_pricing,
})
key = (event.model_profile, quote.tier)
if not has_cache:
summary[key]["estimated"] += 1
summary[key]["estimated_old"] += old
summary[key]["estimated_new"] += new
uncertain = estimated_by_kind[(event.model_profile, event.kind)]
uncertain["rows"] += 1
uncertain["old"] += old
uncertain["new"] += new
bucket = summary[key]
bucket["rows"] += 1
bucket["old"] += old
bucket["new"] += new
changed += 1
if args.apply or args.rollback:
event.cost_cny = new.quantize(Decimal("0.000001"))
event.units = units
if not (args.apply or args.rollback):
s.rollback()
total_old = Decimal("0")
total_new = Decimal("0")
for (profile, tier), item in sorted(summary.items()):
total_old += item["old"]
total_new += item["new"]
print(
f"[INFO] {profile} tier={tier} rows={item['rows']} "
f"estimated={item['estimated']} old={item['old']:.6f} "
f"new={item['new']:.6f} delta={(item['new'] - item['old']):.6f} "
f"estimated_delta={(item['estimated_new'] - item['estimated_old']):.6f}"
)
for (profile, kind), item in sorted(estimated_by_kind.items()):
print(
f"[WARN] estimated profile={profile} kind={kind} rows={item['rows']} "
f"old={item['old']:.6f} new={item['new']:.6f} "
f"delta={(item['new'] - item['old']):.6f}"
)
print(
f"[OK] rows={changed} old={total_old:.6f} new={total_new:.6f} "
f"delta={(total_new - total_old):.6f} mode={mode}"
)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -2,6 +2,12 @@
"capability": "ansys.geometry.inspect@v1",
"display_name": "ANSYS Mechanical 几何检查",
"default_enrollment": false,
"lifecycle": {
"status": "deprecated",
"visibility": "diagnostic_only",
"replacement": "cad.geometry.prepare@v1",
"detail": "保留用于 ANSYS Mechanical 真实导入专项诊断;普通几何准备请使用中性 CAD 能力。"
},
"output_namespace": "ansys",
"input_policy": {
"suffixes": [".step", ".stp", ".x_t", ".x_b", ".iges", ".igs"],

View File

@ -0,0 +1,345 @@
{
"capability": "cad.geometry.prepare@v1",
"display_name": "中性 CAD 几何准备",
"default_enrollment": false,
"lifecycle": {
"status": "active",
"visibility": "standard",
"replacement": null,
"detail": "生成供 ANSYS、Abaqus 等后续任务复用的中性 STEP。"
},
"output_namespace": "cad",
"input_policy": {
"suffixes": [".step", ".stp"],
"max_count": 1,
"max_bytes": 268435456,
"max_total_bytes": 268435456
},
"outputs": {
"geometry": {
"filename": "geometry.step",
"media_type": "model/step",
"relative_path": "geometry.step",
"publish": true,
"required": true
},
"geometry_recipe": {
"filename": "geometry-recipe.json",
"media_type": "application/json",
"relative_path": "geometry-recipe.json",
"publish": true,
"required": true
},
"geometry_manifest": {
"filename": "geometry-manifest.json",
"media_type": "application/json",
"relative_path": "geometry-manifest.json",
"publish": true,
"required": true
},
"geometry_preview": {
"filename": "geometry-preview.png",
"media_type": "image/png",
"relative_path": "geometry-preview.png",
"publish": true,
"required": true
},
"provenance": {
"filename": "provenance.json",
"media_type": "application/json",
"relative_path": ".meta/provenance.json",
"publish": false,
"required": true
}
},
"feature_path": ["operation", "prepare", "type"],
"features": {
"inspect": "1.0.0",
"primitive": "1.0.0",
"recipe": "1.0.0"
},
"summary": {
"title_path": ["operation", "prepare", "title"]
},
"legacy_runtime": null,
"workspace": null,
"request_schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"x-maxBytes": 262144,
"required": ["schema_version", "inputs", "operation", "outputs"],
"additionalProperties": false,
"properties": {
"schema_version": {"const": 1},
"inputs": {
"type": "array",
"maxItems": 1,
"items": {
"type": "object",
"required": ["key", "artifact_id"],
"additionalProperties": false,
"properties": {
"key": {"const": "geometry"},
"artifact_id": {"type": "string", "format": "uuid"}
}
}
},
"operation": {
"type": "object",
"required": ["prepare"],
"additionalProperties": false,
"properties": {
"prepare": {
"oneOf": [
{"$ref": "#/$defs/inspect"},
{"$ref": "#/$defs/primitive_prepare"},
{"$ref": "#/$defs/recipe_prepare"}
]
}
}
},
"outputs": {"type": "array", "maxItems": 0}
},
"$defs": {
"finite": {"type": "number", "minimum": -1000000, "maximum": 1000000},
"positive": {"type": "number", "exclusiveMinimum": 0, "maximum": 1000000},
"nonnegative": {"type": "number", "minimum": 0, "maximum": 1000000},
"vector3": {
"type": "array", "minItems": 3, "maxItems": 3,
"items": {"$ref": "#/$defs/finite"}
},
"direction3": {
"type": "array", "minItems": 3, "maxItems": 3,
"items": {"type": "number", "minimum": -1000000000000, "maximum": 1000000000000}
},
"point3": {
"type": "array", "minItems": 3, "maxItems": 3,
"items": {"$ref": "#/$defs/finite"}
},
"expected_count": {
"type": "integer", "minimum": 1, "maximum": 128,
"description": "Exact selected topology count assertion."
},
"extreme_face": {
"type": "object",
"required": ["type", "axis", "side"],
"additionalProperties": false,
"properties": {
"type": {"const": "extreme_face"},
"axis": {"enum": ["x", "y", "z"]},
"side": {"enum": ["min", "max"]},
"tolerance": {"$ref": "#/$defs/nonnegative"},
"angle_tolerance_degrees": {"type": "number", "minimum": 0, "maximum": 15, "default": 5},
"expected_count": {"$ref": "#/$defs/expected_count"}
}
},
"planar_faces": {
"type": "object",
"required": ["type", "normal", "offset", "tolerance"],
"additionalProperties": false,
"properties": {
"type": {"const": "planar_faces"},
"normal": {"$ref": "#/$defs/direction3"},
"offset": {"$ref": "#/$defs/finite"},
"tolerance": {"$ref": "#/$defs/positive"},
"angle_tolerance_degrees": {"type": "number", "minimum": 0, "maximum": 15, "default": 5},
"expected_count": {"$ref": "#/$defs/expected_count"}
}
},
"surface_faces": {
"type": "object",
"required": ["type", "surface"],
"additionalProperties": false,
"properties": {
"type": {"const": "surface_faces"},
"surface": {"enum": ["plane", "cylinder", "cone", "sphere", "torus", "other"]},
"expected_count": {"$ref": "#/$defs/expected_count"}
}
},
"face_selector": {
"oneOf": [
{"$ref": "#/$defs/extreme_face"},
{"$ref": "#/$defs/planar_faces"},
{"$ref": "#/$defs/surface_faces"}
]
},
"edge_selector": {
"type": "object",
"required": ["type", "curve"],
"additionalProperties": false,
"properties": {
"type": {"const": "curve_edges"},
"curve": {"enum": ["line", "circle", "ellipse", "spline", "other"]},
"expected_count": {"$ref": "#/$defs/expected_count"}
}
},
"solid_selector": {
"type": "object",
"required": ["type"],
"additionalProperties": false,
"properties": {
"type": {"const": "all_solids"},
"expected_count": {"$ref": "#/$defs/expected_count"}
}
},
"region": {
"type": "object",
"required": ["name", "topology", "selector"],
"additionalProperties": false,
"allOf": [
{"if": {"properties": {"topology": {"const": "face"}}, "required": ["topology"]}, "then": {"properties": {"selector": {"$ref": "#/$defs/face_selector"}}}},
{"if": {"properties": {"topology": {"const": "edge"}}, "required": ["topology"]}, "then": {"properties": {"selector": {"$ref": "#/$defs/edge_selector"}}}},
{"if": {"properties": {"topology": {"const": "solid"}}, "required": ["topology"]}, "then": {"properties": {"selector": {"$ref": "#/$defs/solid_selector"}}}}
],
"properties": {
"name": {"type": "string", "minLength": 1, "maxLength": 64, "pattern": "^[A-Za-z][A-Za-z0-9_-]*$"},
"topology": {"enum": ["face", "edge", "solid"]},
"selector": {
"oneOf": [
{"$ref": "#/$defs/face_selector"},
{"$ref": "#/$defs/edge_selector"},
{"$ref": "#/$defs/solid_selector"}
]
}
}
},
"primitive": {
"oneOf": [
{
"type": "object", "required": ["shape", "length", "width", "height"], "additionalProperties": false,
"properties": {"shape": {"const": "box"}, "length": {"$ref": "#/$defs/positive"}, "width": {"$ref": "#/$defs/positive"}, "height": {"$ref": "#/$defs/positive"}}
},
{
"type": "object", "required": ["shape", "radius", "height"], "additionalProperties": false,
"properties": {"shape": {"const": "cylinder"}, "radius": {"$ref": "#/$defs/positive"}, "height": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
},
{
"type": "object", "required": ["shape", "radius", "inner_radius", "height"], "additionalProperties": false,
"properties": {"shape": {"const": "tube"}, "radius": {"$ref": "#/$defs/positive"}, "inner_radius": {"$ref": "#/$defs/positive"}, "height": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
},
{
"type": "object", "required": ["shape", "radius", "height"], "additionalProperties": false,
"properties": {"shape": {"const": "cone"}, "radius": {"$ref": "#/$defs/positive"}, "height": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
},
{
"type": "object", "required": ["shape", "radius1", "radius2", "height"], "additionalProperties": false,
"properties": {"shape": {"const": "frustum"}, "radius1": {"$ref": "#/$defs/nonnegative"}, "radius2": {"$ref": "#/$defs/nonnegative"}, "height": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
},
{
"type": "object", "required": ["shape", "radius"], "additionalProperties": false,
"properties": {"shape": {"const": "sphere"}, "radius": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
},
{
"type": "object", "required": ["shape", "x_min", "x_max", "y_min", "y_max", "z_min", "z_max"], "additionalProperties": false,
"properties": {"shape": {"const": "wedge"}, "x_min": {"$ref": "#/$defs/finite"}, "x_max": {"$ref": "#/$defs/finite"}, "y_min": {"$ref": "#/$defs/finite"}, "y_max": {"$ref": "#/$defs/finite"}, "z_min": {"$ref": "#/$defs/finite"}, "z_max": {"$ref": "#/$defs/finite"}}
},
{
"type": "object", "required": ["shape", "major_radius", "minor_radius"], "additionalProperties": false,
"properties": {"shape": {"const": "torus"}, "major_radius": {"$ref": "#/$defs/positive"}, "minor_radius": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
}
]
},
"profile": {
"type": "object",
"required": ["id", "points"],
"additionalProperties": false,
"properties": {
"id": {"$ref": "#/$defs/name"},
"points": {
"type": "array", "minItems": 3, "maxItems": 512,
"items": {"$ref": "#/$defs/point3"}
}
}
},
"name": {"type": "string", "minLength": 1, "maxLength": 64, "pattern": "^[A-Za-z][A-Za-z0-9_-]*$"},
"step": {
"type": "object",
"required": ["id", "op"],
"additionalProperties": false,
"properties": {
"id": {"$ref": "#/$defs/name"},
"op": {"enum": ["box", "cylinder", "tube", "cone", "frustum", "sphere", "wedge", "torus", "import_step", "extrude", "revolve", "sweep", "loft", "union", "cut", "intersect", "fillet", "chamfer", "shell", "translate", "rotate", "mirror", "linear_pattern", "circular_pattern", "combine"]},
"primitive": {"$ref": "#/$defs/primitive"},
"input": {"const": "geometry"},
"profile": {"$ref": "#/$defs/name"},
"profiles": {"type": "array", "minItems": 2, "maxItems": 32, "uniqueItems": true, "items": {"$ref": "#/$defs/name"}},
"target": {"$ref": "#/$defs/name"},
"targets": {"type": "array", "minItems": 1, "maxItems": 64, "uniqueItems": true, "items": {"$ref": "#/$defs/name"}},
"tools": {"type": "array", "minItems": 1, "maxItems": 64, "uniqueItems": true, "items": {"$ref": "#/$defs/name"}},
"vector": {"$ref": "#/$defs/vector3"},
"axis_start": {"$ref": "#/$defs/point3"},
"axis_end": {"$ref": "#/$defs/point3"},
"angle_degrees": {"type": "number", "exclusiveMinimum": -360, "maximum": 360},
"path": {"type": "array", "minItems": 2, "maxItems": 512, "items": {"$ref": "#/$defs/point3"}},
"selector": {"oneOf": [{"$ref": "#/$defs/face_selector"}, {"$ref": "#/$defs/edge_selector"}]},
"radius": {"$ref": "#/$defs/positive"},
"distance": {"$ref": "#/$defs/positive"},
"thickness": {"$ref": "#/$defs/positive"},
"plane": {"enum": ["XY", "XZ", "YZ"]},
"count": {"type": "integer", "minimum": 2, "maximum": 128},
"spacing": {"$ref": "#/$defs/positive"},
"center": {"$ref": "#/$defs/point3"}
}
},
"base_prepare": {
"type": "object",
"required": ["type", "title", "canonical_unit", "regions"],
"properties": {
"type": {"enum": ["inspect", "primitive", "recipe"]},
"title": {"type": "string", "minLength": 1, "maxLength": 160},
"canonical_unit": {"enum": ["mm", "m"]},
"regions": {"type": "array", "maxItems": 64, "items": {"$ref": "#/$defs/region"}}
}
},
"inspect": {
"allOf": [
{"$ref": "#/$defs/base_prepare"},
{
"type": "object",
"required": ["type", "title", "canonical_unit", "regions", "geometry"],
"additionalProperties": false,
"properties": {
"type": {"const": "inspect"},
"title": true, "canonical_unit": true, "regions": true,
"geometry": {"type": "object", "required": ["input"], "additionalProperties": false, "properties": {"input": {"const": "geometry"}}}
}
}
]
},
"primitive_prepare": {
"allOf": [
{"$ref": "#/$defs/base_prepare"},
{
"type": "object",
"required": ["type", "title", "canonical_unit", "regions", "primitive"],
"additionalProperties": false,
"properties": {"type": {"const": "primitive"}, "title": true, "canonical_unit": true, "regions": true, "primitive": {"$ref": "#/$defs/primitive"}}
}
]
},
"recipe_prepare": {
"allOf": [
{"$ref": "#/$defs/base_prepare"},
{
"type": "object",
"required": ["type", "title", "canonical_unit", "regions", "recipe"],
"additionalProperties": false,
"properties": {
"type": {"const": "recipe"}, "title": true, "canonical_unit": true, "regions": true,
"recipe": {
"type": "object",
"required": ["profiles", "steps", "result"],
"additionalProperties": false,
"properties": {
"profiles": {"type": "array", "maxItems": 64, "items": {"$ref": "#/$defs/profile"}},
"steps": {"type": "array", "minItems": 1, "maxItems": 128, "items": {"$ref": "#/$defs/step"}},
"result": {"$ref": "#/$defs/name"}
}
}
}
}
]
}
}
}
}

View File

@ -0,0 +1,121 @@
{
"capability": "freecad.model.author@v1",
"display_name": "FreeCAD 受控建模",
"default_enrollment": false,
"output_namespace": "freecad",
"input_policy": {
"suffixes": [".step", ".stp"],
"max_count": 1,
"max_bytes": 268435456,
"max_total_bytes": 268435456
},
"outputs": {
"project": {"filename": "model.FCStd", "media_type": "application/vnd.freecad", "relative_path": "model.FCStd", "publish": true, "required": true},
"geometry_step": {"filename": "model.step", "media_type": "model/step", "relative_path": "model.step", "publish": true, "required": true},
"preview": {"filename": "model-preview.png", "media_type": "image/png", "relative_path": "model-preview.png", "publish": true, "required": true},
"model_recipe": {"filename": "model-recipe.json", "media_type": "application/json", "relative_path": "model-recipe.json", "publish": true, "required": true},
"model_manifest": {"filename": "model-manifest.json", "media_type": "application/json", "relative_path": "model-manifest.json", "publish": true, "required": true},
"provenance": {"filename": "provenance.json", "media_type": "application/json", "relative_path": ".meta/provenance.json", "publish": false, "required": true}
},
"feature_path": ["operation", "model", "type"],
"features": {"recipe": "1.0.0"},
"summary": {"title_path": ["operation", "model", "title"]},
"legacy_runtime": null,
"workspace": {
"state_output": "project",
"preview_outputs": ["preview"],
"required_adapter_version": "1.0.0"
},
"request_schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"x-maxBytes": 262144,
"required": ["schema_version", "inputs", "operation", "outputs"],
"additionalProperties": false,
"properties": {
"schema_version": {"const": 1},
"inputs": {
"type": "array", "maxItems": 1,
"items": {
"type": "object", "required": ["key", "artifact_id"], "additionalProperties": false,
"properties": {"key": {"const": "geometry"}, "artifact_id": {"type": "string", "format": "uuid"}}
}
},
"operation": {
"type": "object", "required": ["model"], "additionalProperties": false,
"properties": {"model": {"$ref": "#/$defs/model"}}
},
"outputs": {"type": "array", "maxItems": 0}
},
"$defs": {
"identifier": {"type": "string", "pattern": "^[A-Za-z][A-Za-z0-9_-]{0,63}$"},
"finite": {"type": "number", "minimum": -1000000, "maximum": 1000000},
"positive": {"type": "number", "exclusiveMinimum": 0, "maximum": 1000000},
"parameterRef": {
"type": "object", "required": ["parameter"], "additionalProperties": false,
"properties": {"parameter": {"$ref": "#/$defs/identifier"}}
},
"numberValue": {"oneOf": [{"$ref": "#/$defs/finite"}, {"$ref": "#/$defs/parameterRef"}]},
"positiveValue": {"oneOf": [{"$ref": "#/$defs/positive"}, {"$ref": "#/$defs/parameterRef"}]},
"vector3": {"type": "array", "minItems": 3, "maxItems": 3, "items": {"$ref": "#/$defs/numberValue"}},
"resultRef": {"$ref": "#/$defs/identifier"},
"box": {
"type": "object", "required": ["id", "op", "length", "width", "height"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "box"}, "length": {"$ref": "#/$defs/positiveValue"}, "width": {"$ref": "#/$defs/positiveValue"}, "height": {"$ref": "#/$defs/positiveValue"}}
},
"cylinder": {
"type": "object", "required": ["id", "op", "radius", "height"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "cylinder"}, "radius": {"$ref": "#/$defs/positiveValue"}, "height": {"$ref": "#/$defs/positiveValue"}, "angle_degrees": {"$ref": "#/$defs/positiveValue"}}
},
"cone": {
"type": "object", "required": ["id", "op", "radius1", "radius2", "height"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "cone"}, "radius1": {"$ref": "#/$defs/numberValue"}, "radius2": {"$ref": "#/$defs/numberValue"}, "height": {"$ref": "#/$defs/positiveValue"}, "angle_degrees": {"$ref": "#/$defs/positiveValue"}}
},
"sphere": {
"type": "object", "required": ["id", "op", "radius"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "sphere"}, "radius": {"$ref": "#/$defs/positiveValue"}, "angle1_degrees": {"$ref": "#/$defs/numberValue"}, "angle2_degrees": {"$ref": "#/$defs/numberValue"}, "angle3_degrees": {"$ref": "#/$defs/positiveValue"}}
},
"torus": {
"type": "object", "required": ["id", "op", "major_radius", "minor_radius"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "torus"}, "major_radius": {"$ref": "#/$defs/positiveValue"}, "minor_radius": {"$ref": "#/$defs/positiveValue"}, "angle1_degrees": {"$ref": "#/$defs/numberValue"}, "angle2_degrees": {"$ref": "#/$defs/numberValue"}, "angle3_degrees": {"$ref": "#/$defs/positiveValue"}}
},
"importStep": {
"type": "object", "required": ["id", "op", "input"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "import_step"}, "input": {"const": "geometry"}}
},
"boolean": {
"type": "object", "required": ["id", "op", "target", "tools"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"enum": ["union", "cut", "intersect"]}, "target": {"$ref": "#/$defs/resultRef"}, "tools": {"type": "array", "minItems": 1, "maxItems": 32, "uniqueItems": true, "items": {"$ref": "#/$defs/resultRef"}}}
},
"translate": {
"type": "object", "required": ["id", "op", "target", "vector"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "translate"}, "target": {"$ref": "#/$defs/resultRef"}, "vector": {"$ref": "#/$defs/vector3"}}
},
"rotate": {
"type": "object", "required": ["id", "op", "target", "axis_point", "axis_direction", "angle_degrees"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "rotate"}, "target": {"$ref": "#/$defs/resultRef"}, "axis_point": {"$ref": "#/$defs/vector3"}, "axis_direction": {"$ref": "#/$defs/vector3"}, "angle_degrees": {"$ref": "#/$defs/numberValue"}}
},
"combine": {
"type": "object", "required": ["id", "op", "targets"], "additionalProperties": false,
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "combine"}, "targets": {"type": "array", "minItems": 1, "maxItems": 64, "uniqueItems": true, "items": {"$ref": "#/$defs/resultRef"}}}
},
"step": {"oneOf": [{"$ref": "#/$defs/box"}, {"$ref": "#/$defs/cylinder"}, {"$ref": "#/$defs/cone"}, {"$ref": "#/$defs/sphere"}, {"$ref": "#/$defs/torus"}, {"$ref": "#/$defs/importStep"}, {"$ref": "#/$defs/boolean"}, {"$ref": "#/$defs/translate"}, {"$ref": "#/$defs/rotate"}, {"$ref": "#/$defs/combine"}]},
"preview": {
"type": "object", "additionalProperties": false,
"properties": {"width": {"type": "integer", "minimum": 320, "maximum": 1600}, "height": {"type": "integer", "minimum": 240, "maximum": 1600}, "background": {"type": "string", "pattern": "^#[0-9A-Fa-f]{6}$"}}
},
"model": {
"type": "object", "required": ["type", "recipe_version", "title", "unit", "steps", "result"], "additionalProperties": false,
"properties": {
"type": {"const": "recipe"}, "recipe_version": {"const": 1},
"title": {"type": "string", "minLength": 1, "maxLength": 160},
"unit": {"const": "mm"},
"parameters": {"type": "object", "maxProperties": 64, "propertyNames": {"$ref": "#/$defs/identifier"}, "additionalProperties": {"$ref": "#/$defs/finite"}},
"steps": {"type": "array", "minItems": 1, "maxItems": 128, "items": {"$ref": "#/$defs/step"}},
"result": {"$ref": "#/$defs/resultRef"},
"preview": {"$ref": "#/$defs/preview"}
}
}
}
}
}

View File

@ -55,18 +55,30 @@ test("composer uses one Codex-style action button for stop and queued send", ()
assert.match(html, /\.msg\.queued/);
assert.match(chat, /card\.className = "msg user queued"/);
assert.match(chat, /function composerHasPayload\(\)/);
assert.match(chat, /function isCurrentTaskRunActive\(\)/);
assert.match(chat, /btn\.textContent = "停止"/);
assert.match(chat, /if \(composerHasPayload\(\)\)[\s\S]*queueCurrentMessage\(\)/);
assert.match(chat, /_actionMode === "loading" \|\| _actionMode === "submitting"[\s\S]*btn\.disabled = true/);
assert.match(chat, /setActionMode\("submitting"\)/);
assert.match(chat, /if \(state\.taskId === taskId\) \$\("chat-input"\)\.value = ""/);
assert.match(chat, /if \(state\.taskId === taskId\) setActionMode\("streaming"\)/);
assert.match(chat, /const activeRun = isCurrentTaskRunActive\(\)/);
assert.match(chat, /else if \(activeRun\)[\s\S]*cancelCurrentTask\(\)/);
assert.match(chat, /function queueCurrentMessage\(\)/);
assert.match(chat, /async function dispatchNextQueuedMessage\(taskId\)/);
assert.match(chat, /void dispatchNextQueuedMessage\(ctx\.taskId\)/);
assert.match(chat, /removeQueuedMessage\([\s\S]*streamSse\(r\.events_url, run\)/);
});
test("heavy execution queue exposes the blocking reason", () => {
const chat = readFileSync(new URL("../web/static/js/chat.js", import.meta.url), "utf8");
assert.match(chat, /t === "execution_queue"/);
assert.match(chat, /等待执行容量(当前用户/);
assert.match(chat, /等待执行容量(整机/);
assert.match(chat, /等待执行容量(宿主内存压力)/);
assert.match(chat, /dataset\.runningLabel/);
});
test("automatic title polling starts before a long-running response finishes", () => {
const chat = readFileSync(new URL("../web/static/js/chat.js", import.meta.url), "utf8");
const sendMessage = chat.indexOf("async function sendMessage(");

View File

@ -5,6 +5,7 @@ import test from "node:test";
const filesJs = fs.readFileSync("web/static/js/files.js", "utf8");
const layoutJs = fs.readFileSync("web/static/js/layout.js", "utf8");
const jobsJs = fs.readFileSync("web/static/js/software_jobs.js", "utf8");
const previewJs = fs.readFileSync("web/static/js/preview.js", "utf8");
const pageHtml = fs.readFileSync("web/static/dev.html", "utf8");
test("file menu exposes an explicit confirmed replacement action", () => {
@ -48,6 +49,22 @@ test("software jobs share the embed-safe right workspace", () => {
assert.match(jobsJs, /作业列表加载失败,请点击刷新重试/);
});
test("current task software jobs render in a dedicated result rail", () => {
assert.match(pageHtml, /id="software-result-rail"/);
assert.match(pageHtml, /id="mobile-software-results"/);
assert.match(jobsJs, /task_id: taskJobsTaskId/);
assert.match(jobsJs, /renderTaskResultRail/);
assert.match(jobsJs, /task-selection-changed/);
assert.match(jobsJs, /openSoftwarePreviewGallery/);
});
test("software preview gallery uses authenticated fetch and hides artifact download", () => {
assert.match(previewJs, /export async function openSoftwarePreviewGallery/);
assert.match(previewJs, /"Authorization": "Bearer " \+ state\.token/);
assert.match(previewJs, /\$\("fp-download"\)\.hidden = true/);
assert.match(pageHtml, /software-preview-gallery/);
});
test("mobile navigation distinguishes list, files, and software jobs", () => {
assert.match(pageHtml, /id="mv-tab-left"[^>]*>\s*列表/);
assert.match(pageHtml, /id="mv-tab-right"[^>]*data-right-tab="files"/);

View File

@ -78,6 +78,11 @@ test("renders completed mermaid blocks with strict local configuration", async (
assert.equal(calls.initialize.length, 1);
assert.equal(calls.initialize[0].securityLevel, "strict");
assert.equal(calls.initialize[0].startOnLoad, false);
assert.equal(calls.initialize[0].theme, "base");
assert.equal(calls.initialize[0].themeVariables.primaryColor, "#eaf2ff");
assert.equal(calls.initialize[0].themeVariables.secondaryColor, "#e8f7f3");
assert.equal(calls.initialize[0].themeVariables.tertiaryColor, "#fff4df");
assert.equal(calls.initialize[0].themeVariables.pie5, "#d96c75");
assert.equal(calls.render[0].source, "flowchart LR\nA --> B");
assert.equal(fixture.pre.replacement.className, "mermaid-diagram");
assert.equal(fixture.pre.replacement.children[0].innerHTML, "<svg viewBox='0 0 10 10'></svg>");

View File

@ -0,0 +1,48 @@
"""Admin 存储快照的配额、回收站与物理合计口径。"""
from __future__ import annotations
import sys
import unittest
from datetime import datetime, timezone
from pathlib import Path
from unittest.mock import MagicMock, patch
from uuid import uuid4
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from web.admin import _storage_page
class AdminStorageTests(unittest.TestCase):
def test_storage_page_exposes_trash_and_physical_total(self) -> None:
uid = uuid4()
scanned = datetime(2026, 8, 28, tzinfo=timezone.utc)
totals_result = MagicMock()
totals_result.one.return_value = (1, 100, 40, 2, 1)
rows_result = MagicMock()
rows_result.all.return_value = [
(uid, "user@example.com", "测试用户", "tester", 100, 2, 40, 1, scanned)
]
session = MagicMock()
session.execute.side_effect = [totals_result, rows_result]
with patch("core.agent_builder.load_config", return_value={"quotas": {}}):
page = _storage_page(session, 0, 20)
row = page["rows"][0]
self.assertEqual(row["bytes_used"], 100)
self.assertEqual(row["trash_bytes"], 40)
self.assertEqual(row["trash_file_count"], 1)
self.assertEqual(row["physical_bytes"], 140)
self.assertEqual(page["total"], 1)
self.assertEqual(page["totals"], {
"bytes_used": 100,
"trash_bytes": 40,
"physical_bytes": 140,
"file_count": 2,
"trash_file_count": 1,
})
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,21 @@
from sqlalchemy.dialects.postgresql import dialect, insert as pg_insert
from sqlalchemy import text
from core.storage.models import Artifact
def test_artifact_upsert_partial_index_predicate_is_literal():
statement = pg_insert(Artifact).values(
user_id="00000000-0000-0000-0000-000000000001",
origin_task_id="00000000-0000-0000-0000-000000000002",
current_path="task/report.docx",
label="report",
).on_conflict_do_update(
index_elements=[Artifact.user_id, Artifact.current_path],
index_where=text("status = 'active'"),
set_={"label": "report"},
)
sql = str(statement.compile(dialect=dialect()))
assert "WHERE status = 'active'" in sql
assert "WHERE status = %(" not in sql

View File

@ -3,7 +3,11 @@ import unittest
from pathlib import Path
from core.artifacts import ArtifactPathError
from core.attachments import content_for_model, normalize_attachment_refs
from core.attachments import (
content_for_model,
materialize_native_images,
normalize_attachment_refs,
)
class AttachmentRefTests(unittest.TestCase):
@ -47,3 +51,27 @@ class AttachmentRefTests(unittest.TestCase):
content_for_model("", refs, "实验分析"),
"[用户上传的参考图] 实验分析/figures/a.png",
)
def test_materializes_native_image_without_persisting_internal_fields(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
wd = root / "任务"
wd.mkdir()
(wd / "a.png").write_bytes(b"png")
messages = [{
"role": "user",
"content": "请分析\n\n[用户上传的参考图] 任务/a.png",
"_attachment_refs": [{"path": "a.png", "kind": "image"}],
}]
prepared, count = materialize_native_images(
messages, enabled=True, working_dir=wd, user_root=root,
)
self.assertEqual(count, 1)
self.assertNotIn("_attachment_refs", prepared[0])
self.assertEqual(prepared[0]["content"][0]["type"], "text")
self.assertEqual(prepared[0]["content"][1]["type"], "image_url")
self.assertTrue(
prepared[0]["content"][1]["image_url"]["url"].startswith(
"data:image/png;base64,"
)
)

View File

@ -138,6 +138,10 @@ class BlenderContractTests(unittest.TestCase):
tuple(f"preview_{index:02d}" for index in range(1, 9)),
)
self.assertEqual(contract.required_adapter_version(normalized), "1.0.0")
self.assertEqual(
contract.summarize(normalized)["title"],
"水泥实验室设备布置",
)
self.assertEqual(
set(contract.expected_outputs(normalized)),
{

View File

@ -10,6 +10,7 @@
用两个 broker 实例共享同一 FakeServer 模拟蓝绿两进程
"""
import asyncio
import threading
import unittest
from uuid import uuid4
@ -134,8 +135,36 @@ class TestRedisBroker(unittest.TestCase):
b.emit(tid, {"type": "text", "delta": "x"}) # 不抛
self.assertFalse(b.is_cancelled(tid)) # 降级 False
b.request_cancel(tid) # 不抛
self.assertFalse(b.is_cancelled(tid))
b.close(tid) # 不抛
def test_cancel_uses_local_signal_while_redis_write_is_blocked(self):
"""Redis 卡顿时,同实例 run 无需等待网络超时即可看到停止。"""
class SlowBroken:
def __init__(self):
self.writing = threading.Event()
self.release = threading.Event()
def set(self, *args, **kwargs):
self.writing.set()
self.release.wait(2)
raise ConnectionError("redis down")
def get(self, *args, **kwargs):
raise ConnectionError("redis down")
redis = SlowBroken()
b = RedisRunBroker("redis://fake:6379/0", sync_client=redis,
async_client=fakeredis.aioredis.FakeRedis(decode_responses=True))
tid = uuid4()
worker = threading.Thread(target=b.request_cancel, args=(tid,), daemon=True)
worker.start()
self.assertTrue(redis.writing.wait(1))
self.assertTrue(b.is_cancelled(tid))
redis.release.set()
worker.join(1)
self.assertFalse(b.is_cancelled(tid))
def test_bind_loop_fail_fast_when_unreachable(self):
class Boom:
def ping(self):

389
tests/test_cad_adapter.py Normal file
View File

@ -0,0 +1,389 @@
from __future__ import annotations
import importlib.util
import json
import math
import tempfile
import unittest
from pathlib import Path
from uuid import uuid4
from core.software_contracts import SoftwareContractError, get_contract
ROOT = Path(__file__).resolve().parents[1]
WORKER_PATH = (
ROOT / "windows-node" / "adapters" / "cad.geometry.prepare@v1" / "worker.py"
)
SPEC = importlib.util.spec_from_file_location("zcbot_cad_worker", WORKER_PATH)
assert SPEC and SPEC.loader
worker = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(worker)
HAS_CADQUERY = importlib.util.find_spec("cadquery") is not None
def _request(feature: str, definition: dict, *, regions: list[dict] | None = None) -> dict:
prepare = {
"type": feature,
"title": f"CAD {feature}",
"canonical_unit": "mm",
"regions": regions or [],
}
prepare.update(definition)
return {
"schema_version": 1,
"inputs": [],
"operation": {"prepare": prepare},
"outputs": [],
}
def _primitive(shape: str, **parameters) -> dict:
return _request("primitive", {"primitive": {"shape": shape, **parameters}})
class CadContractTests(unittest.TestCase):
def test_contract_exposes_fixed_features_outputs_and_quotas(self) -> None:
contract = get_contract("cad.geometry.prepare@v1")
self.assertTrue(contract.ordinarily_visible)
self.assertEqual(set(contract.features), {"inspect", "primitive", "recipe"})
self.assertEqual(contract.input_policy["suffixes"], [".step", ".stp"])
request = _primitive("box", length=10, width=20, height=30)
normalized, digest = contract.normalize_request(request)
self.assertEqual(contract.feature(normalized), "primitive")
self.assertEqual(len(digest), 64)
self.assertEqual(
set(contract.expected_outputs(normalized)),
{"geometry", "geometry_recipe", "geometry_manifest", "geometry_preview", "provenance"},
)
geometry_output = contract.outputs["geometry"]
self.assertEqual(geometry_output.filename, "geometry.step")
ansys = get_contract("ansys.mechanical.static_structural@v2")
self.assertIn(Path(geometry_output.filename).suffix, ansys.input_policy["suffixes"])
def test_contract_rejects_scripts_paths_nonfinite_and_missing_units(self) -> None:
contract = get_contract("cad.geometry.prepare@v1")
for key, value in (("script", "anything"), ("path", "C:/secret.step")):
request = _primitive("box", length=10, width=20, height=30)
request["operation"]["prepare"][key] = value
with self.assertRaises(SoftwareContractError):
contract.normalize_request(request)
request = _primitive("box", length=math.inf, width=20, height=30)
with self.assertRaises(SoftwareContractError):
contract.normalize_request(request)
request = _primitive("box", length=math.nan, width=20, height=30)
with self.assertRaisesRegex(SoftwareContractError, "non-finite"):
contract.normalize_request(request)
request = _primitive("box", length=10, width=20, height=30)
del request["operation"]["prepare"]["canonical_unit"]
with self.assertRaises(SoftwareContractError):
contract.normalize_request(request)
for primitive in (
{"shape": "box", "length": 10, "width": 20},
{"shape": "box", "length": 10, "width": 20, "height": 30, "radius": 2},
):
with self.assertRaises(SoftwareContractError):
contract.normalize_request(_request("primitive", {"primitive": primitive}))
def test_ansys_and_cad_planar_selector_contracts_stay_consistent(self) -> None:
cad = get_contract("cad.geometry.prepare@v1").request_schema["$defs"]
ansys = get_contract("ansys.mechanical.static_structural@v2").request_schema["$defs"]
def resolved(value: object, definitions: dict) -> object:
if isinstance(value, list):
return [resolved(item, definitions) for item in value]
if not isinstance(value, dict):
return value
if set(value) == {"$ref"}:
return resolved(definitions[value["$ref"].split("/")[-1]], definitions)
return {
key: resolved(item, definitions)
for key, item in value.items()
if key != "description"
}
for name in ("extreme_face", "planar_faces"):
ansys_name = f"{name}_scope"
self.assertEqual(
resolved(cad[name], cad),
resolved(ansys[ansys_name], ansys),
)
def test_ansys_inspect_is_hidden_diagnostic_with_replacement(self) -> None:
contract = get_contract("ansys.geometry.inspect@v1")
self.assertFalse(contract.ordinarily_visible)
self.assertEqual(contract.lifecycle.status, "deprecated")
self.assertEqual(contract.lifecycle.visibility, "diagnostic_only")
self.assertEqual(contract.lifecycle.replacement, "cad.geometry.prepare@v1")
@unittest.skipUnless(HAS_CADQUERY, "fixed CAD runtime is not installed in project test environment")
class CadKernelTests(unittest.TestCase):
def test_all_required_primitives_have_numeric_oracles(self) -> None:
cases = [
({"shape": "box", "length": 2, "width": 3, "height": 5}, 30.0),
({"shape": "cylinder", "radius": 2, "height": 5}, 20 * math.pi),
({"shape": "tube", "radius": 3, "inner_radius": 2, "height": 5}, 25 * math.pi),
({"shape": "cone", "radius": 3, "height": 4}, 12 * math.pi),
({"shape": "frustum", "radius1": 3, "radius2": 1, "height": 4}, 52 * math.pi / 3),
({"shape": "sphere", "radius": 3}, 36 * math.pi),
({"shape": "wedge", "x_min": 0, "x_max": 4, "y_min": 0, "y_max": 3, "z_min": 0, "z_max": 2}, 12.0),
({"shape": "torus", "major_radius": 5, "minor_radius": 2}, 40 * math.pi**2),
]
for spec, expected in cases:
with self.subTest(spec["shape"]):
shape = worker._primitive(spec)
self.assertAlmostEqual(shape.Volume(), expected, delta=max(1e-8, expected * 1e-8))
def test_recipe_builds_plate_hole_notch_i_beam_and_multi_body(self) -> None:
plate = {
"profiles": [],
"steps": [
{"id": "plate", "op": "box", "primitive": {"shape": "box", "length": 100, "width": 60, "height": 10}},
{"id": "hole", "op": "cylinder", "primitive": {"shape": "cylinder", "radius": 10, "height": 10}},
{"id": "centered_hole", "op": "translate", "target": "hole", "vector": [50, 30, 0]},
{"id": "perforated", "op": "cut", "target": "plate", "tools": ["centered_hole"]},
{"id": "notch", "op": "box", "primitive": {"shape": "box", "length": 20, "width": 20, "height": 10}},
{"id": "placed_notch", "op": "translate", "target": "notch", "vector": [80, 40, 0]},
{"id": "result", "op": "cut", "target": "perforated", "tools": ["placed_notch"]},
],
"result": "result",
}
shape = worker._recipe(plate, None, "mm")
self.assertAlmostEqual(
shape.Volume(),
100 * 60 * 10 - math.pi * 10**2 * 10 - 20 * 20 * 10,
delta=1e-6,
)
i_beam = {
"profiles": [{"id": "i", "points": [[0, 0, 0], [60, 0, 0], [60, 10, 0], [35, 10, 0], [35, 90, 0], [60, 90, 0], [60, 100, 0], [0, 100, 0], [0, 90, 0], [25, 90, 0], [25, 10, 0], [0, 10, 0]]}],
"steps": [{"id": "beam", "op": "extrude", "profile": "i", "vector": [0, 0, 200]}],
"result": "beam",
}
beam = worker._recipe(i_beam, None, "mm")
self.assertAlmostEqual(beam.Volume(), 2000 * 200, delta=1e-6)
multi = {
"profiles": [],
"steps": [
{"id": "coupon", "op": "box", "primitive": {"shape": "box", "length": 10, "width": 10, "height": 20}},
{"id": "top", "op": "translate", "target": "coupon", "vector": [0, 0, 30]},
{"id": "assembly", "op": "combine", "targets": ["coupon", "top"]},
],
"result": "assembly",
}
assembly = worker._recipe(multi, None, "mm")
self.assertEqual(len(assembly.Solids()), 2)
def test_regions_manifest_and_step_roundtrip_are_deterministic(self) -> None:
shape = worker._primitive({"shape": "box", "length": 10, "width": 20, "height": 30})
regions = [{"name": "fixed", "topology": "face", "selector": {"type": "extreme_face", "axis": "z", "side": "min", "expected_count": 1}}]
resolved = worker._regions(shape, regions)
prepare = _primitive("box", length=10, width=20, height=30)["operation"]["prepare"]
first = worker._manifest(shape, prepare, resolved, None)
second = worker._manifest(shape, prepare, resolved, None)
self.assertEqual(first, second)
self.assertEqual(first["topology"]["solids"], 1)
self.assertEqual(first["regions"][0]["count"], 1)
self.assertEqual(first["bounding_box"]["size"], [10.0, 20.0, 30.0])
self.assertAlmostEqual(first["area"], 2200.0, delta=1e-8)
self.assertEqual(first["center_of_mass"], [5.0, 10.0, 15.0])
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "geometry.step"
worker._export_step(shape, path, "mm")
worker._require_step_unit(path)
import cadquery as cq
reopened = cq.importers.importStep(str(path), unit="MM").val()
self.assertAlmostEqual(reopened.Volume(), 6000.0, delta=1e-6)
def test_advanced_recipe_operations_fail_loudly_or_produce_valid_solids(self) -> None:
cases = {
"revolve": {
"profiles": [{"id": "section", "points": [[2, 0, 0], [3, 0, 0], [3, 0, 5], [2, 0, 5]]}],
"steps": [{"id": "result", "op": "revolve", "profile": "section", "axis_start": [0, 0, 0], "axis_end": [0, 0, 1], "angle_degrees": 360}],
"result": "result",
},
"sweep": {
"profiles": [{"id": "section", "points": [[0, -1, -1], [0, 1, -1], [0, 1, 1], [0, -1, 1]]}],
"steps": [{"id": "result", "op": "sweep", "profile": "section", "path": [[0, 0, 0], [10, 0, 0]]}],
"result": "result",
},
"loft": {
"profiles": [
{"id": "a", "points": [[-2, -2, 0], [2, -2, 0], [2, 2, 0], [-2, 2, 0]]},
{"id": "b", "points": [[-1, -1, 10], [1, -1, 10], [1, 1, 10], [-1, 1, 10]]},
],
"steps": [{"id": "result", "op": "loft", "profiles": ["a", "b"]}],
"result": "result",
},
"fillet": {
"profiles": [],
"steps": [
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 10, "width": 10, "height": 10}},
{"id": "result", "op": "fillet", "target": "base", "selector": {"type": "curve_edges", "curve": "line", "expected_count": 12}, "radius": 1},
],
"result": "result",
},
"chamfer": {
"profiles": [],
"steps": [
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 10, "width": 10, "height": 10}},
{"id": "result", "op": "chamfer", "target": "base", "selector": {"type": "curve_edges", "curve": "line", "expected_count": 12}, "distance": 0.5},
],
"result": "result",
},
"shell": {
"profiles": [],
"steps": [
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 10, "width": 10, "height": 10}},
{"id": "result", "op": "shell", "target": "base", "selector": {"type": "extreme_face", "axis": "z", "side": "max", "expected_count": 1}, "thickness": 1},
],
"result": "result",
},
"mirror": {
"profiles": [],
"steps": [
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 2, "width": 3, "height": 4}},
{"id": "result", "op": "mirror", "target": "base", "plane": "YZ"},
],
"result": "result",
},
"linear_pattern": {
"profiles": [],
"steps": [
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 2, "width": 2, "height": 2}},
{"id": "result", "op": "linear_pattern", "target": "base", "vector": [1, 0, 0], "count": 3, "spacing": 4},
],
"result": "result",
},
}
for name, recipe in cases.items():
with self.subTest(name):
shape = worker._recipe(recipe, None, "mm")
self.assertTrue(shape.isValid())
self.assertGreater(shape.Volume(), 0)
invalid = cases["fillet"]
invalid = json.loads(json.dumps(invalid))
invalid["steps"][1]["selector"]["expected_count"] = 11
with self.assertRaisesRegex(RuntimeError, "GEOMETRY_SCOPE_COUNT_MISMATCH"):
worker._recipe(invalid, None, "mm")
def test_flange_circular_holes_and_circular_pattern(self) -> None:
recipe = {
"profiles": [],
"steps": [
{"id": "flange", "op": "cylinder", "primitive": {"shape": "cylinder", "radius": 20, "height": 5}},
{"id": "hole", "op": "cylinder", "primitive": {"shape": "cylinder", "radius": 2, "height": 5}},
{"id": "offset_hole", "op": "translate", "target": "hole", "vector": [12, 0, 0]},
{"id": "holes", "op": "circular_pattern", "target": "offset_hole", "axis_start": [0, 0, 0], "axis_end": [0, 0, 1], "count": 6, "angle_degrees": 360},
{"id": "result", "op": "cut", "target": "flange", "tools": ["holes"]},
],
"result": "result",
}
shape = worker._recipe(recipe, None, "mm")
expected = math.pi * 20**2 * 5 - 6 * math.pi * 2**2 * 5
self.assertAlmostEqual(shape.Volume(), expected, delta=expected * 1e-8)
def test_full_job_emits_fixed_package_and_preview(self) -> None:
request = _request(
"primitive",
{"primitive": {"shape": "cylinder", "radius": 4, "height": 12}},
regions=[{"name": "ends", "topology": "face", "selector": {"type": "surface_faces", "surface": "plane", "expected_count": 2}}],
)
get_contract("cad.geometry.prepare@v1").normalize_request(request)
def execute() -> dict:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "request").mkdir()
(root / "request" / "request.json").write_text(
json.dumps(
{
"job_id": str(uuid4()),
"lease_id": str(uuid4()),
"request_digest": "a" * 64,
"request": request,
}
),
encoding="utf-8",
)
artifacts = worker.run(root)
self.assertEqual(
{item["filename"] for item in artifacts}, set(worker.OUTPUT_MEDIA)
)
self.assertGreater(
(root / "output" / "geometry-preview.png").stat().st_size,
1000,
)
return json.loads(
(root / "output" / "geometry-manifest.json").read_text(
encoding="utf-8"
)
)
first = execute()
second = execute()
self.assertEqual(first, second)
self.assertAlmostEqual(first["volume"], math.pi * 4**2 * 12, delta=1e-6)
def test_external_step_inspect_and_recipe_import(self) -> None:
import cadquery as cq
source = cq.Solid.makeBox(7, 11, 13)
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
input_dir = root / "input" / "geometry"
request_dir = root / "request"
input_dir.mkdir(parents=True)
request_dir.mkdir()
source_path = input_dir / "external.step"
worker._export_step(source, source_path, "mm")
for feature, definition in (
("inspect", {"geometry": {"input": "geometry"}}),
(
"recipe",
{
"recipe": {
"profiles": [],
"steps": [
{"id": "imported", "op": "import_step", "input": "geometry"},
{"id": "moved", "op": "translate", "target": "imported", "vector": [20, 0, 0]},
{"id": "result", "op": "combine", "targets": ["imported", "moved"]},
],
"result": "result",
}
},
),
):
request = _request(feature, definition)
request["inputs"] = [
{"key": "geometry", "artifact_id": str(uuid4())}
]
(request_dir / "request.json").write_text(
json.dumps(
{
"job_id": str(uuid4()),
"lease_id": str(uuid4()),
"request_digest": "b" * 64,
"request": request,
}
),
encoding="utf-8",
)
artifacts = worker.run(root)
self.assertEqual(len(artifacts), 5)
manifest = json.loads(
(root / "output" / "geometry-manifest.json").read_text(
encoding="utf-8"
)
)
expected_volume = 7 * 11 * 13 * (2 if feature == "recipe" else 1)
self.assertAlmostEqual(
manifest["volume"], expected_volume, delta=1e-6
)
if __name__ == "__main__":
unittest.main()

View File

@ -58,7 +58,7 @@ class ContextCompactionTests(unittest.TestCase):
prepared, stats = prepare_messages_with_stats(
messages,
reasoning_replay="tool_calls",
reasoning_replay="tool_turn",
compact_threshold_chars=10_000,
)
@ -78,7 +78,7 @@ class ContextCompactionTests(unittest.TestCase):
}, {"role": "tool", "tool_call_id": "tc1", "content": "ok"}]
prepared = prepare_messages_for_llm(
messages, reasoning_replay="tool_calls",
messages, reasoning_replay="tool_turn",
)
self.assertEqual(prepared[0]["reasoning_content"], "nested reasoning")
@ -91,10 +91,54 @@ class ContextCompactionTests(unittest.TestCase):
"provider_specific_fields": {"signature": "opaque"},
}
prepared = prepare_messages_for_llm([message], reasoning_replay="all")
prepared = prepare_messages_for_llm(
[message], reasoning_replay="conversation"
)
self.assertEqual(prepared[0], message)
def test_conversation_policy_strips_reasoning_from_other_model(self) -> None:
messages = [{
"role": "assistant",
"content": "answer",
"reasoning_content": "deepseek reasoning",
"_model_profile": "deepseek_v4.flash",
}]
prepared = prepare_messages_for_llm(
messages,
reasoning_replay="conversation",
target_model_profile="glm.flash53",
)
self.assertNotIn("reasoning_content", prepared[0])
self.assertNotIn("_model_profile", prepared[0])
def test_tool_turn_keeps_only_reasoning_after_latest_user(self) -> None:
call = lambda cid, reasoning: {
"role": "assistant", "content": None,
"reasoning_content": reasoning,
"_model_profile": "deepseek_v4.flash",
"tool_calls": [{
"id": cid, "type": "function",
"function": {"name": "shell", "arguments": "{}"},
}],
}
messages = [
{"role": "user", "content": "old"},
call("old", "old reasoning"),
{"role": "tool", "tool_call_id": "old", "content": "old result"},
{"role": "assistant", "content": "done"},
{"role": "user", "content": "new"},
call("new", "new reasoning"),
{"role": "tool", "tool_call_id": "new", "content": "new result"},
]
prepared = prepare_messages_for_llm(
messages,
reasoning_replay="tool_turn",
target_model_profile="deepseek_v4.flash",
)
self.assertNotIn("reasoning_content", prepared[1])
self.assertEqual(prepared[5]["reasoning_content"], "new reasoning")
def test_preserves_system_and_recent_messages(self) -> None:
messages = [
{"role": "system", "content": "rules"},

View File

@ -146,7 +146,7 @@ class _FakeLLM:
_FAKE_CAPS = SimpleNamespace(
reliable_context=100, # 极小上限 → 体量必然超阈值,不用造几十万字符
family="deepseek_v4", variant="flash",
reasoning_replay="tool_calls",
reasoning_replay="tool_turn",
input_cny_per_mtoken=1.0, output_cny_per_mtoken=2.0,
)
@ -218,16 +218,24 @@ class MaybeFoldTests(unittest.TestCase):
sess = self._session()
before = list(sess.messages)
events = []
with patch.object(cf, "persist_fold") as persist:
user_id = uuid4()
with patch.object(cf, "persist_fold") as persist, \
patch.object(cf, "record_context_fold_failure") as failure:
result = cf.maybe_fold(
sess, _FakeLLM(" "), _FAKE_CAPS, user_id=uuid4(), emit=events.append
sess, _FakeLLM(" "), _FAKE_CAPS, user_id=user_id, emit=events.append
)
self.assertIsNone(result)
persist.assert_not_called()
failure.assert_called_once_with(
task_id=sess.task_id,
user_id=user_id,
model_profile="deepseek_v4.flash",
reason="empty_summary",
)
self.assertEqual(sess.messages, before) # 内存零污染
self.assertTrue(any(e.get("type") == "warn" for e in events))
def test_fold_input_strips_final_reasoning_but_keeps_tool_reasoning(self) -> None:
def test_fold_input_strips_reasoning_from_completed_turns(self) -> None:
sess = self._session()
sess.messages[2]["reasoning_content"] = "final reasoning"
sess.messages[4] = {
@ -254,7 +262,7 @@ class MaybeFoldTests(unittest.TestCase):
final_msg = next(m for m in sent if m.get("content", "").startswith("a000"))
tool_msg = next(m for m in sent if m.get("tool_calls"))
self.assertNotIn("reasoning_content", final_msg)
self.assertEqual(tool_msg["reasoning_content"], "tool reasoning")
self.assertNotIn("reasoning_content", tool_msg)
if __name__ == "__main__":

View File

@ -12,7 +12,7 @@ from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from core.storage.disk_quota import parse_bytes, scan_user_dir
from core.storage.disk_quota import parse_bytes, scan_user_dir, scan_user_trash
class TestParseBytes(unittest.TestCase):
@ -75,6 +75,7 @@ class TestScanUserDir(unittest.TestCase):
for hidden in (
".zcbot_cache",
".zcbot_external_results",
".zcbot_artifact_trash",
".zcbot_tmp",
".memory",
):
@ -84,6 +85,17 @@ class TestScanUserDir(unittest.TestCase):
self.assertEqual(b, 7)
self.assertEqual(c, 1)
def test_trash_is_excluded_from_quota_and_counted_separately(self):
with tempfile.TemporaryDirectory() as d:
root = Path(d)
(root / "active.bin").write_bytes(b"active")
trash = root / ".zcbot_artifact_trash" / "2026" / "08"
trash.mkdir(parents=True)
(trash / "deleted.bin").write_bytes(b"deleted")
self.assertEqual(scan_user_dir(root), (6, 1))
self.assertEqual(scan_user_trash(root), (7, 1))
if __name__ == "__main__":
unittest.main()

View File

@ -18,6 +18,7 @@ import tempfile
import threading
import time
import unittest
from contextlib import contextmanager
from pathlib import Path
from unittest.mock import MagicMock, patch
from uuid import uuid4
@ -35,6 +36,22 @@ class FakePool:
def __init__(self):
self.ensure_calls = []
self.mark_active_calls = []
self.active = 0
self.capacity = self
self.capacity_notices = []
@contextmanager
def foreground(self, user_id, cancel_check=None, wait_notify=None):
if wait_notify is not None:
for notice in self.capacity_notices:
wait_notify(notice)
yield True
def exec_started(self, user_id):
self.active += 1
def exec_finished(self, user_id):
self.active -= 1
def ensure(self, user_id):
name = f"zcbot-sandbox-{user_id}"
@ -161,6 +178,27 @@ class TestShellExec(unittest.TestCase):
self.assertEqual(pool.ensure_calls, [executor.user_id])
self.assertEqual(pool.mark_active_calls, [executor.user_id])
def test_shell_forwards_capacity_queue_events(self):
executor, pool, _ = make_executor()
pool.capacity_notices = [
{"state": "waiting", "reason": "per_user_limit"},
{"state": "admitted"},
]
events = []
ctx = make_ctx(executor)
ctx.event_emit = events.append
proc = MagicMock()
proc.communicate.return_value = ("ok\n", "")
proc.returncode = 0
with patch("core.executor_docker.subprocess.Popen", return_value=proc):
executor.call_tool("shell", {"command": "echo ok"}, ctx)
self.assertEqual(events, [
{"type": "execution_queue", "state": "waiting", "reason": "per_user_limit"},
{"type": "execution_queue", "state": "admitted"},
])
def test_shell_bad_args(self):
executor, _, _ = make_executor()
ctx = make_ctx(executor)

View File

@ -0,0 +1,234 @@
from __future__ import annotations
import importlib.util
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from core.software_contracts import SoftwareContractError, get_contract
ROOT = Path(__file__).resolve().parents[1]
ADAPTER = ROOT / "windows-node" / "adapters" / "freecad.model.author@v1"
sys.path.insert(0, str(ADAPTER))
SPEC = importlib.util.spec_from_file_location("freecad_launcher", ADAPTER / "worker.py")
assert SPEC and SPEC.loader
launcher = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(launcher)
from recipe import RecipeError, normalize_model
def model() -> dict[str, object]:
return {
"type": "recipe",
"recipe_version": 1,
"title": "受控模型",
"unit": "mm",
"parameters": {"length": 20},
"steps": [
{
"id": "base",
"op": "box",
"length": {"parameter": "length"},
"width": 10,
"height": 5,
},
{"id": "hole", "op": "cylinder", "radius": 2, "height": 5},
{"id": "result", "op": "cut", "target": "base", "tools": ["hole"]},
],
"result": "result",
}
class FreeCadContractTests(unittest.TestCase):
def test_contract_accepts_parameterized_minimal_recipe(self) -> None:
request = {
"schema_version": 1,
"inputs": [],
"operation": {"model": model()},
"outputs": [],
}
normalized, digest = get_contract("freecad.model.author@v1").normalize_request(
request
)
self.assertEqual(normalized["operation"]["model"]["result"], "result")
self.assertEqual(len(digest), 64)
contract = get_contract("freecad.model.author@v1")
self.assertEqual(contract.workspace.state_output, "project")
self.assertEqual(contract.workspace.preview_outputs, ("preview",))
self.assertEqual(
set(contract.expected_outputs(normalized)),
{
"project",
"geometry_step",
"preview",
"model_recipe",
"model_manifest",
"provenance",
},
)
def test_contract_rejects_raw_expression_strings_and_fcstd_inputs(self) -> None:
request = {
"schema_version": 1,
"inputs": [],
"operation": {"model": model()},
"outputs": [],
}
request["operation"]["model"]["steps"][0]["length"] = "length * 2"
with self.assertRaises(SoftwareContractError):
get_contract("freecad.model.author@v1").normalize_request(request)
self.assertEqual(
get_contract("freecad.model.author@v1").input_policy["suffixes"],
[".step", ".stp"],
)
class FreeCadRecipeTests(unittest.TestCase):
def test_parameter_references_are_resolved_to_numbers(self) -> None:
resolved = normalize_model(model(), input_bound=False)
self.assertEqual(resolved["steps"][0]["length"], 20.0)
self.assertNotIn("Spreadsheet", json.dumps(resolved))
def test_forward_reference_and_unbound_step_are_rejected(self) -> None:
value = model()
value["steps"][0] = {
"id": "moved",
"op": "translate",
"target": "later",
"vector": [1, 0, 0],
}
with self.assertRaisesRegex(RecipeError, "moved"):
normalize_model(value, input_bound=False)
imported = {
"type": "recipe",
"recipe_version": 1,
"title": "import",
"unit": "mm",
"steps": [{"id": "part", "op": "import_step", "input": "geometry"}],
"result": "part",
}
with self.assertRaises(RecipeError) as caught:
normalize_model(imported, input_bound=False)
self.assertEqual(caught.exception.code, "FREECAD_STEP_INPUT_NOT_BOUND")
class FreeCadLauncherTests(unittest.TestCase):
def test_version_gate_accepts_only_stable_113_to_before_120(self) -> None:
executable = Path(r"C:\Program Files\FreeCAD 1.1\bin\FreeCADCmd.exe")
for output in ("FreeCAD 1.1.3", "FreeCAD 1.1.9 Revision: 12345"):
with patch.object(
subprocess,
"run",
return_value=subprocess.CompletedProcess([], 0, output, ""),
):
self.assertTrue(launcher._version(executable).startswith("1.1."))
for output in (
"FreeCAD 1.1.2",
"FreeCAD 1.2.0",
"FreeCAD 1.1.3 RC1",
"FreeCAD 1.1.4 weekly",
"FreeCAD 1.1.4, Libs: 1.1.4devR123",
"FreeCAD 1.1.4, Libs: 1.1.4dev.123",
):
with (
patch.object(
subprocess,
"run",
return_value=subprocess.CompletedProcess([], 0, output, ""),
),
self.assertRaises(launcher.AdapterError),
):
launcher._version(executable)
def test_managed_launcher_uses_fake_freecadcmd_with_fixed_inputs(self) -> None:
with tempfile.TemporaryDirectory() as directory:
job_dir = Path(directory) / "含空格 job"
request = {
"schema_version": 1,
"inputs": [],
"operation": {"model": model()},
"outputs": [],
}
record = {
"job_id": "job",
"lease_id": "lease",
"request_digest": "digest",
"request": request,
}
request_path = job_dir / "request" / "request.json"
request_path.parent.mkdir(parents=True)
request_path.write_text(json.dumps(record), encoding="utf-8")
fake = Path(r"C:\Program Files\FreeCAD 1.1\bin\FreeCADCmd.exe")
captured: dict[str, object] = {}
def execute(command, **kwargs):
captured["command"] = command
captured["environment"] = kwargs["env"]
output = job_dir / "output"
metadata = output / ".meta"
for name in (
"model.FCStd",
"model.step",
"model-recipe.json",
"model-manifest.json",
):
(output / name).write_bytes(b"fake-output")
(metadata / "provenance.json").write_bytes(b"{}")
(metadata / "preview-mesh.json").write_text(
json.dumps(
{
"vertices": [[0, 0, 0], [1, 0, 0], [0, 1, 0]],
"triangles": [[0, 1, 2]],
}
),
encoding="utf-8",
)
(metadata / "engine-result.json").write_text(
'{"status":"succeeded"}', encoding="utf-8"
)
return subprocess.CompletedProcess(command, 0)
with (
patch.object(launcher, "_find_freecad", return_value=(fake, "1.1.3")),
patch.object(launcher.subprocess, "run", side_effect=execute),
):
artifacts = launcher.run(job_dir)
command = captured["command"]
environment = captured["environment"]
self.assertEqual(command[0], str(fake))
self.assertIn("--safe-mode", command)
self.assertEqual(Path(command[-1]).name, "freecad_worker.py")
self.assertEqual(
environment["ZCBOT_FREECAD_JOB_DIR"], str(job_dir.resolve())
)
self.assertNotIn("PYTHONPATH", {key.upper() for key in environment})
self.assertEqual(len(artifacts), 6)
def test_launcher_uses_fixed_script_safe_mode_and_isolated_configuration(
self,
) -> None:
source = (ADAPTER / "worker.py").read_text(encoding="utf-8")
for marker in (
"--safe-mode",
"--user-cfg",
"--system-cfg",
"FREECAD_USER_HOME",
"FREECAD_USER_DATA",
"FREECAD_USER_TEMP",
"ZCBOT_FREECAD_JOB_DIR",
):
self.assertIn(marker, source)
for forbidden in ("shell=True", "eval(", "exec(", "requests."):
self.assertNotIn(forbidden, source)
engine = (ADAPTER / "freecad_worker.py").read_text(encoding="utf-8")
self.assertIn("do not rely on __main__", engine)
self.assertNotIn("FreeCADGui", engine)
self.assertNotIn("Spreadsheet", engine)
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,120 @@
from __future__ import annotations
import sys
import unittest
from contextlib import contextmanager
from pathlib import Path
from uuid import uuid4
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
import core.storage.telemetry as telemetry # noqa: E402
class _CaptureSession:
def __init__(self):
self.rows = []
def add(self, row):
self.rows.append(row)
class TestHealthTelemetry(unittest.TestCase):
def setUp(self):
self.session = _CaptureSession()
@contextmanager
def fake_scope():
yield self.session
self.scope_patch = patch.object(telemetry, "session_scope", fake_scope)
self.scope_patch.start()
def tearDown(self):
self.scope_patch.stop()
def _ids(self):
return {"task_id": uuid4(), "user_id": uuid4(), "message_id": uuid4()}
def test_normal_tool_result_does_not_write(self):
ids = self._ids()
written = telemetry.record_tool_failure(
**ids, model_profile="test.fast", tool="read", content="ok",
)
self.assertFalse(written)
self.assertEqual(self.session.rows, [])
def test_tool_failure_is_structured(self):
ids = self._ids()
written = telemetry.record_tool_failure(
**ids,
model_profile="test.fast",
tool="shell",
content="[exit 1]",
arguments={"command": "rg needle docs"},
)
self.assertTrue(written)
row = self.session.rows[0]
self.assertEqual(row.kind, telemetry.KIND_TOOL_FAILURE)
self.assertEqual(row.units["signature"], "exit N (search/no match)")
self.assertEqual(row.units["category"], "failure")
def test_quality_gate_uses_own_kind(self):
ids = self._ids()
telemetry.record_tool_failure(
**ids,
model_profile="test.fast",
tool="shell",
content="[GATE FAIL] svg_quality_checker: overlap\n[exit 1]",
)
self.assertEqual(self.session.rows[0].kind, telemetry.KIND_QUALITY_GATE)
self.assertEqual(self.session.rows[0].units["category"], "quality_gate")
def test_quality_gate_is_detected_before_trailing_command_hint(self):
ids = self._ids()
telemetry.record_tool_failure(
**ids,
model_profile="deepseek_v4.flash",
tool="shell",
content=(
"[stdout]\n[ERROR] slide.svg - Failed\n"
"[GATE FAIL] svg_quality_checker: 1 error\n"
"python scripts/svg_to_pptx.py \".\"\n[exit 1]"
),
)
self.assertEqual(self.session.rows[0].kind, telemetry.KIND_QUALITY_GATE)
self.assertEqual(self.session.rows[0].units["category"], "quality_gate")
def test_control_events_are_structured(self):
ids = self._ids()
common = {
"task_id": ids["task_id"], "user_id": ids["user_id"],
"model_profile": "test.fast",
}
telemetry.record_run_stopped(**common, reason="max_iterations")
telemetry.record_agent_guard(
**common, tool="edit", guard="same_error", count=4,
signature="old_str not found",
)
telemetry.record_context_fold_failure(
**common, reason="empty_summary",
)
telemetry.record_quality_gate(
**common, tool="shell", gate="pptx_full_page_image",
)
self.assertEqual(
[row.kind for row in self.session.rows],
[
telemetry.KIND_RUN_STOPPED,
telemetry.KIND_AGENT_GUARD,
telemetry.KIND_CONTEXT_FOLD_FAILURE,
telemetry.KIND_QUALITY_GATE,
],
)
self.assertEqual(self.session.rows[0].units["category"], "agent_control")
if __name__ == "__main__":
unittest.main()

View File

@ -75,6 +75,22 @@ class LLMKwargsTests(unittest.TestCase):
kwargs["extra_body"], {"thinking": {"type": "enabled"}}
)
def test_glm_preserved_thinking_is_nested_in_thinking_body(self) -> None:
caps = ModelCapabilities(
family="glm", variant="flash53", model_id="zai/glm-5.3-flash",
api_key_env="TEST_LLM_API_KEY", thinking_enabled=True,
thinking_transport="extra_body", thinking_clear=False,
default_reasoning_effort="high",
)
with patch.dict(os.environ, {"TEST_LLM_API_KEY": "test-key"}):
kwargs = LLM(caps)._build_kwargs(
[{"role": "user", "content": "hello"}], None, None, "high"
)
self.assertEqual(kwargs["extra_body"], {
"thinking": {"type": "enabled", "clear_thinking": False},
"reasoning_effort": "high",
})
def test_invalid_transport_is_rejected_before_request(self) -> None:
llm = self._llm(
family="test", thinking_enabled=True, thinking_transport="unknown"
@ -94,10 +110,13 @@ class LLMKwargsTests(unittest.TestCase):
self.assertEqual(caps.reasoning_effort_levels, ["low", "high", "max"])
self.assertEqual(caps.default_reasoning_effort, "high")
self.assertEqual(caps.max_output, 8192)
self.assertEqual(caps.output_cny_per_mtoken, 2.0)
self.assertEqual(caps.cache_hit_cny_per_mtoken, 0.02)
self.assertEqual(caps.output_cny_per_mtoken, 4.752)
self.assertEqual(caps.cache_hit_cny_per_mtoken, 0.0504)
self.assertEqual(
caps.pricing["periods"][0]["revision"], "deepseek-v4-20260816"
)
self.assertEqual(caps.thinking_transport, "extra_body")
self.assertEqual(caps.reasoning_replay, "tool_calls")
self.assertEqual(caps.reasoning_replay, "tool_turn")
def test_other_controllable_profiles_declare_transport(self) -> None:
models_dir = Path(__file__).resolve().parents[1] / "config" / "models"
@ -105,8 +124,12 @@ class LLMKwargsTests(unittest.TestCase):
glm = ModelCapabilities.load("glm.pro52", models_dir)
doubao = ModelCapabilities.load("doubao.turbo", models_dir)
self.assertFalse(glm.thinking_enabled)
self.assertEqual(glm.profile, "glm.flash53")
self.assertTrue(glm.thinking_enabled)
self.assertEqual(glm.thinking_transport, "extra_body")
self.assertFalse(glm.thinking_clear)
self.assertEqual(glm.reasoning_replay, "conversation")
self.assertTrue(glm.native_image_input)
self.assertTrue(doubao.thinking_enabled)
self.assertEqual(doubao.thinking_transport, "extra_body")
self.assertEqual(doubao.default_reasoning_effort, "")

View File

@ -1,9 +1,12 @@
"""look_at_image 的问题收敛与兼容兜底测试(不碰网络和数据库)。"""
from __future__ import annotations
import tempfile
import unittest
import uuid
from contextlib import ExitStack
from pathlib import Path
from types import SimpleNamespace
from unittest import mock
from core.ark_client import ArkConfig
@ -27,6 +30,69 @@ class LookAtImageQuestionTests(unittest.TestCase):
user_root=Path("."),
)
def test_native_vision_model_omits_look_tool_but_keeps_document_ocr(self):
from core.tool_registry import ToolContext, build_tools
ark = ArkConfig(
api_key="test",
base_url="https://example.invalid",
raw={"vision": {"test": {"model_id": "vision-test"}}},
)
def build(native_image_input: bool) -> dict:
with tempfile.TemporaryDirectory() as tmp, ExitStack() as stack:
root = Path(tmp)
stack.enter_context(mock.patch(
"core.tool_registry._external_system_status_available",
return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry._external_systems_available",
return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry.smtp_configured", return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry.wechat_push_available", return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry.lfasr_configured", return_value=False,
))
stack.enter_context(mock.patch(
"core.tool_registry.BochaConfig.load", return_value=None,
))
return build_tools(ToolContext(
tool_base=root,
ur_path=root,
working_dir_path=root,
task_id=uuid.uuid4(),
uid=uuid.uuid4(),
cfg={},
caps=SimpleNamespace(
enable_run_python=False,
native_image_input=native_image_input,
),
skills=SimpleNamespace(skills={}),
cancel_check=None,
scheduled_run=True,
deferred_actions=SimpleNamespace(),
ark_cfg=ark,
img_provider="",
img_key="",
img_cfg=None,
img_provider_cfg=None,
video_variant="",
office_to_pdf_available=False,
))
text_only = build(False)
native = build(True)
self.assertIn("look_at_image", text_only)
self.assertIn("read_document", text_only)
self.assertNotIn("look_at_image", native)
self.assertIn("read_document", native)
def _execute_and_question(self, question=None) -> str:
captured = {}

View File

@ -35,6 +35,7 @@ def _make_loop(stream_results, nonstream_results):
loop.caps = SimpleNamespace(reliable_context=64_000, family="test", variant="t")
loop.session = SimpleNamespace(messages=[], task_id="test-task")
loop.user_id = "test-user" # 无 DB 环境:_log_malformed_args 的落库路径应静默跳过
loop.user_root = None
# salvage(0.58.24)接在畸形重试前:空 schemas → 未知工具无白名单 → _try_salvage 返 False,
# 这些用例本就测「salvage 救不了 → 落非流式重试」,给个空 executor 即可走到该分支。
loop.executor = SimpleNamespace(schemas=lambda: [])
@ -56,8 +57,8 @@ def _make_loop(stream_results, nonstream_results):
class TestMalformedRetry(unittest.TestCase):
def test_deepseek_write_name_aborts_before_buffered_reasoning_is_emitted(self):
"""真实 collect 路径见到 write 名即关流,之前的 reasoning 不重复上屏"""
def test_deepseek_write_name_resets_streamed_reasoning_before_reroute(self):
"""真实 collect 路径实时发 reasoning见到 write 后重置再改走非流式"""
reasoning = SimpleNamespace(
choices=[SimpleNamespace(delta=SimpleNamespace(
reasoning_content="thinking", content=None, tool_calls=None,
@ -86,7 +87,44 @@ class TestMalformedRetry(unittest.TestCase):
with self.assertRaises(PreferNonstreamToolCall):
loop._collect_stream_once([])
self.assertEqual(loop.events, [])
self.assertEqual(loop.events, [
{"type": "reasoning", "delta": "thinking"},
{"type": "reasoning_reset"},
])
def test_deepseek_reasoning_is_emitted_before_output_route_is_known(self):
"""普通 DeepSeek 回答不能等首个正文 token 才释放已经到达的推理。"""
reasoning = SimpleNamespace(
choices=[SimpleNamespace(delta=SimpleNamespace(
reasoning_content="thinking", content=None, tool_calls=None,
))]
)
text = SimpleNamespace(
choices=[SimpleNamespace(delta=SimpleNamespace(
reasoning_content=None, content="answer", tool_calls=None,
))]
)
loop = object.__new__(AgentLoop)
loop.caps = SimpleNamespace(
family="deepseek_v4", default_reasoning_effort=None,
)
loop.executor = SimpleNamespace(schemas=lambda: [])
loop.cancel_check = None
loop.events = []
loop._emit = loop.events.append
loop.llm = SimpleNamespace(
chat_stream=lambda **_kwargs: iter([reasoning, text]),
)
with unittest.mock.patch(
"core.loop.litellm.stream_chunk_builder", return_value=object()
):
loop._collect_stream_once([])
self.assertEqual(loop.events, [
{"type": "reasoning", "delta": "thinking"},
{"type": "text", "delta": "answer"},
])
def test_preferred_tool_reroutes_before_arguments_stream(self):
"""首包识别 write/edit 后直接非流式,不把它算作一次畸形失败。"""

View File

@ -13,11 +13,12 @@ from core.loop import AgentLoop
class _Session:
def __init__(self, messages=None):
self.task_id = uuid4()
self.messages = list(messages or [])
self.appended = []
self.append_artifacts = []
def append(self, message, *, artifact_refs=None):
def append(self, message, *, artifact_refs=None, **_kwargs):
self.messages.append(message)
self.appended.append(message)
self.append_artifacts.append(artifact_refs)
@ -101,6 +102,48 @@ class PersistedTurnTests(unittest.TestCase):
"version": 1, "scope": "working_dir", "path": "report.pdf",
}])
def test_failed_tool_result_records_health_event(self) -> None:
session = _Session([{"role": "user", "content": "执行"}])
executor = MagicMock()
executor.call_tool.return_value = SimpleNamespace(
content="[Error] command failed", artifacts=(),
)
loop = AgentLoop(
llm=MagicMock(), executor=executor, session=session,
capabilities=SimpleNamespace(
max_iterations=2, family="test", variant="model",
input_cny_per_mtoken=0, output_cny_per_mtoken=0,
cache_hit_cny_per_mtoken=0, pricing={},
),
user_id=uuid4(), working_dir=Path("."),
)
loop._maybe_fold_context = MagicMock()
tool_call = SimpleNamespace(
id="call-1",
function=SimpleNamespace(name="shell", arguments='{"command":"false"}'),
)
responses = [
SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(
content="", tool_calls=[tool_call],
))], usage=None,
),
SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(
content="已说明失败", tool_calls=None,
))], usage=None,
),
]
loop._stream_llm = MagicMock(side_effect=[(r, False) for r in responses])
with patch("core.loop.record_chat_usage"), \
patch("core.loop.record_tool_failure") as failure:
result = loop.run_persisted_turn()
self.assertEqual(result, "已说明失败")
failure.assert_called_once()
self.assertEqual(failure.call_args.kwargs["tool"], "shell")
self.assertEqual(failure.call_args.kwargs["content"], "[Error] command failed")
if __name__ == "__main__":
unittest.main()

98
tests/test_pricing.py Normal file
View File

@ -0,0 +1,98 @@
from contextlib import contextmanager
from datetime import datetime, timezone
from decimal import Decimal
from pathlib import Path
from types import SimpleNamespace
import unittest
from unittest.mock import patch
from uuid import uuid4
from core.capabilities import ModelCapabilities
from core.pricing import calculate_chat_cost, resolve_chat_price
from core.storage.usage import record_chat_usage
MODELS_DIR = Path(__file__).resolve().parents[1] / "config" / "models"
class PricingTests(unittest.TestCase):
def test_deepseek_flash_selects_peak_and_off_peak(self) -> None:
caps = ModelCapabilities.load("deepseek_v4.flash", MODELS_DIR)
# 2026-08-17 是周一01:00 UTC 为峰时起点04:00 已回到谷时。
peak = resolve_chat_price(
caps.pricing,
occurred_at=datetime(2026, 8, 17, 1, 0, tzinfo=timezone.utc),
)
off_peak = resolve_chat_price(
caps.pricing,
occurred_at=datetime(2026, 8, 17, 4, 0, tzinfo=timezone.utc),
)
self.assertIsNotNone(peak)
self.assertIsNotNone(off_peak)
assert peak is not None and off_peak is not None
self.assertEqual(peak.tier, "peak")
self.assertEqual(peak.input_per_mtoken, Decimal("0.44"))
self.assertEqual(off_peak.tier, "default")
self.assertEqual(off_peak.output_per_mtoken, Decimal("0.66"))
def test_deepseek_weekend_is_off_peak(self) -> None:
caps = ModelCapabilities.load("deepseek_v4.pro", MODELS_DIR)
quote = resolve_chat_price(
caps.pricing,
occurred_at=datetime(2026, 8, 22, 2, 0, tzinfo=timezone.utc),
)
assert quote is not None
self.assertEqual(quote.tier, "default")
def test_usd_quote_calculates_cache_split_in_cny(self) -> None:
caps = ModelCapabilities.load("deepseek_v4.flash", MODELS_DIR)
quote = resolve_chat_price(
caps.pricing,
occurred_at=datetime(2026, 8, 17, 1, 0, tzinfo=timezone.utc),
)
assert quote is not None
result = calculate_chat_cost(
quote,
prompt_tokens=1_000_000,
completion_tokens=500_000,
cache_hit_tokens=800_000,
)
# miss:0.2*0.44*7.2 + hit:0.8*0.014*7.2 + out:0.5*1.32*7.2
self.assertEqual(result.total_cny, Decimal("5.466240"))
self.assertEqual(result.cache_miss_tokens, 200_000)
def test_local_catalog_precedes_litellm_cost_map(self) -> None:
caps = ModelCapabilities.load("deepseek_v4.flash", MODELS_DIR)
added = []
class FakeSession:
def add(self, row):
added.append(row)
@contextmanager
def fake_scope():
yield FakeSession()
with (
patch("core.storage.usage.session_scope", fake_scope),
patch("core.storage.usage._safe_chat_cost_usd", return_value=Decimal("999")),
):
cost = record_chat_usage(
task_id=None,
user_id=uuid4(),
message_id=None,
model_profile="deepseek_v4.flash",
prompt_tokens=1_000_000,
completion_tokens=0,
pricing=caps.pricing,
occurred_at=datetime(2026, 8, 17, 4, 0, tzinfo=timezone.utc),
response=SimpleNamespace(created=None),
)
self.assertEqual(cost, Decimal("1.584000"))
self.assertEqual(added[0].units["pricing_revision"], "deepseek-v4-20260816")
self.assertEqual(added[0].units["price_tier"], "default")
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,274 @@
import json
import os
import unittest
from contextlib import contextmanager
from datetime import datetime, timedelta, timezone
from importlib import util
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import MagicMock, Mock, patch
from uuid import uuid4
import httpx
from core.capabilities import ModelCapabilities
from core.llm import LLM
from core.provider_credentials.registry import BY_ID
from core.provider_credentials.runtime import resolve_credentials
from core.provider_credentials.service import (
ProviderCredentialError,
RevisionConflict,
_hint,
_notification_transition,
_row_payload,
replace_credentials,
)
from core.provider_credentials.testing import (
TestResult,
classify_response,
test_provider,
)
def response(status, body):
content = json.dumps(body).encode() if isinstance(body, dict) else str(body).encode()
return httpx.Response(status, content=content)
class ProviderRegistryTests(unittest.TestCase):
def test_credential_groups_are_explicit(self):
self.assertEqual(
[f.name for f in BY_ID["xfyun_iat"].fields],
["appid", "api_key", "api_secret"],
)
self.assertEqual(
[f.name for f in BY_ID["xfyun_lfasr"].fields],
["appid", "secret_key"],
)
self.assertNotIn("ZCBOT_DB_URL", [f.env for p in BY_ID.values() for f in p.fields])
def test_hint_only_reveals_last_four(self):
self.assertEqual(_hint("sk-abcdefgh"), "***efgh")
self.assertNotIn("abcdef", _hint("sk-abcdefgh"))
class ProviderClassificationTests(unittest.TestCase):
def test_http_402_and_429_are_not_conflated(self):
self.assertEqual(classify_response(402, "").status, "exhausted")
self.assertEqual(classify_response(429, "rate limit").status, "unreachable")
def test_deepseek_30_yuan_boundary(self):
def request_at(amount):
return lambda *a, **k: response(200, {
"is_available": True,
"balance_infos": [{"currency": "CNY", "total_balance": str(amount)}],
})
creds = {"api_key": "candidate-secret"}
self.assertEqual(
test_provider("deepseek", creds, request=request_at("29.99")).status,
"low_balance",
)
self.assertEqual(
test_provider("deepseek", creds, request=request_at("30.00")).status,
"normal",
)
def test_candidate_secret_is_not_returned_in_detail(self):
secret = "candidate-super-secret"
result = test_provider(
"deepseek", {"api_key": secret},
request=lambda *a, **k: response(401, {"message": secret}),
)
self.assertEqual(result.status, "auth_error")
self.assertNotIn(secret, result.detail)
class ProviderResolutionTests(unittest.TestCase):
def test_database_precedes_environment(self):
row = SimpleNamespace(credentials={"api_key": "cipher"})
class Result:
def scalar_one_or_none(self):
return row
class Session:
def execute(self, _statement):
return Result()
@contextmanager
def scope():
yield Session()
with (
patch.dict(os.environ, {"DEEPSEEK_API_KEY": "env-key"}),
patch("core.storage.session_scope", scope),
patch("core.provider_credentials.runtime.decrypt_secret", return_value="db-key") as decrypt,
):
resolved = resolve_credentials("deepseek")
self.assertEqual((resolved.source, resolved.values["api_key"]), ("database", "db-key"))
decrypt.assert_called_once_with("cipher", aad="provider:deepseek:api_key")
def test_environment_fallback_without_database(self):
with (
patch.dict(os.environ, {"DEEPSEEK_API_KEY": "env-key"}),
patch("core.storage.session_scope", side_effect=RuntimeError("no db")),
):
resolved = resolve_credentials("deepseek")
self.assertEqual((resolved.source, resolved.values), ("env", {"api_key": "env-key"}))
def test_each_xfyun_group_resolves_its_declared_fields(self):
env = {
"XFYUN_APPID": "app", "XFYUN_API_KEY": "iat-key",
"XFYUN_API_SECRET": "iat-secret", "XFYUN_LFASR_SECRET_KEY": "lf-key",
}
with patch.dict(os.environ, env), patch("core.storage.session_scope", side_effect=RuntimeError("no db")):
self.assertEqual(set(resolve_credentials("xfyun_iat").values), {"appid", "api_key", "api_secret"})
self.assertEqual(set(resolve_credentials("xfyun_lfasr").values), {"appid", "secret_key"})
class LLMCredentialLifecycleTests(unittest.TestCase):
def _caps(self):
return ModelCapabilities(
family="deepseek_v4", model_id="deepseek/model",
api_key_env="DEEPSEEK_API_KEY", thinking_transport="none",
)
def test_created_llm_uses_new_key_on_next_request(self):
state = {"key": "old-key"}
llm = LLM(self._caps(), credential_resolver=lambda _env: state["key"])
first = llm._build_kwargs([], None, None, None)
state["key"] = "new-key"
second = llm._build_kwargs([], None, None, None)
self.assertEqual(first["api_key"], "old-key")
self.assertEqual(second["api_key"], "new-key")
def test_built_stream_request_keeps_key_snapshot(self):
state = {"key": "old-key"}
calls = []
llm = LLM(self._caps(), credential_resolver=lambda _env: state["key"])
def completion(**kwargs):
calls.append(kwargs["api_key"])
state["key"] = "new-key"
return iter([{"chunk": 1}, {"chunk": 2}])
with patch("core.llm.litellm.completion", side_effect=completion):
self.assertEqual(len(list(llm.chat_stream([]))), 2)
self.assertEqual(calls, ["old-key"])
class ProviderMutationTests(unittest.TestCase):
def test_missing_master_key_refuses_database_save(self):
with (
patch("core.provider_credentials.service.crypto_configured", return_value=False),
self.assertRaisesRegex(ProviderCredentialError, "MASTER_KEY"),
):
replace_credentials(
"deepseek", {"api_key": "candidate"}, expected_revision=0,
updated_by=uuid4(),
)
def test_failed_candidate_never_opens_database_transaction(self):
with (
patch("core.provider_credentials.service.crypto_configured", return_value=True),
patch("core.provider_credentials.service.test_provider",
return_value=TestResult("auth_error", "认证失败")),
patch("core.provider_credentials.service.session_scope") as scope,
self.assertRaises(ProviderCredentialError),
):
replace_credentials(
"deepseek", {"api_key": "bad-key"}, expected_revision=0,
updated_by=uuid4(),
)
scope.assert_not_called()
def test_revision_conflict_does_not_overwrite(self):
class Session:
def execute(self, _statement):
return SimpleNamespace(rowcount=0)
@contextmanager
def scope():
yield Session()
with (
patch("core.provider_credentials.service.crypto_configured", return_value=True),
patch("core.provider_credentials.service.test_provider",
return_value=TestResult("normal", "正常")),
patch("core.provider_credentials.service.encrypt_secret", return_value="cipher"),
patch("core.provider_credentials.service.session_scope", scope),
self.assertRaises(RevisionConflict),
):
replace_credentials(
"deepseek", {"api_key": "new-key"}, expected_revision=7,
updated_by=uuid4(),
)
def test_alert_cooldown_and_recovery(self):
now = datetime.now(timezone.utc)
low = TestResult("low_balance", "余额低")
self.assertEqual(_notification_transition("normal", None, low, now), "low_balance")
self.assertIsNone(_notification_transition("low_balance", now, low, now))
self.assertEqual(
_notification_transition("low_balance", now - timedelta(days=1), low, now),
"low_balance",
)
self.assertEqual(
_notification_transition("auth_error", now, TestResult("normal", "正常"), now),
"recovered",
)
def test_admin_payload_contains_no_ciphertext(self):
provider = BY_ID["deepseek"]
row = SimpleNamespace(
credentials={"api_key": "ciphertext-secret"},
credential_hint={"api_key": "***1234"}, revision=2,
test_status="normal", test_detail="正常", last_tested_at=None,
balance_amount=None, balance_currency=None,
)
with patch("core.provider_credentials.service.resolve_credentials") as resolve:
resolve.return_value = SimpleNamespace(source="database", values={"api_key": "plain-secret"})
payload = _row_payload(provider, row)
encoded = json.dumps(payload, ensure_ascii=False)
self.assertNotIn("ciphertext-secret", encoded)
self.assertNotIn("plain-secret", encoded)
self.assertIn("***1234", encoded)
class ProviderLeaderTests(unittest.TestCase):
def test_unclaimed_round_does_not_test(self):
connection = Mock()
connection.exec_driver_sql.return_value.scalar.return_value = False
engine = MagicMock()
engine.connect.return_value.__enter__.return_value = connection
with (
patch("core.provider_credentials.monitor.get_engine", return_value=engine),
patch("core.provider_credentials.monitor.due_provider_ids") as due,
):
from core.provider_credentials.monitor import run_due_checks
self.assertEqual(run_due_checks(), 0)
due.assert_not_called()
class ProviderMigrationTests(unittest.TestCase):
def test_0039_creates_single_encrypted_credentials_table(self):
path = Path(__file__).resolve().parents[1] / "db" / "migrations" / "versions" / "20260902_1000_0039_provider_credentials.py"
spec = util.spec_from_file_location("migration_0039_test", path)
module = util.module_from_spec(spec)
assert spec.loader is not None
spec.loader.exec_module(module)
captured = {}
def create_table(name, *items):
captured["name"] = name
captured["columns"] = {item.name for item in items if hasattr(item, "name")}
with patch.object(module.op, "create_table", side_effect=create_table):
module.upgrade()
self.assertEqual(captured["name"], "provider_credentials")
self.assertTrue({"provider_id", "credentials", "credential_hint", "revision"} <= captured["columns"])
self.assertNotIn("api_key", captured["columns"])
if __name__ == "__main__":
unittest.main()

View File

@ -98,6 +98,21 @@ class TestSalvage(unittest.TestCase):
raw = '前缀{"path": "/a", "content": "x"}尾部残渣'
self.assertIsNone(salvage_tool_arguments(raw, WRITE_KEYS))
def test_repeated_identical_json_with_truncated_copy_is_salvaged(self):
"""wire 重复同一参数且末尾又截断时,可用两个一致的完整副本恢复。"""
good = {"command": "rg -n 'needle' /workspace/report.md"}
encoded = json.dumps(good, ensure_ascii=False)
raw = encoded + encoded + encoded[:18]
self.assertEqual(salvage_tool_arguments(raw, SHELL_KEYS), good)
def test_distinct_complete_objects_with_trailing_garbage_are_rejected(self):
"""存在两个不同的合法参数对象时语义不唯一,不能猜测执行哪一个。"""
first = json.dumps({"command": "first"})
second = json.dumps({"command": "second"})
self.assertIsNone(
salvage_tool_arguments(first + second + "{truncated", SHELL_KEYS)
)
def test_clean_json_at_char0(self):
"""char-0 就是完好 JSON(理论上不会进 salvage,但函数应幂等返回它)。"""
raw = '{"path": "/a", "content": "x"}'

View File

@ -0,0 +1,213 @@
from __future__ import annotations
import json
import os
import tempfile
import threading
import time
import unittest
from datetime import datetime, timezone
from types import SimpleNamespace
from pathlib import Path
from unittest.mock import MagicMock, patch
from uuid import uuid4
from core.sandbox.capacity import ExecCapacity
from core.sandbox.pool import SandboxPool
from core.sandbox.package_scans import persist_scan
from deploy.sandbox.package_scan import scan_packages
from core import procs
from web.admin import _sandbox_package_stats
class CapacityTests(unittest.TestCase):
def test_env_overrides_capacity_config(self):
with tempfile.TemporaryDirectory() as td, patch.dict(os.environ, {
"ZCBOT_MAX_ACTIVE_EXECS": "5", "ZCBOT_MAX_BACKGROUND_EXECS": "3",
"ZCBOT_MAX_ACTIVE_EXECS_PER_USER": "1", "ZCBOT_MIN_MEM_AVAILABLE": "2g",
}, clear=False):
cap = ExecCapacity(Path(td), {"max_active_execs": 9, "max_background_execs": 8,
"max_active_execs_per_user": 7, "min_mem_available": "4g"})
self.assertEqual((cap.max_active, cap.max_background, cap.max_per_user, cap.min_mem_available),
(5, 3, 1, 2 * 1024**3))
def test_config_cannot_raise_hard_limits(self):
with tempfile.TemporaryDirectory() as td:
cap = ExecCapacity(Path(td), {"max_active_execs": 99, "max_background_execs": 99,
"max_active_execs_per_user": 99})
self.assertEqual((cap.max_active, cap.max_background, cap.max_per_user), (6, 4, 3))
def test_global_per_user_and_background_limits(self):
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=10**12):
cap = ExecCapacity(Path(td), {"max_active_execs": 3, "max_background_execs": 1, "max_active_execs_per_user": 2})
a = cap.try_acquire("u1", "background", lease_id="a")
b = cap.try_acquire("u1", "foreground", lease_id="b")
self.assertIsNotNone(a); self.assertIsNotNone(b)
self.assertIsNone(cap.try_acquire("u1", "foreground", lease_id="c"))
self.assertIsNone(cap.try_acquire("u2", "background", lease_id="d"))
self.assertIsNotNone(cap.try_acquire("u2", "foreground", lease_id="e"))
self.assertIsNone(cap.try_acquire("u3", "foreground", lease_id="f"))
def test_cross_instance_state_and_cancel_queue(self):
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=10**12):
one = ExecCapacity(Path(td), {"max_active_execs": 1})
two = ExecCapacity(Path(td), {"max_active_execs": 1})
lease = one.try_acquire("u1", "foreground", lease_id="held")
cancelled = threading.Event()
result = []
t = threading.Thread(target=lambda: result.append(two.acquire_foreground("u2", cancelled.is_set)))
t.start(); time.sleep(.15); cancelled.set(); t.join(2)
self.assertEqual(result, [None])
self.assertEqual(two.snapshot()["foreground_queued"], 0)
one.release(lease)
def test_foreground_queue_reports_limit_and_admission(self):
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=10**12):
cap = ExecCapacity(Path(td), {"max_active_execs": 2, "max_active_execs_per_user": 1})
held = cap.try_acquire("u1", "background", lease_id="held")
notices = []
result = []
t = threading.Thread(
target=lambda: result.append(cap.acquire_foreground("u1", wait_notify=notices.append))
)
t.start(); time.sleep(.15)
self.assertEqual(notices, [{
"state": "waiting", "reason": "per_user_limit",
"user_running": 1, "user_limit": 1,
"global_running": 1, "global_limit": 2,
}])
cap.release(held); t.join(2)
self.assertEqual(len(result), 1)
self.assertIsNotNone(result[0])
self.assertEqual(notices[-1], {"state": "admitted"})
cap.release(result[0])
def test_immediate_foreground_admission_does_not_report_queue(self):
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=10**12):
cap = ExecCapacity(Path(td), {"max_active_execs": 1})
notices = []
lease = cap.acquire_foreground("u1", wait_notify=notices.append)
self.assertIsNotNone(lease)
self.assertEqual(notices, [])
cap.release(lease)
def test_memory_pressure_pauses_new_admission(self):
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=100):
cap = ExecCapacity(Path(td), {"min_mem_available": "1g"})
self.assertIsNone(cap.try_acquire("u", "foreground"))
self.assertTrue(cap.snapshot()["memory_paused"])
class PoolTests(unittest.TestCase):
def test_defaults_and_docker_tmpfs(self):
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.pool.subprocess.run") as run:
run.return_value = MagicMock(returncode=0, stdout="")
pool = SandboxPool(Path(td) / "users")
self.assertEqual((pool.memory, pool.cpus, pool.idle_ttl, pool.tmp_size), ("4g", "2.0", 600, "1g"))
pool._docker_run(uuid4(), "box")
argv = run.call_args_list[-1].args[0]
self.assertIn("/tmp:exec,size=1g,mode=1777", argv)
def test_active_exec_not_reaped(self):
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.pool._container_running", return_value=True), patch("core.sandbox.pool.subprocess.run") as run:
pool = SandboxPool(Path(td) / "users", idle_ttl=1)
uid = uuid4(); pool._last_active[uid] = int(time.time()) - 10
pool.exec_started(uid)
self.assertEqual(pool.reap_idle(), [])
run.assert_not_called()
def test_scan_failure_does_not_block_idle_removal(self):
with tempfile.TemporaryDirectory() as td, \
patch("core.sandbox.package_scans.scan_and_persist", side_effect=RuntimeError("db down")), \
patch("core.sandbox.pool.subprocess.run", return_value=MagicMock(returncode=0)):
pool = SandboxPool(Path(td) / "users", idle_ttl=1)
uid = uuid4(); pool._last_active[uid] = int(time.time()) - 10
self.assertEqual(pool.reap_idle(), [f"zcbot-sandbox-{uid}"])
def test_finalizer_helper_swallows_scanner_error(self):
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.package_scans.scan_and_persist", side_effect=RuntimeError("db down")):
pool = SandboxPool(Path(td) / "users")
pool._scan_before_remove("box", uuid4(), "foreground")
class PackageScannerTests(unittest.TestCase):
def test_empty_scan_does_not_touch_database(self):
persist_scan(container_session_id="empty", user_id=uuid4(), execution_kind="foreground",
result={"packages": [], "image_digest": "sha256:x", "python_version": "3.12", "total_installed_bytes": 0})
def test_added_override_bytes_and_symlink_escape(self):
with tempfile.TemporaryDirectory() as td:
root = Path(td) / "lib"; site = root / "python3.12" / "site-packages"; site.mkdir(parents=True)
pkg = site / "demo.py"; pkg.write_text("abc", encoding="utf-8")
dist = site / "Demo-2.dist-info"; dist.mkdir()
(dist / "METADATA").write_text("Name: Demo\nVersion: 2\n", encoding="utf-8")
(dist / "REQUESTED").write_text("", encoding="utf-8")
(dist / "RECORD").write_text("demo.py,,3\n", encoding="utf-8")
rows = scan_packages(root, {"demo": "1"})
self.assertEqual(rows[0]["change"], "override")
self.assertEqual(rows[0]["installed_bytes"], 3)
self.assertTrue(rows[0]["direct_requested"])
self.assertEqual(scan_packages(root, {"demo": "2"})[0]["change"], "reinstall")
self.assertEqual(scan_packages(root, {})[0]["change"], "added")
def test_metadata_symlink_is_ignored(self):
with tempfile.TemporaryDirectory() as td:
root = Path(td) / "lib"; site = root / "python3.12" / "site-packages"; site.mkdir(parents=True)
outside = Path(td) / "outside"; outside.write_text("Name: Bad\nVersion: 1\n")
dist = site / "Bad.dist-info"; dist.mkdir()
try:
(dist / "METADATA").symlink_to(outside)
except OSError:
self.skipTest("symlink unavailable")
self.assertEqual(scan_packages(root, {}), [])
class ProcQueueTests(unittest.TestCase):
def test_queued_status_and_cancel(self):
with tempfile.TemporaryDirectory() as td:
d = Path(td) / "p"; d.mkdir()
meta = {"proc_id": "p1", "backend": "docker", "state": "queued"}
procs.write_meta(d, meta)
self.assertEqual(procs.status_of(meta, d)[0], "queued")
self.assertEqual(procs.kill_proc(meta, d), "已取消排队")
self.assertEqual(procs.status_of(procs.read_meta(d), d), ("finished", 137))
def test_start_queued_respects_capacity_then_runs(self):
with tempfile.TemporaryDirectory() as td:
d = Path(td) / "p"; d.mkdir(); (d / "runner.sh").write_text("true")
uid = uuid4()
meta = {"proc_id": "p1", "task_id": "t1", "backend": "docker", "state": "queued",
"user_id": str(uid), "cwd": "/workspace/demo"}
procs.write_meta(d, meta)
pool = MagicMock(); pool.capacity.try_acquire.return_value = None
self.assertFalse(procs.start_queued_docker(meta, d, pool))
pool.capacity.try_acquire.return_value = "bg-p1"
pool.run_proc_container.return_value = "zcbot-proc-p1"
with patch("core.procs.subprocess.run", return_value=MagicMock(returncode=0, stderr="")):
self.assertTrue(procs.start_queued_docker(meta, d, pool))
saved = procs.read_meta(d)
self.assertEqual(saved["state"], "running")
self.assertEqual(saved["capacity_lease_id"], "bg-p1")
class AdminAggregationTests(unittest.TestCase):
def test_package_sessions_are_aggregated_by_name_version(self):
scans = [
SimpleNamespace(container_session_id="s1", user_id=uuid4(), execution_kind="foreground",
packages=[{"name":"Demo","version":"1","direct_requested":True,"installed_bytes":100,"change":"added","base_version":None}],
finished_at=datetime.now(timezone.utc)),
SimpleNamespace(container_session_id="s2", user_id=uuid4(), execution_kind="background",
packages=[{"name":"Demo","version":"1","direct_requested":False,"installed_bytes":300,"change":"added","base_version":None}],
finished_at=datetime.now(timezone.utc)),
]
session = MagicMock(); session.execute.return_value.scalars.return_value.all.return_value = scans
row = _sandbox_package_stats(session, None)["rows"][0]
self.assertEqual(row["session_count"], 2)
self.assertEqual(row["user_count"], 2)
self.assertEqual(row["foreground_sessions"], 1)
self.assertEqual(row["background_sessions"], 1)
self.assertEqual(row["direct_sessions"], 1)
self.assertEqual(row["average_installed_bytes"], 200)
if __name__ == "__main__":
unittest.main()

View File

@ -59,7 +59,7 @@ def _job(action: str, *, status: str = "succeeded"):
class SoftwareFollowupTests(unittest.TestCase):
def test_report_claim_persists_fixed_assistant_message_with_artifacts(self):
def test_report_claim_completes_without_persisting_chat_message(self):
job = _job("report")
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
session = _Session([job.task_id, task, job])
@ -68,20 +68,12 @@ class SoftwareFollowupTests(unittest.TestCase):
def scope():
yield session
with (
patch("web.software_followups.session_scope", scope),
patch("web.software_followups.allocate_message_idx", return_value=7),
):
with patch("web.software_followups.session_scope", scope):
claim = claim_followup(job.job_id)
self.assertEqual(claim.action, "report")
self.assertEqual(job.followup_status, "completed")
self.assertEqual(len(session.added), 1)
message = session.added[0]
self.assertEqual(message.payload["role"], "assistant")
self.assertIn(str(job.job_id), message.payload["content"])
self.assertEqual(len(message.artifact_refs), 1)
self.assertEqual(message.artifact_refs[0]["label"], "figure.png")
self.assertEqual(session.added, [])
def test_analyze_claim_persists_internal_turn_and_locks_task(self):
job = _job("analyze")
@ -120,7 +112,7 @@ class SoftwareFollowupTests(unittest.TestCase):
self.assertEqual(job.followup_status, "pending")
self.assertFalse(session.added)
def test_failed_job_persists_fixed_assistant_message_without_artifacts(self):
def test_failed_job_completes_without_persisting_chat_message(self):
job = _job("analyze", status="failed")
job.error = {"code": "COLUMN_MISSING", "detail": " Y2\n does not exist "}
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
@ -130,21 +122,14 @@ class SoftwareFollowupTests(unittest.TestCase):
def scope():
yield session
with (
patch("web.software_followups.session_scope", scope),
patch("web.software_followups.allocate_message_idx", return_value=9),
):
with patch("web.software_followups.session_scope", scope):
claim = claim_followup(job.job_id)
self.assertEqual(claim.action, "report")
self.assertEqual(job.followup_status, "completed")
message = session.added[0]
self.assertEqual(message.payload["role"], "assistant")
self.assertIn("专业软件任务执行失败", message.payload["content"])
self.assertIn("Y2 does not exist", message.payload["content"])
self.assertEqual(message.artifact_refs, [])
self.assertEqual(session.added, [])
def test_cancelled_job_persists_fixed_assistant_message(self):
def test_cancelled_job_completes_without_persisting_chat_message(self):
job = _job("analyze", status="cancelled")
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
session = _Session([job.task_id, task, job])
@ -153,19 +138,17 @@ class SoftwareFollowupTests(unittest.TestCase):
def scope():
yield session
with (
patch("web.software_followups.session_scope", scope),
patch("web.software_followups.allocate_message_idx", return_value=10),
):
with patch("web.software_followups.session_scope", scope):
claim = claim_followup(job.job_id)
self.assertEqual(claim.action, "report")
self.assertIn("专业软件任务已取消", session.added[0].payload["content"])
self.assertEqual(job.followup_status, "completed")
self.assertEqual(session.added, [])
def test_busy_task_leaves_failed_notification_pending(self):
def test_busy_task_does_not_delay_non_chat_result(self):
job = _job("report", status="failed")
task = SimpleNamespace(task_id=job.task_id, run_status="running")
session = _Session([job.task_id, task])
session = _Session([job.task_id, task, job])
@contextmanager
def scope():
@ -174,8 +157,8 @@ class SoftwareFollowupTests(unittest.TestCase):
with patch("web.software_followups.session_scope", scope):
claim = claim_followup(job.job_id)
self.assertIsNone(claim)
self.assertEqual(job.followup_status, "pending")
self.assertEqual(claim.action, "report")
self.assertEqual(job.followup_status, "completed")
self.assertFalse(session.added)

View File

@ -42,16 +42,24 @@ class SoftwareJobToolTests(unittest.TestCase):
)
self.assertEqual(result["capabilities"][0]["capability"], "origin.plot@v2")
self.assertEqual(result["capabilities"][0]["available_nodes"], 1)
self.assertEqual(
result["capabilities"][1]["capability"],
"ansys.geometry.inspect@v1",
capabilities = {item["capability"] for item in result["capabilities"]}
self.assertIn("cad.geometry.prepare@v1", capabilities)
self.assertIn("ansys.mechanical.static_structural@v2", capabilities)
self.assertNotIn("ansys.geometry.inspect@v1", capabilities)
with patch("tools.software_jobs.list_nodes", return_value=nodes):
diagnostic = json.loads(
SoftwareCapabilityListTool(self.user_id, self.task_id).execute(
include_diagnostic=True
)
self.assertEqual(result["capabilities"][1]["available_nodes"], 0)
self.assertEqual(
result["capabilities"][2]["capability"],
"ansys.mechanical.static_structural@v2",
)
self.assertEqual(result["capabilities"][2]["available_nodes"], 0)
inspect = next(
item for item in diagnostic["capabilities"]
if item["capability"] == "ansys.geometry.inspect@v1"
)
self.assertEqual(inspect["lifecycle"], "deprecated")
self.assertEqual(inspect["visibility"], "diagnostic_only")
self.assertEqual(inspect["replacement"], "cad.geometry.prepare@v1")
def test_submit_injects_current_user_and_task(self):
created = {"job_id": str(uuid4()), "status": "queued"}

View File

@ -34,6 +34,7 @@ from core.software_nodes import (
_verify_secret,
create_enrollment,
delete_node,
list_nodes,
update_node_runtime,
)
from web.routers.software_nodes import NodeConnectionManager, _bearer
@ -87,6 +88,8 @@ class SoftwareNodeSecurityTests(unittest.TestCase):
"ansys.geometry.inspect@v1",
"ansys.mechanical.static_structural@v2",
"blender.scene.author@v3",
"cad.geometry.prepare@v1",
"freecad.model.author@v1",
},
)
self.assertEqual(set(result["capabilities"]), set(enrollment.allowed_capabilities))
@ -122,6 +125,30 @@ class SoftwareNodeSecurityTests(unittest.TestCase):
)
self.assertIsNotNone(node.last_seen_at)
@patch("core.software_nodes.session_scope")
def test_admin_node_list_includes_active_job_count(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
node_id = uuid4()
node = type("Node", (), {
"node_id": node_id,
"name": "绘图节点",
"status": "online",
"capabilities": ["origin.plot@v2"],
"node_version": "0.2.0",
"os_version": "Windows",
"runtime": {},
"last_seen_at": datetime.now(timezone.utc),
})()
node_result = MagicMock()
node_result.scalars.return_value.all.return_value = [node]
count_result = MagicMock()
count_result.all.return_value = [(node_id, 1)]
session.execute.side_effect = [node_result, count_result]
result = list_nodes()
self.assertEqual(result[0]["active_job_count"], 1)
class SoftwareNodeConnectionTests(unittest.IsolatedAsyncioTestCase):
async def test_new_connection_replaces_old_without_removing_new(self) -> None:
@ -799,6 +826,11 @@ class SoftwareJobProtocolTests(unittest.TestCase):
self.assertEqual(offer["node_id"], node.node_id)
self.assertEqual(offer["payload"]["job_id"], str(jobs[1].job_id))
self.assertEqual(offer["payload"]["request"]["operation"]["plot"]["type"], "line")
self.assertEqual(
offer["payload"]["request_summary"]["display_name"],
"Origin 科研绘图",
)
self.assertIn("title", offer["payload"]["request_summary"])
def test_input_download_rechecks_file_digest(self) -> None:
source = (

View File

@ -33,6 +33,8 @@ class StaticVendorTests(unittest.TestCase):
self.assertIn('id="right-tab-jobs"', html)
self.assertIn('id="right-jobs-panel"', html)
self.assertIn('id="software-job-badge"', html)
self.assertIn(".workspace-badge[hidden]", html)
self.assertIn(".sj-entry-dot.succeeded, .sj-entry-dot.failed", html)
self.assertIn(".right-workspace-panel {", html)
self.assertLess(
html.index('id="right-tab-jobs"'),
@ -50,6 +52,7 @@ class StaticVendorTests(unittest.TestCase):
self.assertIn("续接自 Job ${sourceJobId.slice(0, 8)}", source)
self.assertIn("<dt>来源 Job</dt>", source)
self.assertIn("refreshCurrentTaskAfterSoftwareJob(job.task_id)", source)
self.assertNotIn('else if (active.length) entryDot.classList.add("active")', source)
self.assertIn("job.execution_runtime", source)
self.assertIn("执行版本未记录", source)
self.assertIn("执行版本</dt><dd>${escapeHtml(jobRuntimeText(job))}", source)
@ -69,7 +72,7 @@ class StaticVendorTests(unittest.TestCase):
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")
self.assertIn('id="node-enrollment-modal" class="modal"', html)
self.assertIn("生成 Windows Node 注册码", html)
self.assertIn("生成专业软件节点注册码", html)
self.assertIn('"/v1/admin/software-node-enrollments"', admin_js)
self.assertNotIn('capabilities: ["origin.plot@v2"]', admin_js)
self.assertIn("节点能力由客户端自动发现并上报", html)
@ -87,10 +90,80 @@ class StaticVendorTests(unittest.TestCase):
self.assertIn('apiSend("PATCH", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
self.assertIn('apiSend("DELETE", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
self.assertIn("最近心跳", admin_js)
self.assertIn("活动任务", admin_js)
self.assertIn("node.active_job_count", admin_js)
self.assertIn("重新启用", admin_js)
self.assertIn("永久删除", admin_js)
self.assertNotIn("localStorage.setItem", admin_js)
def test_admin_uses_compact_overview_and_grouped_workspace_tabs(self) -> None:
html = ADMIN_HTML.read_text(encoding="utf-8")
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")
self.assertIn('id="ops-summary" class="ops-summary"', admin_js)
self.assertIn(".ops-summary {", html)
self.assertIn('class="summary-primary"', admin_js)
self.assertIn('class="summary-secondary"', admin_js)
labels = (
"任务运行",
"执行容量",
"专业软件节点",
"工具健康",
"活跃用户",
"Token 与缓存",
"成本",
"存储",
)
positions = []
for label in labels:
self.assertIn(f'label: "{label}"', admin_js)
positions.append(admin_js.index(f'label: "{label}"'))
self.assertEqual(positions, sorted(positions))
self.assertIn("function renderRuntimeDetails()", admin_js)
self.assertIn("function renderCapacitySection()", admin_js)
self.assertIn("function renderUsageSection(d)", admin_js)
for tab in ("运行", "用量", "用户", "资源", "集成", "质量"):
self.assertIn(f'"{tab}"', admin_js)
self.assertIn('class="workspace-tabs"', admin_js)
self.assertIn('role="tabpanel"', admin_js)
self.assertIn('tab: "run", target: "s-runtime"', admin_js)
self.assertIn('tab: "run", target: "s-sandbox-capacity"', admin_js)
self.assertIn('tab: "resources", target: "s-windows-node"', admin_js)
self.assertIn('tab: "quality", target: "s-toolfail"', admin_js)
self.assertIn('tab: "users", target: "s-users"', admin_js)
self.assertIn('tab: "usage", target: "s-usage"', admin_js)
self.assertIn('tab: "resources", target: "s-storage"', admin_js)
self.assertIn('unit: softwareNodesLoaded ? "在线 / 总数"', admin_js)
self.assertIn('value: fmtTokens(tokens7d), unit: "近 7 天"', admin_js)
self.assertIn("storageTotals.physical_bytes", admin_js)
self.assertIn("repeat(auto-fit, minmax(190px, 1fr))", html)
self.assertIn('apiGet("/v1/admin/storage/users?page=0&page_size=1")', admin_js)
self.assertIn('id="s-sandbox-capacity"', admin_js)
self.assertIn('id="s-sandbox-packages"', admin_js)
self.assertIn('sandboxCapacityData = await apiGet("/v1/admin/sandbox/capacity")', admin_js)
self.assertNotIn("containerStatusHTML(capacity)", admin_js)
self.assertIn('data-summary-tab="${escapeHtml(tab)}"', admin_js)
self.assertIn("function renderCapacityDrawer()", admin_js)
self.assertIn("function openCapacityDrawer()", admin_js)
self.assertIn("const TOOL_HEALTH_REFRESH_MS = 90000", admin_js)
self.assertIn("if (toolFailuresRequest) return toolFailuresRequest", admin_js)
self.assertIn("loadToolFailures();", admin_js)
self.assertNotIn('activeTab === "quality"', admin_js)
self.assertIn('item.category === "failure"', admin_js)
self.assertIn('item.category === "agent_control"', admin_js)
self.assertIn("代理控制与上下文异常(近 24h", admin_js)
self.assertIn('id="capacity-drawer" class="capacity-drawer"', html)
self.assertIn(".capacity-drawer-panel {", html)
def test_changelog_loads_incremental_pages(self) -> None:
source = (JS_DIR / "changelog.js").read_text(encoding="utf-8")
self.assertIn("const PAGE_SIZE = 5", source)
self.assertIn("offset=${_nextOffset}", source)
self.assertIn("d.has_more", source)
self.assertIn("加载更多", source)
self.assertNotIn("/v1/changelog?limit=50", source)
def test_dev_html_uses_local_markdown_vendor_assets(self) -> None:
html = DEV_HTML.read_text(encoding="utf-8")
@ -140,6 +213,17 @@ class StaticVendorTests(unittest.TestCase):
self.assertIn(":focus-visible", css)
self.assertIn("@media (max-width: 640px)", css)
def test_admin_entry_lives_in_main_conversation_panel(self) -> None:
html = DEV_HTML.read_text(encoding="utf-8")
header = html.split("<header>", 1)[1].split("</header>", 1)[0]
conversation_head = html.split('id="pane-mid"', 1)[1].split(
'id="chat-meta"', 1
)[0]
self.assertNotIn('id="hd-admin"', header)
self.assertIn('id="hd-admin"', conversation_head)
self.assertIn('href="/static/admin.html"', conversation_head)
def test_file_toolbar_keeps_hidden_toggle_and_groups_add_actions(self) -> None:
html = DEV_HTML.read_text(encoding="utf-8")
files_js = (JS_DIR / "files.js").read_text(encoding="utf-8")
@ -191,6 +275,14 @@ class StaticVendorTests(unittest.TestCase):
self.assertNotIn('<div class="role">科研助手</div>', frontend)
self.assertIn(".tool-call.activity > summary", css)
self.assertIn("#task-progress-dock .task-progress", css)
self.assertIn(".conversation-rail-caption", html)
self.assertIn('id="conversation-side-rail"', html)
self.assertLess(html.index('id="msg-outline-rail"'), html.index('id="software-result-rail"'))
self.assertIn(".rail-count.attention", html)
self.assertIn("toggleConversationRailPinned", frontend)
self.assertIn("syncConversationSideRail", frontend)
self.assertIn("点击固定展开或收起导航", frontend)
self.assertIn("个新结果", frontend)
def test_frontend_surfaces_context_and_cache_stats(self) -> None:
src = _frontend_source()
@ -211,6 +303,13 @@ class StaticVendorTests(unittest.TestCase):
self.assertIn("await yieldToStreamPaint()", chat_js)
self.assertIn('document.visibilityState === "hidden"', chat_js)
def test_sse_consumer_resets_abandoned_reasoning_stream(self) -> None:
chat_js = (JS_DIR / "chat.js").read_text(encoding="utf-8")
self.assertIn("function resetReasonSeg(run)", chat_js)
self.assertIn('t === "reasoning_reset"', chat_js)
self.assertIn("resetReasonSeg(ctx)", chat_js)
def test_custom_task_name_is_optional_with_explicit_directory(self) -> None:
html = DEV_HTML.read_text(encoding="utf-8")
newtask_js = (JS_DIR / "newtask.js").read_text(encoding="utf-8")

View File

@ -32,6 +32,27 @@ class StorageMigrationTests(unittest.TestCase):
self.assertIn("ix_scheduled_jobs_due_active", rendered)
self.assertIn("USING brin", rendered)
def test_0040_upgrade_compiles_health_event_indexes(self) -> None:
statements: list[str] = []
def capture(sql, *multiparams, **params):
statements.append(str(sql.compile(dialect=postgresql.dialect())))
engine = create_mock_engine("postgresql+psycopg://", capture)
operations = Operations(MigrationContext.configure(engine.connect()))
migration = importlib.import_module(
"db.migrations.versions.20260903_1400_0040_tool_health_indexes"
)
with patch.object(migration, "op", operations):
migration.upgrade()
rendered = "\n".join(statements)
self.assertIn("ix_usage_health_kind_created", rendered)
self.assertIn("ux_usage_tool_failure_message", rendered)
self.assertIn("tool_failure", rendered)
self.assertIn("agent_guard", rendered)
self.assertIn("context_fold_failure", rendered)
if __name__ == "__main__":
unittest.main()

View File

@ -75,6 +75,15 @@ class TestSystemPromptPaths(unittest.TestCase):
self.assertIn(str(wd), prompt)
self.assertNotIn("/workspace/数据资源展示", prompt)
def test_mermaid_visual_guidance_is_backend_independent(self):
for backend in ("docker", "host"):
with self.subTest(backend=backend):
prompt, _, _ = _build(backend)
self.assertIn("## Mermaid 视觉规范", prompt)
self.assertIn("`classDef` 按角色分组", prompt)
self.assertIn("不要输出全灰单色图", prompt)
self.assertIn("用户指定品牌色", prompt)
def test_rename_guard_only_when_tool_is_available(self):
interactive, _, _ = _build("docker", allow_rename=True)
scheduled, _, _ = _build("docker", allow_rename=False)

View File

@ -11,36 +11,67 @@ from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
import core.toolfail as tf # noqa: E402
from core.storage.telemetry import ( # noqa: E402
KIND_AGENT_GUARD,
KIND_EMPTY_RESPONSE,
KIND_RUN_ERROR,
KIND_TOOL_FAILURE,
KIND_TOOL_MALFORMED,
)
from core.tool_failure import structured_failure # noqa: E402
NOW = datetime.now(timezone.utc)
class _FakeSession:
"""按 SQL 关键字分流四段查询:messages / tool_malformed / run_error / empty_response。"""
"""工具健康只允许一次 usage_events 查询"""
def __init__(self, msg_rows, malformed_rows, run_rows, empty_rows):
self._msg_rows = msg_rows
self._malformed_rows = malformed_rows
self._run_rows = run_rows
self._empty_rows = empty_rows
def __init__(self, rows):
self._rows = rows
self.calls = 0
def execute(self, clause, params=None):
sql = str(clause)
if "tool_malformed" in sql:
rows = self._malformed_rows
elif "run_error" in sql:
rows = self._run_rows
elif "empty_response" in sql:
rows = self._empty_rows
else:
rows = self._msg_rows
return SimpleNamespace(fetchall=lambda: rows)
self.calls += 1
if "from usage_events" not in sql or "from messages" in sql:
raise AssertionError(sql)
return SimpleNamespace(fetchall=lambda: self._rows)
def _structured_row(row):
task_id, user_id, created_at, tool, content = row[:5]
arguments = None
if len(row) > 6 and isinstance(row[6], dict):
calls = row[6].get("tool_calls") or []
if calls:
arguments = (calls[0].get("function") or {}).get("arguments")
units = structured_failure(tool, content, arguments=arguments)
assert units is not None
return (
KIND_TOOL_FAILURE, task_id, user_id, created_at, None, tool,
None, None, None, units["failure_kind"], units["signature"],
units["category"], units["sample"],
)
def _scan(msg_rows, malformed_rows, run_rows=None, empty_rows=None, **kw):
event_rows = [_structured_row(row) for row in msg_rows]
event_rows.extend((
KIND_TOOL_MALFORMED, task_id, user_id, created_at, None, tool,
err, head, tail, None, None, None, None,
) for task_id, user_id, created_at, tool, err, head, tail in malformed_rows)
event_rows.extend((
KIND_RUN_ERROR, task_id, user_id, created_at, None, None,
err, None, None, None, None, None, None,
) for task_id, user_id, created_at, err in (run_rows or []))
event_rows.extend((
KIND_EMPTY_RESPONSE, task_id, user_id, created_at, model_profile, None,
None, None, None, None, None, None, None,
) for task_id, user_id, created_at, model_profile in (empty_rows or []))
@contextmanager
def fake_scope():
yield _FakeSession(msg_rows, malformed_rows, run_rows or [], empty_rows or [])
yield _FakeSession(event_rows)
with patch.object(tf, "session_scope", fake_scope):
return tf.scan_tool_failures(**kw)
@ -73,8 +104,8 @@ class TestToolfailMalformed(unittest.TestCase):
out = _scan([], rows, days=7, min_count=2, min_tasks=2)
self.assertEqual(len(out), 1)
def test_merges_with_message_clusters(self):
"""两段数据源并存互不干扰:messages 报错与 malformed 各出各的 cluster"""
def test_merges_structured_and_malformed_clusters(self):
"""同表不同 kind 并存互不干扰:普通失败与 malformed 各出一组"""
msg_rows = [
("t1", "u1", NOW, "glob", "[Error] base path not found"),
("t2", "u2", NOW, "glob", "[Error] base path not found"),
@ -184,6 +215,25 @@ class TestToolfailEmptyResponse(unittest.TestCase):
self.assertEqual(out, [])
class TestStructuredHealthEvents(unittest.TestCase):
def test_agent_control_is_separate_from_operational_failure(self):
rows = [(
KIND_AGENT_GUARD, f"t{i}", "u1", NOW, "deepseek_v4.pro", "edit",
None, None, None, "guard", "same_error: old_str not found",
"agent_control", "same_error, count=4",
) for i in range(2)]
@contextmanager
def fake_scope():
yield _FakeSession(rows)
with patch.object(tf, "session_scope", fake_scope):
out = tf.scan_tool_failures(days=1, min_count=2, min_tasks=2)
self.assertEqual(len(out), 1)
self.assertEqual(out[0]["category"], "agent_control")
self.assertEqual(out[0]["kind"], "guard")
class TestToolWireHealth(unittest.TestCase):
def test_aggregates_recovery_rates_and_totals(self):
rows = [
@ -226,6 +276,34 @@ class TestToolWireHealth(unittest.TestCase):
self.assertIsNone(out["total"]["recovery_rate_24h"])
class TestToolHealthCache(unittest.TestCase):
def setUp(self):
tf._clear_tool_health_cache()
def tearDown(self):
tf._clear_tool_health_cache()
def test_failure_scan_is_reused_within_ttl(self):
expected = [{"tool": "shell"}]
with patch.object(tf, "scan_tool_failures", return_value=expected) as scan:
first = tf.scan_tool_failures_cached(days=7, min_count=3, min_tasks=1)
second = tf.scan_tool_failures_cached(days=7, min_count=3, min_tasks=1)
self.assertIs(first, expected)
self.assertIs(second, expected)
scan.assert_called_once_with(days=7, min_count=3, min_tasks=1)
def test_wire_scan_normalizes_window_before_caching(self):
expected = {"days": 90, "rows": [], "total": {}}
with patch.object(tf, "scan_tool_wire_health", return_value=expected) as scan:
first = tf.scan_tool_wire_health_cached(days=365)
second = tf.scan_tool_wire_health_cached(days=90)
self.assertIs(first, expected)
self.assertIs(second, expected)
scan.assert_called_once_with(days=90)
class TestProviderCriticalAlert(unittest.TestCase):
def setUp(self):
tf._alerted_at.clear()

View File

@ -194,8 +194,16 @@ class TasksCrudTests(unittest.TestCase):
tid = _client.post("/v1/tasks", json={"name": "状态闸任务"}, headers=_AUTH).json()["task_id"]
# idle 时 cancel → 409
self.assertEqual(_client.post(f"/v1/tasks/{tid}/cancel", headers=_AUTH).status_code, 409)
# running 时 clear → 409;回 idle 后 clear → 200 且归零
# running 时 cancel 可重试:第一次切 cancelling第二次幂等补发信号。
_set_run_status(tid, "running")
with patch("web.routers.messages.broker.request_cancel") as request_cancel:
first = _client.post(f"/v1/tasks/{tid}/cancel", headers=_AUTH)
second = _client.post(f"/v1/tasks/{tid}/cancel", headers=_AUTH)
self.assertEqual(first.status_code, 202, first.text)
self.assertEqual(second.status_code, 202, second.text)
self.assertEqual(request_cancel.call_count, 2)
# cancelling 时 clear → 409;回 idle 后 clear → 200 且归零
self.assertEqual(_client.post(f"/v1/tasks/{tid}/clear", headers=_AUTH).status_code, 409)
_set_run_status(tid, "idle")
d = _client.post(f"/v1/tasks/{tid}/clear", headers=_AUTH).json()

View File

@ -73,11 +73,17 @@ class PublicEndpointTests(unittest.TestCase):
self.assertIn("brand", body)
def test_changelog_public(self):
r = _client.get("/v1/changelog?limit=2")
r = _client.get("/v1/changelog?limit=2&offset=0")
self.assertEqual(r.status_code, 200)
self.assertIn("entries", r.json())
first = r.json()
self.assertEqual(len(first["entries"]), 2)
self.assertGreaterEqual(first["total"], 2)
self.assertEqual(first["next_offset"], 2)
second = _client.get("/v1/changelog?limit=2&offset=2").json()
self.assertNotEqual(first["entries"][0]["version"], second["entries"][0]["version"])
# limit 越界 clamp,不 500
self.assertEqual(_client.get("/v1/changelog?limit=9999").status_code, 200)
self.assertEqual(_client.get("/v1/changelog?offset=-1").status_code, 200)
def test_root_redirects_to_dev_spa(self):
r = _client.get("/", follow_redirects=False)
@ -111,6 +117,12 @@ class AuthGateTests(unittest.TestCase):
("POST", "/v1/tasks"),
("POST", "/v1/asr/transcribe"),
("GET", "/v1/admin/overview"),
("GET", "/v1/admin/provider-credentials"),
("PUT", "/v1/admin/provider-credentials/deepseek"),
("POST", "/v1/admin/provider-credentials/deepseek/test"),
("DELETE", "/v1/admin/provider-credentials/deepseek"),
("GET", "/v1/admin/sandbox/capacity"),
("GET", "/v1/admin/sandbox/packages"),
("GET", "/v1/admin/software-nodes"),
("GET", "/v1/software-jobs"),
("POST", "/v1/software-jobs/00000000-0000-0000-0000-000000000000/cancel"),

30
tests/test_web_sinks.py Normal file
View File

@ -0,0 +1,30 @@
from __future__ import annotations
import unittest
from uuid import uuid4
from web.sinks import WebEventSink
class _Broker:
def __init__(self) -> None:
self.events: list[tuple] = []
def emit(self, task_id, event) -> None:
self.events.append((task_id, event))
class WebEventSinkTests(unittest.TestCase):
def test_done_is_deferred_to_worker_close(self) -> None:
broker = _Broker()
task_id = uuid4()
sink = WebEventSink(broker, task_id)
sink.emit({"type": "text", "delta": "ok"})
sink.emit({"type": "done"})
self.assertEqual(broker.events, [(task_id, {"type": "text", "delta": "ok"})])
if __name__ == "__main__":
unittest.main()

View File

@ -9,17 +9,28 @@ PROJECT = ROOT / "Zcbot.WindowsNode"
class WindowsNodeSourceTests(unittest.TestCase):
def test_project_targets_net10_windows_forms_with_json_schema_validator(self) -> None:
def test_project_targets_net10_wpf_with_windows_forms_interop(self) -> None:
tree = ET.parse(PROJECT / "Zcbot.WindowsNode.csproj")
root = tree.getroot()
self.assertEqual(root.findtext("./PropertyGroup/TargetFramework"), "net10.0-windows")
self.assertEqual(root.findtext("./PropertyGroup/UseWindowsForms"), "true")
self.assertEqual(root.findtext("./PropertyGroup/UseWPF"), "true")
self.assertEqual(root.findtext("./PropertyGroup/OutputType"), "WinExe")
self.assertEqual(
root.findtext("./PropertyGroup/ApplicationIcon"), "Assets\\zcbot.ico"
)
self.assertEqual(
[item.attrib["Include"] for item in root.findall("./ItemGroup/Resource")],
["Assets\\zcbot.ico"],
)
icon = PROJECT / "Assets" / "zcbot.ico"
self.assertTrue(icon.is_file())
self.assertEqual(icon.read_bytes()[:6], b"\x00\x00\x01\x00\x07\x00")
packages = root.findall("./ItemGroup/PackageReference")
self.assertEqual([item.attrib["Include"] for item in packages], ["JsonSchema.Net"])
def test_node_protocol_and_secret_storage_markers_are_present(self) -> None:
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs"))
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.rglob("*.cs"))
for marker in (
"v1/software-nodes/enroll",
"v1/software-nodes/connect",
@ -29,16 +40,129 @@ class WindowsNodeSourceTests(unittest.TestCase):
"SetAccessRuleProtection(isProtected: true",
'"adapter.json"',
"NotifyIcon",
"ConfigurationForm",
"MainWindow",
"TrayIconFactory.Create",
'"--headless"',
):
self.assertIn(marker, source)
def test_gui_and_headless_share_the_composition_root(self) -> None:
program = (PROJECT / "Program.cs").read_text(encoding="utf-8")
composition = (
PROJECT / "Bootstrap" / "NodeCompositionRoot.cs"
).read_text(encoding="utf-8")
controller = (
PROJECT / "Application" / "NodeApplicationController.cs"
).read_text(encoding="utf-8")
self.assertEqual(program.count("new NodeCompositionRoot"), 2)
self.assertIn("CreateApplicationController()", program)
self.assertIn("RunHeadlessAsync(shutdown.Token)", program)
self.assertIn("new App(compositionRoot.CreateApplicationController())", program)
self.assertIn("return app.Run()", program)
self.assertNotIn("new TrayApplicationContext", program)
self.assertIn("new NodeConnectionLoop(config, Paths", composition)
self.assertIn("stopTask ??= StopOnceAsync(mode)", controller)
self.assertIn("await StopConnectionAsync()", controller)
def test_wpf_overview_registration_and_settings_use_view_model_commands(self) -> None:
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
encoding="utf-8"
)
view_model = (
PROJECT / "Presentation" / "ViewModels" / "MainWindowViewModel.cs"
).read_text(encoding="utf-8")
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
encoding="utf-8"
)
startup = (PROJECT / "StartupRegistration.cs").read_text(encoding="utf-8")
for binding in (
"ShowOverviewCommand",
"ShowSettingsCommand",
"RegisterCommand",
"ReconnectCommand",
"ResetIdentityCommand",
"IsStartupEnabled",
"DataRootPage.CurrentRoot",
):
self.assertIn(f"{{Binding {binding}", xaml)
self.assertIn('PasswordChanged="EnrollmentCodeBox_OnPasswordChanged"', xaml)
self.assertIn("CopyDiagnosticsCommand", xaml)
self.assertNotIn("OpenLegacyCommand", xaml + view_model)
self.assertNotIn("NodeApplicationController", xaml + view_model)
self.assertIn("interface IStartupRegistration", startup)
self.assertIn("RegisterRequested += RegisterFromWpfAsync", tray)
self.assertIn("ReconnectRequested += ReconnectFromWpfAsync", tray)
self.assertIn("ResetIdentityRequested += ResetIdentityFromWpfAsync", tray)
def test_wpf_software_page_uses_runtime_services_and_tray_dialogs(self) -> None:
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
encoding="utf-8"
)
page = (
PROJECT / "Presentation" / "ViewModels" / "SoftwarePageViewModel.cs"
).read_text(encoding="utf-8")
service = (
PROJECT / "Presentation" / "Services" / "SoftwareManagementService.cs"
).read_text(encoding="utf-8")
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
encoding="utf-8"
)
for binding in (
"ShowSoftwareCommand",
"SoftwarePage.RefreshCommand",
"SoftwarePage.CancelInstallCommand",
"SoftwarePage.Cards",
"ChooseLocationCommand",
"ClearLocationCommand",
"InstallRuntimeCommand",
):
self.assertIn(f"{{Binding {binding}", xaml)
self.assertIn("ISoftwareManagementService", page + service)
self.assertIn("SoftwareLocationService.SaveConfiguredPath", service)
self.assertIn("SoftwareLocationService.ClearConfiguredPath", service)
self.assertIn("ManagedRuntimeInstaller.InstallAsync", service)
self.assertIn("AdapterCatalog.HasActiveExecution", service)
self.assertIn("ChooseLocationRequested += ChooseSoftwareLocation", tray)
self.assertIn("ConfirmInstallRequested += ConfirmRuntimeInstall", tray)
self.assertNotIn("OpenFileDialog", page + service)
def test_wpf_theme_has_selected_navigation_status_and_responsive_actions(self) -> None:
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
encoding="utf-8"
)
theme = (
PROJECT / "Presentation" / "Themes" / "LightTheme.xaml"
).read_text(encoding="utf-8")
self.assertGreaterEqual(xaml.count("<ToggleButton"), 3)
self.assertIn('IsChecked="{Binding IsOverviewPage, Mode=OneWay}"', xaml)
self.assertIn('IsChecked="{Binding IsSoftwarePage, Mode=OneWay}"', xaml)
self.assertIn('IsChecked="{Binding IsSettingsPage, Mode=OneWay}"', xaml)
self.assertIn('Binding="{Binding StatusTone}"', xaml)
self.assertIn("SoftwarePage.IsBusy", xaml)
self.assertGreaterEqual(xaml.count("<WrapPanel"), 3)
self.assertIn("HasOperationMessage", xaml)
for token in (
"SidebarSelectedBrush",
"SuccessSurfaceBrush",
"WarningSurfaceBrush",
"ErrorSurfaceBrush",
"CardBorderStyle",
"SectionTitleStyle",
):
self.assertIn(token, theme)
self.assertIn('<ControlTemplate TargetType="Button">', theme)
self.assertIn('<ControlTemplate TargetType="ToggleButton">', theme)
def test_workspace_storage_is_global_by_workspace_not_user_directory(self) -> None:
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
store = (PROJECT / "WorkspaceStore.cs").read_text(encoding="utf-8")
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
encoding="utf-8"
)
self.assertIn('Path.Combine(root, "workspaces")', models)
self.assertIn('Binding(job).WorkspaceId.ToString("D")', store)
self.assertNotIn("UserId", store)
@ -48,7 +172,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertIn("workspaceStore.Restore", runner)
def test_runtime_auto_reports_all_discovered_adapter_capabilities(self) -> None:
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
config_store = (PROJECT / "NodeConfigStore.cs").read_text(encoding="utf-8")
self.assertIn(
"var installedCapabilities = adapters.All.Select(item => item.Capability).ToArray()",
@ -62,27 +186,42 @@ class WindowsNodeSourceTests(unittest.TestCase):
def test_node_does_not_expose_arbitrary_execution_primitives(self) -> None:
source = "\n".join(
path.read_text(encoding="utf-8")
for path in PROJECT.glob("*.cs")
for path in PROJECT.rglob("*.cs")
if path.name not in {
"AdapterProcessRunner.cs",
"AdapterExecutionService.cs",
"ProcessSupervisor.cs",
"ConfigurationForm.cs",
"SoftwareRuntimeManager.cs",
"TrayHost.cs",
"SoftwareLocationService.cs",
"ManagedRuntimeInstaller.cs",
}
)
for forbidden in ("Process.Start", "cmd.exe", "powershell.exe", "LabTalk"):
self.assertNotIn(forbidden, source)
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
self.assertIn('"explorer.exe"', form)
self.assertIn("startInfo.ArgumentList.Add(dataRoot.Text)", form)
self.assertNotIn("UseShellExecute = true", form)
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
encoding="utf-8"
)
self.assertIn('"explorer.exe"', tray)
self.assertIn("startInfo.ArgumentList.Add(path)", tray)
self.assertNotIn("UseShellExecute = true", tray)
def test_data_root_is_configurable_and_migrated_before_switching(self) -> None:
settings = (PROJECT / "NodeDataRoot.cs").read_text(encoding="utf-8")
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
page = (
PROJECT / "Presentation" / "ViewModels" / "DataRootPageViewModel.cs"
).read_text(encoding="utf-8")
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
encoding="utf-8"
)
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
controller = (
PROJECT / "Application" / "NodeApplicationController.cs"
).read_text(encoding="utf-8")
composition = (
PROJECT / "Bootstrap" / "NodeCompositionRoot.cs"
).read_text(encoding="utf-8")
self.assertIn('"ZCBOT_WINDOWS_NODE_DATA_DIR"', settings)
self.assertIn("Environment.SpecialFolder.LocalApplicationData", settings)
@ -99,15 +238,13 @@ class WindowsNodeSourceTests(unittest.TestCase):
settings.index("VerifyTree(source, staging"),
settings.index("Directory.Move(staging, target)"),
)
self.assertIn("DataRootMigrationRequested", form)
self.assertIn("jobInbox.HasPendingJobs", form)
self.assertIn(
"NodeDataRootSettings.SaveUserRoot(result.TargetDirectory)", tray
)
self.assertIn("MigrationRequested", page)
self.assertIn("service.HasPendingJobs", page)
self.assertIn("saveDataRoot(result.TargetDirectory)", controller)
self.assertIn("NodeDataRootSettings.SaveUserRoot", composition)
self.assertIn("controller.MigrateDataRootAsync(target)", tray)
self.assertIn("jobPipelines.Values.Concat(exportPipelines.Values)", connection)
self.assertIn(
"Path.GetPathRoot(NodePaths.ForCurrentMachine().RootDirectory)", connection
)
self.assertIn("Path.GetPathRoot(paths.RootDirectory)", connection)
def test_config_field_names_do_not_serialize_plain_node_token(self) -> None:
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
@ -116,28 +253,46 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertNotIn("string NodeToken", stored_record)
def test_configuration_ui_never_displays_or_copies_token(self) -> None:
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
self.assertIn("CreateTextBox(usePassword: true)", form)
self.assertIn("UseSystemPasswordChar = usePassword", form)
self.assertNotIn("NodeToken", form)
self.assertIn("Clipboard.SetText(BuildDiagnosticText(currentConfig))", form)
diagnostics = form.split("private string BuildDiagnosticText", 1)[1].split(
"private async Task RunAnsysAcceptanceAsync", 1
)[0]
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
encoding="utf-8"
)
view_model = (
PROJECT / "Presentation" / "ViewModels" / "MainWindowViewModel.cs"
).read_text(encoding="utf-8")
diagnostic_service = (
PROJECT / "Application" / "DiagnosticService.cs"
).read_text(encoding="utf-8")
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
encoding="utf-8"
)
self.assertIn("<PasswordBox", xaml)
self.assertNotIn("NodeToken", xaml + view_model + diagnostic_service)
self.assertIn("Clipboard.SetText(text)", tray)
diagnostics = diagnostic_service.split("public string Build", 1)[1]
self.assertNotIn("NodeToken", diagnostics)
self.assertIn("内容不包含 Node Token", form)
self.assertIn("清除本机身份", form)
self.assertIn("管理后台删除或禁用云端旧节点", form)
self.assertIn("内容不包含 Node Token", view_model)
self.assertIn("清除本机身份", xaml)
self.assertIn("管理后台删除或禁用云端旧节点", tray)
def test_ansys_acceptance_ui_runs_only_fixed_bundled_suite_before_gate(self) -> None:
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
encoding="utf-8"
)
view_model = (
PROJECT / "Presentation" / "ViewModels" / "AnsysAcceptanceViewModel.cs"
).read_text(encoding="utf-8")
service = (
PROJECT / "Application" / "AnsysAcceptanceService.cs"
).read_text(encoding="utf-8")
adapters = (PROJECT / "NodeAdapters.cs").read_text(encoding="utf-8")
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
self.assertIn("ANSYS Mechanical 真机验收", form)
self.assertIn("运行内置基准验收", form)
self.assertIn("AcceptanceReportPassed", form)
self.assertIn('"ZCBOT_ANSYS_242_VALIDATED", "1"', form)
self.assertIn("EnvironmentVariableTarget.Machine", form)
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
encoding="utf-8"
)
self.assertIn("ANSYS Mechanical 真机验收", xaml)
self.assertIn("运行内置基准验收", xaml)
self.assertIn("ReportPassed", service)
self.assertIn('GateEnvironmentVariable,\n "1"', service)
self.assertIn("EnvironmentVariableTarget.Machine", service)
self.assertIn("SupportsLocalAcceptance", adapters)
self.assertIn('Path.Combine(descriptor.DirectoryPath, "acceptance.py")', runner)
self.assertIn(
@ -145,114 +300,106 @@ class WindowsNodeSourceTests(unittest.TestCase):
)
self.assertIn('["--work-root", root, "--repeat", "20"', runner)
self.assertIn("Adapter acceptance cannot run while a dispatched job is active", runner)
acceptance = form.split("private async Task RunAnsysAcceptanceAsync", 1)[1].split(
"private void EnableAnsysGate", 1
)[0]
self.assertNotIn("new OpenFileDialog", acceptance)
self.assertNotIn("Process.Start", acceptance)
self.assertIn("service.RunAsync(workRoot", view_model)
self.assertNotIn("new OpenFileDialog", view_model + service)
self.assertNotIn("Process.Start", view_model + service)
def test_professional_software_ui_uses_fixed_per_runtime_installers(self) -> None:
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
manager = (PROJECT / "SoftwareRuntimeManager.cs").read_text(encoding="utf-8")
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
encoding="utf-8"
)
service = (
PROJECT / "Presentation" / "Services" / "SoftwareManagementService.cs"
).read_text(encoding="utf-8")
catalog = (PROJECT / "Runtime" / "SoftwareCatalog.cs").read_text(encoding="utf-8")
manager = (PROJECT / "Runtime" / "SoftwareLocationService.cs").read_text(
encoding="utf-8"
)
installer = (PROJECT / "Runtime" / "ManagedRuntimeInstaller.cs").read_text(
encoding="utf-8"
)
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
encoding="utf-8"
)
self.assertIn('CreateSectionTitle("专业软件")', form)
self.assertIn('CreateButton("自动检测", 92)', form)
self.assertIn('CreateButton("选择位置", 92)', form)
self.assertIn('CreateButton("安装 / 更新环境", 150', form)
self.assertIn("panel.SetColumnSpan(path, 2)", form)
self.assertIn("path.ReadOnly = true", form)
software_panel = form.split("private Control CreateSoftwarePanel", 1)[1].split(
"private void RefreshSoftwareCards", 1
)[0]
self.assertIn("panel.SetColumnSpan(status, 2)", software_panel)
self.assertIn("panel.SetColumnSpan(actions, 2)", software_panel)
self.assertIn("SoftwareRuntimeManager.InstallRuntimeAsync", form)
self.assertIn("item.HasActiveJobs", form)
for runtime_id in ('"origin"', '"ansys"', '"blender"'):
self.assertIn(runtime_id, manager)
self.assertIn('Text="本机专业软件"', xaml)
self.assertIn('Content="自动检测"', xaml)
self.assertIn('Content="选择位置"', xaml)
self.assertIn('Content="安装 / 更新环境"', xaml)
self.assertIn('IsReadOnly="True"', xaml)
self.assertIn("ManagedRuntimeInstaller.InstallAsync", service)
self.assertIn("item.HasActiveJobs", service)
for runtime_id in ('"origin"', '"ansys"', '"blender"', '"freecad"', '"cad"'):
self.assertIn(runtime_id, catalog)
self.assertIn("SoftwarePathKind.None", catalog)
self.assertIn('"https://pypi.org/simple/"', catalog)
self.assertIn("definition.DefaultIndexUrl ?? DefaultIndexUrl", installer)
self.assertIn("runtime-only 软件不接受应用路径", manager)
self.assertIn('@"Software\\Zcbot\\WindowsNode\\Software"', manager)
self.assertIn("RegistryHive.LocalMachine", manager)
self.assertIn("RegistryHive.CurrentUser", manager)
self.assertIn('"ZCBOT_BLENDER_EXE"', manager)
self.assertIn('"AWP_ROOT242"', manager)
self.assertIn('"ZCBOT_ORIGIN_EXE"', manager)
self.assertIn('"Origin*.exe"', manager)
self.assertIn('"ZCBOT_BLENDER_EXE"', catalog)
self.assertIn('"ZCBOT_FREECAD_CMD"', catalog)
self.assertIn('"FreeCADCmd.exe"', catalog)
self.assertIn('"AWP_ROOT242"', catalog)
self.assertIn('"ZCBOT_ORIGIN_EXE"', catalog)
self.assertIn('"Origin*.exe"', catalog)
self.assertIn("OriginExecutables", manager)
self.assertIn('Path.Combine(runtimes, definition.RuntimeId)', manager)
self.assertIn("ArgumentList.Add(argument)", manager)
self.assertNotIn('"cmd.exe"', manager.lower())
self.assertNotIn('"powershell.exe"', manager.lower())
self.assertIn("SoftwareRuntimeManager.ApplyProcessEnvironment", runner)
self.assertIn('Path.Combine(runtimes, definition.RuntimeId)', installer)
self.assertIn("ArgumentList.Add(argument)", installer)
self.assertNotIn('"cmd.exe"', installer.lower())
self.assertNotIn('"powershell.exe"', installer.lower())
self.assertIn("SoftwareLocationService.ApplyProcessEnvironment", runner)
self.assertIn("AdapterCatalog.HasActiveExecution", installer)
def test_configuration_window_is_resizable_and_dpi_safe(self) -> None:
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
self.assertIn("ClientSize = new Size(1080, 1050)", form)
self.assertIn("MinimumSize = new Size(760, 560)", form)
self.assertIn("FormBorderStyle.Sizable", form)
self.assertIn("AutoScaleMode.Dpi", form)
self.assertIn("AutoScroll = true", form)
self.assertNotIn("MaximumSize = new Size(410", form)
self.assertIn("注册码默认 10 分钟有效", form)
self.assertIn("成功注册一次后立即失效", form)
self.assertIn("CreateSection", form)
self.assertIn("注册并连接", form)
self.assertNotIn("ContentWidth", form)
self.assertIn("new Padding(24, 20, 24, 20)", form)
self.assertIn("new ColumnStyle(SizeType.Percent, 100)", form)
self.assertIn('AddTab(tabs, "节点概览")', form)
self.assertIn('AddTab(tabs, "专业软件")', form)
self.assertIn('AddTab(tabs, "本机任务")', form)
self.assertIn('AddTab(tabs, "运行设置")', form)
self.assertIn("SizeMode = TabSizeMode.FillToRight", form)
self.assertIn("new NavigationTabControl", form)
self.assertIn("DrawMode = TabDrawMode.OwnerDrawFixed", form)
self.assertIn("Appearance = TabAppearance.FlatButtons", form)
self.assertNotIn("protected override void OnResize", form)
self.assertIn("tab.Controls.Add(content)", form)
self.assertIn("CreateSectionTitle(\"节点信息\")", form)
self.assertIn("CreateSubsectionTitle(\"可用软件\")", form)
self.assertIn("heading.Controls.Add(state, 1, 0)", form)
self.assertNotIn("headingHint", form)
self.assertIn("panel.SetColumnSpan(path, 2)", form)
self.assertIn("panel.SetColumnSpan(status, 2)", form)
self.assertIn("panel.SetColumnSpan(actions, 2)", form)
self.assertIn("BorderedTableLayoutPanel", form)
self.assertIn(
'runtime.Software == "OriginPro" ? "Origin" : runtime.Software', form
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
encoding="utf-8"
)
self.assertIn("FormatCapabilitySummary(adapters.All)", form)
self.assertIn("GroupBy(entry => entry.SoftwareName", form)
self.assertIn("GroupBy(entry => entry.CapabilityName", form)
self.assertIn("ShortCapabilityName(adapter.DisplayName, softwareName)", form)
self.assertIn('CreateButton("立即重连", 112, primary: true)', form)
self.assertIn("ReconnectRequested?.Invoke()", form)
self.assertIn("registrationCard.Visible = !registered", form)
self.assertIn("reconnect.Visible = registered", form)
self.assertIn("resetIdentity.Visible = registered", form)
self.assertIn('CreateSectionTitle("本机任务")', form)
self.assertIn("DataGridView", form)
self.assertIn("jobInbox.ReadJobSnapshots()", form)
self.assertIn('"software_running" => "软件执行中"', form)
self.assertNotIn("HttpClient", form)
self.assertIn('Width="1040"', xaml)
self.assertIn('MinWidth="760"', xaml)
self.assertIn('MinHeight="560"', xaml)
self.assertIn("<ScrollViewer", xaml)
self.assertIn('Content="注册并连接"', xaml)
self.assertIn('Content="节点概览"', xaml)
self.assertIn('Content="专业软件"', xaml)
self.assertIn('Content="本机任务"', xaml)
self.assertIn('Content="运行设置"', xaml)
self.assertIn("<DataGrid", xaml)
self.assertFalse((PROJECT / "ConfigurationForm.cs").exists())
self.assertFalse((PROJECT / "TrayApplicationContext.cs").exists())
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
self.assertIn("form.ReconnectRequested += RestartConnection", tray)
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
encoding="utf-8"
)
self.assertNotIn("legacyForm", tray)
self.assertIn("statusItem.Text = CompactMenuText(status.Message, 24)", tray)
self.assertIn('singleLine[..(length - 1)] + ""', tray)
def test_tray_exit_waits_for_or_cancels_active_jobs(self) -> None:
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
encoding="utf-8"
)
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
host_loop = (PROJECT / "Host" / "NodeConnectionLoop.cs").read_text(
encoding="utf-8"
)
controller = (
PROJECT / "Application" / "NodeApplicationController.cs"
).read_text(encoding="utf-8")
self.assertIn("private async Task ExitNodeAsync()", tray)
self.assertIn("if (exiting || config is null", tray)
self.assertIn("ReadJobSnapshots()", tray)
self.assertIn("if (exiting)", tray)
self.assertIn("controller.ActiveJobCount", tray)
self.assertIn("MessageBoxButtons.YesNoCancel", tray)
self.assertIn("connectionLoop?.CancelActiveJobsForExit()", tray)
self.assertIn("NodeShutdownMode.CancelJobs", tray)
self.assertIn("tray.ContextMenuStrip.Enabled = false", tray)
self.assertLess(tray.index("await connectionTask"), tray.index("ExitThread();"))
self.assertIn("internal void CancelActiveJobsForExit()", connection)
self.assertLess(tray.index("await controller.StopAsync"), tray.index("shutdown();"))
self.assertIn("stopTask ??= StopOnceAsync(mode)", controller)
self.assertIn("connection?.CancelActiveJobsForExit()", controller)
self.assertIn("await StopConnectionAsync()", controller)
self.assertIn("public void CancelActiveJobsForExit()", host_loop)
self.assertIn("coordinator.CancelActiveJobsForExit()", host_loop)
self.assertIn("adapters.Find(job.Capability)?.Cancel(job.JobId)", connection)
self.assertIn("inputDownloader.DownloadAsync(job, forcedStop.Token)", connection)
self.assertIn('"NODE_EXIT_CANCELLED"', connection)
@ -260,11 +407,20 @@ class WindowsNodeSourceTests(unittest.TestCase):
def test_local_job_monitor_is_persisted_and_software_neutral(self) -> None:
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
monitor = (
PROJECT / "Application" / "JobMonitorService.cs"
).read_text(encoding="utf-8")
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
models = (PROJECT / "JobMonitorModels.cs").read_text(encoding="utf-8")
self.assertIn('"state.json"', inbox)
self.assertIn("ReadJobSnapshots", inbox)
self.assertIn("AtomicWrite(path, content, overwrite: true)", inbox)
repository = (PROJECT / "Jobs" / "JobRepository.cs").read_text(
encoding="utf-8"
)
state_machine = (PROJECT / "Jobs" / "JobStateMachine.cs").read_text(
encoding="utf-8"
)
self.assertIn('"state.json"', monitor)
self.assertIn("ReadSnapshots", monitor)
self.assertIn("AtomicFile.Write(path, content, overwrite: true)", repository)
for stage in (
"accepted",
"downloading_inputs",
@ -275,26 +431,36 @@ class WindowsNodeSourceTests(unittest.TestCase):
"failed",
"cancelled",
):
self.assertIn(f'"{stage}"', inbox + connection)
self.assertIn(
f'"{stage}"',
inbox + monitor + repository + state_machine + connection,
)
self.assertIn("JobDisplaySnapshot", models)
self.assertNotIn("origin_running", inbox + connection + models)
self.assertNotIn("origin_running", inbox + monitor + connection + models)
def test_startup_and_runtime_installation_are_owned_by_the_ui(self) -> None:
startup = (PROJECT / "StartupRegistration.cs").read_text(encoding="utf-8")
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
manager = (PROJECT / "SoftwareRuntimeManager.cs").read_text(encoding="utf-8")
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
encoding="utf-8"
)
view_model = (
PROJECT / "Presentation" / "ViewModels" / "MainWindowViewModel.cs"
).read_text(encoding="utf-8")
installer = (PROJECT / "Runtime" / "ManagedRuntimeInstaller.cs").read_text(
encoding="utf-8"
)
self.assertFalse((ROOT / "install-windows-node.bat").exists())
self.assertIn('Text = "登录 Windows 后自动启动节点"', form)
self.assertIn("StartupRegistration.SetEnabled", form)
self.assertIn('Content="登录 Windows 后自动启动节点"', xaml)
self.assertIn("startupRegistration.SetEnabled", view_model)
self.assertIn('@"Software\\Microsoft\\Windows\\CurrentVersion\\Run"', startup)
self.assertIn("Environment.ProcessPath", startup)
self.assertIn("FindPython312Async", manager)
self.assertIn('new BootstrapPython("py.exe", ["-3.12"])', manager)
self.assertIn('new BootstrapPython("python.exe", [])', manager)
self.assertIn('"ZCBOT_PIP_INDEX_URL"', manager)
self.assertIn("FindPython312Async", installer)
self.assertIn('new BootstrapPython("py.exe", ["-3.12"])', installer)
self.assertIn('new BootstrapPython("python.exe", [])', installer)
self.assertIn('"ZCBOT_PIP_INDEX_URL"', installer)
self.assertIn(
'"https://pypi.tuna.tsinghua.edu.cn/simple/"', manager
'"https://pypi.tuna.tsinghua.edu.cn/simple/"', installer
)
def test_publish_output_contains_the_complete_node_payload(self) -> None:
@ -331,6 +497,22 @@ class WindowsNodeSourceTests(unittest.TestCase):
"adapters\\blender.scene.author@v3\\blender.scene.author.v3.json",
project,
)
self.assertIn(
"..\\..\\software-contracts\\cad.geometry.prepare.v1.json",
project,
)
self.assertIn(
"adapters\\cad.geometry.prepare@v1\\cad.geometry.prepare.v1.json",
project,
)
self.assertIn(
"..\\..\\software-contracts\\freecad.model.author.v1.json",
project,
)
self.assertIn(
"adapters\\freecad.model.author@v1\\freecad.model.author.v1.json",
project,
)
self.assertNotIn("software-contracts\\%(Filename)%(Extension)", project)
self.assertFalse((ROOT / "install-windows-node.ps1").exists())
self.assertFalse((ROOT / "install-origin-runtime.ps1").exists())
@ -354,30 +536,36 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertFalse((ROOT / "package-windows-node.ps1").exists())
def test_auth_rejection_is_distinct_from_http_websocket_handshake_failure(self) -> None:
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
self.assertIn('socket.HttpStatusCode is HttpStatusCode.Unauthorized', connection)
self.assertIn('or HttpStatusCode.Forbidden', connection)
self.assertIn("WebSocket 握手被拒绝,请检查服务端或反向代理", connection)
self.assertIn("catch (NodeEndpointException exception)", connection)
self.assertIn("throw new NodeEndpointException", connection)
self.assertIn("(int?)result.CloseStatus == 4003", connection)
self.assertIn("节点身份已被服务端拒绝", connection)
self.assertNotIn("Node credentials were rejected", connection)
client = (PROJECT / "Protocol" / "NodeProtocolClient.cs").read_text(
encoding="utf-8"
)
session = (PROJECT / "Host" / "NodeSession.cs").read_text(encoding="utf-8")
self.assertIn('socket.HttpStatusCode is HttpStatusCode.Unauthorized', client)
self.assertIn('or HttpStatusCode.Forbidden', client)
self.assertIn("WebSocket 握手被拒绝,请检查服务端或反向代理", session)
self.assertIn("catch (NodeEndpointException exception)", session)
self.assertIn("throw new NodeEndpointException", client)
self.assertIn("(int?)result.CloseStatus == 4003", client)
self.assertIn("节点身份已被服务端拒绝", client)
self.assertNotIn("Node credentials were rejected", client + session)
def test_adapter_runtime_probe_is_worker_owned_and_reported(self) -> None:
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
encoding="utf-8"
)
worker = (
ROOT / "adapters" / "origin.plot@v2" / "worker.py"
).read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
self.assertIn('r"Origin.ApplicationSI\\CLSID"', worker)
self.assertIn('["--probe"]', runner)
self.assertIn('root.GetProperty("adapter_version")', runner)
self.assertIn('startInfo.Environment["PYTHONUTF8"] = "1"', runner)
self.assertIn('startInfo.Environment["PYTHONIOENCODING"] = "utf-8"', runner)
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
self.assertIn('AppendLine($"Adapter: {adapter.AdapterVersion}")', form)
self.assertIn('$"{group.Key} · 版本未知"', form)
diagnostics = (
PROJECT / "Application" / "DiagnosticService.cs"
).read_text(encoding="utf-8")
self.assertIn('AppendLine($"Adapter: {adapter.AdapterVersion}")', diagnostics)
self.assertIn(
"&& !jobInbox.HasPendingJobs",
connection,
@ -385,7 +573,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertIn("originAdapter?.HasActiveJobs", connection)
self.assertIn(
"ReadRecoverableJobs().Any(item => item.Terminal is null)",
(PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8"),
(PROJECT / "Jobs" / "JobRepository.cs").read_text(encoding="utf-8"),
)
self.assertNotIn("CreateInstance", worker)
for marker in (
@ -398,7 +586,22 @@ class WindowsNodeSourceTests(unittest.TestCase):
def test_job_offer_is_persisted_before_acceptance(self) -> None:
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
monitor = (
PROJECT / "Application" / "JobMonitorService.cs"
).read_text(encoding="utf-8")
repository = (PROJECT / "Jobs" / "JobRepository.cs").read_text(
encoding="utf-8"
)
recovery = (PROJECT / "Jobs" / "JobRecoveryService.cs").read_text(
encoding="utf-8"
)
atomic_file = (PROJECT / "Persistence" / "AtomicFile.cs").read_text(
encoding="utf-8"
)
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
protocol_client = (PROJECT / "Protocol" / "NodeProtocolClient.cs").read_text(
encoding="utf-8"
)
adapters = (PROJECT / "NodeAdapters.cs").read_text(encoding="utf-8")
self.assertIn("adapters.Find(capability)", inbox)
self.assertIn("adapter.ValidateRequest(request)", inbox)
@ -411,44 +614,51 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertNotIn('AppContext.BaseDirectory, "software-contracts"', adapters)
self.assertNotIn("IsValidOriginRequest", inbox)
self.assertNotIn("IsValidPlot", inbox)
self.assertIn('root.TryGetProperty("input_transfers"', inbox)
self.assertIn('root.TryGetProperty("input_transfers"', inbox + monitor)
self.assertIn('root.TryGetProperty("request_summary"', inbox + monitor)
self.assertIn("ReadDisplayTitle(root)", monitor)
self.assertIn("foreach (var action in operation.EnumerateObject())", monitor)
self.assertIn("transfers.GetArrayLength() > 16", inbox)
self.assertNotIn("transfers.GetArrayLength() is < 1 or > 16", inbox)
self.assertIn('"input", key, filename', inbox)
self.assertIn("FileOptions.WriteThrough", inbox)
self.assertIn("stream.Flush(flushToDisk: true)", inbox)
self.assertIn('"input", key, filename', repository)
self.assertIn("FileOptions.WriteThrough", atomic_file)
self.assertIn("stream.Flush(flushToDisk: true)", atomic_file)
new_record = inbox.split("var record =", 1)[1].split("private static JsonElement?", 1)[0]
self.assertLess(
new_record.index("AtomicWrite(requestPath, record"),
new_record.index("AtomicFile.Write(requestPath, record"),
new_record.index("JobOfferResult.Accept"),
)
self.assertIn('offerResult.Accepted ? "job_accept" : "job_reject"', connection)
self.assertIn("sendLock.WaitAsync", connection)
self.assertIn("sendLock.WaitAsync", protocol_client)
self.assertIn("!jobInbox.HasPendingJobs", connection)
self.assertIn("capability_runtime = capabilityRuntime", connection)
self.assertIn("adapter.RunAsync(job)", connection)
self.assertIn("ReportRecoverableJobsAsync", connection)
self.assertIn("ConcurrentDictionary<Guid, Task> jobPipelines", connection)
self.assertIn("StartJobPipeline(socket, acceptedJob)", connection)
self.assertIn("StartJobPipeline(client, acceptedJob)", connection)
self.assertIn("inputDownloader.DownloadAsync(job, forcedStop.Token)", connection)
self.assertNotIn("CreateLinkedTokenSource(cancellationToken, forcedStop.Token)", connection)
self.assertIn('stage = "uploading_outputs"', connection)
self.assertIn('stage = "software_running"', connection)
self.assertNotIn('stage = "origin_running"', connection)
self.assertIn("&& !job.UploadComplete", connection)
self.assertIn("&& !job.CloudTerminal", connection)
self.assertIn("JobRecoveryAction.ResumeOutputUpload", connection)
self.assertIn("job.UploadComplete || job.CloudTerminal", recovery)
self.assertIn(
"await Task.WhenAll(jobPipelines.Values.Concat(exportPipelines.Values).ToArray())",
connection,
)
self.assertIn("StartJobPipeline(socket, job)", connection)
self.assertIn("StartJobPipeline(client, job, recovery: true)", connection)
self.assertIn('stage = "downloading_inputs"', connection)
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', inbox)
self.assertIn("AtomicWrite(requestPath, updated, overwrite: true)", inbox)
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', repository)
self.assertIn("AtomicFile.Write(requestPath, updated, overwrite: true)", inbox)
downloader = (PROJECT / "JobInputDownloader.cs").read_text(encoding="utf-8")
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', downloader)
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', downloader)
http_client = (PROJECT / "Transfers" / "NodeHttpClient.cs").read_text(
encoding="utf-8"
)
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', http_client)
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', http_client)
self.assertIn("SendWithRetryAsync", downloader)
self.assertIn("HttpCompletionOption.ResponseHeadersRead", downloader)
self.assertIn("IncrementalHash.CreateHash", downloader)
self.assertIn("total > expectedSize", downloader)
@ -456,8 +666,13 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertNotIn("Process.Start", downloader)
def test_adapter_worker_launch_is_manifest_driven_and_terminal_driven(self) -> None:
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
encoding="utf-8"
)
supervisor = (PROJECT / "Adapters" / "ProcessSupervisor.cs").read_text(
encoding="utf-8"
)
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
project = (PROJECT / "Zcbot.WindowsNode.csproj").read_text(encoding="utf-8")
worker = (
ROOT / "adapters" / "origin.plot@v2" / "worker.py"
@ -473,8 +688,8 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', runner)
self.assertIn('"NODE_RESTARTED_DURING_JOB"', runner)
self.assertIn("CancellationTokenSource.CreateLinkedTokenSource", runner)
self.assertIn("process.Kill(entireProcessTree: true)", runner)
self.assertIn('type.GetString() == "job_cancel"', connection)
self.assertIn("process.Kill(entireProcessTree: true)", supervisor)
self.assertIn('message.Type == "job_cancel"', connection)
self.assertIn('"cancelled", "USER_CANCELLED"', connection)
self.assertIn("..\\adapters\\**\\*", project)
self.assertIn("descriptor.Manifest.WorkerTimeoutMinutes", runner)
@ -495,24 +710,29 @@ class WindowsNodeSourceTests(unittest.TestCase):
def test_origin_adapter_can_be_packaged_without_building_node(self) -> None:
script = (ROOT / "package-origin-adapter.bat").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
self.assertIn('"adapters\\origin.plot@v2\\adapter.json"', script)
self.assertIn('"adapters\\origin.plot@v2\\worker.py"', script)
self.assertIn('"adapters\\origin.plot@v2\\acceptance.py"', script)
self.assertIn('"..\\software-contracts\\origin.plot.v2.json"', script)
self.assertNotIn("dotnet", script.lower())
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
self.assertIn("Contract SHA-256: {adapter.ContractSha256}", form)
self.assertIn("Workspace protocol:", form)
self.assertIn("Workspace state:", form)
diagnostics = (
PROJECT / "Application" / "DiagnosticService.cs"
).read_text(encoding="utf-8")
self.assertIn("Contract SHA-256: {adapter.ContractSha256}", diagnostics)
self.assertIn("Workspace protocol:", diagnostics)
self.assertIn("Workspace state:", diagnostics)
self.assertIn("workspace_protocol_version", connection)
self.assertIn("contract_sha256", connection)
uploader = (PROJECT / "JobOutputUploader.cs").read_text(encoding="utf-8")
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', uploader)
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', uploader)
self.assertIn('DefaultRequestHeaders.Add("X-Lease-Id"', uploader)
http_client = (PROJECT / "Transfers" / "NodeHttpClient.cs").read_text(
encoding="utf-8"
)
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', http_client)
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', http_client)
self.assertIn('request.Headers.Add("X-Lease-Id"', http_client)
self.assertIn("SHA256.HashDataAsync", uploader)
self.assertIn("upload-complete.json", connection + uploader)
self.assertIn("adapter.PreviewOutputIds", connection)
@ -536,11 +756,17 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertNotIn('".tmp-" + Guid.NewGuid()', uploader)
self.assertNotIn("Process.Start", uploader)
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
self.assertIn("_ = form.Handle", tray)
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
encoding="utf-8"
)
controller = (
PROJECT / "Application" / "NodeApplicationController.cs"
).read_text(encoding="utf-8")
self.assertNotIn("legacyForm", tray)
self.assertIn("window.ShowAndActivate()", tray)
self.assertIn(
'UpdateStatus(NodeStatus.Create(NodeState.Connecting, "正在连接 zcbot…"))',
tray,
'Publish(NodeStatus.Create(NodeState.Connecting, "正在连接 zcbot…"))',
controller,
)
def test_origin_analysis_adapter_can_be_packaged_without_building_node(self) -> None:
@ -572,9 +798,36 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertIn("tar.exe -a -c -f", script)
self.assertNotIn("dotnet", script.lower())
def test_cad_adapter_can_be_packaged_without_building_node(self) -> None:
script = (ROOT / "package-cad-adapter.bat").read_text(encoding="utf-8")
worker = (
ROOT / "adapters" / "cad.geometry.prepare@v1" / "worker.py"
).read_text(encoding="utf-8")
self.assertIn("adapters\\cad.geometry.prepare@v1\\adapter.json", script)
self.assertIn("adapters\\cad.geometry.prepare@v1\\worker.py", script)
self.assertIn("adapters\\cad.geometry.prepare@v1\\requirements.txt", script)
self.assertIn("..\\software-contracts\\cad.geometry.prepare.v1.json", script)
self.assertIn("tar.exe -a -c -f", script)
self.assertNotIn("dotnet", script.lower())
self.assertIn("INPUT_PATH_ESCAPES_JOB", worker)
self.assertIn("OUTPUT_PATH_ESCAPES_JOB", worker)
self.assertIn("STEP_ROUNDTRIP_VOLUME_MISMATCH", worker)
for forbidden in ("subprocess", "eval(", "exec(", "os.system", "requests."):
self.assertNotIn(forbidden, worker)
def test_freecad_adapter_can_be_packaged_without_building_node(self) -> None:
script = (ROOT / "package-freecad-adapter.bat").read_text(encoding="utf-8")
for filename in ("adapter.json", "worker.py", "freecad_worker.py", "recipe.py", "preview_renderer.py", "requirements.txt", "acceptance.py"):
self.assertIn(filename, script)
self.assertIn("..\\software-contracts\\freecad.model.author.v1.json", script)
self.assertIn("tar.exe -a -c -f", script)
self.assertNotIn("dotnet", script.lower())
def test_local_job_monitor_never_shows_update_before_acceptance(self) -> None:
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
self.assertIn("state.UpdatedAt > acceptedAt", inbox)
monitor = (
PROJECT / "Application" / "JobMonitorService.cs"
).read_text(encoding="utf-8")
self.assertIn("state.UpdatedAt > acceptedAt", monitor)
if __name__ == "__main__":

Some files were not shown because too many files have changed in this diff Show More