Compare commits
35 Commits
codex/orig
...
main
| Author | SHA1 | Date |
|---|---|---|
|
|
5b052c464b | |
|
|
ca73a4affb | |
|
|
09cb7d327b | |
|
|
4af4816aa2 | |
|
|
2c6091db03 | |
|
|
dbaa9faa6e | |
|
|
2ff648afe6 | |
|
|
e82b6842c7 | |
|
|
6b4c54a1fe | |
|
|
7b541a2256 | |
|
|
11108f31c2 | |
|
|
aa8ccec738 | |
|
|
a1068f0ec1 | |
|
|
f50bfc1bcf | |
|
|
572da9a365 | |
|
|
605e4eecfe | |
|
|
e47ff1f942 | |
|
|
7917331e5a | |
|
|
37aa83f8fb | |
|
|
38deea7d1d | |
|
|
0d5cbb8407 | |
|
|
e05de152ad | |
|
|
08754be1a4 | |
|
|
7ffa9eb645 | |
|
|
3bed132c30 | |
|
|
510033d06f | |
|
|
7f439a91d5 | |
|
|
0bc6d695ef | |
|
|
a7c3ee8b16 | |
|
|
b6d0e34b51 | |
|
|
bff6125cd1 | |
|
|
9728fc3dd2 | |
|
|
a475207781 | |
|
|
b0a7a86aa8 | |
|
|
a219f8a0c0 |
38
CHANGELOG.md
38
CHANGELOG.md
|
|
@ -6,6 +6,44 @@
|
||||||
> 开发中的用户文案可先写入 `## Unreleased`;该区不会被前端解析,正式发布时再替换为数字版本和日期。
|
> 开发中的用户文案可先写入 `## Unreleased`;该区不会被前端解析,正式发布时再替换为数字版本和日期。
|
||||||
> 工程口径的完整记录见 `PROGRESS.md` / git log。
|
> 工程口径的完整记录见 `PROGRESS.md` / git log。
|
||||||
|
|
||||||
|
## Unreleased
|
||||||
|
|
||||||
|
- 修复深度思考或事件流重连期间点击“停止”偶尔没有反应的问题;停止信号会在状态落库前立即送达,重复停止也会可靠补发。
|
||||||
|
|
||||||
|
## 0.71.0 — 2026-09-03
|
||||||
|
|
||||||
|
- 提高工具参数异常时的自动恢复率,产物发布和工具健康分类也更加稳定准确。
|
||||||
|
|
||||||
|
- 新增 FreeCAD 受控建模:可通过参数化 Recipe 组合基础体、布尔运算、变换和 STEP 导入,交付可继续调整的 FCStd 工程、STEP、模型清单与无显卡预览,并支持版本续作和失败回滚。
|
||||||
|
|
||||||
|
- 管理后台会异步汇总工具健康大数,并避免短时间内重复请求;工具异常统一按结构化事件统计,代理主动保护与真正故障分开展示,多人同时查看或自动刷新时不再反复扫描历史消息。
|
||||||
|
|
||||||
|
- 单用户可同时运行的重型任务由 2 个提升到 3 个;任务等待执行容量时,对话会直接说明是当前用户、整机或宿主内存限制,获得槽位后自动继续。
|
||||||
|
|
||||||
|
- 管理员现在可在管理后台安全录入、测试和更换模型、媒体、检索及语音服务凭据,并查看来源、脱敏尾号和可用状态;数据库凭据可随时删除并回退原有环境配置。DeepSeek 余额低于 30 元、额度耗尽或认证失败时会主动提醒。
|
||||||
|
|
||||||
|
- 新增中性 CAD 几何准备:可生成或检查 STEP,组合基础体、截面、布尔、圆角、阵列等受控建模步骤,并交付几何清单和预览,结果可继续用于 ANSYS、Abaqus 等分析任务。
|
||||||
|
|
||||||
|
- Mermaid 图表采用更清晰的科研配色与更精致的图框,新生成的流程图还会按数据、处理、判断、结果等角色使用协调的语义色,减少单调的灰白图。
|
||||||
|
|
||||||
|
- 管理后台改为常驻的紧凑总览与“运行、用量、用户、资源、集成、质量”分组工作区;总览卡可直接定位到同口径详情,执行容量仍可从侧栏快速查看,桌面端和移动端都能更快进入具体管理内容。
|
||||||
|
|
||||||
|
## 0.70.0 — 2026-09-02
|
||||||
|
|
||||||
|
- 容器长任务会在整机繁忙时安全排队,后台任务可查看或取消排队状态;管理员可在“容器状态”查看实时容量,并通过“容器依赖”分析用户临时安装的 Python 依赖及占用。
|
||||||
|
|
||||||
|
- 软件作业与轮次导航统一收在对话右侧,并增加醒目的数量徽章;既可悬停速览,也可固定展开,进行中或未读作业会直接通过数字提醒。
|
||||||
|
|
||||||
|
- 模型体验进一步升级:思考过程恢复实时显示,新版 GLM 向默认档位开放,并支持直接理解上传图片和延续长工具任务的分析状态。
|
||||||
|
|
||||||
|
- 专业软件节点启用完整的新管理界面,节点状态、首次注册、软件环境、本机任务、ANSYS 验收和数据目录迁移均可直接管理;应用图标、导航、状态提示和窄窗口布局也更清晰,关闭窗口后仍会继续在托盘运行。
|
||||||
|
|
||||||
|
## 0.69.1 — 2026-08-27
|
||||||
|
|
||||||
|
- 专业软件任务的进度与结果改为在当前对话左侧集中堆叠,不再用多条结果消息打断正文;可直接打开 Blender 等软件的多张预览,手机端从“软件结果”入口统一查看。
|
||||||
|
|
||||||
|
- 修复专业软件已经出图后,对话偶尔仍持续显示“思考中”的问题。
|
||||||
|
|
||||||
## 0.69.0 — 2026-08-26
|
## 0.69.0 — 2026-08-26
|
||||||
|
|
||||||
- Origin 新增受控科研分析,可进行数据体检、归一化、平滑、求导、积分和线性拟合,并交付结果表、诊断信息及可继续检查的 Origin 工程。
|
- Origin 新增受控科研分析,可进行数据体检、归一化、平滑、求导、积分和线性拟合,并交付结果表、诊断信息及可继续检查的 Origin 工程。
|
||||||
|
|
|
||||||
59
DESIGN.md
59
DESIGN.md
|
|
@ -136,7 +136,7 @@ Eval 与生产 core 解耦,通过现有 `/v1` API 创建专用任务、监听
|
||||||
|
|
||||||
默认 `deepseek_v4.flash`;复杂 bug / 终稿升 pro + reasoning_effort=max;fallback 手动切 Claude。成本量级:修 bug flash ~$0.01 / 完整申报书 flash ~$0.30(pro-max ~$1.5,Opus ~$10+)。99% 任务 flash 够用。
|
默认 `deepseek_v4.flash`;复杂 bug / 终稿升 pro + reasoning_effort=max;fallback 手动切 Claude。成本量级:修 bug flash ~$0.01 / 完整申报书 flash ~$0.30(pro-max ~$1.5,Opus ~$10+)。99% 任务 flash 够用。
|
||||||
|
|
||||||
模型思考参数由 profile 统一表达:`thinking_enabled` 只表示开关,`thinking_transport` 只表示已验证的传输协议,`reasoning_effort` 只表示开启后的推理强度,`reasoning_replay` 只表示历史 reasoning 的 provider 回传策略(`none` / `tool_calls` / `all`);`core/llm_params.py` 是请求参数构造唯一入口,`core/context.py` 是历史消息清洗唯一入口。原始 assistant 响应仍完整落库供展示与导出,发模型前才按 profile 裁剪,且上下文统计、压缩与折叠都使用裁剪后的请求视图。DeepSeek、GLM、方舟当前共享 `extra_body` 协议,DeepSeek V4 仅为带工具调用的 assistant 消息保留 reasoning,未验证网关明确用 `none`、不猜参数协议,主循环不再按 family 分支。`/v1/models` 只返回语义明确的 `thinking_enabled`。
|
模型思考参数由 profile 统一表达:`thinking_enabled` 只表示开关,`thinking_transport` 只表示已验证的传输协议,`reasoning_effort` 只表示开启后的推理强度,`thinking_clear` 表示 provider 是否清除历史思考,`reasoning_replay` 表示状态生命周期(`none` / `tool_turn` / `conversation` / `provider_managed`);`core/llm_params.py` 是请求参数构造唯一入口,`core/context.py` 是历史消息清洗唯一入口。原始 assistant 响应完整落库,provider-bound 副本只向相同生产模型回放未改写的 reasoning:DeepSeek V4 仅保留当前用户轮次的工具链状态,GLM-5.3 Flash 在同模型会话内保留完整状态,未验证网关明确用 `none`,未来签名/加密 block 走 `provider_managed`。模型切换、上下文折叠和普通压缩都在同一入口应用隔离;上下文统计使用裁剪后的请求视图,原生图片 token 不反向污染 chars/token 校准。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -174,7 +174,7 @@ Eval 与生产 core 解耦,通过现有 `/v1` API 创建专用任务、监听
|
||||||
|
|
||||||
**无感部署(蓝绿,0.41)**:生产双 systemd 实例(`ZCBOT_INSTANCE=blue/green`)+ nginx upstream 切流(对外仍 8765),取代单实例 restart 的 503 窗口。双实例并存的互踩由三件事消解:`tasks.run_owner`(0020)让 reaper 只收自己色、sandbox 容器名/label 带实例色各管各的、微信长轮询 PG advisory lock 选主;定时任务 claim 本就 SKIP LOCKED 天然安全。部署编排在 `deploy/update_bluegreen.sh`(流程/兜底见 RUN)。
|
**无感部署(蓝绿,0.41)**:生产双 systemd 实例(`ZCBOT_INSTANCE=blue/green`)+ nginx upstream 切流(对外仍 8765),取代单实例 restart 的 503 窗口。双实例并存的互踩由三件事消解:`tasks.run_owner`(0020)让 reaper 只收自己色、sandbox 容器名/label 带实例色各管各的、微信长轮询 PG advisory lock 选主;定时任务 claim 本就 SKIP LOCKED 天然安全。部署编排在 `deploy/update_bluegreen.sh`(流程/兜底见 RUN)。
|
||||||
|
|
||||||
**broker 外置(Redis pub/sub,✅ 0.42)**:0.41 落地时 event/cancel broker 留在进程内,代价是切换窗口内"刷新看不到旧实例 run 直播 / 停止送达不到"两个边缘。2026-07-06 重评(用户量已非个位数)决定实施 —— ①部署时总有 in-flight run,窗口边缘从偶发变常态;②单进程逼近天花板后,最近的扩容手段是稳态双实例同时接流量,broker 外置是它的硬前提(否则 POST 与 SSE 落不同实例直播全瞎)。选 Redis 不选 PG LISTEN/NOTIFY(token 级 delta 全过主库太吵:NOTIFY 全局队列 + 8KB payload 限制,把实时路径耦到 PG)、不选 nginx sticky hash(upstream 变更时 hash 重排,恰在部署窗口失效,治标)。实现(`web/broker.py`,LocalRunBroker/RedisRunBroker 同接口鸭子类型):`ZCBOT_REDIS_URL` env 开关,不设即进程内(dev 零影响);event 走 `PUBLISH zcbot:ev:<tid>` + 单条 async pubsub reader 路由到本地订阅 queue(Redis 只管跨进程一跳,fan-out 最后一公里仍在进程内);done = SETEX key(60s)+ publish 双通道,订阅先挂 channel 再查 key 不漏;cancel = SETEX/GET/DEL key,loop 在 chunk 间 poll(localhost ~0.1ms)。取消另以共享 PG 中已经提交的 `tasks.run_status='cancelling'` 作可靠兜底:运行线程每秒低频读取一次,未配置 Redis 的蓝绿窗口或 Redis 运行中故障时仍可跨实例停止;PG 不承载 token 直播事件。容错纪律:启动 ping 不通 fail-fast(同 sandbox init),运行中失败 10s 节流 log + 降级丢帧,reader 1s 退避重连。Redis 不解决的:run 本体仍绑进程(进程死 run 死,drain/reaper 不变)、线程池上限(调 `ZCBOT_RUN_MAX_WORKERS` 的事)。
|
**broker 外置(Redis pub/sub,✅ 0.42)**:0.41 落地时 event/cancel broker 留在进程内,代价是切换窗口内"刷新看不到旧实例 run 直播 / 停止送达不到"两个边缘。2026-07-06 重评(用户量已非个位数)决定实施 —— ①部署时总有 in-flight run,窗口边缘从偶发变常态;②单进程逼近天花板后,最近的扩容手段是稳态双实例同时接流量,broker 外置是它的硬前提(否则 POST 与 SSE 落不同实例直播全瞎)。选 Redis 不选 PG LISTEN/NOTIFY(token 级 delta 全过主库太吵:NOTIFY 全局队列 + 8KB payload 限制,把实时路径耦到 PG)、不选 nginx sticky hash(upstream 变更时 hash 重排,恰在部署窗口失效,治标)。实现(`web/broker.py`,LocalRunBroker/RedisRunBroker 同接口鸭子类型):`ZCBOT_REDIS_URL` env 开关,不设即进程内(dev 零影响);event 走 `PUBLISH zcbot:ev:<tid>` + 单条 async pubsub reader 路由到本地订阅 queue(Redis 只管跨进程一跳,fan-out 最后一公里仍在进程内);done = SETEX key(60s)+ publish 双通道,订阅先挂 channel 再查 key 不漏;cancel 在鉴权后先置本实例 Event,再 SETEX/GET/DEL key 跨实例传递,loop 在 chunk 间优先 poll 本地信号(通常约 100ms 内退出,不等待 Redis 或 DB 往返)。取消另以共享 PG 中已经提交的 `tasks.run_status='cancelling'` 作可靠兜底:运行线程每秒低频读取一次,未配置 Redis 的蓝绿窗口或 Redis 运行中故障时仍可跨实例停止;PG 不承载 token 直播事件。容错纪律:启动 ping 不通 fail-fast(同 sandbox init),运行中失败 10s 节流 log + 降级丢帧,reader 1s 退避重连。Redis 不解决的:run 本体仍绑进程(进程死 run 死,drain/reaper 不变)、线程池上限(调 `ZCBOT_RUN_MAX_WORKERS` 的事)。
|
||||||
|
|
||||||
### 7.1 心智模型:Task 一等公民 + Dir 文件副视图
|
### 7.1 心智模型:Task 一等公民 + Dir 文件副视图
|
||||||
|
|
||||||
|
|
@ -184,7 +184,7 @@ Eval 与生产 core 解耦,通过现有 `/v1` API 创建专用任务、监听
|
||||||
|
|
||||||
**对话产物与生命周期(0025/0028/0033)**:真实文件仍是内容事实源;`artifacts` 表记录已发布产物的稳定身份和生命周期,包含 user-root 相对当前路径、来源 task、复制来源、可空的 `software_job_id`、哈希/大小及 active/deleted、回收路径。`software_job_id` 不设外键,非空即表示该正式产物由对应专业软件 Job 生成;复制品保留该来源,Job 或 task 生命周期结束也不抹除来源标识。新 `messages.artifact_refs` 使用 `{version:2, artifact_id, scope:"working_dir", path:"reports/a.pdf", label?:"最终报告"}`;`path` 是兼容快照,预览/下载优先按 `artifact_id` 找当前路径,因此移动或重命名后历史卡片仍有效。version 1 和 `NULL` 旧消息继续走原 task-scoped 兼容链。普通源码树、中间文件、`.meta/` 技术信息和配套资源不登记;agent 仅用 `publish_artifacts` 显式提升少量最终文件。移动保持身份,复制为每个副本创建新身份并记录直接来源;删除将文件移入 `.zcbot_artifact_trash/` 并软删记录,普通文件仍物理删除。
|
**对话产物与生命周期(0025/0028/0033)**:真实文件仍是内容事实源;`artifacts` 表记录已发布产物的稳定身份和生命周期,包含 user-root 相对当前路径、来源 task、复制来源、可空的 `software_job_id`、哈希/大小及 active/deleted、回收路径。`software_job_id` 不设外键,非空即表示该正式产物由对应专业软件 Job 生成;复制品保留该来源,Job 或 task 生命周期结束也不抹除来源标识。新 `messages.artifact_refs` 使用 `{version:2, artifact_id, scope:"working_dir", path:"reports/a.pdf", label?:"最终报告"}`;`path` 是兼容快照,预览/下载优先按 `artifact_id` 找当前路径,因此移动或重命名后历史卡片仍有效。version 1 和 `NULL` 旧消息继续走原 task-scoped 兼容链。普通源码树、中间文件、`.meta/` 技术信息和配套资源不登记;agent 仅用 `publish_artifacts` 显式提升少量最终文件。移动保持身份,复制为每个副本创建新身份并记录直接来源;删除将文件移入 `.zcbot_artifact_trash/` 并软删记录,普通文件仍物理删除。
|
||||||
|
|
||||||
**用户消息附件(0031)**:`messages.attachment_refs` 与助手产物分开表达输入附件,元素为 `{version:1, scope:"working_dir", path, label, kind, media_type, size_bytes}`;文件仍是事实源,不登记为已发布 artifact,也不承诺独立生命周期。`payload.content` 只保存用户自然语言,模型上下文在内存中按附件类型补兼容提示,避免 UI 协议污染正文。`NULL` 表示旧客户端/旧历史,前端继续解析正文标记;`[]` 表示新消息明确无附件。独立 attachment 表暂缓,只有出现跨消息复用、稳定身份、版本快照或附件级审计需求时再抽象 `file_assets + message_attachments`。
|
**用户消息附件(0031)**:`messages.attachment_refs` 与助手产物分开表达输入附件,元素为 `{version:1, scope:"working_dir", path, label, kind, media_type, size_bytes}`;文件仍是事实源,不登记为已发布 artifact,也不承诺独立生命周期。`payload.content` 只保存用户自然语言,Session 内存补兼容路径提示并保留私有附件引用;发模型前按 `input_modalities` 决定纯文本退化或把图片物化为 OpenAI-compatible Base64 `image_url` block,Base64 永不落库、物化仍经过 working_dir/user_root 越界与大小校验。原生看图模型不注册 `look_at_image`,`read_document` 继续承担扫描 PDF 等文档摄取。`NULL` 表示旧客户端/旧历史,`[]` 表示新消息明确无附件;独立 attachment 表仍待跨消息复用、稳定身份、版本快照或附件级审计需求出现后再抽象。
|
||||||
|
|
||||||
### 7.2 资源模型(/v1)
|
### 7.2 资源模型(/v1)
|
||||||
|
|
||||||
|
|
@ -208,7 +208,7 @@ Admin GET /v1/admin/*(require_admin;overview + usage/models|users + storage/
|
||||||
Export GET /v1/tasks/{id}/export(docx)
|
Export GET /v1/tasks/{id}/export(docx)
|
||||||
```
|
```
|
||||||
|
|
||||||
**SSE 事件**:`run_start / llm_start / text{delta} / reasoning{delta}(thinking 模型推理流,前端灰色折叠卡)/ progress_snapshot{run_id,steps,waiting}(当前 user message 即 run 边界,从 messages 投影恢复)/ tool_call / tool_result(预览,完整走 DB)/ llm_end / model_switch / warn{msg}(熔断·重复拦截·折叠失败等运行时提醒)/ context_fold{phase,...}(§8.8 Phase 2 折叠 start/done)/ cancelled / error / done`。`task_progress` 每次提交完整步骤快照,前端整体替换;旧 `set_plan/update_step` 仅在历史投影时兼容。进度 dock 是运行状态而非历史消息:活跃时紧凑显示,正常完成后隐藏,`ask_user` 等待确认/取消/异常时折叠保留。fan-out:每订阅独立 queue;迟到订阅先从 PG 恢复当前 run 最新进度,终态迟到订阅立收 done。普通直播事件不持久化(messages 走 PG)。
|
**SSE 事件**:`run_start / llm_start / text{delta} / reasoning{delta}(thinking 模型推理流,前端灰色折叠卡)/ reasoning_reset(传输层放弃当前响应并重试时清除已直播推理)/ progress_snapshot{run_id,steps,waiting}(当前 user message 即 run 边界,从 messages 投影恢复)/ tool_call / tool_result(预览,完整走 DB)/ llm_end / model_switch / warn{msg}(熔断·重复拦截·折叠失败等运行时提醒)/ context_fold{phase,...}(§8.8 Phase 2 折叠 start/done)/ cancelled / error / done`。`task_progress` 每次提交完整步骤快照,前端整体替换;旧 `set_plan/update_step` 仅在历史投影时兼容。进度 dock 是运行状态而非历史消息:活跃时紧凑显示,正常完成后隐藏,`ask_user` 等待确认/取消/异常时折叠保留。fan-out:每订阅独立 queue;迟到订阅先从 PG 恢复当前 run 最新进度,终态迟到订阅立收 done。普通直播事件不持久化(messages 走 PG)。
|
||||||
**版本化**:`/v1` minor 半年兼容,major 6 个月 deprecation。**CORS**:本地 `*`,部署收紧。
|
**版本化**:`/v1` minor 半年兼容,major 6 个月 deprecation。**CORS**:本地 `*`,部署收紧。
|
||||||
|
|
||||||
### 7.3 认证
|
### 7.3 认证
|
||||||
|
|
@ -244,6 +244,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
- working_dir 存相对 ROOT posix 串,读写统一过 `core/paths.py`;入口 `validate_task_name` 拒空/`/\NUL`/`.` 起头。
|
- working_dir 存相对 ROOT posix 串,读写统一过 `core/paths.py`;入口 `validate_task_name` 拒空/`/\NUL`/`.` 起头。
|
||||||
- `auto_title_pending`(0023)只是一轮 UI 命名闸,不是 task 状态机;旧创建入口/存量行恒 false,快速入口首发后消费,人工改名优先清闸。
|
- `auto_title_pending`(0023)只是一轮 UI 命名闸,不是 task 状态机;旧创建入口/存量行恒 false,快速入口首发后消费,人工改名优先清闸。
|
||||||
- **0004 简化**:runs 表只写不读、独立 run 实体在单活形态下冗余 → 合并 `run_status/run_error` 入 tasks;需要前端关联本轮时直接复用该轮 user message UUID 为 `run_id`,不恢复 runs 表。**0006**:`tasks.model_profile` 为 source-of-truth(PATCH 切、下条 send 生效);usage_events 重建 v2 多态形态,统计 source-of-truth;tasks 三列保留作粗概览。run_status 终态:ok 收回 idle,error(出错)与 cancelled(用户停止)是持久终态 —— 前端 `renderPersistedRunTerminal` 据此在每次重渲后补持久卡(扛过收尾 loadMessages 整屏重建),刷新/切任务仍在;下次起新 run(post_message 写 running)覆盖清掉。
|
- **0004 简化**:runs 表只写不读、独立 run 实体在单活形态下冗余 → 合并 `run_status/run_error` 入 tasks;需要前端关联本轮时直接复用该轮 user message UUID 为 `run_id`,不恢复 runs 表。**0006**:`tasks.model_profile` 为 source-of-truth(PATCH 切、下条 send 生效);usage_events 重建 v2 多态形态,统计 source-of-truth;tasks 三列保留作粗概览。run_status 终态:ok 收回 idle,error(出错)与 cancelled(用户停止)是持久终态 —— 前端 `renderPersistedRunTerminal` 据此在每次重渲后补持久卡(扛过收尾 loadMessages 整屏重建),刷新/切任务仍在;下次起新 run(post_message 写 running)覆盖清掉。
|
||||||
|
- **版本化模型计费**(2026-08-30):provider 响应的 usage token 是用量事实源,直连模型 YAML 中带生效时间/币种/来源的 `pricing.periods` 是价格事实源;LiteLLM price map 仅给没有本地价目的网关模型兜底,避免第三方价目滞后覆盖已核对的国内厂商价格。`core/pricing.py` 按调用时间选择价格版本和 DeepSeek 峰谷时段,统一拆缓存命中/未命中/输出成本;主对话、标题、折叠、知识库摘要与 prompt 润色共用 `record_chat_usage`。每行 `usage_events.units` 快照 revision/tier/原币单价/汇率/分项成本,无 schema migration;公开按量价是产品估算口径,赠送额度、套餐和商务折扣以 provider 账单为最终财务口径。历史调价只通过默认 dry-run、显式 apply 且保存旧值的重算工具修正,不让新配置静默改写旧账。
|
||||||
- **0029 消息序号**:`tasks.next_message_idx` 在 task 行锁下统一分配 `messages.idx`,Web、agent 与渠道追加不再各自维护序号或依赖冲突重试;分配时仍与 `max(idx)` 校准,允许蓝绿发布窗口内旧实例继续写入。清空消息与计数器在同一事务归零。
|
- **0029 消息序号**:`tasks.next_message_idx` 在 task 行锁下统一分配 `messages.idx`,Web、agent 与渠道追加不再各自维护序号或依赖冲突重试;分配时仍与 `max(idx)` 校准,允许蓝绿发布窗口内旧实例继续写入。清空消息与计数器在同一事务归零。
|
||||||
- **No-subtask**:同 user 下前缀互含即拒(归一 posix 后 Python 端比对);同 working_dir 允许。
|
- **No-subtask**:同 user 下前缀互含即拒(归一 posix 后 Python 端比对);同 working_dir 允许。
|
||||||
- **文件面板先备料**:user_root 与普通目录都可显式新建直接子目录;创建成功后前端进入该目录,用户可先上传/选入资料,再把顶层空目录选作新对话 working_dir。目录 leaf 复用 `validate_task_name`,不允许借 UI 创建点目录或路径式名称。
|
- **文件面板先备料**:user_root 与普通目录都可显式新建直接子目录;创建成功后前端进入该目录,用户可先上传/选入资料,再把顶层空目录选作新对话 working_dir。目录 leaf 复用 `validate_task_name`,不允许借 UI 创建点目录或路径式名称。
|
||||||
|
|
@ -252,11 +253,11 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
|
|
||||||
### 7.5 沙盒:Per-user 容器 + Per-tool exec
|
### 7.5 沙盒:Per-user 容器 + Per-tool exec
|
||||||
|
|
||||||
选型:**每 user 长驻容器**(文件模型本就以 user root 为安全边界,per-task 会切碎共享工作区)+ **每 tool 一次 docker exec**(exec 级 timeout/cwd/统计)+ 空闲 5 分钟回收 + bind mount user root→`/workspace`。
|
选型:**每 user 长驻容器**(文件模型本就以 user root 为安全边界,per-task 会切碎共享工作区)+ **每 tool 一次 docker exec**(exec 级 timeout/cwd/统计)+ 无执行活动 10 分钟回收 + bind mount user root→`/workspace`。
|
||||||
|
|
||||||
**边界划分**:Control plane 留宿主(auth/DB/files 校验/SSE/LLM/受控 web 工具/配额审计),Execution plane 进容器(shell/run_python/任意生成代码)。目标不是"所有操作进容器",是"所有不可信执行不能在宿主"——否则凭据反被带进执行面。
|
**边界划分**:Control plane 留宿主(auth/DB/files 校验/SSE/LLM/受控 web 工具/配额审计),Execution plane 进容器(shell/run_python/任意生成代码)。目标不是"所有操作进容器",是"所有不可信执行不能在宿主"——否则凭据反被带进执行面。
|
||||||
|
|
||||||
**硬限制**:cgroup CPU/mem、pids-limit、exec timeout、并发数、read-only rootfs、tmpfs /tmp、no-new-privileges、drop ALL caps、非 root、`--shm-size`。**软配额**:按 user 计 DB(磁盘/LLM cost/wall time/流量/并发),超额 429。**网络**:默认 deny outbound,搜索抓取走宿主受控工具。
|
**硬限制**:单容器 4 GiB/2 CPU、pids 1024、`/dev/shm` 512 MiB、`/tmp` 1 GiB、exec timeout、read-only rootfs、no-new-privileges、drop ALL caps、非 root。整机重型执行由宿主共享文件账本 + advisory lock 统一准入:前后台合计 6、后台最多 4、单用户合计 3;蓝绿/多 Web 实例共享同一账本,前台排队不计 timeout 且可取消,并通过 SSE 明示单用户、整机、内存压力或队列顺序等等待原因;宿主 MemAvailable 低于阈值时暂停新放行、不杀存量。轻量 fs 工具不占重型槽。普通容器显式跟踪 active exec,reaper 只回收无 active exec 且超 TTL 的容器。**软配额**:按 user 计 DB(磁盘/LLM cost/wall time/流量),超额 429。**网络**:默认 deny outbound,搜索抓取走宿主受控工具。
|
||||||
|
|
||||||
**落地清单(Stage C 硬协议,实施按此对账)**:
|
**落地清单(Stage C 硬协议,实施按此对账)**:
|
||||||
1. **网络 blocklist 硬编码段**(任一缺失=未完成):`169.254/16`(metadata)、内网三段、CGNAT `100.64/10`;**PG 实际 IP 单独再 block**(belt-and-suspenders)。**容器自身 loopback(`-o lo`)显式放行**——netns 隔离下容器内 127.0.0.1 到不了宿主,DROP 它无安全收益且误伤容器内 IPC(2026-07 实锤:puppeteer↔chromium DevTools 走 127.0.0.1,被 DROP 导致 mermaid 渲染 90 天 0 成功)。
|
1. **网络 blocklist 硬编码段**(任一缺失=未完成):`169.254/16`(metadata)、内网三段、CGNAT `100.64/10`;**PG 实际 IP 单独再 block**(belt-and-suspenders)。**容器自身 loopback(`-o lo`)显式放行**——netns 隔离下容器内 127.0.0.1 到不了宿主,DROP 它无安全收益且误伤容器内 IPC(2026-07 实锤:puppeteer↔chromium DevTools 走 127.0.0.1,被 DROP 导致 mermaid 渲染 90 天 0 成功)。
|
||||||
|
|
@ -275,7 +276,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
| gVisor → Firecracker/e2b | 合规客户 / 单机 100+ user / 兼容墙 | 每 VM 100MB+ 不划算;e2b 与 storage_root 自持冲突 |
|
| gVisor → Firecracker/e2b | 合规客户 / 单机 100+ user / 兼容墙 | 每 VM 100MB+ 不划算;e2b 与 storage_root 自持冲突 |
|
||||||
| docker exec → 容器内 tool-runner RPC | exec 开销 >30% 持续两周 / 长驻服务 / 单轮 >20 次调用 | 自管清理+观测损失 >> 200ms×N;美学统一 ≠ 理由 |
|
| docker exec → 容器内 tool-runner RPC | exec 开销 >30% 持续两周 / 长驻服务 / 单轮 >20 次调用 | 自管清理+观测损失 >> 200ms×N;美学统一 ≠ 理由 |
|
||||||
|
|
||||||
**Image 体积 / 多 user 资源 / 加包**(2026-05-28):① image 大 ≠ 运行时吃资源(layer 共享、不 exec 只是磁盘字节);② 瓶颈在并发 exec 不在 idle 容器,杠杆全在运行时限制;③ 新增依赖 = base 收敛 + **per-user venv**(`<user_root>/.venv/`,bind mount 回收不丢;不放共享 volume——install 脚本是任意代码,破坏隔离)+ 使用频次沉淀进 base。
|
**Image 体积 / 多 user 资源 / 加包**(2026-09-01 修订):① image 大 ≠ 运行时吃资源(layer 共享、不 exec 只是磁盘字节);② 瓶颈在并发 exec 不在 idle 容器,杠杆在共享准入与 cgroup;③ requirements 保持全局 site-packages,非 root + readonly rootfs + `HOME=/tmp` 使普通 pip 临时落到 `/tmp/.local`、cache 落 `/tmp/.cache`。不建 venv、不持久化用户包、不加额外包目录;容器删除前由镜像内可信扫描器只解析 `.dist-info`,与构建期基础清单比较 added/override/reinstall,空扫描不入库、失败不阻塞删除。Admin 只分析依赖会话,不自动改基础镜像。
|
||||||
|
|
||||||
### 7.6 / 7.7 改造项与阶段
|
### 7.6 / 7.7 改造项与阶段
|
||||||
|
|
||||||
|
|
@ -298,7 +299,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
- **path-as-identity 而非 folder_id**:folder 真实存在于 FS,folder_id 是第二份 source of truth;rename 走 DB-aware 同事务 cascade。
|
- **path-as-identity 而非 folder_id**:folder 真实存在于 FS,folder_id 是第二份 source of truth;rename 走 DB-aware 同事务 cascade。
|
||||||
- **files API 单一 mutation 入口**(2026-05-18):"顶层目录分支"从数据状态派生而非客户端意图,放服务端才有强制力;双命名空间(/folders vs /files)把分支搬给 client,失强制力且端点翻倍。
|
- **files API 单一 mutation 入口**(2026-05-18):"顶层目录分支"从数据状态派生而非客户端意图,放服务端才有强制力;双命名空间(/folders vs /files)把分支搬给 client,失强制力且端点翻倍。
|
||||||
- **task 软删除(2026-06-17 推翻 hard cascade)**:公测后对话轨迹是训练/研究语料,`deleted_at` 置位 + restore,避免用户误删立即永久丢失。**当前实现仍无限期保留软删数据**,物理清理仅有管理员手段;后续生命周期已定为“软删除后保留 30 天再物理清理”(待容量信号实施,见 §8.5),届时恢复能力明确限于宽限期内。
|
- **task 软删除(2026-06-17 推翻 hard cascade)**:公测后对话轨迹是训练/研究语料,`deleted_at` 置位 + restore,避免用户误删立即永久丢失。**当前实现仍无限期保留软删数据**,物理清理仅有管理员手段;后续生命周期已定为“软删除后保留 30 天再物理清理”(待容量信号实施,见 §8.5),届时恢复能力明确限于宽限期内。
|
||||||
- **文件留存**:普通用户文件仍是 FS 直接删除;已写入结构化 `messages.artifact_refs` 的已发布产物,在统一 files delete 入口删除时原子移动到用户根目录的平台隐藏区 `.zcbot_artifact_trash/`,原路径与历史卡片立即表现为已删除。递归目录只回收其中 artifact,其他文件照常删除;回收内容仍计入用户配额。上传入口默认以原子独占创建保留同名两份,只有用户从文件菜单明确选择并确认「替换文件」时才允许原子覆盖;替换保持原路径和 artifact 身份、内容更新。该机制仍不防 agent/shell 绕过 files API 或整盘损坏。完整地基仍采用 restic/borg 定时增量备份(与应用解耦,捕获删除+覆盖+所有写入口),后续容量需要时再补 `data_events` 用户意图事件和回收区清理/恢复管理。
|
- **文件留存**:普通用户文件仍是 FS 直接删除;已写入结构化 `messages.artifact_refs` 的已发布产物,在统一 files delete 入口删除时原子移动到用户根目录的平台隐藏区 `.zcbot_artifact_trash/`,原路径与历史卡片立即表现为已删除。递归目录只回收其中 artifact,其他文件照常删除;回收内容不占用户逻辑配额,但后台扫描在 `user_disk_usage` 中单列其字节数和文件数,Admin 展示“配额占用 / 回收站 / 物理合计”,避免释放用户额度后失去服务器真实容量可见性。上传入口默认以原子独占创建保留同名两份,只有用户从文件菜单明确选择并确认「替换文件」时才允许原子覆盖;替换保持原路径和 artifact 身份、内容更新。该机制仍不防 agent/shell 绕过 files API 或整盘损坏。完整地基仍采用 restic/borg 定时增量备份(与应用解耦,捕获删除+覆盖+所有写入口),后续容量需要时再补 `data_events` 用户意图事件和回收区清理/恢复管理。
|
||||||
- **0004 删 runs/usage_events 旧表**:只写不读的死代码;代价是失历史 run 元数据,真要细粒度审计再补(届时是新需求非技术债)。
|
- **0004 删 runs/usage_events 旧表**:只写不读的死代码;代价是失历史 run 元数据,真要细粒度审计再补(届时是新需求非技术债)。
|
||||||
- **本地也用 PG 不用 SQLite**:dogfood ≡ 真实路径;Docker 已是必然依赖;双 adapter 维护税 > 一次性配置。
|
- **本地也用 PG 不用 SQLite**:dogfood ≡ 真实路径;Docker 已是必然依赖;双 adapter 维护税 > 一次性配置。
|
||||||
- **API-only,UI 由 platform 实现**(2026-05-15):本仓库再维护一套 UI 是双套浪费;SSE payload 从 HTML 切 JSON;沉淀的 sink/broker/路径安全全保留。**dev SPA 留一份**作 dogfood 主路径(SSE 调试 curl/Swagger 都覆盖不了)。
|
- **API-only,UI 由 platform 实现**(2026-05-15):本仓库再维护一套 UI 是双套浪费;SSE payload 从 HTML 切 JSON;沉淀的 sink/broker/路径安全全保留。**dev SPA 留一份**作 dogfood 主路径(SSE 调试 curl/Swagger 都覆盖不了)。
|
||||||
|
|
@ -316,7 +317,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
|
|
||||||
### 8.1 图像理解 + Seedream i2i(✅ 2026-06-16)
|
### 8.1 图像理解 + Seedream i2i(✅ 2026-06-16)
|
||||||
|
|
||||||
缺口:主模型纯文本;t2i 无法"改已生成图"或"读上传图"。**选 E+C 组合**:seedream 加 `reference_images` 走 i2i + 新增 `look_at_image`(Doubao Seed 2.0 Lite,一次读图 <¥0.01)让 DeepSeek 自决何时"借眼睛"。**不选 A**(主模型换多模态:V4 code/tool-calling 是核心,换=降能力+改 loop 引 multimodal,工程 5×);**不选 B**(每条消息隐式 vision 路由:烧 token+失 agentic 控制权)。关键实测:ARK 接受 base64 data URL → 内网无需对象存储。**升级到 A 的信号**:用户要"贴图直接对话读图"成高频——当前假设"图是工具调用对象"而非"对话内容"。
|
历史基线是纯文本主模型配 `look_at_image` 借眼睛,生图改图独立走 `reference_images`。2026-08-27 随 GLM-5.3 Flash 原生多模态进入默认档位,升级信号已满足:附件输入改为 capability-driven,原生图片模型直接收到用户上传图片,纯文本模型继续使用路径提示与辅助视觉工具;不会把所有文件或工具产物隐式送入模型,避免无关 token、越界读取和缓存污染。生图/改图仍是显式外部动作,原生理解不替代图像生成工具;运行中产生的新图片若未作为后续用户附件加入,不自动注入下一轮。
|
||||||
|
|
||||||
### 8.2 Token 优化与上下文治理(✅ 2026-06-04 起)
|
### 8.2 Token 优化与上下文治理(✅ 2026-06-04 起)
|
||||||
|
|
||||||
|
|
@ -330,7 +331,7 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
|
|
||||||
**优雅 drain 已是单实例上限**(SIGTERM 拒新 run + 等收尾);先撞的瓶颈是线程池(每活跃 run 占 1 线程)。落地排序:① 轻量监控(显式 executor + 60s `[stats]` 周期日志——要历史峰值不是快照);② 按数据决策扩容;③ `--reload` 缩 503 窗。**不做监控界面**:运维健康是少数标量,日志够;业务分析走 DB SQL。界面阶梯:日志 → /v1/stats → Grafana → 只读 dashboard,现停第一级。无感换版已由蓝绿落地(0.41)、broker 外置 Redis 已实施(0.42,均见 §7.0);扩容路径:调大线程池 → 稳态双实例分流(broker 已外置,纯 nginx 配置动作)。
|
**优雅 drain 已是单实例上限**(SIGTERM 拒新 run + 等收尾);先撞的瓶颈是线程池(每活跃 run 占 1 线程)。落地排序:① 轻量监控(显式 executor + 60s `[stats]` 周期日志——要历史峰值不是快照);② 按数据决策扩容;③ `--reload` 缩 503 窗。**不做监控界面**:运维健康是少数标量,日志够;业务分析走 DB SQL。界面阶梯:日志 → /v1/stats → Grafana → 只读 dashboard,现停第一级。无感换版已由蓝绿落地(0.41)、broker 外置 Redis 已实施(0.42,均见 §7.0);扩容路径:调大线程池 → 稳态双实例分流(broker 已外置,纯 nginx 配置动作)。
|
||||||
|
|
||||||
**工具失败聚集的信号分层(2026-07-29)**:7 天窗口继续保留取证,但管理端默认判断对象是“近 24h 仍活跃且跨 ≥2 task”的系统性故障;单 task 反复试错、按设计非零退出的质量门、近 24h 已归零的历史尾巴分别展示,避免正常迭代挤占故障榜首。API 保留低阈值全量 `clusters`,只加 `category=failure|quality_gate`,分区属于前端读侧语义,不删除既有字段。空输出 shell 非零退出从相邻 assistant tool_call 提取稳定命令类别(如 search/no match、dependency probe),只改善签名,不持久化第二份命令事实。RepeatGuard 只把 `run_python` 的 traceback + 非零退出纳入同错 streak;不泛化到 shell,避免 grep 未命中和质量门复检被误拦。provider wire 健康另走只读派生端点,按 `model_profile+tool` 对已有 `tool_salvaged/tool_malformed` 事件计算 24h/窗口抢救率;它是失败数的分母与趋势解释,不混进 cluster,不新增表/索引,也不反向改变并行调用、salvage 或重试策略。
|
**工具健康事件与信号分层(2026-09-03)**:`usage_events` 是工具健康的唯一事实源,不再从 `messages.payload` 回扫或补算;普通 tool result 在落消息后仅当分类为失败时写 `tool_failure`(质量门写 `quality_gate`),并在写入时完成稳定签名、shell 命令类别和类别分类。切换前仅存在于 messages 的普通失败历史有意不迁移、不纳入新口径。畸形参数、抢救成功、run 终态错误沿用 `tool_malformed/tool_salvaged/run_error/empty_response`;主动停止、重复调用硬拦、上下文折叠失败分别记 `run_stopped/agent_guard/context_fold_failure`。这些控制事件统一 `category=agent_control`,与 `category=failure|quality_gate` 分栏,管理页“工具健康”大数只计真实失败,避免把保护机制触发误报为平台故障。7 天窗口继续保留取证,默认判断对象是近 24h 活跃信号;单 task、跨 task、质量门、控制事件和已安静历史分别展示。provider wire 健康仍按 `model_profile+tool` 对 `tool_salvaged/tool_malformed` 计算 24h/窗口抢救率,不混入失败数。读侧合并为一次带 `(kind, created_at)` 部分索引的事件查询;Admin 首次进入即加载,之后 90 秒节流并由服务端短时单飞缓存兜底。
|
||||||
|
|
||||||
### 8.5 定时任务(✅ 2026-06-18)
|
### 8.5 定时任务(✅ 2026-06-18)
|
||||||
|
|
||||||
|
|
@ -406,15 +407,15 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
**决策**:把长进程从 zcbot 进程树上摘下来,OS 就是"任务组件",不引 Celery/RQ/队列。`shell`/`run_python` 加 `background=true`(**模型判断**:预计 >~1min 走后台;前台超时报错里提示改后台——判断错了纠错路径只有一步;用户显式指令永远优先)。状态协议**纯文件**:`<user_root>/.zcbot_procs/<task_id>/<proc_id>/{proc.json, output.log, exit_code}`——dotfile 用户不可见(同 `.zcbot_tmp` 惯例),`exit_code` 文件出现是唯一终态信号,状态判定全靠文件 + 现场探测(pid / 容器 running),**无常驻登记,天然扛重启**。查询/终止走配套 `check_process` 工具(host in-process,两种 backend 通吃)。
|
**决策**:把长进程从 zcbot 进程树上摘下来,OS 就是"任务组件",不引 Celery/RQ/队列。`shell`/`run_python` 加 `background=true`(**模型判断**:预计 >~1min 走后台;前台超时报错里提示改后台——判断错了纠错路径只有一步;用户显式指令永远优先)。状态协议**纯文件**:`<user_root>/.zcbot_procs/<task_id>/<proc_id>/{proc.json, output.log, exit_code}`——dotfile 用户不可见(同 `.zcbot_tmp` 惯例),`exit_code` 文件出现是唯一终态信号,状态判定全靠文件 + 现场探测(pid / 容器 running),**无常驻登记,天然扛重启**。查询/终止走配套 `check_process` 工具(host in-process,两种 backend 通吃)。
|
||||||
|
|
||||||
- **host backend**:detach 独立 wrapper(`core/proc_wrapper.py`,stdlib-only,sys.executable 直跑不依赖 PYTHONPATH):限时(默认 7200s / cap 86400s)、超时杀进程树记 124、日志截尾 10MB、最后写 exit_code。
|
- **host backend**:detach 独立 wrapper(`core/proc_wrapper.py`,stdlib-only,sys.executable 直跑不依赖 PYTHONPATH):限时(默认 7200s / cap 86400s)、超时杀进程树记 124、日志截尾 10MB、最后写 exit_code。
|
||||||
- **docker backend**:**专用容器** `zcbot-proc-<id>`(pool.run_proc_container,同款硬化 + iptables init),`product=proc` + 无 instance label —— 与 sandbox 容器的 idle reaper / shutdown_all 生命周期**解耦**,dockerd 托管,蓝绿切换/实例重启不中断。不用 `docker exec -d` 进 sandbox 容器:idle 5min reaper + 启动 shutdown_all 会把长进程随容器带走。
|
- **docker backend**:**专用容器** `zcbot-proc-<id>`(pool.run_proc_container,同款硬化 + iptables init),`product=proc` + 无 instance label —— 与普通 sandbox 的 idle reaper / shutdown_all 生命周期**解耦**,dockerd 托管,蓝绿切换/实例重启不中断。无容量时先写 `state=queued`,调度器获得共享前后台槽后才启动,故排队时间不计运行 timeout;`check_process` 兼容 queued/running/finished/lost,并可取消 queued。
|
||||||
- **回收**:check_process 见终态顺手 rm 容器;web lifespan 每小时 `procs.sweep`(终态目录 7d TTL / exited 孤儿容器),幂等,蓝绿双实例同时跑无害。
|
- **回收**:后台仍运行即有活动,不适用普通容器 idle TTL;check_process 见终态顺手扫描临时包并 rm 容器,web lifespan 约 30 秒 sweep/dispatch(终态目录仍保留 7d),幂等,蓝绿双实例以 proc 目录锁避免重复启动。运行中容器由 Docker 托管,服务重启恢复 queued 并保留 running。
|
||||||
- **通知/可视**:不做服务端推送 —— 前端轮询 `GET /v1/procs`(用户级,纯文件读取,仅有 running proc 时 5s 一拉):`[Background]` 工具结果卡本身活化(spinner+跳秒+停止按钮,与前台工具卡同体验,历史重渲同样恢复;`POST .../procs/<id>/kill`)、running→终态弹 toast(跨 task 也提醒,点击跳转)。proc 完成时刻往往没有活跃 run,SSE 通道根本不在,轮询是诚实的选型。
|
- **通知/可视**:不做服务端推送 —— 前端轮询 `GET /v1/procs`(用户级,纯文件读取,仅有 running proc 时 5s 一拉):`[Background]` 工具结果卡本身活化(spinner+跳秒+停止按钮,与前台工具卡同体验,历史重渲同样恢复;`POST .../procs/<id>/kill`)、running→终态弹 toast(跨 task 也提醒,点击跳转)。proc 完成时刻往往没有活跃 run,SSE 通道根本不在,轮询是诚实的选型。
|
||||||
|
|
||||||
### 8.13 扫描件 PDF 直读:方舟文档理解,不接外部 OCR(✅ 2026-07-21)
|
### 8.13 扫描件 PDF 直读:方舟文档理解,不接外部 OCR(✅ 2026-07-21)
|
||||||
|
|
||||||
缺口:markitdown 只抽 PDF 文本层,扫描件(老标准/检测报告/红头指南,建材院高频)转出为空=死路。**选复用 seed-2.0-lite 的方舟文档理解**(chat file 内容块,PDF 整本 base64 内联)新增 `read_document`:零新供应商(敏感文档不出已有豆包面)、零新基础设施、记账复用 vision 通道;实测 ~1300 输入 token/页(约 1 厘/页)、100 页全覆盖、17MB 内联可用。**不选专用解析 API**(MinerU/Textin:版面还原最好,但申报书/专利底稿要上传新第三方 + 免费额度政策不稳);**不选本地 OCR**(PaddleOCR 类:镜像塞推理依赖,需求未量化前过度投资);**不选 file_url/file_id 传址**(前者要给用户文件开免认证公网直链=新安全面、开发机 NAT 后还跑不通;后者要接 TOS 多落一份存储;base64 是零新增面的唯一形态,行业惯例 chat 端点也不收 multipart)。防上下文爆:多页 OCR 强制 `save_md` 落盘只返预览。**升级信号**:>100 页/>30MB 巨件成高频 → 接 TOS 走 file_id;要高保真版面/公式还原 → 再评 MinerU。probe/smoke 留仓(`scripts/probe_ark_doc.py` / `smoke_read_document.py`)。
|
缺口:markitdown 只抽 PDF 文本层,扫描件(老标准/检测报告/红头指南,建材院高频)转出为空=死路。**选复用 seed-2.0-lite 的方舟文档理解**(chat file 内容块,PDF 整本 base64 内联)新增 `read_document`:零新供应商(敏感文档不出已有豆包面)、零新基础设施、记账复用 vision 通道;实测 ~1300 输入 token/页(约 1 厘/页)、100 页全覆盖、17MB 内联可用。**不选专用解析 API**(MinerU/Textin:版面还原最好,但申报书/专利底稿要上传新第三方 + 免费额度政策不稳);**不选本地 OCR**(PaddleOCR 类:镜像塞推理依赖,需求未量化前过度投资);**不选 file_url/file_id 传址**(前者要给用户文件开免认证公网直链=新安全面、开发机 NAT 后还跑不通;后者要接 TOS 多落一份存储;base64 是零新增面的唯一形态,行业惯例 chat 端点也不收 multipart)。防上下文爆:多页 OCR 强制 `save_md` 落盘只返预览。**升级信号**:>100 页/>30MB 巨件成高频 → 接 TOS 走 file_id;要高保真版面/公式还原 → 再评 MinerU。probe/smoke 留仓(`scripts/probe_ark_doc.py` / `smoke_read_document.py`)。
|
||||||
- **对话锁(前端)**:bg proc 运行期间该 task 的 composer 锁定(发送→停止,Enter 拦截),观感与前台执行完全一致 —— 后台化的收益定位为「进程扛超时/服务重启」,**不改变"一个任务同时只做一件事"的对话心智**;完成的那次轮询解锁 + toast「可继续对话」。锁只在前端,服务端不 409:「停止」入口必须可达,且多设备/渠道绕过前端锁属可接受边缘(等的是同一个进程,发了消息也不冲突)。
|
- **对话锁(前端)**:bg proc 运行期间该 task 的 composer 锁定(发送→停止,Enter 拦截),观感与前台执行完全一致 —— 后台化的收益定位为「进程扛超时/服务重启」,**不改变"一个任务同时只做一件事"的对话心智**;完成的那次轮询解锁 + toast「可继续对话」。锁只在前端,服务端不 409:「停止」入口必须可达,且多设备/渠道绕过前端锁属可接受边缘(等的是同一个进程,发了消息也不冲突)。
|
||||||
- **防失控**:每用户并发 running 上限(`ZCBOT_MAX_BG_PROCS` 默 3);前台默认超时不放大(它是逼模型做前台/后台选择的杠杆)。
|
- **防失控**:前后台统一使用整机 6 / 后台 4 / 单用户 3 的共享准入;前台默认超时不放大(它是逼模型做前台/后台选择的杠杆)。
|
||||||
|
|
||||||
**边界(防滑坡)**:只覆盖「单个本地长进程」。①**外部异步作业**(seedance 等 submit/poll 形态)不进这里——工具内轮询 + `resume_task_id` 续查已够;②**job 链/依赖/自动重试**不做——那是 workflow 引擎,编排的唯一归属是 agent loop(模型 check 后自己决定下一步),同 §6 拒绝编排的理由;③**完成后自动续跑 run**不做——zcbot 的长任务产物多为终点交付物(与 Claude Code"build 是中间步骤"不同),自动续跑=无人在场烧 token,通知给人、下一步由人/下次对话决定。
|
**边界(防滑坡)**:只覆盖「单个本地长进程」。①**外部异步作业**(seedance 等 submit/poll 形态)不进这里——工具内轮询 + `resume_task_id` 续查已够;②**job 链/依赖/自动重试**不做——那是 workflow 引擎,编排的唯一归属是 agent loop(模型 check 后自己决定下一步),同 §6 拒绝编排的理由;③**完成后自动续跑 run**不做——zcbot 的长任务产物多为终点交付物(与 Claude Code"build 是中间步骤"不同),自动续跑=无人在场烧 token,通知给人、下一步由人/下次对话决定。
|
||||||
|
|
||||||
|
|
@ -471,10 +472,14 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
|
|
||||||
云端控制面使用独立的 `software_node_enrollments` 与 `software_nodes`,不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵,数据库只保存 SHA-256 摘要;节点注册在行锁事务中校验有效期、预期名称和服务端支持的能力,成功后原子消费。注册码只建立节点身份,不要求管理员逐项授权专业软件能力。每个节点获得独立高熵 Token,数据库只保存 bcrypt 强哈希,明文仅在注册响应出现一次。
|
云端控制面使用独立的 `software_node_enrollments` 与 `software_nodes`,不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵,数据库只保存 SHA-256 摘要;节点注册在行锁事务中校验有效期、预期名称和服务端支持的能力,成功后原子消费。注册码只建立节点身份,不要求管理员逐项授权专业软件能力。每个节点获得独立高熵 Token,数据库只保存 bcrypt 强哈希,明文仅在注册响应出现一次。
|
||||||
|
|
||||||
专业软件采用“共享能力契约 + 本机 adapter”边界。仓库根目录 `software-contracts/*.json` 是语言无关的声明式源码事实源,描述 capability、请求 JSON Schema、输入额度、输出 manifest、feature 与最低 adapter 版本;Core 按文件签名热加载契约,只负责身份、账本、调度、传输、摘要与最终发布,不包含 Origin 或其他软件的操作分支。热加载先完整校验新快照再原子切换,写入中或非法版本继续使用上一份有效快照。Windows Node 是稳定的可信宿主,只负责持久化 job 目录、下载/上传、恢复、取消,以及从程序目录 `adapters/*/adapter.json` 发现 adapter;发布过程把对应源码合同复制进各 adapter 目录,Host 只读取 manifest 同目录指向的这一份合同并做通用 JSON Schema 校验,不再读取共享合同目录或采用本地优先 fallback。这样完整 Node 包与独立 adapter 包具有相同部署单元,manifest、合同、Worker 和依赖必须整体更新,避免旧合同静默覆盖新合同。manifest 声明 capability、实际 adapter 版本、运行类型、入口和契约文件,并可声明 1–1440 分钟的单任务超时;旧 manifest 缺省时保持 30 分钟,版本与时限不编译进 EXE。Node 在心跳中上报实际加载合同的 SHA-256 和 Workspace 协议版本,服务端只向明确兼容的节点调度 Workspace Job。Host 另提供本机运维层:按 `runtime_id` 合并软件卡,允许管理员自动检测或选择预定义应用位置,并只按内置软件清单及固定 requirements 单独构建受管 runtime;UI 不接受任意安装命令,任务请求仍不能影响解释器、应用路径或安装过程。Host 将已验证的位置以固定环境变量注入对应 Worker,Worker 的 `--probe` 继续承担最终软件版本、SDK/COM 和执行门健康校验。Worker 可以是受管 Python 脚本或独立 EXE,进程只接收 job 目录,并通过 `state.json`、`terminal.json` 与固定输出目录交接;Python、COM 或某个 SDK 都不属于通用任务协议。当前 Origin、ANSYS 和 Blender adapter 使用 Python 只是实现选择。
|
专业软件采用“共享能力契约 + 本机 adapter”边界。仓库根目录 `software-contracts/*.json` 是语言无关的声明式源码事实源,描述 capability、请求 JSON Schema、输入额度、输出 manifest、feature 与最低 adapter 版本;Core 按文件签名热加载契约,只负责身份、账本、调度、传输、摘要与最终发布,不包含 Origin 或其他软件的操作分支。契约可附带 `active/deprecated` 生命周期、`standard/diagnostic_only` 可见性和替代 capability;普通能力枚举隐藏诊断入口,显式诊断查询与历史 Job 读取仍保留。热加载先完整校验新快照再原子切换,写入中或非法版本继续使用上一份有效快照。Windows Node 是稳定的可信宿主,只负责持久化 job 目录、下载/上传、恢复、取消,以及从程序目录 `adapters/*/adapter.json` 发现 adapter;发布过程把对应源码合同复制进各 adapter 目录,Host 只读取 manifest 同目录指向的这一份合同并做通用 JSON Schema 校验,不再读取共享合同目录或采用本地优先 fallback。这样完整 Node 包与独立 adapter 包具有相同部署单元,manifest、合同、Worker 和依赖必须整体更新,避免旧合同静默覆盖新合同。manifest 声明 capability、实际 adapter 版本、运行类型、入口和契约文件,并可声明 1–1440 分钟的单任务超时;旧 manifest 缺省时保持 30 分钟,版本与时限不编译进 EXE。Node 在心跳中上报实际加载合同的 SHA-256 和 Workspace 协议版本,服务端只向明确兼容的节点调度 Workspace Job。Host 另提供本机运维层:按 `runtime_id` 合并软件卡,桌面软件允许管理员自动检测或选择预定义应用位置,runtime-only 软件使用 `SoftwarePathKind.None` 且不接受路径;两类软件都只按内置清单及固定 requirements 单独构建受管 runtime。UI 不接受任意安装命令,任务请求仍不能影响解释器、应用路径或安装过程。Host 将已验证的位置以固定环境变量注入对应 Worker,Worker 的 `--probe` 继续承担最终软件版本、SDK/COM 或内核 roundtrip 健康校验。Worker 可以是受管 Python 脚本或独立 EXE,进程只接收 job 目录,并通过 `state.json`、`terminal.json` 与固定输出目录交接;Python、COM 或某个 SDK 都不属于通用任务协议。当前 Origin、ANSYS、Blender 和中性 CAD adapter 使用 Python只是实现选择。
|
||||||
|
|
||||||
|
Core 在 Job offer 中附带由 capability 合同生成的 `request_summary`,供本机任务列表统一显示标题;Host 不解析各软件私有请求结构,旧落盘 Job 只做通用兼容读取。
|
||||||
|
|
||||||
扩展现有 capability 的 feature 时修改共享契约、对应 Worker 和测试,不修改 Core 调度、Job 生命周期或 Node EXE;增加新专业软件时新增契约与一个 adapter 目录。Cloud 会在下一次契约查询时获得新校验和工具 schema;Node 在启动时从本地目录获得能力、版本和入口,并在首次连接及每次心跳中上报全部已发现 adapter。服务端用本次上报中自己具有共享契约的 capability 原子替换节点能力列表,未知能力不进入调度;管理端只读展示该结果,不保存人工能力选择。现有节点因此不需要清身份或重新注册,升级 Node 后重启一次即可在下一次心跳同步。当前选择明确的运维边界:zcbot 契约可热更新,不要求重启;本机 adapter 更新时先退出托盘 Node,整体替换 adapter 目录后重新启动,不重装、不替换 Node EXE,也不建设自动更新平台。当前 Node 仍按整机单执行槽保守串行,未来只有真实并行软件需求出现时,才把 slot 账本升级为 per-capability 租约,而不改变 Job 协议。
|
扩展现有 capability 的 feature 时修改共享契约、对应 Worker 和测试,不修改 Core 调度、Job 生命周期或 Node EXE;增加新专业软件时新增契约与一个 adapter 目录。Cloud 会在下一次契约查询时获得新校验和工具 schema;Node 在启动时从本地目录获得能力、版本和入口,并在首次连接及每次心跳中上报全部已发现 adapter。服务端用本次上报中自己具有共享契约的 capability 原子替换节点能力列表,未知能力不进入调度;管理端只读展示该结果,不保存人工能力选择。现有节点因此不需要清身份或重新注册,升级 Node 后重启一次即可在下一次心跳同步。当前选择明确的运维边界:zcbot 契约可热更新,不要求重启;本机 adapter 更新时先退出托盘 Node,整体替换 adapter 目录后重新启动,不重装、不替换 Node EXE,也不建设自动更新平台。当前 Node 仍按整机单执行槽保守串行,未来只有真实并行软件需求出现时,才把 slot 账本升级为 per-capability 租约,而不改变 Job 协议。
|
||||||
|
|
||||||
|
桌面宿主采用 WPF `Application` 持有唯一 GUI 消息循环,WPF 主窗口只通过 ViewModel/Command 消费应用服务与控制器状态;WinForms 被限制在 `NotifyIcon`、文件/目录选择、确认框和剪贴板互操作,不再存在第二套配置窗口或 Host 生命周期。托盘宿主负责把后台状态经 Dispatcher 投递到 WPF、控制窗口显隐,并复用应用控制器完成注册、重连、清身份、数据目录迁移与安全退出。CLI 参数在 WPF 初始化前分流,因此 `run --headless` 不创建窗口或托盘。节点概览、首次注册、登录自启动、专业软件/runtime、本机任务、ANSYS 固定验收和数据目录迁移均已进入 WPF;`SoftwareManagementService`、`JobMonitorService`、`AnsysAcceptanceService`、`DataRootManagementService` 与 `DiagnosticService` 分别封装固定业务边界,ViewModel 不解析 Job 落盘格式、不访问注册表、不创建 Worker,也不直接持有应用控制器。
|
||||||
|
|
||||||
Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/software-nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活,新连接关闭旧连接;`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态,再关闭现有连接;断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。
|
Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/software-nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活,新连接关闭旧连接;`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态,再关闭现有连接;断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。
|
||||||
|
|
||||||
第二阶段已增加 `software_jobs`(专业软件任务)账本与 `origin.plot@v2` 的 offer/accept 骨架。用户只能在本人 task 下以幂等键提交固定 schema;云端规范化请求并记录 SHA-256,按当前进程真实在线、能力匹配、健康且有空闲 slot 的 Node 创建短期 offer。Node 再次校验 schema、图形类型和输出格式,使用 write-through、flush 与原子 rename 先落本机任务目录,再回 `job_accept`;重复 job 只有 digest 一致才接受。过期或发送失败的 offer 回到队列,lease、Node 和 digest 不匹配的响应被拒绝。Node 接收后云端进入 `dispatched` 而非 `running`,并将 slot 降为 0;只有固定 Worker 真正启动后才进入软件无关的 `software_running`,具体软件和操作由 capability/request 表达。
|
第二阶段已增加 `software_jobs`(专业软件任务)账本与 `origin.plot@v2` 的 offer/accept 骨架。用户只能在本人 task 下以幂等键提交固定 schema;云端规范化请求并记录 SHA-256,按当前进程真实在线、能力匹配、健康且有空闲 slot 的 Node 创建短期 offer。Node 再次校验 schema、图形类型和输出格式,使用 write-through、flush 与原子 rename 先落本机任务目录,再回 `job_accept`;重复 job 只有 digest 一致才接受。过期或发送失败的 offer 回到队列,lease、Node 和 digest 不匹配的响应被拒绝。Node 接收后云端进入 `dispatched` 而非 `running`,并将 slot 降为 0;只有固定 Worker 真正启动后才进入软件无关的 `software_running`,具体软件和操作由 capability/request 表达。
|
||||||
|
|
@ -487,13 +492,23 @@ Origin 科研分析以独立 `origin.analysis@v1` capability 承载,不把数
|
||||||
|
|
||||||
Recipe 是专业软件的声明式目标状态,不是任意脚本或逐次鼠标命令。`origin.plot@v2` 以新增 `plot.type=recipe + recipe_version=1` 复用已验证的 `layout/panels/series/axis/legend` 组合结构;旧 `multi_panel` 与 Recipe 进入同一个受控执行器,单图和特殊统计图继续保留兼容入口。模型可以组合契约允许的原子图形能力,但不能提供 Python、LabTalk、解释器、模板名或文件路径。其他软件可采用同一通用 Job 外壳和各自的声明式 Recipe,不建设跨软件万能 DSL;API/SDK executor 优先,未来的 UI Automation 或 Computer Use 仅作为 adapter 内部执行后端,不改变云端 Recipe。
|
Recipe 是专业软件的声明式目标状态,不是任意脚本或逐次鼠标命令。`origin.plot@v2` 以新增 `plot.type=recipe + recipe_version=1` 复用已验证的 `layout/panels/series/axis/legend` 组合结构;旧 `multi_panel` 与 Recipe 进入同一个受控执行器,单图和特殊统计图继续保留兼容入口。模型可以组合契约允许的原子图形能力,但不能提供 Python、LabTalk、解释器、模板名或文件路径。其他软件可采用同一通用 Job 外壳和各自的声明式 Recipe,不建设跨软件万能 DSL;API/SDK executor 优先,未来的 UI Automation 或 Computer Use 仅作为 adapter 内部执行后端,不改变云端 Recipe。
|
||||||
|
|
||||||
|
`cad.geometry.prepare@v1` 是独立、无状态、runtime-only 的中性几何入口,以固定 CadQuery 2.8.0 / OCP 7.9.3.1 生成 STEP;它只负责检查、基础体和受控有序 Recipe,不承载材料、网格、接触、边界条件或求解。Recipe 覆盖显式坐标闭合截面、拉伸/旋转/扫掠/放样、布尔、圆角/倒角/抽壳、变换、线性/圆周阵列、STEP import 与多实体组合,所有引用、轮廓平面性、形状有效性、实体上限和选择数量都机械校验。输出固定为 STEP、规范化 Recipe、确定性几何 manifest、无 GUI PNG 预览和 provenance;manifest 只保存单位、几何量、拓扑/表面摘要和已解析区域,不把 OCCT、ANSYS 或 Abaqus 的临时实体 ID 作为跨软件事实。区域选择器以几何语义描述 face/edge/solid;其中 `extreme_face` 与 `planar_faces` 和 ANSYS 静力 v2 保持同构并由合同测试守护。输入首版只接受显式声明单位的 STEP/STP,统一转换到请求必填的 `canonical_unit`,缺失或歧义单位直接失败。
|
||||||
|
|
||||||
|
首版不输出 GLB:当前稳定事实源和跨求解器交换面是 STEP,现有 GLB 链路依赖 Blender/装配导出,未在独立 CAD runtime 上形成同等稳定的 roundtrip 与打包验证;后续只有在 CAD runtime 内可独立验证时再作为新增可选产物。
|
||||||
|
|
||||||
|
`freecad.model.author@v1` 是面向已安装 FreeCAD 1.1 稳定版的有状态受控建模入口,只接受 `>=1.1.3,<1.2.0`,不接受 RC、weekly 或 experimental 构建。受管 `freecad` Python runtime 只负责发现 `FreeCADCmd.exe`、版本门禁、任务级配置隔离和 Pillow 软件渲染;固定 FreeCAD 侧 Worker 在应用自带 Python 中调用 `FreeCAD/Part`,不把 FreeCAD 安装进 venv,也不依赖 FreeCADGui/GPU。每个 Job 固定启用 safe mode、独立 user/system cfg 与 user data/temp 目录,并清除不可信 Python 搜索路径;请求不能影响脚本、程序路径、命令行、本机路径、URL、宏、插件或 Workbench。
|
||||||
|
|
||||||
|
FreeCAD v1 的 Recipe 是模型唯一事实源,每次 revision 都从空文档按完整目标状态重建,而不是继续编辑上一版 FCStd 中的隐式状态;轻量命名参数只在 adapter 内解析为有限数后写入 Shape,不生成 Spreadsheet、alias 或原生 Expression。首版操作限制为 box/cylinder/cone/sphere/torus、STEP import、union/cut/intersect、translate/rotate/combine;输入只允许零或一个已登记 STEP/STP,外部 FCStd 与更高阶特征留待后续。固定输出 `model.FCStd`、`model.step`、无 GUI 等轴测预览、Recipe、manifest 和 provenance;发布前必须完成 recompute、Shape 有效性、FCStd 重开与 STEP 重开,并对实体数、包围盒和体积做容差比较。Workspace 继续只管理 current/rollback 两代,续作沿用通用 `software_job_revise`,不新增协议或数据库状态。
|
||||||
|
|
||||||
|
`ansys.geometry.inspect@v1` 继续保留原合同和 adapter,专门验证 Mechanical 真实导入后的 GeoData、单位与 Named Selection;它至少保留一个发布周期,但生命周期标为 deprecated/diagnostic-only,并指向中性 CAD 入口。普通几何准备不再推荐占用 ANSYS 许可证,显式诊断请求与历史 Job/旧节点协议不受影响。
|
||||||
|
|
||||||
Blender 入口为 `blender.scene.author@v3`,唯一场景类型是 `recipe`;它是 Blender 专属的声明式静态场景目标状态,不是跨软件万能 DSL,也不是 Python、逐步按键或任意节点脚本。Recipe v1 组合带稳定 ID、父子层级和集合归属的 geometry/generator/text/empty 对象:geometry 覆盖 box/cylinder/cone/sphere/torus/plane、二维轮廓拉伸、轴向旋转、路径扫掠和有壁厚直管,受控 modifier 覆盖 boolean/bevel/mirror/array/solidify,并统一声明材质、变换、灯光、世界背景和 1–8 个相机视图;`rotary_kiln` 只是 generator 组件,可与其他对象处于同一场景,不再占据顶层 capability。adapter 1.0.0 把合同校验后的 Recipe 编译为固定 `bpy` 调用,限制请求 256 KiB、200 个声明对象、展开后 1000 个 Blender 对象、100 万 mesh polygon、材质/灯光/视图和修改器数量;Worker 还机械校验 ID 唯一性、引用、父子关系、布尔目标、管壁和请求预览的一致性。输出固定包含 `.blend`、首张 PNG、场景清单和溯源,可再选择最多七张 PNG 与 GLB;预览用 `preview_01`–`preview_08` 稳定槽位,视图业务名称记录在 manifest。
|
Blender 入口为 `blender.scene.author@v3`,唯一场景类型是 `recipe`;它是 Blender 专属的声明式静态场景目标状态,不是跨软件万能 DSL,也不是 Python、逐步按键或任意节点脚本。Recipe v1 组合带稳定 ID、父子层级和集合归属的 geometry/generator/text/empty 对象:geometry 覆盖 box/cylinder/cone/sphere/torus/plane、二维轮廓拉伸、轴向旋转、路径扫掠和有壁厚直管,受控 modifier 覆盖 boolean/bevel/mirror/array/solidify,并统一声明材质、变换、灯光、世界背景和 1–8 个相机视图;`rotary_kiln` 只是 generator 组件,可与其他对象处于同一场景,不再占据顶层 capability。adapter 1.0.0 把合同校验后的 Recipe 编译为固定 `bpy` 调用,限制请求 256 KiB、200 个声明对象、展开后 1000 个 Blender 对象、100 万 mesh polygon、材质/灯光/视图和修改器数量;Worker 还机械校验 ID 唯一性、引用、父子关系、布尔目标、管壁和请求预览的一致性。输出固定包含 `.blend`、首张 PNG、场景清单和溯源,可再选择最多七张 PNG 与 GLB;预览用 `preview_01`–`preview_08` 稳定槽位,视图业务名称记录在 manifest。
|
||||||
|
|
||||||
Scene Recipe 每次按完整目标状态从 factory-empty 场景确定性重建,不把上一版 `.blend` 中的隐式手工状态当作事实源;Workspace 继续保存当前与 rollback 两份工程供结果留存和失败恢复,完整 Recipe 保存在 Job 请求中作为续作事实源。为避免旧的回转窑专用状态与新语义混用,本次按用户明确授权直接移除 v2 合同与调度,旧 v2 Workspace 文件不删除但不能作为 v3 source。目标执行版本仍为 Blender 5.2 LTS;受管 Python 只负责发现并以 `--background --factory-startup --disable-autoexec --python-exit-code 1` 拉起已安装的 Blender,实际 `bpy` 在 Blender 自带 Python 内执行,请求不能提供脚本、命令行、本机路径、URL、插件、driver 或任意 Geometry Nodes。
|
Scene Recipe 每次按完整目标状态从 factory-empty 场景确定性重建,不把上一版 `.blend` 中的隐式手工状态当作事实源;Workspace 继续保存当前与 rollback 两份工程供结果留存和失败恢复,完整 Recipe 保存在 Job 请求中作为续作事实源。为避免旧的回转窑专用状态与新语义混用,本次按用户明确授权直接移除 v2 合同与调度,旧 v2 Workspace 文件不删除但不能作为 v3 source。目标执行版本仍为 Blender 5.2 LTS;受管 Python 只负责发现并以 `--background --factory-startup --disable-autoexec --python-exit-code 1` 拉起已安装的 Blender,实际 `bpy` 在 Blender 自带 Python 内执行,请求不能提供脚本、命令行、本机路径、URL、插件、driver 或任意 Geometry Nodes。
|
||||||
|
|
||||||
专业软件的连续加工以独立 `software_workspaces` 为稳定边界,Job 只记录一次操作。首个 Job 创建独立 `workspace_id`,两者不共用 ID;服务端保存 user 归属、capability、`home_node_id`、`head_job_id`、容量与留存元数据,Node 物理目录只使用 `<data_root>/workspaces/<workspace_id>/`,不再按 user 建上层目录。全新节点的 `data_root` 默认是当前专用账号的 `%LocalAppData%/Zcbot/WindowsNode`,以便无需提权即可从 UI 完成首次配置;升级节点若已有 `%ProgramData%/Zcbot/WindowsNode` 则继续原位使用,不做隐式迁移。专用运行账号可在 UI 中选择其他本机固定磁盘,机器级 `ZCBOT_WINDOWS_NODE_DATA_DIR` 拥有最高优先级并使 UI 只读;账号级选择保存在 HKCU,避免把根目录定位信息放入其自身。迁移先停止调度并等待本机任务及导出收尾,在目标同级临时目录复制并逐文件校验后才原子切换,旧目录不自动删除。这样用户统计集中在数据库完成,节点只管理实际落到本机的 workspace,也避免多节点各自维护一套用户目录树。Workspace 一旦首次调度即粘在 home node;后续 Job 必须以当前 head 为 source、串行回到同一节点,节点离线时等待而不复制工程或静默改派。
|
专业软件的连续加工以独立 `software_workspaces` 为稳定边界,Job 只记录一次操作。首个 Job 创建独立 `workspace_id`,两者不共用 ID;服务端保存 user 归属、capability、`home_node_id`、`head_job_id`、容量与留存元数据,Node 物理目录只使用 `<data_root>/workspaces/<workspace_id>/`,不再按 user 建上层目录。全新节点的 `data_root` 默认是当前专用账号的 `%LocalAppData%/Zcbot/WindowsNode`,以便无需提权即可从 UI 完成首次配置;升级节点若已有 `%ProgramData%/Zcbot/WindowsNode` 则继续原位使用,不做隐式迁移。专用运行账号可在 UI 中选择其他本机固定磁盘,机器级 `ZCBOT_WINDOWS_NODE_DATA_DIR` 拥有最高优先级并使 UI 只读;账号级选择保存在 HKCU,避免把根目录定位信息放入其自身。迁移先停止调度并等待本机任务及导出收尾,在目标同级临时目录复制并逐文件校验后才原子切换,旧目录不自动删除。这样用户统计集中在数据库完成,节点只管理实际落到本机的 workspace,也避免多节点各自维护一套用户目录树。Workspace 一旦首次调度即粘在 home node;后续 Job 必须以当前 head 为 source、串行回到同一节点,节点离线时等待而不复制工程或静默改派。
|
||||||
|
|
||||||
Node 对每个 Workspace 只保留 `current` 和 `rollback` 两代。续作开始前把 `current` 原子切换为 `rollback`,Worker 打开该工程并将新状态写入 Job 输出区;成功后输出区提升为新 `current`,失败或进程中断则恢复 `rollback`。因此连续加工不会为每个 Job 永久复制一份工程,物理空间上限约为当前工程加一份回滚工程;Job 账本仍完整保留参数、摘要和 local manifest。`software_job_revise(source_job_id, operation, outputs)` 只允许从 Workspace 当前 head 继续,复用已登记输入并重新经过当前契约校验。当前 Origin adapter 以 OPJU 为 workspace state;ANSYS 静力 capability 仍是无状态一次性执行,契约中的 `workspace: null` 明确保持原发布流程。
|
Node 对每个 Workspace 只保留 `current` 和 `rollback` 两代。续作开始前把 `current` 原子切换为 `rollback`,Worker 打开该工程并将新状态写入 Job 输出区;成功后输出区提升为新 `current`,失败或进程中断则恢复 `rollback`。因此连续加工不会为每个 Job 永久复制一份工程,物理空间上限约为当前工程加一份回滚工程;Job 账本仍完整保留参数、摘要和 local manifest。`software_job_revise(source_job_id, operation, outputs)` 只允许从 Workspace 当前 head 继续,复用已登记输入并重新经过当前契约校验。当前 Origin adapter 以 OPJU、Blender 以 BLEND、FreeCAD 以 FCStd 为 workspace state;ANSYS 静力 capability 仍是无状态一次性执行,契约中的 `workspace: null` 明确保持原发布流程。
|
||||||
|
|
||||||
科研统计图继续以独立 feature 增量扩展同一契约:`box/histogram/violin` 的系列只绑定原始 Y 列,统计规则由 Origin 固定模板决定;`violin` 把跨输入的观测列复制进受控全 Y 展示表,通过与菜单等价的固定 `worksheet -p 206 Violin` 一次性提交完整选择,使 Origin 正确建立分类轴和每列一组的核密度分布。原始输入表仍保留,默认以分类标签替代图例,并固定预留底部空间显示分类标签与轴标题;首版拒绝逐系列样式,因为原生多 Y 小提琴在图层内是单个统计对象,不能稳定映射普通曲线样式。`bubble` 增加正数 `size` 数据角色,由受控 modifier column 驱动符号尺寸;`band` 要求同一输入的 `x/y/lower/upper`,先画上下界并填充到下一曲线,再叠加中心线,Worker 在打开 Origin 前拒绝非有限尺寸、非正尺寸和倒置边界。`stacked_column/stacked_area/stacked_bar` 统一把两条以上 XY 系列复制进内部连续 XYY 工作表,严格校验横坐标相同后建立 plot group,并只执行 Worker 内置的固定累计图层命令;请求不能提供命令、模板或工作表范围。材料谱图使用独立 `stacked_line` feature:请求显式给出 `stack_gap_percent`,Worker 保留原始输入表,并在内部展示表中把每条曲线的最小值对齐为自身基线,再按全体最大 Y 跨度及间隔百分比逐条纵向偏移;各系列可拥有不同 X 采样点,展示纵轴不支持对数尺度。默认在曲线右端直接标注系列名并隐藏易遮挡谱线的图例,用户显式提供 `legend` 时仍按请求显示;单图标题按实际画布宽度定位。该实现不依赖节点上的 `OFFSETSTACKY.OTP` 或隐式主题,避免模板版本差异,同时 OPJU 中仍可追溯未变换数据。`area/polar/pie` 继续使用固定 Origin 类型 ID。未经过目标 Origin 版本真机验证的统计属性不进入公共 schema,避免暴露看似可配但不能稳定复现的参数。
|
科研统计图继续以独立 feature 增量扩展同一契约:`box/histogram/violin` 的系列只绑定原始 Y 列,统计规则由 Origin 固定模板决定;`violin` 把跨输入的观测列复制进受控全 Y 展示表,通过与菜单等价的固定 `worksheet -p 206 Violin` 一次性提交完整选择,使 Origin 正确建立分类轴和每列一组的核密度分布。原始输入表仍保留,默认以分类标签替代图例,并固定预留底部空间显示分类标签与轴标题;首版拒绝逐系列样式,因为原生多 Y 小提琴在图层内是单个统计对象,不能稳定映射普通曲线样式。`bubble` 增加正数 `size` 数据角色,由受控 modifier column 驱动符号尺寸;`band` 要求同一输入的 `x/y/lower/upper`,先画上下界并填充到下一曲线,再叠加中心线,Worker 在打开 Origin 前拒绝非有限尺寸、非正尺寸和倒置边界。`stacked_column/stacked_area/stacked_bar` 统一把两条以上 XY 系列复制进内部连续 XYY 工作表,严格校验横坐标相同后建立 plot group,并只执行 Worker 内置的固定累计图层命令;请求不能提供命令、模板或工作表范围。材料谱图使用独立 `stacked_line` feature:请求显式给出 `stack_gap_percent`,Worker 保留原始输入表,并在内部展示表中把每条曲线的最小值对齐为自身基线,再按全体最大 Y 跨度及间隔百分比逐条纵向偏移;各系列可拥有不同 X 采样点,展示纵轴不支持对数尺度。默认在曲线右端直接标注系列名并隐藏易遮挡谱线的图例,用户显式提供 `legend` 时仍按请求显示;单图标题按实际画布宽度定位。该实现不依赖节点上的 `OFFSETSTACKY.OTP` 或隐式主题,避免模板版本差异,同时 OPJU 中仍可追溯未变换数据。`area/polar/pie` 继续使用固定 Origin 类型 ID。未经过目标 Origin 版本真机验证的统计属性不进入公共 schema,避免暴露看似可配但不能稳定复现的参数。
|
||||||
|
|
||||||
|
|
@ -503,7 +518,7 @@ Workspace capability 的默认完成协议不发布正式产物。Node 只自动
|
||||||
|
|
||||||
第六阶段增加用户级 Job 中心与 Agent typed tools。`software_capability_list` 只暴露固定能力及当前在线空闲节点数,`software_job_submit/status/cancel` 在构造时绑定当前 user/task,模型不能跨用户或跨对话指定归属。`software_job_submit` 的唯一入口为通用 `inputs[] + operation + outputs[]`;输入和输出使用任务内稳定 key,具体 selector、type、format 和 options 由 capability 校验。`register_artifact` 是普通文件获得输入身份的唯一入口,并明确返回 UUID。右下角 Job 中心按用户聚合各对话任务,活动期短轮询、空闲期降频;终态变化通知用户,成功任务可回到原对话发起分析。取消采用协作协议:未派发任务直接终止,已派发任务先进入 `cancelling`,云端通过 WebSocket 发送并在心跳时重放 `job_cancel`,Node 杀死固定 Worker 进程树后回报 `cancelled`;终态写入仍由云端账本裁决。
|
第六阶段增加用户级 Job 中心与 Agent typed tools。`software_capability_list` 只暴露固定能力及当前在线空闲节点数,`software_job_submit/status/cancel` 在构造时绑定当前 user/task,模型不能跨用户或跨对话指定归属。`software_job_submit` 的唯一入口为通用 `inputs[] + operation + outputs[]`;输入和输出使用任务内稳定 key,具体 selector、type、format 和 options 由 capability 校验。`register_artifact` 是普通文件获得输入身份的唯一入口,并明确返回 UUID。右下角 Job 中心按用户聚合各对话任务,活动期短轮询、空闲期降频;终态变化通知用户,成功任务可回到原对话发起分析。取消采用协作协议:未派发任务直接终止,已派发任务先进入 `cancelling`,云端通过 WebSocket 发送并在心跳时重放 `job_cancel`,Node 杀死固定 Worker 进程树后回报 `cancelled`;终态写入仍由云端账本裁决。
|
||||||
|
|
||||||
Job 完成后的对话闭环复用 `software_jobs` 账本,不增加通用事件表。提交时用 `completion_action=report|analyze` 区分“只报告产物”和“报告并分析”,未明确分析时默认 `report`;成功终态将 `followup_status` 置为 `pending`。分发器只在原 task 空闲时领取:`report` 直接写入带 artifact refs 的固定 assistant 消息,不产生模型费用;`analyze` 写入前端隐藏的内部完成事件,并复用 task 单活锁、持久消息和 SSE run 自动续跑 Agent。`pending/running/completed/failed` 状态使服务重启、对话繁忙和重复完成回执均不会造成重复回复;Job 中心的手动深入分析通过同一入口重新排入分析,而不是模拟点击或只预填输入框。
|
Job 完成后的交互闭环复用 `software_jobs` 账本,不增加通用事件表,也不把每个软件结果复制为新的聊天消息。提交时用 `completion_action=report|analyze` 区分“只展示结果”和“结果完成后继续分析”,未明确分析时默认 `report`;终态将 `followup_status` 置为 `pending`。`report` 不依赖原 task 是否空闲,分发器只完成回调账本;当前 task 的左侧结果轨按 Job 独立堆叠运行状态和结果,成功预览可直接打开,移动端收敛为结果入口。右侧 Job 中心仍是跨 task 的完整管理面,主消息流只保留用户与 Agent 的叙事;历史 `software_job_report` 消息继续兼容读取,但不再新增。只有 `analyze` 需要等待原 task 空闲,随后写入前端隐藏的内部完成事件,并复用 task 单活锁、持久消息和 SSE run 自动续跑 Agent。`pending/running/completed/failed` 状态使服务重启、对话繁忙和重复完成回执均不会造成重复分析;Job 中心的手动深入分析通过同一入口重新排入分析,而不是模拟点击或只预填输入框。
|
||||||
|
|
||||||
ANSYS 能力固定面向 Windows 上的 Mechanical 2024 R2(revision 242),并以共享 GeoData 层拆成检查和求解两个声明式入口。`ansys.geometry.inspect@v1` 只导入一份已登记 STEP/Parasolid/IGES,枚举 assembly/part/body/face、几何单位、包围盒、面质心/法向/面积、已有 Named Selection,并输出 JSON manifest、整体预览和溯源;它不划分网格或求解。`ansys.mechanical.static_structural@v2` 接受显式单位、内置 Structural Steel、全局网格、固定支撑、力/压力和三类结果,边界区域可引用导入的 Named Selection,也可用全局轴极值平面或带法向/偏置/容差的平面查询;查询只在 Mechanical 导入后的真实 GeoData 上解析,按实体 ID 创建项目内 Named Selection,可用 `expected_count` 硬断言选择数量,并把最终面、来源和选区预览写入结果。两者都不接受 Mechanical 工程、Python、APDL、Journal、URL 或本机路径;静力输出固定为 MECHDAT、汇总/结果表、选区及应力/变形图、solver log 和 `.meta` 溯源。
|
ANSYS 能力固定面向 Windows 上的 Mechanical 2024 R2(revision 242),并以共享 GeoData 层拆成检查和求解两个声明式入口。`ansys.geometry.inspect@v1` 只导入一份已登记 STEP/Parasolid/IGES,枚举 assembly/part/body/face、几何单位、包围盒、面质心/法向/面积、已有 Named Selection,并输出 JSON manifest、整体预览和溯源;它不划分网格或求解。`ansys.mechanical.static_structural@v2` 接受显式单位、内置 Structural Steel、全局网格、固定支撑、力/压力和三类结果,边界区域可引用导入的 Named Selection,也可用全局轴极值平面或带法向/偏置/容差的平面查询;查询只在 Mechanical 导入后的真实 GeoData 上解析,按实体 ID 创建项目内 Named Selection,可用 `expected_count` 硬断言选择数量,并把最终面、来源和选区预览写入结果。两者都不接受 Mechanical 工程、Python、APDL、Journal、URL 或本机路径;静力输出固定为 MECHDAT、汇总/结果表、选区及应力/变形图、solver log 和 `.meta` 溯源。
|
||||||
|
|
||||||
|
|
@ -519,10 +534,20 @@ ANSYS 能力固定面向 Windows 上的 Mechanical 2024 R2(revision 242),
|
||||||
|
|
||||||
当前网关与 zcbot 共用 host,但 HTML 响应同时用 CSP `sandbox` 和 iframe sandbox 强制 opaque origin,且不授予 `allow-same-origin`;这一约束在签名 URL 被顶层打开时同样生效,项目代码不能读取主站 localStorage/JWT。`Referrer-Policy: no-referrer` 防能力 URL 随外链泄露,资源 URL 自带同一签名所以 ES module 不依赖认证 Cookie。将来需要 LocalStorage、Service Worker 或全栈服务器时,升级条件是独立 wildcard preview origin + 专用只挂载当前项目目录的容器 + HTTP/WebSocket/SSE 代理、健康检查和空闲回收,而不是放宽当前同源 sandbox。
|
当前网关与 zcbot 共用 host,但 HTML 响应同时用 CSP `sandbox` 和 iframe sandbox 强制 opaque origin,且不授予 `allow-same-origin`;这一约束在签名 URL 被顶层打开时同样生效,项目代码不能读取主站 localStorage/JWT。`Referrer-Policy: no-referrer` 防能力 URL 随外链泄露,资源 URL 自带同一签名所以 ES module 不依赖认证 Cookie。将来需要 LocalStorage、Service Worker 或全栈服务器时,升级条件是独立 wildcard preview origin + 专用只挂载当前项目目录的容器 + HTTP/WebSocket/SSE 代理、健康检查和空闲回收,而不是放宽当前同源 sandbox。
|
||||||
|
|
||||||
|
### 8.18 Provider API 凭据动态控制面(implementation,2026-09-02)
|
||||||
|
|
||||||
|
模型、媒体、平台数据与语音 Provider 采用代码内静态可信注册表:Provider id、显示名、凭据字段组、固定测试协议、余额能力和巡检频率均由代码定义,Admin 只能提交凭据值,不能编辑目标 URL、`api_base`、模型、协议或请求模板。这一控制面与可编辑的用户外部系统 definition 分离,避免把平台根凭据送往管理员可变目标。
|
||||||
|
|
||||||
|
`provider_credentials` 每个 Provider 只保存一套当前凭据。各字段以 `provider + field` 为 AES-GCM AAD 独立加密,未配置 `ZCBOT_CREDENTIAL_MASTER_KEY` 时拒绝写数据库,不存在明文降级。数据库覆盖优先;没有完整覆盖时沿用既有 env,模型能力、价格、`model_id` 和 `api_base` 仍以 `config/models` / `config/media` 为事实源。
|
||||||
|
|
||||||
|
凭据生命周期采用请求级解析:LLM 在每次构造新的 provider 请求时解析一次,媒体、检索和语音在每次新 HTTP/WebSocket 会话开始时解析;因此更新后下一次请求立即生效,已经发出的 HTTP 或流式响应保持启动时的 Key。候选凭据先在内存中测试,认证成功(含低余额告警)后才以 revision 乐观并发原子替换;失败不改旧密文。
|
||||||
|
|
||||||
|
Provider 测试将 402/明确额度不足、认证失败、网络不可达和普通 429 分开分类。DeepSeek 读取官方 CNY 余额,`< ¥30` 为低余额,30 分钟巡检;其他 Provider 没有官方余额字段时只展示认证/连通状态。业务调用的明确 402、额度耗尽或认证失败进入同一状态与提醒入口。后台一轮以 PostgreSQL advisory lock 在蓝绿实例间选主,首次异常立即邮件,持续异常最多每日一次,恢复后重置提醒状态。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 附录:DeepSeek V4 关键事实(2026-04-24)
|
## 附录:DeepSeek V4 关键事实(2026-04-24)
|
||||||
|
|
||||||
- V4-Pro:1.6T/49B 激活,1M context,SWE-Bench 80.6;V4-Flash:284B/13B 激活,1M context
|
- V4-Pro:1.6T/49B 激活,1M context,SWE-Bench 80.6;V4-Flash:284B/13B 激活,1M context
|
||||||
- 推理:non-thinking / thinking / thinking-max;价格 in ~$0.145/M、out ~$1.74/M(约 Opus 1/6)
|
- 推理:non-thinking / thinking / thinking-max;2026-08-16 起按工作日 UTC 峰谷计费,具体单价以官方价格页与版本化模型档案为准
|
||||||
- 旧 `deepseek-chat/reasoner` 已于 2026-07 下线,全库仅存 `deepseek-v4-flash/pro`
|
- 旧 `deepseek-chat/reasoner` 已于 2026-07 下线,全库仅存 `deepseek-v4-flash/pro`
|
||||||
|
|
|
||||||
44
PROGRESS.md
44
PROGRESS.md
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`。
|
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`。
|
||||||
|
|
||||||
最后更新:2026-08-26(Origin 科研分析与平台来源升级,bump 0.69.0)
|
最后更新:2026-09-03(Unreleased:工具健康统一事件口径)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -20,6 +20,38 @@
|
||||||
---
|
---
|
||||||
## 已完成关键能力
|
## 已完成关键能力
|
||||||
|
|
||||||
|
- **09-03 / 0.71.0 / 工具链可靠性收敛**:tool arguments salvage 新增“至少两个完整且完全一致副本 + 尾部截断副本”的保守恢复,语义不一致仍拒绝。修复 artifacts 部分唯一索引谓词被参数化后 PostgreSQL 无法匹配 `ON CONFLICT` 的问题,并让输出中任意行首 `[GATE FAIL]` 均按质量门记录,避免 SVG/PPT 质检占用真实故障大数。无 schema/migration/API 变化,生产失败历史未改写。
|
||||||
|
|
||||||
|
- **09-03 / 0.71.0 / 工具健康统一写入 usage_events**:普通工具失败在 tool result 落消息后结构化写 `tool_failure`,并新增 `run_stopped`、`agent_guard`、`context_fold_failure`、`quality_gate` 四类任务异常事件;工具健康聚合彻底移除 messages JSONB 回扫,只用一次 usage_events 查询,切换前的普通失败历史不迁移。Admin 大数只计真实 failure,代理主动停止/重复保护/上下文异常改为独立控制区,质量门继续单列。新增 0040 migration,为健康 kind 时间窗和普通失败 message 去重补部分索引。
|
||||||
|
|
||||||
|
- **09-03 / 0.71.0 / 工具健康监控请求收敛**:Admin 首次进入时异步汇总工具健康大数,之后固定按 90 秒节流刷新并对进行中请求单飞,不再每 10 秒重复扫描近 7 天记录。服务端按查询参数增加 90 秒进程内单飞缓存,多页面或多管理员并发时共享一次扫描,日巡检继续直读不受缓存影响。本提交本身未新增表、字段、migration;结构化事件口径由后续同日改动完成。
|
||||||
|
|
||||||
|
- **09-03 / 0.71.0 / 管理后台紧凑总览与分组工作区**:Admin 由纵向长页改为顶部常驻的两级总览和“运行、用量、用户、资源、集成、质量”六个页签,四项运行状态使用主卡、四项运营指标收进紧凑指标栏;点击总览可切换页签并精确定位同口径详情,任务、容量、用户活跃、Token/成本和存储均补齐详情指标。页签 hash 支持刷新与前进后退,桌面端页签吸顶,移动端顶栏操作收进菜单、运营指标与页签可横向滑动;1440px/390px 浏览器渲染、六页签与深链交互检查通过,无 API、schema、migration 或运行方式变化。
|
||||||
|
|
||||||
|
- **09-02 / 0.71.0 / 单用户重型执行容量与排队提示**:共享执行容量保持整机 6、后台 4,单用户上限由 2 提升到 3;前台重型任务首次等待时通过 SSE 告知单用户、整机、内存压力或队列顺序原因,放行后恢复正常执行提示,排队仍不计入命令 timeout 且可取消。无 schema、migration 或 API 变化。
|
||||||
|
|
||||||
|
- **09-02 / 0.71.0 / Admin Provider API 凭据管理与余额提醒**:新增代码内静态可信 Provider 注册表和单行当前凭据控制面,0039 以 AES-GCM 字段级 AAD 密文保存;Admin 可查看 DB/env 来源与尾号、先测后原子替换、手动测试和删除覆盖,LLM/媒体/平台来源/语音均按新请求热解析。DeepSeek 每 30 分钟检查官方 CNY 余额,低于 ¥30 及明确 402/认证失败进入统一状态与邮件提醒,蓝绿实例以 PostgreSQL advisory lock 单轮选主;持续异常每日最多一次、恢复重置。未连接生产数据库、未调用真实第三方 API。
|
||||||
|
|
||||||
|
- **09-03 / 0.71.0 / FreeCAD Windows Node 接入**:新增 `freecad.model.author@v1` / adapter 1.0.0 与独立 `freecad` runtime,目标仅为正式版 FreeCAD `>=1.1.3,<1.2.0`。两层 Worker 由受管 Python 负责 `FreeCADCmd.exe` 发现、版本门禁、safe mode/独立配置与 Pillow 等轴测渲染,由 FreeCAD 自带 Python 固定调用 Part;Recipe 每次从空文档完整重建,参数引用先解析为数值,不写 Spreadsheet/Expression。首版覆盖五类基础体、STEP import、三类布尔、平移/旋转/组合,固定交付 FCStd、STEP、Recipe、manifest、预览和 provenance,并执行 recompute、Shape 有效性、FCStd/STEP 重开及实体数/包围盒/体积容差校验。Windows Node 增加 FreeCAD 软件卡、UI/环境变量/PATH/卸载注册表/标准目录检测与独立/完整包集成;纯 Recipe、假版本 probe、合同、源码和打包自动测试已落地,FreeCAD 1.1.3 真机 acceptance 待目标机执行,未伪造通过。
|
||||||
|
|
||||||
|
- **09-02 / 0.71.0 / 中性 CAD 几何准备**:新增无状态 `cad.geometry.prepare@v1`,以独立 `cad` runtime 和固定 CadQuery 2.8.0/OCP 7.9.3.1 生成供 ANSYS/Abaqus 复用的 STEP;首版含 inspect、八类基础体和受控 Recipe,覆盖显式二维截面、拉伸/旋转/扫掠/放样、布尔、圆角/倒角/抽壳、变换、阵列、STEP import 与多实体组合。固定交付 STEP、规范化 Recipe、几何 manifest、无 GUI 预览和 provenance,加入单位、请求/输入/步骤/点/实体/区域/超时与有限数防线,区域选择与 ANSYS v2 语义一致且支持 `expected_count`。Windows Node 新增 runtime-only 软件卡和基本体/STEP roundtrip probe;通用合同生命周期让旧 `ansys.geometry.inspect@v1` 停止普通推荐、保留显式诊断与历史兼容。本地数值 oracle、STEP 重开、确定性 manifest、区域与固定打包已验证;ANSYS/Abaqus 真实导入仍待目标机验收,未伪造通过。
|
||||||
|
|
||||||
|
- **09-02 / 0.71.0 / 管理后台运行总览分层**:Admin 八张摘要卡按当前运行与运营资源重排,主指标统一为近期或实时口径,累计与低频明细降级;执行容量新增可自动刷新的右侧详情抽屉,集中展示前后台队列、容器生命周期、单用户占用与宿主资源,并移除物理占用与逻辑配额口径不一致的存储进度条。无 API、schema、migration 或运行方式变化。
|
||||||
|
|
||||||
|
- **09-02 / 0.70.0 / 管理与更新日志界面收敛**:Admin 顶部“容器状态”直接承载前后台执行、排队、回收、宿主资源与单用户占用,移除重复的独立容量区块,并统一采用“容器依赖”“专业软件节点”用户文案;用户版更新日志只保留可感知变化,公开接口新增向后兼容的 offset 分页元数据,前端每页加载 5 个版本并按需继续加载。
|
||||||
|
|
||||||
|
- **09-01 / 0.70.0 / Sandbox 执行容量与临时依赖观测**:默认单容器资源提升到 4 GiB/2 CPU、pids 1024、shm 512 MiB、tmp 1 GiB,普通容器无活动 10 分钟回收;新增跨蓝绿/多 Web 实例共享的整机 6、后台 4、单用户 2 重型槽位,前台可取消排队且排队不计 timeout,活动计数防长命令被 reaper 误删,内存压力只暂停新放行。Docker 后台 proc 无槽时持久化 queued、约 30 秒恢复调度,运行中仍由 Docker 托管,终态目录继续保留 7 天。镜像构建基础 Python 包清单,容器删除前可信扫描 `/tmp/.local` 的 `.dist-info` 并以单会话表幂等记录非空差异;Admin 顶部“容器状态”集中展示实时容量,“容器依赖”提供 7/30 天/全部依赖会话统计。新增 0038 migration,未连接或迁移生产数据库、未启动真实专业软件。
|
||||||
|
|
||||||
|
- **08-31 / 0.70.0 / Windows Node 完整 WPF 管理界面**:本机任务、ANSYS 固定验收、诊断复制和数据目录迁移全部进入 WPF;任务页每秒刷新本地持久化记录并保持选择,验收支持进度、停止、通过报告校验与机器级执行门,数据迁移继续经过活动任务检查、停连、复制、SHA-256 校验、失败回滚和重启。新增 `JobMonitorService`、`AnsysAcceptanceService`、`DataRootManagementService` 与 `DiagnosticService`,删除经典 `ConfigurationForm` 和旧 `TrayApplicationContext`,WinForms 只保留托盘及受控系统对话框互操作;117 项 .NET 行为测试、28 项 Windows Node 源码专项、格式与 diff 检查通过,未启动专业软件、连接生产服务或读写真实节点数据。
|
||||||
|
|
||||||
|
- **08-31 / 0.70.0 / Windows Node Host/WPF 重构阶段 0–6 第二切片**:WPF 在节点概览、首次注册和运行设置基础上新增专业软件页,Origin、ANSYS、Blender 卡片通过 `SoftwareManagementService` 统一获取位置、runtime 和 probe 快照,可选择或恢复自动检测位置、安装/更新独立 Python 3.12 环境并取消安装。选择对话框仍限制在 Tray WinForms 互操作层,ViewModel 不访问注册表或启动进程;全机执行门会禁用安装,`ManagedRuntimeInstaller` 入口继续二次拒绝活动任务并保持临时验证、原子替换和失败 rollback。同步完成 WPF 视觉打磨:主题 token、圆角控件模板、导航选中态、状态色带/徽标、软件可用性标识、忙碌进度和窄窗口操作换行统一落地。ANSYS 验收、本机任务和数据目录迁移仍从经典配置进入。73 项 .NET 行为测试、191 项 Origin/ANSYS/Blender/合同/Node/UI 专项、格式与 diff 检查、Release 发布包及隔离 GUI/headless 烟测通过,未安装 runtime、启动专业软件、连接生产服务或读写真实节点数据。
|
||||||
|
|
||||||
|
- **08-30 / 0.70.0 / 国内模型版本化计费与历史矫正工具**:新增本地价格目录解析,直连 DeepSeek、豆包、智谱不再被 LiteLLM 滞后价目覆盖;DeepSeek 按 2026-08-16 官方新价及工作日 UTC 峰谷窗口计费,豆包使用人民币固定价,GLM 绑定国内 BigModel 标准按量价。主对话、Prompt 润色、自动标题、上下文折叠和知识库摘要统一提取缓存 usage 并保存价格 revision/tier/币种/汇率/分项快照;新增默认 dry-run、显式确认 apply/rollback 的历史重算脚本,可针对调价后 DeepSeek 事件审计并矫正成本。无 schema/migration,未连接或写入生产数据库。
|
||||||
|
|
||||||
|
- **08-27 / 0.70.0 / GLM-5.3 Flash + 通用 reasoning/多模态能力**:下架 GLM-5.1/5.2 的可选入口并以隐藏别名将存量 `glm.pro/pro52` 统一解析到 `glm.flash53`,Flash53 加入默认与专业档位;模型能力新增原生输入模态和 `none/tool_turn/conversation/provider_managed` reasoning 生命周期,历史状态仅向同一生产模型回放,DeepSeek 限当前工具轮、GLM 保留同模型会话并发送 `clear_thinking=false`。结构化图片附件仍以文件引用为事实源,只在 provider 请求边界安全物化为 Base64 `image_url`,GLM 不再注册 `look_at_image`,文档读取与生图/视频工具保持独立;无 schema/migration/依赖变化,未调用真实模型或连接生产数据库。
|
||||||
|
|
||||||
|
- **08-27 / 0.69.1 / 专业软件结果轨与终态收敛**:当前对话的专业软件 Job 按独立状态点堆叠在聊天左侧,悬停展示名称与进度,成功的 Workspace 任务可直接打开多预览画廊,移动端收敛为“软件结果”入口;主消息流只保留用户与 Agent 叙事,默认 `report` 完成后只更新 Job 账本和视图,显式 `analyze` 才重新进入 Agent 对话。Web SSE 终态统一延后到任务状态落库之后发送,结果刷新会清理不再活动的 live card,避免软件已出图但“思考中”被重新点亮。既有历史软件报告消息继续兼容显示,无 schema、migration 或 HTTP API 变化。
|
||||||
|
|
||||||
- **08-26 / 0.69.0 / Origin 科研分析 MVP**:新增非默认的 `origin.analysis@v1` / adapter 0.1.0,与稳定的 `origin.plot@v2` 分离;首批提供数据体检、最大值/面积/参考点归一化、固定局部多项式平滑、一/二阶导数、区间/累计梯形积分和带 95% 置信区间的线性拟合,固定输出可编辑 OPJU、CSV、诊断、分析规格与溯源,可选 XLSX,线性拟合工程同时保留 Origin 原生报告。六个合成真值用例在 Origin 2024 / originpro 1.1.15 / numpy 2.2.6 上完成生产 Worker、数值 oracle、OPJU 重开和进程释放门;前三例来自组合执行的逐例通过记录,后三例分别通过,目标机 COM 完整退出约需 2 分钟。Origin/合同/Job/Node 专项 159 项 unittest、Release .NET build、完整 Node publish 与独立 adapter ZIP 校验通过,未连接或写入数据库。
|
- **08-26 / 0.69.0 / Origin 科研分析 MVP**:新增非默认的 `origin.analysis@v1` / adapter 0.1.0,与稳定的 `origin.plot@v2` 分离;首批提供数据体检、最大值/面积/参考点归一化、固定局部多项式平滑、一/二阶导数、区间/累计梯形积分和带 95% 置信区间的线性拟合,固定输出可编辑 OPJU、CSV、诊断、分析规格与溯源,可选 XLSX,线性拟合工程同时保留 Origin 原生报告。六个合成真值用例在 Origin 2024 / originpro 1.1.15 / numpy 2.2.6 上完成生产 Worker、数值 oracle、OPJU 重开和进程释放门;前三例来自组合执行的逐例通过记录,后三例分别通过,目标机 COM 完整退出约需 2 分钟。Origin/合同/Job/Node 专项 159 项 unittest、Release .NET build、完整 Node publish 与独立 adapter ZIP 校验通过,未连接或写入数据库。
|
||||||
|
|
||||||
- **08-26 / 0.69.0 / platform_sources bounded context**:将 paper_server、内部材料库和 Materials Project 收敛为独立顶层包,以显式可信 Provider 列表统一可用性与生命周期,单来源配置/装配失败不阻断其他来源;`core/tool_registry.py` 只保留一个构建入口,与用户连接 `external_systems` 完全独立。模型工具统一改为来源明确的 `paper_server_*`、`materials_library_*`、`materials_project_*`,保留材料库批量检索、MP 专用 SDK/CIF/entries 语义及 paper_server 安全下载边界;部署环境变量仍是事实源,无数据库迁移、无版本提升。
|
- **08-26 / 0.69.0 / platform_sources bounded context**:将 paper_server、内部材料库和 Materials Project 收敛为独立顶层包,以显式可信 Provider 列表统一可用性与生命周期,单来源配置/装配失败不阻断其他来源;`core/tool_registry.py` 只保留一个构建入口,与用户连接 `external_systems` 完全独立。模型工具统一改为来源明确的 `paper_server_*`、`materials_library_*`、`materials_project_*`,保留材料库批量检索、MP 专用 SDK/CIF/entries 语义及 paper_server 安全下载边界;部署环境变量仍是事实源,无数据库迁移、无版本提升。
|
||||||
|
|
@ -124,6 +156,7 @@
|
||||||
- **08-12 / 0.64.1 / PDF/PPT 连续预览滚轮与触摸修复**:外层预览的防滚动穿透逻辑不再拦截来自 PDF 内层 viewport 的普通 wheel 事件,保留图片 Ctrl+滚轮缩放和其他预览的边界隔离;PDF viewport 同步声明触摸平移与 WebView/iOS 惯性滚动,使鼠标滚轮、触控板和手机上下滑动都直接驱动连续页列表。Node 前端预览 11 项、JavaScript 语法及 diff 检查通过;无 schema、migration、HTTP API、依赖或运行方式变化,未连接生产 DB。
|
- **08-12 / 0.64.1 / PDF/PPT 连续预览滚轮与触摸修复**:外层预览的防滚动穿透逻辑不再拦截来自 PDF 内层 viewport 的普通 wheel 事件,保留图片 Ctrl+滚轮缩放和其他预览的边界隔离;PDF viewport 同步声明触摸平移与 WebView/iOS 惯性滚动,使鼠标滚轮、触控板和手机上下滑动都直接驱动连续页列表。Node 前端预览 11 项、JavaScript 语法及 diff 检查通过;无 schema、migration、HTTP API、依赖或运行方式变化,未连接生产 DB。
|
||||||
|
|
||||||
- **08-12 / 0.64.0 / artifact 稳定身份 + 隐藏回收**:新增 0028 `artifacts` 生命周期表并回填存量结构化引用;新发布消息写带 `artifact_id` 的 v2 引用,移动/重命名保持身份,复制创建独立身份并记录来源,历史卡片按身份解析最新路径。删除已发布产物时移动到用户隐藏目录 `.zcbot_artifact_trash/` 并软删记录,普通文件仍物理删除;Python 35 项(测试库门控 1 skip)、Node 前端 11 项、mypy、Alembic 单 head、编译、Ruff 致命规则及 diff 检查通过,未连接或写入生产 DB。
|
- **08-12 / 0.64.0 / artifact 稳定身份 + 隐藏回收**:新增 0028 `artifacts` 生命周期表并回填存量结构化引用;新发布消息写带 `artifact_id` 的 v2 引用,移动/重命名保持身份,复制创建独立身份并记录来源,历史卡片按身份解析最新路径。删除已发布产物时移动到用户隐藏目录 `.zcbot_artifact_trash/` 并软删记录,普通文件仍物理删除;Python 35 项(测试库门控 1 skip)、Node 前端 11 项、mypy、Alembic 单 head、编译、Ruff 致命规则及 diff 检查通过,未连接或写入生产 DB。
|
||||||
|
- **08-28 / 0.70.0 / 回收站与用户配额分离**:0037 为 `user_disk_usage` 增加回收站字节与文件数快照;后台扫描从用户配额中排除 `.zcbot_artifact_trash`,同时保留其物理占用统计,Admin 分列展示配额占用、回收站和物理合计。相关 pytest 48 项与 41 个子测试、Python 编译、JavaScript 语法、Alembic 单 head、Ruff 致命规则及 diff 检查通过;本机 Docker daemon 未运行,DB 集成测试按安全门控跳过,未连接或写入生产 DB。
|
||||||
|
|
||||||
- **08-12 / 0.63.11 / PDF/PPT 连续预览滚动布局修复**:补齐 PDF 预览根容器的纵向 Flex 布局与可收缩高度约束,使内部连续页 viewport 获得实际可用高度和独立滚动区域,不再被外层 `overflow:hidden` 裁断;回归测试同步锁定根容器、viewport 与页列表三层布局契约。Node 前端预览 11 项、JavaScript 语法及 diff 检查通过;无 schema、migration、HTTP API、依赖或运行方式变化,未连接生产 DB。
|
- **08-12 / 0.63.11 / PDF/PPT 连续预览滚动布局修复**:补齐 PDF 预览根容器的纵向 Flex 布局与可收缩高度约束,使内部连续页 viewport 获得实际可用高度和独立滚动区域,不再被外层 `overflow:hidden` 裁断;回归测试同步锁定根容器、viewport 与页列表三层布局契约。Node 前端预览 11 项、JavaScript 语法及 diff 检查通过;无 schema、migration、HTTP API、依赖或运行方式变化,未连接生产 DB。
|
||||||
|
|
||||||
|
|
@ -428,7 +461,9 @@ core/llm_transport.py 438 ← wire 层健壮性:畸形/吐空检测+留
|
||||||
core/tool_registry.py 264 ← 声明式工具注册表((组名,gate,factory);secret/host 工具按实际能力 gate)
|
core/tool_registry.py 264 ← 声明式工具注册表((组名,gate,factory);secret/host 工具按实际能力 gate)
|
||||||
core/context.py 95 ← LLM 调用前压缩旧 tool / load_skill 消息(带压力门槛),保 tool_call 协议字段
|
core/context.py 95 ← LLM 调用前压缩旧 tool / load_skill 消息(带压力门槛),保 tool_call 协议字段
|
||||||
core/external_systems/*.py ← 外部系统目录/用户授权/凭据加密 + 通用 OpenAPI/MCP connector
|
core/external_systems/*.py ← 外部系统目录/用户授权/凭据加密 + 通用 OpenAPI/MCP connector
|
||||||
|
core/provider_credentials/*.py ← 静态平台 Provider 目录、DB/env resolver、候选测试、状态提醒与蓝绿巡检
|
||||||
core/software_nodes.py ← Windows Node 注册码、身份认证与运行状态
|
core/software_nodes.py ← Windows Node 注册码、身份认证与运行状态
|
||||||
|
core/software_contracts.py ← 专业软件合同热加载、生命周期/可见性与 feature 兼容门
|
||||||
core/sinks.py 101
|
core/sinks.py 101
|
||||||
core/paths.py 50 ← task_dir db form 归一
|
core/paths.py 50 ← task_dir db form 归一
|
||||||
core/probe.py 243
|
core/probe.py 243
|
||||||
|
|
@ -445,10 +480,10 @@ core/ark_client.py 105 ← 火山方舟 HTTP 客户端
|
||||||
core/asr_xfyun.py 170 ← 讯飞语音听写 IAT wss 客户端(整段 PCM→文本;web 语音输入用,diag: scripts/diag_asr.py)
|
core/asr_xfyun.py 170 ← 讯飞语音听写 IAT wss 客户端(整段 PCM→文本;web 语音输入用,diag: scripts/diag_asr.py)
|
||||||
core/asr_lfasr.py 250 ← 讯飞录音文件转写 LFASR 客户端(异步订单 + 说话人分离;transcribe_audio 工具底座,diag: scripts/diag_lfasr.py)
|
core/asr_lfasr.py 250 ← 讯飞录音文件转写 LFASR 客户端(异步订单 + 说话人分离;transcribe_audio 工具底座,diag: scripts/diag_lfasr.py)
|
||||||
core/agent_builder.py 649 ← 装配 lib:build_agent/system prompt(工具注册块已迁 tool_registry)
|
core/agent_builder.py 649 ← 装配 lib:build_agent/system prompt(工具注册块已迁 tool_registry)
|
||||||
core/executor.py / sandbox/{network,pool}.py / executor_docker.py ← Executor ABC + Docker per-user 容器池
|
core/executor.py / sandbox/{network,pool,capacity,package_scans}.py / executor_docker.py / procs.py ← Executor ABC + Docker per-user 容器池 + 宿主共享执行槽 + bg proc 文件队列 + 临时依赖扫描
|
||||||
tools/{base,output,fs,shell,run_python,skill_tool,skill_authoring,media_common,seedream,seedance,gpt_image,look_at_image,read_document,image_ref,web_search,web_fetch,documents,materials_project,transcribe_audio,office_to_pdf,external_systems}.py ← media_common=媒体五工具共享原语;external_systems=host-side 外部系统元工具
|
tools/{base,output,fs,shell,run_python,skill_tool,skill_authoring,media_common,seedream,seedance,gpt_image,look_at_image,read_document,image_ref,web_search,web_fetch,documents,materials_project,transcribe_audio,office_to_pdf,external_systems}.py ← media_common=媒体五工具共享原语;external_systems=host-side 外部系统元工具
|
||||||
main.py ~210 ← 入口:web / db / probe / user / sandbox check
|
main.py ~210 ← 入口:web / db / probe / user / sandbox check
|
||||||
db/migrations/versions/ 0001-0030
|
db/migrations/versions/ 0001-0039
|
||||||
web/app.py ~210 ← 工厂 + lifespan 编排(07-23 拆分;路由在 routers/,协程在 background 等)
|
web/app.py ~210 ← 工厂 + lifespan 编排(07-23 拆分;路由在 routers/,协程在 background 等)
|
||||||
web/routers/*.py ← 含 external_systems 用户连接与 software_nodes 节点路由
|
web/routers/*.py ← 含 external_systems 用户连接与 software_nodes 节点路由
|
||||||
web/{background,scheduler_runner,wechat_runner}.py ← lifespan 后台协程按域析出
|
web/{background,scheduler_runner,wechat_runner}.py ← lifespan 后台协程按域析出
|
||||||
|
|
@ -456,7 +491,8 @@ web/{runs,common,schemas,model_gate,userfiles}.py ← BG worker/共享 helper/
|
||||||
web/auth.py ~190 ← 邮箱密码 + platform_key → JWT
|
web/auth.py ~190 ← 邮箱密码 + platform_key → JWT
|
||||||
web/broker.py / sinks.py / pptx_render.py / admin.py
|
web/broker.py / sinks.py / pptx_render.py / admin.py
|
||||||
web/static/dev.html + admin.html + ui.css + js/*.js ← dev SPA 与管理后台零构建前端
|
web/static/dev.html + admin.html + ui.css + js/*.js ← dev SPA 与管理后台零构建前端
|
||||||
windows-node/ ← .NET 10 WinForms 托盘节点、注册与登录后自启动
|
software-contracts/ ← 语言无关专业软件合同(含 FreeCAD、中性 CAD 与生命周期)
|
||||||
|
windows-node/ ← .NET 10 WPF/托盘节点、固定 adapter、FreeCAD/CAD runtime 与发布脚本
|
||||||
web/static/vendor/ ~1 MB ← jszip / docx-preview / xlsx
|
web/static/vendor/ ~1 MB ← jszip / docx-preview / xlsx
|
||||||
tests/ 360 项 ← golden/rendering 基线 + 路由两套(nodb/db)+ DB 套件(ZCBOT_TEST_DB_URL 门控)
|
tests/ 360 项 ← golden/rendering 基线 + 路由两套(nodb/db)+ DB 套件(ZCBOT_TEST_DB_URL 门控)
|
||||||
─────────────────────────────────
|
─────────────────────────────────
|
||||||
|
|
|
||||||
92
RUN.md
92
RUN.md
|
|
@ -74,13 +74,13 @@
|
||||||
# SMTP_PASSWORD=<授权码/应用专用密码,非登录密码>
|
# SMTP_PASSWORD=<授权码/应用专用密码,非登录密码>
|
||||||
# SMTP_FROM=you@qq.com # 可选,默认取 SMTP_USER
|
# SMTP_FROM=you@qq.com # 可选,默认取 SMTP_USER
|
||||||
# SMTP_FROM_NAME=总院科研辅助智能体 # 可选,发件人显示名,默"总院科研辅助智能体"(不暴露内部代号)
|
# SMTP_FROM_NAME=总院科研辅助智能体 # 可选,发件人显示名,默"总院科研辅助智能体"(不暴露内部代号)
|
||||||
# 工具失败聚集巡检(0.58.11,反 9dcae061"挂 90 天无人知"):web 进程每天扫近 7 天
|
# 工具失败聚集巡检(0.58.11,2026-09-03 改为只扫 usage_events):web 进程每天扫近 7 天
|
||||||
# tool 错误消息 + 被丢弃的畸形 tool_call 参数(0.58.19,kind=malformed),同签名 >=5 次
|
# 结构化工具/运行异常事件(含畸形 tool_call 参数),同签名 >=5 次
|
||||||
# 且跨 >=2 task 判聚集 → 发下面邮箱(复用上面 SMTP_*;未配邮箱或 SMTP 则只打日志
|
# 且跨 >=2 task 判聚集 → 发下面邮箱(复用上面 SMTP_*;未配邮箱或 SMTP 则只打日志
|
||||||
# [toolfail] 行)。**只发近 24h 仍活跃的聚集**(0.58.19):去重集是内存态、每次部署清零,
|
# [toolfail] 行)。**只发近 24h 仍活跃的聚集**(0.58.19):去重集是内存态、每次部署清零,
|
||||||
# 不加活跃过滤的话高频部署期每次重启都会把 7 天窗口内早已安静的存量聚集重发一遍。
|
# 不加活跃过滤的话高频部署期每次重启都会把 7 天窗口内早已安静的存量聚集重发一遍。
|
||||||
# admin 页仍取低阈值全量,分区展示:近24h跨任务系统性故障 / 单任务反复失败 /
|
# admin 页仍取低阈值全量,分区展示:近24h跨任务系统性故障 / 单任务反复失败 /
|
||||||
# 按设计拦截的质量门 / 已安静历史;修复后看对应项进入“已安静”。同区的“工具
|
# 代理控制与上下文异常 / 按设计拦截的质量门 / 已安静历史;修复后看对应项进入“已安静”。同区的“工具
|
||||||
# 调用链路健康”按模型档+工具展示 tool_salvaged/tool_malformed 的24h/7d抢救率。
|
# 调用链路健康”按模型档+工具展示 tool_salvaged/tool_malformed 的24h/7d抢救率。
|
||||||
# ZCBOT_DEVELOPER_EMAIL=dev@example.com
|
# ZCBOT_DEVELOPER_EMAIL=dev@example.com
|
||||||
# ZCBOT_TOOLFAIL_SCAN_INTERVAL=86400 # 秒,默 86400;<=0 关掉巡检
|
# ZCBOT_TOOLFAIL_SCAN_INTERVAL=86400 # 秒,默 86400;<=0 关掉巡检
|
||||||
|
|
@ -93,9 +93,8 @@
|
||||||
# 单次 LLM 请求超时(秒),默 600(与 litellm 默认一致,显式化 + 可调)。长思考模型
|
# 单次 LLM 请求超时(秒),默 600(与 litellm 默认一致,显式化 + 可调)。长思考模型
|
||||||
# 被掐("600s 无字节 → run 标 error")再调大;流式正常出 chunk 不触发。
|
# 被掐("600s 无字节 → run 标 error")再调大;流式正常出 chunk 不触发。
|
||||||
# ZCBOT_LLM_TIMEOUT_S=600
|
# ZCBOT_LLM_TIMEOUT_S=600
|
||||||
# 后台进程(bg proc,DESIGN §8.12;shell/run_python background=true):每用户并发上限,默 3。
|
# 后台进程与前台重型执行统一受 sandbox.max_* / ZCBOT_MAX_* 共享容量控制。
|
||||||
# 状态锚 <user_root>/.zcbot_procs/,后台默认限时 7200s(工具 timeout 参数可调,cap 86400)。
|
# 状态锚 <user_root>/.zcbot_procs/,后台默认限时 7200s(工具 timeout 参数可调,cap 86400)。
|
||||||
# ZCBOT_MAX_BG_PROCS=3
|
|
||||||
# 静态多文件 Web 项目预览会话 TTL(秒),默认 7 天,平台限制 5 分钟~30 天。
|
# 静态多文件 Web 项目预览会话 TTL(秒),默认 7 天,平台限制 5 分钟~30 天。
|
||||||
# 改后重启 web 生效;只影响新建/再次发布的预览。
|
# 改后重启 web 生效;只影响新建/再次发布的预览。
|
||||||
# ZCBOT_WEB_PREVIEW_TTL_SECONDS=604800
|
# ZCBOT_WEB_PREVIEW_TTL_SECONDS=604800
|
||||||
|
|
@ -141,7 +140,8 @@
|
||||||
# ZCBOT_CREDENTIAL_MASTER_KEY=<至少 32 字符随机串>
|
# ZCBOT_CREDENTIAL_MASTER_KEY=<至少 32 字符随机串>
|
||||||
```
|
```
|
||||||
> litellm 在 import 时副作用加载 .env;入口走 `main.py`,`.env` 自动生效。直跑 `python -c "from core.storage import ..."` 不经 litellm 链路时记得自己 `import litellm` 触发,或手动 `export ZCBOT_DB_URL=...`。
|
> litellm 在 import 时副作用加载 .env;入口走 `main.py`,`.env` 自动生效。直跑 `python -c "from core.storage import ..."` 不经 litellm 链路时记得自己 `import litellm` 触发,或手动 `export ZCBOT_DB_URL=...`。
|
||||||
- **平台托管来源配置**:`PAPER_SERVER_*`、`DOCUMENT_SEARCH_*`、`MP_API_KEY` 由宿主部署环境提供,修改后重启 web 生效;不进数据库、用户连接、prompt、tool result、`run_python` 或 Docker sandbox。若需求升级为用户级凭据/per-user grant,接入 `external_systems`;若需要在线动态 definition、revision/reverify 或 OAuth,先设计独立 `platform_sources` 控制面,不直接扩展当前静态 env registry。
|
- **平台 Provider 凭据**:`DEEPSEEK_API_KEY`、`ZHIPUAI_API_KEY`、`ARK_API_KEY`、`UNIFYLLM_API_KEY`、`LOCAL_LLM_API_KEY`、`BOCHA_API_KEY`、`PAPER_SERVER_API_KEY`、`DOCUMENT_SEARCH_API_KEY`、`MP_API_KEY` 和两组 `XFYUN_*` 可继续由宿主 env 提供,也可在 0039 migration 后由 Admin「API 凭据」加密覆盖。数据库覆盖在下一次新外部请求生效;删除覆盖立即回退当前 env。URL、模型和协议继续由代码/YAML 管理,Admin 不可编辑。凭据不进入 prompt、tool result、`run_python` 或 Docker sandbox。
|
||||||
|
- **Provider 凭据部署**:先配置至少 32 字符的 `ZCBOT_CREDENTIAL_MASTER_KEY`(轮换沿用 `ZCBOT_CREDENTIAL_KEY_ID` / `ZCBOT_CREDENTIAL_PREVIOUS_KEYS`),再执行 `.venv/Scripts/python.exe main.py db upgrade head` 创建 `provider_credentials`。蓝绿部署须先迁移、后启动新代码。未配置 master key 时 env 调用保持可用,但 Admin 保存会明确拒绝;master key 丢失或 AAD 不匹配时已有 DB 覆盖不会静默回退 env,应恢复正确 keyring 或在可解密后删除覆盖。
|
||||||
- **依赖**:`pip install -r requirements.txt`(已在 `.venv` 里;含 `bcrypt`、`segno`、`cryptography`)。
|
- **依赖**:`pip install -r requirements.txt`(已在 `.venv` 里;含 `bcrypt`、`segno`、`cryptography`)。
|
||||||
- **微信接入(ClawBot,§8.7)**:① `main.py db upgrade head` 带上 migration `0012`;② `.env` 设 `ZCBOT_WECHAT_BOT_ENABLED=1` + `ZCBOT_WECHAT_SECRET_KEY=<串>`;③ 用户登录后点**左栏 rail「微信」按钮**(`/static/wechat_bind.html` 仍保留作独立/嵌入入口)扫码绑定(需个人微信 8.0.70+ 且灰度到 ClawBot 插件)。绑定后在微信「微信 ClawBot」对话即走 zcbot;**主动推送需用户近 24h 在微信开口过一次**(冷启动/超期推不出,退邮件兜底)。**支持语音消息**(voice_item SILK v3 → pilk 解码 → 讯飞 IAT 转写进对话,回执「🎤 已识别:…」;需 `XFYUN_*` 三件套 + ffmpeg + pilk,pilk 随 requirements 装)。
|
- **微信接入(ClawBot,§8.7)**:① `main.py db upgrade head` 带上 migration `0012`;② `.env` 设 `ZCBOT_WECHAT_BOT_ENABLED=1` + `ZCBOT_WECHAT_SECRET_KEY=<串>`;③ 用户登录后点**左栏 rail「微信」按钮**(`/static/wechat_bind.html` 仍保留作独立/嵌入入口)扫码绑定(需个人微信 8.0.70+ 且灰度到 ClawBot 插件)。绑定后在微信「微信 ClawBot」对话即走 zcbot;**主动推送需用户近 24h 在微信开口过一次**(冷启动/超期推不出,退邮件兜底)。**支持语音消息**(voice_item SILK v3 → pilk 解码 → 讯飞 IAT 转写进对话,回执「🎤 已识别:…」;需 `XFYUN_*` 三件套 + ffmpeg + pilk,pilk 随 requirements 装)。
|
||||||
- **企业微信(渠道 B,纯推送,§8.7)**:① 管理员建自建应用 → 填 `WECOM_CORPID/AGENTID/SECRET`(+ 可见范围含目标用户);② `main.py db upgrade head`。**绑定两条路,任选**:
|
- **企业微信(渠道 B,纯推送,§8.7)**:① 管理员建自建应用 → 填 `WECOM_CORPID/AGENTID/SECRET`(+ 可见范围含目标用户);② `main.py db upgrade head`。**绑定两条路,任选**:
|
||||||
|
|
@ -156,7 +156,7 @@
|
||||||
- **channel 长会话上下文(微信/企业微信通用,0019)**:常驻会话不再无限膨胀。① **自动分段**——入站时距上次消息超过 `config.json` 的 `channel.session_gap_hours`(默 **6** 小时,设 `<=0` 关闭)→ 软重置:只把「最后一条 user 消息起」喂模型(保留上一轮做续聊锚点),之前的历史仍全留 DB,网页端照旧翻完整记录;② **手动新话题**——用户在微信/企业微信里直接发「新话题 / 新会话 / `/new` / 清空上下文」→ 硬重置,彻底从零(回执提示已归档)。两者都**不删任何消息**,只移动「喂给模型的窗口起点」`tasks.context_base_idx`。网页端「清空对话」(`POST /v1/tasks/{id}/clear`)仍整清并把 base 归 0。需 `main.py db upgrade head` 带上 `0019`。
|
- **channel 长会话上下文(微信/企业微信通用,0019)**:常驻会话不再无限膨胀。① **自动分段**——入站时距上次消息超过 `config.json` 的 `channel.session_gap_hours`(默 **6** 小时,设 `<=0` 关闭)→ 软重置:只把「最后一条 user 消息起」喂模型(保留上一轮做续聊锚点),之前的历史仍全留 DB,网页端照旧翻完整记录;② **手动新话题**——用户在微信/企业微信里直接发「新话题 / 新会话 / `/new` / 清空上下文」→ 硬重置,彻底从零(回执提示已归档)。两者都**不删任何消息**,只移动「喂给模型的窗口起点」`tasks.context_base_idx`。网页端「清空对话」(`POST /v1/tasks/{id}/clear`)仍整清并把 base 归 0。需 `main.py db upgrade head` 带上 `0019`。
|
||||||
- **PG**:`ZCBOT_DB_URL` 必填。本地 docker compose / 远端 dev / 生产任选;未设置时启动清晰报错,不引导 docker(§7.4)。
|
- **PG**:`ZCBOT_DB_URL` 必填。本地 docker compose / 远端 dev / 生产任选;未设置时启动清晰报错,不引导 docker(§7.4)。
|
||||||
- **OpenAPI / MCP 外部系统**:① `.env` 配置独立的 `ZCBOT_CREDENTIAL_MASTER_KEY`,可选 `ZCBOT_CREDENTIAL_KEY_ID` 标识当前密钥;轮换时把旧 key 以 JSON 对象放入 `ZCBOT_CREDENTIAL_PREVIOUS_KEYS`,待用户凭据完成重写后再移除。② 执行 `main.py db upgrade head`。③ admin 进入管理后台「外部系统」,选择通用 OpenAPI 或通用 MCP;具体 MES/ERP/LIMS 都作为数据库 definition 配置,不新增专用 provider。MCP 填写与登录 Base URL 同源的 Streamable HTTP URL,可选填写期望 Server 名称;连接后以 `tools/list` 为事实源。④ 普通用户点击左栏 **「外部」**,页面按 definition 动态显示用户名密码、API Key 或 Bearer Token;目标、Server 身份、登录、认证绑定或 TLS 变化后保留密文并暂停调用,重新测试成功后恢复。OpenAPI spec 和 MCP tool catalog 只在进程内按连接身份有界缓存,登录与业务响应均限长,普通用户和模型不能传任意 URL。
|
- **OpenAPI / MCP 外部系统**:① `.env` 配置独立的 `ZCBOT_CREDENTIAL_MASTER_KEY`,可选 `ZCBOT_CREDENTIAL_KEY_ID` 标识当前密钥;轮换时把旧 key 以 JSON 对象放入 `ZCBOT_CREDENTIAL_PREVIOUS_KEYS`,待用户凭据完成重写后再移除。② 执行 `main.py db upgrade head`。③ admin 进入管理后台「外部系统」,选择通用 OpenAPI 或通用 MCP;具体 MES/ERP/LIMS 都作为数据库 definition 配置,不新增专用 provider。MCP 填写与登录 Base URL 同源的 Streamable HTTP URL,可选填写期望 Server 名称;连接后以 `tools/list` 为事实源。④ 普通用户点击左栏 **「外部」**,页面按 definition 动态显示用户名密码、API Key 或 Bearer Token;目标、Server 身份、登录、认证绑定或 TLS 变化后保留密文并暂停调用,重新测试成功后恢复。OpenAPI spec 和 MCP tool catalog 只在进程内按连接身份有界缓存,登录与业务响应均限长,普通用户和模型不能传任意 URL。
|
||||||
- **Artifact 生命周期(0028)**:部署本版本必须先执行 `.venv/Scripts/python.exe main.py db upgrade head`。migration 会从存量 `messages.artifact_refs` 回填 active artifact 身份;删除后的已发布产物保存在用户根目录隐藏区 `.zcbot_artifact_trash/`,默认不自动清理且继续计入磁盘配额。普通文件删除语义不变。
|
- **Artifact 生命周期(0028/0037)**:部署本版本必须先执行 `.venv/Scripts/python.exe main.py db upgrade head`。migration 会从存量 `messages.artifact_refs` 回填 active artifact 身份;删除后的已发布产物保存在用户根目录隐藏区 `.zcbot_artifact_trash/`,默认不自动清理、不计用户配额,但其物理占用由后台扫描单列并展示在 Admin。普通文件删除语义不变。
|
||||||
- **用户消息结构化附件(0031)**:部署新版前执行 `.venv/Scripts/python.exe main.py db upgrade head`,为 `messages` 增加 nullable `attachment_refs`。migration 不回写或改动存量正文;旧客户端仍可只传 `content`,新版会发送结构化 `attachments`。
|
- **用户消息结构化附件(0031)**:部署新版前执行 `.venv/Scripts/python.exe main.py db upgrade head`,为 `messages` 增加 nullable `attachment_refs`。migration 不回写或改动存量正文;旧客户端仍可只传 `content`,新版会发送结构化 `attachments`。
|
||||||
- **旧 `factory_mes` definition 一次性转换**:新版代码不再识别 `factory_mes`;部署时保持旧服务进程运行,先从新代码目录执行数据脚本,转换成功后再重启到新版。脚本不加载 `.env`、不读取 `ZCBOT_DB_URL`,只认显式的 `ZCBOT_MIGRATION_DB_URL`;默认 dry-run,检查同名冲突与配置合法性。确认输出后加 `--apply`,脚本把 definition 转为 `generic_openapi + query`、物化 JWT/提示/只读 POST 配置并同步 active connection revision,不解密或改写用户凭据。
|
- **旧 `factory_mes` definition 一次性转换**:新版代码不再识别 `factory_mes`;部署时保持旧服务进程运行,先从新代码目录执行数据脚本,转换成功后再重启到新版。脚本不加载 `.env`、不读取 `ZCBOT_DB_URL`,只认显式的 `ZCBOT_MIGRATION_DB_URL`;默认 dry-run,检查同名冲突与配置合法性。确认输出后加 `--apply`,脚本把 definition 转为 `generic_openapi + query`、物化 JWT/提示/只读 POST 配置并同步 active connection revision,不解密或改写用户凭据。
|
||||||
```powershell
|
```powershell
|
||||||
|
|
@ -177,7 +177,7 @@
|
||||||
容器留着复用即可(alpine 版很轻);schema 升级后重跑一次 `db upgrade head`。
|
容器留着复用即可(alpine 版很轻);schema 升级后重跑一次 `db upgrade head`。
|
||||||
- **Auth env**:`PLATFORM_KEY` + `JWT_SECRET` 任一缺失 web 启动 fail-fast。生成随机串:`python -c "import secrets; print(secrets.token_urlsafe(48))"`。
|
- **Auth env**:`PLATFORM_KEY` + `JWT_SECRET` 任一缺失 web 启动 fail-fast。生成随机串:`python -c "import secrets; print(secrets.token_urlsafe(48))"`。
|
||||||
- **用户管理**(`users.email/password_hash/role`,0005 UNIQUE(email)、0009 role):dev SPA 登录后端。发用户两条路径任选:CLI `main.py user add`(下方),或在登录页右下角"+ 管理员添加用户"链接(需先设 `ZCBOT_ADMIN_TOKEN` env,弹窗输入 email/密码/管理员口令/角色)。撤用户 `DELETE FROM users WHERE email=...`(先 DELETE 该 user 的 tasks)。**用户自助改密**:登录后顶栏「改密码」按钮(走 `POST /v1/auth/change_password`,需知道旧密码);改邮箱 / 用户忘了旧密码无法自助 → 手动 SQL(见故障兜底)。
|
- **用户管理**(`users.email/password_hash/role`,0005 UNIQUE(email)、0009 role):dev SPA 登录后端。发用户两条路径任选:CLI `main.py user add`(下方),或在登录页右下角"+ 管理员添加用户"链接(需先设 `ZCBOT_ADMIN_TOKEN` env,弹窗输入 email/密码/管理员口令/角色)。撤用户 `DELETE FROM users WHERE email=...`(先 DELETE 该 user 的 tasks)。**用户自助改密**:登录后顶栏「改密码」按钮(走 `POST /v1/auth/change_password`,需知道旧密码);改邮箱 / 用户忘了旧密码无法自助 → 手动 SQL(见故障兜底)。
|
||||||
- **角色与管理后台**(`users.role` ∈ `user`/`admin`):admin 才显顶栏"管理"入口 → `/static/admin.html`(非 admin 403)。页面:左侧目录(点击滚到对应区)+ 运行态/任务/用户用量/按模型/各用户用量/存储;「按模型」「各用户用量」支持时间筛选(全部/近7天/近30天)+ 排序(按成本/按用量),「各用户用量」「存储」分页;顶栏「导出 PDF」走浏览器打印(在打印对话框选"另存为 PDF",列表取前 10)。提管理员 `main.py user role --email X --role admin`(改完即时生效,role 走 DB 查不进 JWT)。`ZCBOT_ADMIN_TOKEN` 是另一回事(发用户共享口令),与 role 互不相干。
|
- **角色与管理后台**(`users.role` ∈ `user`/`admin`):admin 才显顶栏"管理"入口 → `/static/admin.html`(非 admin 403)。页面顶部常驻紧凑总览,下方按“运行/用量/用户/资源/集成/质量”页签切换;点击总览卡会进入对应页签并定位同口径详情。执行容量也可从详情侧栏查看;「按模型」「各用户用量」支持时间筛选(全部/近7天/近30天)+ 排序(按成本/按用量),「各用户用量」「存储」分页;顶栏「导出 PDF」走浏览器打印(在打印对话框选"另存为 PDF",列表取前 10)。提管理员 `main.py user role --email X --role admin`(改完即时生效,role 走 DB 查不进 JWT)。`ZCBOT_ADMIN_TOKEN` 是另一回事(发用户共享口令),与 role 互不相干。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -215,7 +215,7 @@ python -m venv .venv
|
||||||
```bash
|
```bash
|
||||||
# 模型能力对账(费 token)
|
# 模型能力对账(费 token)
|
||||||
.venv/Scripts/python.exe main.py probe --model deepseek_v4.flash
|
.venv/Scripts/python.exe main.py probe --model deepseek_v4.flash
|
||||||
.venv/Scripts/python.exe main.py probe --model glm.pro # 智谱 GLM-5.1(走 litellm zai provider + 国内站 bigmodel.cn)
|
.venv/Scripts/python.exe main.py probe --model glm.flash53 # 智谱 GLM-5.3 Flash(原生多模态,国内站 bigmodel.cn)
|
||||||
.venv/Scripts/python.exe main.py probe --model local.r1 # 内网 DeepSeek-R1(满血,128K),涉密任务用;需 .env 设 LOCAL_LLM_API_KEY
|
.venv/Scripts/python.exe main.py probe --model local.r1 # 内网 DeepSeek-R1(满血,128K),涉密任务用;需 .env 设 LOCAL_LLM_API_KEY
|
||||||
.venv/Scripts/python.exe main.py probe --model local.qwen3 # 内网 Qwen3-30B-A3B(MoE,原生 32K);共享 LOCAL_LLM_API_KEY
|
.venv/Scripts/python.exe main.py probe --model local.qwen3 # 内网 Qwen3-30B-A3B(MoE,原生 32K);共享 LOCAL_LLM_API_KEY
|
||||||
|
|
||||||
|
|
@ -361,6 +361,10 @@ $env:ZCBOT_EVAL_TOKEN = "<dedicated-eval-user-jwt>"
|
||||||
| `GET /v1/skills/{name}` | 返某 skill 完整 SKILL.md 正文(前端「技能」modal 点开查看);同名按 user wins | 必填 |
|
| `GET /v1/skills/{name}` | 返某 skill 完整 SKILL.md 正文(前端「技能」modal 点开查看);同名按 user wins | 必填 |
|
||||||
| `DELETE /v1/skills/{name}` | 删当前 user 私有 skill(`.skills/<name>/` 整目录);只删 user 源,内置不可删 → 404;`.skills` 文件面板隐藏,这是 UI 上删自己 skill 的唯一入口 | 必填 |
|
| `DELETE /v1/skills/{name}` | 删当前 user 私有 skill(`.skills/<name>/` 整目录);只删 user 源,内置不可删 → 404;`.skills` 文件面板隐藏,这是 UI 上删自己 skill 的唯一入口 | 必填 |
|
||||||
| `GET /v1/external-system-providers` | 列管理员已启用的外部系统目录和安全的动态凭据字段声明;不返回完整配置或密钥 | 必填 |
|
| `GET /v1/external-system-providers` | 列管理员已启用的外部系统目录和安全的动态凭据字段声明;不返回完整配置或密钥 | 必填 |
|
||||||
|
| `GET /v1/admin/provider-credentials` | Admin 查看静态 Provider 目录、来源、脱敏尾号、测试/余额/提醒状态;不返回密文或明文 | admin |
|
||||||
|
| `PUT /v1/admin/provider-credentials/{provider_id}` | Admin 提交完整凭据组与 `expected_revision`;候选先测试,成功后原子替换 | admin |
|
||||||
|
| `POST /v1/admin/provider-credentials/{provider_id}/test` | Admin 测试当前 DB/env 凭据;标记 billable 的 Provider 前端会二次确认 | admin |
|
||||||
|
| `DELETE /v1/admin/provider-credentials/{provider_id}` | Admin 按 revision 删除数据库覆盖并回退 env | admin |
|
||||||
| `GET/POST /v1/external-systems` | 列当前用户连接 / 新建并在线验证连接;创建 body `{definition_id,name,credentials}`,旧 Factory `{username,password}` 请求继续兼容 | 必填 |
|
| `GET/POST /v1/external-systems` | 列当前用户连接 / 新建并在线验证连接;创建 body `{definition_id,name,credentials}`,旧 Factory `{username,password}` 请求继续兼容 | 必填 |
|
||||||
| `PUT /v1/external-systems/{id}/credentials` | 用 `{credentials}` 重新提交并在线验证当前用户连接;凭据不提供读取接口,旧用户名/密码格式继续兼容 | 必填 |
|
| `PUT /v1/external-systems/{id}/credentials` | 用 `{credentials}` 重新提交并在线验证当前用户连接;凭据不提供读取接口,旧用户名/密码格式继续兼容 | 必填 |
|
||||||
| `POST /v1/external-systems/{id}/test` | 用已保存密文凭据测试登录和 Swagger 可读性,并更新连接状态 | 必填 |
|
| `POST /v1/external-systems/{id}/test` | 用已保存密文凭据测试登录和 Swagger 可读性,并更新连接状态 | 必填 |
|
||||||
|
|
@ -392,7 +396,7 @@ $env:ZCBOT_EVAL_TOKEN = "<dedicated-eval-user-jwt>"
|
||||||
| `GET /v1/models` | 列 chat LLM 模型清单(扫 `config/models/*.yaml`),前端顶栏切换 / 新建对话框下拉用 | 必填 |
|
| `GET /v1/models` | 列 chat LLM 模型清单(扫 `config/models/*.yaml`),前端顶栏切换 / 新建对话框下拉用 | 必填 |
|
||||||
| `GET /v1/image_models` | 列图像生成 variant 清单(扫 `config/media/doubao.yaml` image 段),前端"生图"下拉用;yaml 无 image variant → 空列表 → UI 隐藏下拉 | 必填 |
|
| `GET /v1/image_models` | 列图像生成 variant 清单(扫 `config/media/doubao.yaml` image 段),前端"生图"下拉用;yaml 无 image variant → 空列表 → UI 隐藏下拉 | 必填 |
|
||||||
|
|
||||||
**SSE 事件**(每帧 `event: <type>` + `data: <JSON>`):建连时若当前 run 已发布计划,先补 `progress_snapshot{run_id,steps,waiting}` → `run_start{}` → `llm_start{}` → `text{delta}` / `tool_call{name,args,args_preview}` / `tool_result{name,preview,truncated}` → `llm_end{prompt_tokens,completion_tokens}` → `done{}`;cancel 走 `cancelled{}` 后随 `done{}` 收流;异常走 `error{msg}`。`task_progress` 新协议每次携带完整 `steps`,客户端整体替换;消息分页响应也附加 `progress_snapshot`,刷新不依赖当前 30 条窗口。`waiting=true` 表示本轮已调用 `ask_user` 等待确认;正常完成回看时隐藏进度,等待/取消/异常则折叠保留。30s 无 event 服务端发 `: ping` 心跳。nginx 反代记得关 buffering(响应头已带 `X-Accel-Buffering: no` 默认起效)。
|
**SSE 事件**(每帧 `event: <type>` + `data: <JSON>`):建连时若当前 run 已发布计划,先补 `progress_snapshot{run_id,steps,waiting}` → `run_start{}` → `llm_start{}` → `text{delta}` / `tool_call{name,args,args_preview}` / `execution_queue{state,reason,...}` / `tool_result{name,preview,truncated}` → `llm_end{prompt_tokens,completion_tokens}` → `done{}`;`execution_queue` 仅在前台重型工具确实等待共享容量时发送 `waiting`,放行后发送 `admitted`。cancel 走 `cancelled{}` 后随 `done{}` 收流;异常走 `error{msg}`。`task_progress` 新协议每次携带完整 `steps`,客户端整体替换;消息分页响应也附加 `progress_snapshot`,刷新不依赖当前 30 条窗口。`waiting=true` 表示本轮已调用 `ask_user` 等待确认;正常完成回看时隐藏进度,等待/取消/异常则折叠保留。30s 无 event 服务端发 `: ping` 心跳。nginx 反代记得关 buffering(响应头已带 `X-Accel-Buffering: no` 默认起效)。
|
||||||
|
|
||||||
**SSE 客户端注意**:浏览器原生 `EventSource` 不支持自定义 header,无法塞 Bearer token。要么 `fetch + ReadableStream` 自解 SSE 帧(dev.html 走的就是这条),要么后端日后加 `?token=...` query(目前不支持,避免 token 进 access log)。
|
**SSE 客户端注意**:浏览器原生 `EventSource` 不支持自定义 header,无法塞 Bearer token。要么 `fetch + ReadableStream` 自解 SSE 帧(dev.html 走的就是这条),要么后端日后加 `?token=...` query(目前不支持,避免 token 进 access log)。
|
||||||
|
|
||||||
|
|
@ -678,12 +682,18 @@ sudo -u zcbot docker network create zcbot-sandbox-net
|
||||||
# 容器 runtime(切 gVisor 用 runsc,Firecracker 用 kata;默 runc)
|
# 容器 runtime(切 gVisor 用 runsc,Firecracker 用 kata;默 runc)
|
||||||
# ZCBOT_SANDBOX_RUNTIME=
|
# ZCBOT_SANDBOX_RUNTIME=
|
||||||
# 空闲多少秒回收(默 300)
|
# 空闲多少秒回收(默 300)
|
||||||
# ZCBOT_SANDBOX_IDLE_TTL=300
|
# ZCBOT_SANDBOX_IDLE_TTL=600
|
||||||
# 资源限制(优先级 env > yaml `sandbox.*` > 默);改后重启 web 新起容器生效
|
# 资源限制(优先级 env > yaml `sandbox.*` > 默);改后重启 web 新起容器生效
|
||||||
# ZCBOT_SANDBOX_MEMORY=2g
|
# ZCBOT_SANDBOX_MEMORY=4g
|
||||||
# ZCBOT_SANDBOX_CPUS=1.0
|
# ZCBOT_SANDBOX_CPUS=2.0
|
||||||
# ZCBOT_SANDBOX_PIDS_LIMIT=256
|
# ZCBOT_SANDBOX_PIDS_LIMIT=1024
|
||||||
# ZCBOT_SANDBOX_SHM_SIZE=512m # chromium/mmdc 渲 mermaid 的 /dev/shm(docker 默 64MB 不够会挂超时)
|
# ZCBOT_SANDBOX_SHM_SIZE=512m # chromium/mmdc 渲 mermaid 的 /dev/shm(docker 默 64MB 不够会挂超时)
|
||||||
|
# ZCBOT_SANDBOX_TMP_SIZE=1g
|
||||||
|
# ZCBOT_MAX_ACTIVE_EXECS=6
|
||||||
|
# ZCBOT_MAX_BACKGROUND_EXECS=4
|
||||||
|
# ZCBOT_MAX_ACTIVE_EXECS_PER_USER=3
|
||||||
|
# 三个并发 env 只用于向下收紧,代码硬上限固定为 6 / 4 / 3。
|
||||||
|
# ZCBOT_MIN_MEM_AVAILABLE=1g # 低于阈值暂停新放行,不杀正在执行的任务
|
||||||
# PG 实际 IP,逗号分隔。defense-in-depth ── 即便落内网三段(§7.5 #1),
|
# PG 实际 IP,逗号分隔。defense-in-depth ── 即便落内网三段(§7.5 #1),
|
||||||
# init.sh 再加一遍 DROP 规则。生产部署必填。
|
# init.sh 再加一遍 DROP 规则。生产部署必填。
|
||||||
ZCBOT_PG_IPS=10.1.2.3,10.1.2.4
|
ZCBOT_PG_IPS=10.1.2.3,10.1.2.4
|
||||||
|
|
@ -705,7 +715,7 @@ ZCBOT_SANDBOX_BACKEND=docker .venv/bin/python main.py web
|
||||||
# 触发任一 shell / run_python 消息后,容器应已起
|
# 触发任一 shell / run_python 消息后,容器应已起
|
||||||
sudo -u zcbot docker ps --filter label=zcbot.product=sandbox
|
sudo -u zcbot docker ps --filter label=zcbot.product=sandbox
|
||||||
# 应看到 zcbot-sandbox-<your-uid>,STATUS = Up ...
|
# 应看到 zcbot-sandbox-<your-uid>,STATUS = Up ...
|
||||||
# 5 分钟无新消息后 reaper 自动 rm
|
# 10 分钟无执行活动后 reaper 自动 rm
|
||||||
```
|
```
|
||||||
|
|
||||||
也可直接起一个测试容器单验 hardening(不依赖 web 进程):
|
也可直接起一个测试容器单验 hardening(不依赖 web 进程):
|
||||||
|
|
@ -717,10 +727,10 @@ sudo -u zcbot docker run -d \
|
||||||
--label zcbot.product=sandbox \
|
--label zcbot.product=sandbox \
|
||||||
--label zcbot.user_id=$USER_ID \
|
--label zcbot.user_id=$USER_ID \
|
||||||
--network zcbot-sandbox-net \
|
--network zcbot-sandbox-net \
|
||||||
--read-only --tmpfs /tmp:exec,size=512m,mode=1777 \
|
--read-only --tmpfs /tmp:exec,size=1g,mode=1777 --shm-size=512m \
|
||||||
--cap-drop=ALL --cap-add=NET_ADMIN \
|
--cap-drop=ALL --cap-add=NET_ADMIN \
|
||||||
--security-opt=no-new-privileges \
|
--security-opt=no-new-privileges \
|
||||||
--pids-limit=256 --memory=2g --cpus=1.0 \
|
--pids-limit=1024 --memory=4g --cpus=2.0 \
|
||||||
-v /opt/zcbot/workspace/users/$USER_ID:/workspace \
|
-v /opt/zcbot/workspace/users/$USER_ID:/workspace \
|
||||||
-e ZCBOT_PG_IPS=10.1.2.3 \
|
-e ZCBOT_PG_IPS=10.1.2.3 \
|
||||||
zcbot-sandbox:latest
|
zcbot-sandbox:latest
|
||||||
|
|
@ -823,23 +833,29 @@ docker rm -f zcbot-sandbox-<user-uuid>
|
||||||
docker rm -f $(docker ps -aq -f label=zcbot.product=sandbox) 2>/dev/null
|
docker rm -f $(docker ps -aq -f label=zcbot.product=sandbox) 2>/dev/null
|
||||||
```
|
```
|
||||||
|
|
||||||
磁盘配额查看(0008 表):
|
磁盘配额与回收站查看(0008/0037 表):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 当前所有 user 用量
|
# 当前所有 user 的配额占用、回收站与文件物理合计
|
||||||
psql "$ZCBOT_DB_URL" -c "
|
psql "$ZCBOT_DB_URL" -c "
|
||||||
SELECT user_id, bytes_used/1024/1024 AS mb, file_count, scanned_at
|
SELECT user_id,
|
||||||
|
bytes_used/1024/1024 AS quota_mb,
|
||||||
|
trash_bytes/1024/1024 AS trash_mb,
|
||||||
|
(bytes_used+trash_bytes)/1024/1024 AS physical_mb,
|
||||||
|
file_count, trash_file_count, scanned_at
|
||||||
FROM user_disk_usage ORDER BY bytes_used DESC;"
|
FROM user_disk_usage ORDER BY bytes_used DESC;"
|
||||||
|
|
||||||
# 强制扫描某 user(改 yaml interval 太久不耐烦时)
|
# 强制扫描某 user(改 yaml interval 太久不耐烦时)
|
||||||
.venv/bin/python -c "
|
.venv/bin/python -c "
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
from core.storage.disk_quota import scan_user_dir, upsert_user_usage
|
from core.storage.disk_quota import scan_user_dir, scan_user_trash, upsert_user_usage
|
||||||
uid = UUID('<your-user-uuid>')
|
uid = UUID('<your-user-uuid>')
|
||||||
b, c = scan_user_dir(Path('/home/lighthouse/zcbot/workspace/users') / str(uid))
|
root = Path('/home/lighthouse/zcbot/workspace/users') / str(uid)
|
||||||
upsert_user_usage(uid, b, c)
|
b, c = scan_user_dir(root)
|
||||||
print(f'{b/1024/1024:.1f} MB / {c} files')"
|
tb, tc = scan_user_trash(root)
|
||||||
|
upsert_user_usage(uid, b, c, tb, tc)
|
||||||
|
print(f'quota={b/1024/1024:.1f} MB, trash={tb/1024/1024:.1f} MB')"
|
||||||
```
|
```
|
||||||
|
|
||||||
DNS / resolv.conf 文件位置(host 侧):
|
DNS / resolv.conf 文件位置(host 侧):
|
||||||
|
|
@ -979,7 +995,7 @@ sudo xfs_quota -x -c "limit -p bhard=10g zcbot_<user_uuid>" /opt
|
||||||
| prod 想把 workspace 落独立数据盘 | **别用 env / 别指 ROOT 外绝对路径**(workspace 锚定 ROOT,ROOT 外会让文件面板 / agent / 新建 task 三家分叉)。用 **bind mount** 把 `/data/...` 接到 `ROOT/workspace`,逻辑路径不变,DB 不用改。详「workspace 落独立数据盘」段 |
|
| prod 想把 workspace 落独立数据盘 | **别用 env / 别指 ROOT 外绝对路径**(workspace 锚定 ROOT,ROOT 外会让文件面板 / agent / 新建 task 三家分叉)。用 **bind mount** 把 `/data/...` 接到 `ROOT/workspace`,逻辑路径不变,DB 不用改。详「workspace 落独立数据盘」段 |
|
||||||
| 文件面板"目录尚未创建"但文件确实在 / agent 写的文件面板看不到 | workspace 被指到了 ROOT 外(旧 `ZCBOT_WORKSPACE_DIR` 绝对路径残留)→ 文件面板走 `resolve_workspace` 看一处、agent 走 DB `from_db_path`(锚 ROOT)看另一处。删掉 env、改用 bind mount(见上段),三家归一 |
|
| 文件面板"目录尚未创建"但文件确实在 / agent 写的文件面板看不到 | workspace 被指到了 ROOT 外(旧 `ZCBOT_WORKSPACE_DIR` 绝对路径残留)→ 文件面板走 `resolve_workspace` 看一处、agent 走 DB `from_db_path`(锚 ROOT)看另一处。删掉 env、改用 bind mount(见上段),三家归一 |
|
||||||
| `docker run zcbot-sandbox:latest` 报 `Unable to find image` | 镜像没 build。`sudo -u zcbot docker build -f deploy/sandbox/Dockerfile --build-arg HOST_UID=$(id -u zcbot) --build-arg HOST_GID=$(id -g zcbot) -t zcbot-sandbox:latest .` |
|
| `docker run zcbot-sandbox:latest` 报 `Unable to find image` | 镜像没 build。`sudo -u zcbot docker build -f deploy/sandbox/Dockerfile --build-arg HOST_UID=$(id -u zcbot) --build-arg HOST_GID=$(id -g zcbot) -t zcbot-sandbox:latest .` |
|
||||||
| 后台进程(bg proc)疑似残留 / 想手工排查 | docker 模式:`docker ps --filter label=zcbot.product=proc` 列在跑的 proc 容器,`docker rm -f zcbot-proc-<id>` 手杀;host 模式:状态锚在 `<user_root>/.zcbot_procs/<task_id>/<proc_id>/`(proc.json 有 pid,exit_code 文件在 = 已结束)。web 进程每小时 sweep 自动回收(终态目录 7 天 TTL);正常终止走前端停止按钮 / `check_process(action="kill")` |
|
| 后台进程(bg proc)疑似残留 / 想手工排查 | docker 模式:`docker ps --filter label=zcbot.product=proc` 列在跑的 proc 容器;状态锚在 `<user_root>/.zcbot_procs/<task_id>/<proc_id>/`,`proc.json` 的 `state=queued` 表示等待共享容量,`exit_code` 存在表示已结束。web 约每 30 秒调度 queued 并回收终态容器,终态目录仍保留 7 天;正常终止/取消排队走前端停止按钮或 `check_process(action="kill")`。若手工 `docker rm -f`,状态会收敛为 lost。 |
|
||||||
| 后台进程状态显示 lost | 进程没留退出码就没了 —— 宿主重启 / OOM killer / docker daemon 重启把它带走(bg proc 扛 zcbot 重启和蓝绿,但不扛宿主级重启)。output.log 保留到中断为止,需重新 background=true 发起 |
|
| 后台进程状态显示 lost | 进程没留退出码就没了 —— 宿主重启 / OOM killer / docker daemon 重启把它带走(bg proc 扛 zcbot 重启和蓝绿,但不扛宿主级重启)。output.log 保留到中断为止,需重新 background=true 发起 |
|
||||||
| 镜像 build pip 报 `THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE`(本仓 requirements 未钉 hash) | **不是被篡改、也不是 require-hashes**:镜像 index 声明的 wheel hash 与它实际吐出的文件字节不符 = 该镜像存的文件损坏 / 截断(2026-06-03 腾讯源就这么坏过 litellm-1.87.0)。换源重 build:`PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple/ sudo -E bash deploy/update.sh`。验真伪:`https://pypi.org/pypi/<pkg>/<ver>/json` 看官方 sha256 是哪边对。与下面"版本滞后(Could not find)"是两回事 |
|
| 镜像 build pip 报 `THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE`(本仓 requirements 未钉 hash) | **不是被篡改、也不是 require-hashes**:镜像 index 声明的 wheel hash 与它实际吐出的文件字节不符 = 该镜像存的文件损坏 / 截断(2026-06-03 腾讯源就这么坏过 litellm-1.87.0)。换源重 build:`PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple/ sudo -E bash deploy/update.sh`。验真伪:`https://pypi.org/pypi/<pkg>/<ver>/json` 看官方 sha256 是哪边对。与下面"版本滞后(Could not find)"是两回事 |
|
||||||
| 镜像 build pip 报 `ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', ...)` | 境内访问 PyPI 抖动。加 `--build-arg PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple/`(清华,现默认)或腾讯 / 阿里源,详 RUN.md「镜像构建」段。Dockerfile 已把 pip timeout 拉到 60s,主因仍是源不通而非超时 |
|
| 镜像 build pip 报 `ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', ...)` | 境内访问 PyPI 抖动。加 `--build-arg PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple/`(清华,现默认)或腾讯 / 阿里源,详 RUN.md「镜像构建」段。Dockerfile 已把 pip timeout 拉到 60s,主因仍是源不通而非超时 |
|
||||||
|
|
@ -1015,6 +1031,8 @@ sudo xfs_quota -x -c "limit -p bhard=10g zcbot_<user_uuid>" /opt
|
||||||
| `.pptx` 预览首次慢几秒 | 正常 —— soffice 冷启 + 转换 ~2-4s,转完缓存到源同目录 `.preview/<stem>.<hash>.pdf`,再点即时。源文件一改(mtime/size 变)hash 变、自动重转 |
|
| `.pptx` 预览首次慢几秒 | 正常 —— soffice 冷启 + 转换 ~2-4s,转完缓存到源同目录 `.preview/<stem>.<hash>.pdf`,再点即时。源文件一改(mtime/size 变)hash 变、自动重转 |
|
||||||
| `materials_project_*` tool 没出现在对话里 | `.env` 没设 `MP_API_KEY`,build_agent 跳过注册。设了重启 web 即可;Materials Project 联网查询走 host-side tool,离线 pymatgen 不受影响。 |
|
| `materials_project_*` tool 没出现在对话里 | `.env` 没设 `MP_API_KEY`,build_agent 跳过注册。设了重启 web 即可;Materials Project 联网查询走 host-side tool,离线 pymatgen 不受影响。 |
|
||||||
| 豆包调价了 | 改 `config/media/doubao.yaml` 的 `price_cny_per_image` 一行 → 重启 web。**历史 usage_events 不受影响**(units jsonb 里有当时单价 snapshot,聚合查仍按旧价);新写入按新价。涨价瞬间到改 YAML 中间这段记账偏低,开发期接受 |
|
| 豆包调价了 | 改 `config/media/doubao.yaml` 的 `price_cny_per_image` 一行 → 重启 web。**历史 usage_events 不受影响**(units jsonb 里有当时单价 snapshot,聚合查仍按旧价);新写入按新价。涨价瞬间到改 YAML 中间这段记账偏低,开发期接受 |
|
||||||
|
| 文本模型调价了 | 在 `config/models/<family>.yaml` 的 `pricing.periods` 追加带 `revision/effective_from/source_url` 的新价格版本并重启 web;直连模型以本地版本化价格为准,LiteLLM 只兜底无本地价格的网关。新事件会在 `usage_events.units` 快照版本、时段、币种、汇率和分项成本,历史记录不会被配置静默改写。 |
|
||||||
|
| DeepSeek 调价后的历史成本不准 | 先显式设置目标库 `ZCBOT_DB_URL`,运行 `.venv/Scripts/python.exe scripts/reprice_usage.py --from 2026-08-16T16:00:00Z` 查看 dry-run;核对脱敏 DB 目标、行数和差额后,才追加 `--apply --confirm APPLY_DEEPSEEK_REPRICE`。脚本单事务、按 revision 幂等,并保存 `previous_cost_cny`;回退用相同参数加 `--rollback --confirm APPLY_DEEPSEEK_REPRICE`。缺缓存明细的旧辅助调用按未命中估算并标记。 |
|
||||||
| `kill -HUP <pid>` 后 `/openapi.json` 没新接口 | uvicorn **不响应 SIGHUP**(没装 handler,落 Python 默认终止;Windows 上信号本身无效)。Ubuntu 上用 `systemctl restart zcbot`,或 unit 加 `--reload` 让 uvicorn 监听文件自动重起(见"部署"段)。验证:`curl -s http://127.0.0.1:8765/openapi.json \| python3 -c 'import sys,json;print([p for p in json.load(sys.stdin)["paths"] if "auth" in p])'` |
|
| `kill -HUP <pid>` 后 `/openapi.json` 没新接口 | uvicorn **不响应 SIGHUP**(没装 handler,落 Python 默认终止;Windows 上信号本身无效)。Ubuntu 上用 `systemctl restart zcbot`,或 unit 加 `--reload` 让 uvicorn 监听文件自动重起(见"部署"段)。验证:`curl -s http://127.0.0.1:8765/openapi.json \| python3 -c 'import sys,json;print([p for p in json.load(sys.stdin)["paths"] if "auth" in p])'` |
|
||||||
| `systemctl restart zcbot` 要等几十秒才退 | 正常 —— 优雅 drain 在等在跑的 run 收尾(`shutdown.drain_timeout` 默 30s),没在跑 run 时秒退。journal 出现 `[shutdown] draining N in-flight run(s)` 即正常。真急(不在乎杀掉在跑 run):`systemctl kill -s KILL zcbot` |
|
| `systemctl restart zcbot` 要等几十秒才退 | 正常 —— 优雅 drain 在等在跑的 run 收尾(`shutdown.drain_timeout` 默 30s),没在跑 run 时秒退。journal 出现 `[shutdown] draining N in-flight run(s)` 即正常。真急(不在乎杀掉在跑 run):`systemctl kill -s KILL zcbot` |
|
||||||
| 部署后在跑的对话被标 `error: server restarted before run finished` | 该 run 在 drain 期内没收尾、cancel 也没在 `cancel_grace` 内退,被 SIGKILL 后下次启动 reaper 标的。多半是 run 卡在不 poll cancel 的长动作(如单次超长 docker exec)或 `TimeoutStopSec` 配得比 drain 预算还小被提前 SIGKILL。先核对 unit `TimeoutStopSec > drain_timeout + cancel_grace`;真有超长 run 把 `drain_timeout` 调大 |
|
| 部署后在跑的对话被标 `error: server restarted before run finished` | 该 run 在 drain 期内没收尾、cancel 也没在 `cancel_grace` 内退,被 SIGKILL 后下次启动 reaper 标的。多半是 run 卡在不 poll cancel 的长动作(如单次超长 docker exec)或 `TimeoutStopSec` 配得比 drain 预算还小被提前 SIGKILL。先核对 unit `TimeoutStopSec > drain_timeout + cancel_grace`;真有超长 run 把 `drain_timeout` 调大 |
|
||||||
|
|
@ -1101,21 +1119,29 @@ Zcbot.WindowsNode.exe
|
||||||
|
|
||||||
Windows Node 不需要安装脚本。解压发布包后,使用实际运行 Node 的专用 Windows 账号直接启动 `Zcbot.WindowsNode.exe`:在主窗口注册节点,按需勾选“登录 Windows 后自动启动节点”,再到“专业软件”区域逐项配置并安装本机实际拥有的软件。开机启动保存为当前账号的 Windows `Run` 项,不安装 Windows Service,也不要求管理员创建计划任务。
|
Windows Node 不需要安装脚本。解压发布包后,使用实际运行 Node 的专用 Windows 账号直接启动 `Zcbot.WindowsNode.exe`:在主窗口注册节点,按需勾选“登录 Windows 后自动启动节点”,再到“专业软件”区域逐项配置并安装本机实际拥有的软件。开机启动保存为当前账号的 Windows `Run` 项,不安装 Windows Service,也不要求管理员创建计划任务。
|
||||||
|
|
||||||
专业软件 runtime 安装要求系统能通过 `py -3.12` 或 PATH 中的 `python.exe` 找到 Python 3.12。依赖默认显式使用清华 PyPI 镜像;如需切换官方源或院内镜像,在启动 Node 前设置 `ZCBOT_PIP_INDEX_URL`。解释器位于 `<data_root>\runtimes\origin|ansys|blender\Scripts\python.exe`;如需由集中运维提供既有受管解释器,机器级 `ZCBOT_ADAPTER_ORIGIN_PYTHON`、`ZCBOT_ADAPTER_ANSYS_PYTHON` 或 `ZCBOT_ADAPTER_BLENDER_PYTHON` 继续兼容,旧名 `ZCBOT_ORIGIN_PYTHON` 暂时兼容。`node.json`、可恢复任务、workspace 和 runtime 集中保存在同一数据根目录,不会因替换程序目录而丢失。任务请求无权选择解释器、脚本、安装依赖或应用路径。
|
专业软件 runtime 安装要求系统能通过 `py -3.12` 或 PATH 中的 `python.exe` 找到 Python 3.12。既有桌面软件依赖默认显式使用清华 PyPI 镜像;中性 CAD 因二进制 wheel 完整性要求默认使用官方 PyPI。统一覆盖源或切换院内镜像时,在启动 Node 前设置 `ZCBOT_PIP_INDEX_URL`,但不得绕过 pip 哈希校验。解释器位于 `<data_root>\runtimes\origin|ansys|blender|freecad|cad\Scripts\python.exe`;如需由集中运维提供既有受管解释器,机器级 `ZCBOT_ADAPTER_ORIGIN_PYTHON`、`ZCBOT_ADAPTER_ANSYS_PYTHON`、`ZCBOT_ADAPTER_BLENDER_PYTHON`、`ZCBOT_ADAPTER_FREECAD_PYTHON` 或 `ZCBOT_ADAPTER_CAD_PYTHON` 可指定固定解释器,旧名 `ZCBOT_ORIGIN_PYTHON` 暂时兼容。`node.json`、可恢复任务、workspace 和 runtime 集中保存在同一数据根目录,不会因替换程序目录而丢失。任务请求无权选择解释器、脚本、安装依赖或应用路径。
|
||||||
|
|
||||||
数据根目录可在 Node 主窗口“运行设置”中迁移到其他本机固定磁盘。已有未完成任务时 UI 会拒绝迁移;开始后先停止接收新任务,把整个根目录复制到目标同级临时目录并逐文件 SHA-256 校验,成功后才保存账号级路径并重启。旧目录不会自动删除,确认新目录正常后再由管理员手工清理。机器级 `ZCBOT_WINDOWS_NODE_DATA_DIR` 的优先级高于 UI 配置,存在时 UI 只读;设置或修改该环境变量后必须重启 Node,并由管理员自行保证完整数据已经迁入。软件卡安装的 runtime 始终跟随当前数据根目录。
|
数据根目录可在 Node 主窗口“运行设置”中迁移到其他本机固定磁盘。已有未完成任务时 UI 会拒绝迁移;开始后先停止接收新任务,把整个根目录复制到目标同级临时目录并逐文件 SHA-256 校验,成功后才保存账号级路径并重启。旧目录不会自动删除,确认新目录正常后再由管理员手工清理。机器级 `ZCBOT_WINDOWS_NODE_DATA_DIR` 的优先级高于 UI 配置,存在时 UI 只读;设置或修改该环境变量后必须重启 Node,并由管理员自行保证完整数据已经迁入。软件卡安装的 runtime 始终跟随当前数据根目录。
|
||||||
|
|
||||||
Workspace 目录固定为 `<data_root>\workspaces\<workspace-id>\`,不使用 user ID 上层目录。`current` 是当前可续作工程,`rollback` 只保留上一版用于失败恢复;每次成功续作替换 `current`,不会按 Job 永久堆积工程副本。不要手工移动 workspace 到其他节点:服务端会把后续 Job 固定调度到其 home node,节点离线时任务保持等待。备份或迁移节点时必须把数据库中的 workspace 归属和该目录作为一个整体处理。
|
Workspace 目录固定为 `<data_root>\workspaces\<workspace-id>\`,不使用 user ID 上层目录。`current` 是当前可续作工程,`rollback` 只保留上一版用于失败恢复;每次成功续作替换 `current`,不会按 Job 永久堆积工程副本。不要手工移动 workspace 到其他节点:服务端会把后续 Job 固定调度到其 home node,节点离线时任务保持等待。备份或迁移节点时必须把数据库中的 workspace 归属和该目录作为一个整体处理。
|
||||||
|
|
||||||
主窗口“专业软件”按 Origin、ANSYS Mechanical 2024 R2 和 Blender 分别管理应用位置与隔离 runtime。每张软件卡都可单独执行“安装/更新运行环境”,只会读取安装包内固定 requirements,并先在临时目录完成后再替换该软件的 `runtimes/<runtime_id>`;有对应活动任务时拒绝替换。安装要求本机存在 Python 3.12,可通过 `py -3.12` 或 PATH 中的 `python.exe` 找到。三种软件均可用“选择位置”保存当前专用账号的配置,点“自动检测”会清除手工值并重新探测;Origin 按 `ZCBOT_ORIGIN_EXE`、COM 注册、Windows 卸载注册表和 OriginLab 标准目录查找 `Origin*.exe`,ANSYS 与 Blender 继续按各自环境变量、注册表、PATH 或标准目录查找。环境变量 `ZCBOT_ORIGIN_EXE`、`ZCBOT_BLENDER_EXE` 与 `AWP_ROOT242` 继续兼容。选择 Origin 程序只确定安装位置和版本读取来源,实际执行仍要求 Origin COM probe 通过。配置或安装完成后不必重新注册节点,后续 probe/心跳会刷新能力状态。
|
主窗口“专业软件”按 Origin、ANSYS Mechanical 2024 R2、Blender、FreeCAD 和中性 CAD 分别管理隔离 runtime;前四者同时管理桌面应用位置,中性 CAD 是 runtime-only 软件卡,不显示也不接受“选择位置/自动检测”。每张软件卡都可单独执行“安装/更新运行环境”,只会读取安装包内固定 requirements,并先在临时目录完成后再替换该软件的 `runtimes/<runtime_id>`;有对应活动任务时拒绝替换。安装要求本机存在 Python 3.12,可通过 `py -3.12` 或 PATH 中的 `python.exe` 找到。Origin 按 `ZCBOT_ORIGIN_EXE`、COM 注册、Windows 卸载注册表和 OriginLab 标准目录查找 `Origin*.exe`,ANSYS、Blender 与 FreeCAD 继续按各自环境变量、注册表、PATH 或标准目录查找。环境变量 `ZCBOT_ORIGIN_EXE`、`ZCBOT_BLENDER_EXE`、`ZCBOT_FREECAD_CMD` 与 `AWP_ROOT242` 继续兼容;FreeCAD 位置必须是 `FreeCADCmd.exe`。选择 Origin 程序只确定安装位置和版本读取来源,实际执行仍要求 Origin COM probe 通过;CAD probe 会验证固定依赖、基本体和 STEP roundtrip。配置或安装完成后不必重新注册节点,后续 probe/心跳会刷新能力状态。
|
||||||
|
|
||||||
Blender 能力为 `blender.scene.author@v3`,目标节点版本为 Blender 5.2 LTS。请求固定使用 `scene.type=recipe`、`recipe_version=1`,通过 `objects[]` 组合 geometry/generator/text/empty:geometry 支持 box、cylinder、cone、sphere、torus、plane、extrude、revolve、sweep 和 pipe,generator 当前提供可与其他对象组合的 `rotary_kiln`;对象可声明父级、集合、材质、变换及 boolean/bevel/mirror/array/solidify 修改器。`render.views` 提供 1–8 个透视或正交相机,每个视图绑定唯一的 `preview_01`–`preview_08`,必须包含 `preview_01`;首张预览固定生成,其余视图必须在 `outputs` 请求同名 PNG,GLB 仍按需请求。任务同时生成 `scene.blend`、`scene-manifest.json`、`provenance.json` 和可选 `scene.glb`。
|
Blender 能力为 `blender.scene.author@v3`,目标节点版本为 Blender 5.2 LTS。请求固定使用 `scene.type=recipe`、`recipe_version=1`,通过 `objects[]` 组合 geometry/generator/text/empty:geometry 支持 box、cylinder、cone、sphere、torus、plane、extrude、revolve、sweep 和 pipe,generator 当前提供可与其他对象组合的 `rotary_kiln`;对象可声明父级、集合、材质、变换及 boolean/bevel/mirror/array/solidify 修改器。`render.views` 提供 1–8 个透视或正交相机,每个视图绑定唯一的 `preview_01`–`preview_08`,必须包含 `preview_01`;首张预览固定生成,其余视图必须在 `outputs` 请求同名 PNG,GLB 仍按需请求。任务同时生成 `scene.blend`、`scene-manifest.json`、`provenance.json` 和可选 `scene.glb`。
|
||||||
|
|
||||||
Recipe 是完整目标状态,每次从空场景重建;修改已有 v3 结果时,从 `software_job_status` 的 `editable_request` 形成完整下一版 `operation/outputs` 后调用 `software_job_revise`。v2 回转窑合同已破坏性移除,旧 Workspace 不能作为 v3 source,节点磁盘上的旧 `.blend` 不会自动删除;确需保留时应在部署前从 `<data_root>\workspaces\<workspace-id>\current\scene.blend` 备份。Node 会优先使用 UI 中指定的 `blender.exe`,否则从 `ZCBOT_BLENDER_EXE`、Windows 卸载注册表、PATH 或 `%ProgramFiles%\Blender Foundation\Blender *\blender.exe` 探测 Blender。启动器固定使用 Blender 后台、factory startup、禁止自动脚本和 Python 异常退出码,渲染优先使用 5.2 的 Eevee Next 标识并向旧 Eevee 标识回退;不读取用户启动文件,也不接受请求传入 Python、插件、命令行、URL、本机路径、driver 或任意节点脚本。
|
Recipe 是完整目标状态,每次从空场景重建;修改已有 v3 结果时,从 `software_job_status` 的 `editable_request` 形成完整下一版 `operation/outputs` 后调用 `software_job_revise`。v2 回转窑合同已破坏性移除,旧 Workspace 不能作为 v3 source,节点磁盘上的旧 `.blend` 不会自动删除;确需保留时应在部署前从 `<data_root>\workspaces\<workspace-id>\current\scene.blend` 备份。Node 会优先使用 UI 中指定的 `blender.exe`,否则从 `ZCBOT_BLENDER_EXE`、Windows 卸载注册表、PATH 或 `%ProgramFiles%\Blender Foundation\Blender *\blender.exe` 探测 Blender。启动器固定使用 Blender 后台、factory startup、禁止自动脚本和 Python 异常退出码,渲染优先使用 5.2 的 Eevee Next 标识并向旧 Eevee 标识回退;不读取用户启动文件,也不接受请求传入 Python、插件、命令行、URL、本机路径、driver 或任意节点脚本。
|
||||||
|
|
||||||
ANSYS 能力为 `ansys.geometry.inspect@v1` 和 `ansys.mechanical.static_structural@v2`,只面向 Mechanical 2024 R2(revision 242)。adapter 0.3.0 使用 PyMechanical 0.11.3:前者导入几何并输出面清单、单位、包围盒、Named Selection 和预览,后者可按已有 Named Selection、全局轴极值面或平面法向/偏置/容差建立支撑与载荷选区,再执行真实静力求解。v2 直接替换旧 v1,旧请求不会在新版节点调度。Node 会从 `adapters/` 自动发现并在每次心跳上报全部能力,现有节点不需要清身份或重新注册,但必须整体替换发布目录并重启。默认 probe 仍返回不可用;在目标机器完成真实许可证与基准算例验收前,不得设置 `ZCBOT_ANSYS_242_VALIDATED=1`。
|
FreeCAD 能力为 `freecad.model.author@v1`,只支持正式版 `>=1.1.3,<1.2.0`,RC、weekly、experimental、旧版和 1.2+ 都会在 probe 阶段保持不可用。请求固定使用 `model.type=recipe`、`recipe_version=1`、`unit=mm`,可声明最多 64 个有限数参数,并在步骤数值位置用 `{"parameter":"name"}` 引用;adapter 会先解析成具体数值,FCStd 不绑定 Spreadsheet 或 Expression。首版步骤为 box、cylinder、cone、sphere、torus、import_step、union、cut、intersect、translate、rotate、combine,只能绑定零或一个已登记 STEP/STP,不能提交 FCStd。固定输出 `model.FCStd`、`model.step`、`model-preview.png`、`model-recipe.json`、`model-manifest.json` 和隐藏 `.meta/provenance.json`;预览自动回传,其余按 Workspace/导出机制处理。续作从 `editable_request` 形成完整下一版 Recipe 后调用 `software_job_revise`,每版重新构建并由通用 current/rollback 管理。
|
||||||
|
|
||||||
普通几何先提交 `ansys.geometry.inspect@v1`,`outputs` 传空数组;成功后以 `geometry-manifest.json` 的单位、包围盒、面质心/法向/面积和 `geometry-preview.png` 判断边界区域。规则板件可在静力 v2 中使用 `{"type":"extreme_face","axis":"x","side":"min","expected_count":1}`;已知平面可使用 `{"type":"planar_faces","normal":[1,0,0],"offset":200,"tolerance":0.01,"expected_count":1}`,其中偏置和容差沿用 `analysis.unit_system` 的长度单位;已有 CAD Named Selection 则使用 `{"type":"named_selection","name":"fixed_mount"}`。每个 boundary/load 还必须提供唯一的 ASCII `name`,用于 Mechanical 树、反力结果和溯源。`expected_count` 可省略,但自动化生产请求应尽量提供;数量不符、没有平面、查询为空或名称冲突都会在网格和求解前失败。复杂装配或歧义区域不得只凭预览猜测,应回到 CAD 建立 Named Selection 后再提交。
|
FreeCAD 启动器只接受 UI 配置或 `ZCBOT_FREECAD_CMD` 指向的 `FreeCADCmd.exe`,否则按 PATH、卸载注册表与 `%ProgramFiles%\FreeCAD*` 标准目录探测。每个 Job 固定使用 `--safe-mode`、独立 `--user-cfg/--system-cfg` 与 `FREECAD_USER_HOME/FREECAD_USER_DATA/FREECAD_USER_TEMP`,并清理外部 Python 搜索路径;请求不能指定脚本、程序、命令行、URL、本机路径、宏、插件或 Workbench。FreeCAD 自带 Python 完成 recompute、Shape 校验、FCStd 保存重开和 STEP 导出重开,受管 Python/Pillow 根据 tessellation 生成无 GUI/GPU 的等轴测预览。
|
||||||
|
|
||||||
|
ANSYS 能力为 `ansys.geometry.inspect@v1` 和 `ansys.mechanical.static_structural@v2`,只面向 Mechanical 2024 R2(revision 242)。adapter 0.3.0 使用 PyMechanical 0.11.3:前者保留为真实 Mechanical 导入专项诊断,已标记 deprecated/diagnostic-only,普通能力枚举会指向 `cad.geometry.prepare@v1`;其外部合同、adapter、历史 Job 和明确诊断调用仍兼容至少一个发布周期。后者可按已有 Named Selection、全局轴极值面或平面法向/偏置/容差建立支撑与载荷选区,再执行真实静力求解。Node 会从 `adapters/` 自动发现并在每次心跳上报全部能力,现有节点不需要清身份或重新注册,但必须整体替换发布目录并重启。默认 probe 仍返回不可用;在目标机器完成真实许可证与基准算例验收前,不得设置 `ZCBOT_ANSYS_242_VALIDATED=1`。
|
||||||
|
|
||||||
|
普通几何先提交 `cad.geometry.prepare@v1`,`outputs` 传空数组;`inspect` 必须绑定一个已登记 STEP/STP,`primitive` 不带输入,`recipe` 可选绑定一个供 `import_step` 引用的输入。三种 feature 都必须显式给 `canonical_unit=mm|m`,不会猜测缺失或歧义 STEP 单位;固定输出为 `geometry.step`、`geometry-recipe.json`、`geometry-manifest.json`、`geometry-preview.png` 和 `.meta/provenance.json`。成功后的 `geometry.step` 会登记为普通 Artifact,可直接作为后续 `ansys.mechanical.static_structural@v2` 或 Abaqus capability 的 `geometry` 输入,不新增数据库关系。
|
||||||
|
|
||||||
|
CAD primitive 覆盖 box、cylinder、tube、cone/frustum、sphere、wedge、torus。Recipe 只接受合同列出的有序步骤和稳定 ASCII ID:基础体、显式三维坐标且共面的二维闭合轮廓、extrude/revolve/sweep/loft、union/cut/intersect、fillet/chamfer/shell、translate/rotate/mirror、linear/circular pattern、STEP import 和 combine。请求上限为 256 KiB、一个 256 MiB STEP、128 步、64 轮廓、每轮廓 512 点、64 区域和 256 个结果实体,adapter 超时 15 分钟;数值必须有限且坐标/尺寸受合同范围约束。任何未知引用、退化轮廓、空选区、`expected_count` 不符、无效布尔结果或 STEP roundtrip 数值偏差都会让 Job 清晰失败,不静默跳过。
|
||||||
|
|
||||||
|
规则板件可在 CAD 区域和静力 v2 中一致使用 `{"type":"extreme_face","axis":"x","side":"min","expected_count":1}`;已知平面可使用 `{"type":"planar_faces","normal":[1,0,0],"offset":200,"tolerance":0.01,"expected_count":1}`。CAD 还支持按 surface type 选 face、按 curve type 选 edge 和选择全部 solid;manifest 记录的是选择器、数量及几何摘要,不记录 OCCT/Mechanical 内部实体 ID。静力请求中的偏置和容差仍沿用 `analysis.unit_system` 长度单位;已有 CAD Named Selection 可直接使用 `{"type":"named_selection","name":"fixed_mount"}`。复杂装配或歧义区域不得只凭预览猜测,应在明确几何语义或 Named Selection 后再提交。
|
||||||
|
|
||||||
验收使用安装包自带的 NIST FTC-11 AP203 几何,adapter 通过与生产请求相同的 `planar_faces` 法向、偏置、容差及 `expected_count=1` 建立 `fixed_face`、`load_face`;不再走验收专用选面旁路。先退出托盘 Node、整体替换新版发布目录并启动,在 ANSYS 软件卡中安装/更新运行环境,确认能力仍显示待验收。随后在“ANSYS Mechanical 真机验收”卡片点击“运行内置基准验收”,只需选择报告父目录;UI 会使用固定参数执行一次完整进程树取消和连续 20 次求解,不能通过界面改变脚本、次数、试件或执行门条件。验收成功后先在许可证管理端确认席位均已归还,再点击“开启 ANSYS 执行门”;该按钮只会在本次 `acceptance-report.json` 明确通过后启用,写入机器级环境变量需要管理员权限。随后从托盘完全退出并重新启动 Node。内置 STEP 来源于 NIST MBE PMI Validation and Conformance Testing Project,可不受限制地使用,文件头保留来源说明。
|
验收使用安装包自带的 NIST FTC-11 AP203 几何,adapter 通过与生产请求相同的 `planar_faces` 法向、偏置、容差及 `expected_count=1` 建立 `fixed_face`、`load_face`;不再走验收专用选面旁路。先退出托盘 Node、整体替换新版发布目录并启动,在 ANSYS 软件卡中安装/更新运行环境,确认能力仍显示待验收。随后在“ANSYS Mechanical 真机验收”卡片点击“运行内置基准验收”,只需选择报告父目录;UI 会使用固定参数执行一次完整进程树取消和连续 20 次求解,不能通过界面改变脚本、次数、试件或执行门条件。验收成功后先在许可证管理端确认席位均已归还,再点击“开启 ANSYS 执行门”;该按钮只会在本次 `acceptance-report.json` 明确通过后启用,写入机器级环境变量需要管理员权限。随后从托盘完全退出并重新启动 Node。内置 STEP 来源于 NIST MBE PMI Validation and Conformance Testing Project,可不受限制地使用,文件头保留来源说明。
|
||||||
|
|
||||||
|
|
@ -1152,12 +1178,16 @@ Web 用户登录后,文件栏 Job 中心会聚合本人最近任务。活动
|
||||||
|
|
||||||
常用二维扩展单图 type 包括 `area`、`polar`、`pie`,均使用 `input/x/y`;其中 polar 的 X 是角度(度)、Y 是半径,pie 的 X 是扇区标签、Y 是数值,两者使用 Origin 原生坐标/标签系统且不接受笛卡尔 `x_axis/y_axis/z_axis` 参数。`stacked_area`、`stacked_bar` 与 `stacked_column` 一样至少提供两条 `input/x/y`。堆叠系列可以来自不同输入,但行数和逐行 X 值必须完全一致,否则 Worker 在启动绘图前以 `STACKED_PLOT_X_VALUES_MISMATCH` 拒绝,避免错位累计。统计图与这些扩展图暂不混入 `multi_panel.series[].kind`。
|
常用二维扩展单图 type 包括 `area`、`polar`、`pie`,均使用 `input/x/y`;其中 polar 的 X 是角度(度)、Y 是半径,pie 的 X 是扇区标签、Y 是数值,两者使用 Origin 原生坐标/标签系统且不接受笛卡尔 `x_axis/y_axis/z_axis` 参数。`stacked_area`、`stacked_bar` 与 `stacked_column` 一样至少提供两条 `input/x/y`。堆叠系列可以来自不同输入,但行数和逐行 X 值必须完全一致,否则 Worker 在启动绘图前以 `STACKED_PLOT_X_VALUES_MISMATCH` 拒绝,避免错位累计。统计图与这些扩展图暂不混入 `multi_panel.series[].kind`。
|
||||||
|
|
||||||
专业软件契约的源码事实源位于 `software-contracts/*.json`。zcbot 服务端会在文件变化后校验并热加载完整契约快照,正常修改契约不需要重启服务;非法或尚未写完的文件不会替换上一份有效快照。Node 发布时把对应事实源复制进 `adapters/<capability>/`,运行时只读取 manifest 同目录指向的合同,不存在共享目录 fallback;因此完整 Node 包与独立 adapter 包具有相同结构,更新时必须整体替换 adapter 目录并重启 Node。Node 心跳上报实际合同 SHA-256 和 Workspace 协议版本,服务端不会把 Workspace Job 派给未明确声明支持的节点。manifest 可用 `worker_timeout_minutes` 为每种软件声明 1–1440 分钟的执行上限;旧 manifest 未声明时继续使用 30 分钟。Worker 可以是受管 Python 脚本或独立 EXE。
|
专业软件契约的源码事实源位于 `software-contracts/*.json`。zcbot 服务端会在文件变化后校验并热加载完整契约快照,正常修改契约不需要重启服务;非法或尚未写完的文件不会替换上一份有效快照。可选 `lifecycle` 声明状态、普通/诊断可见性及替代 capability;缺失时按 active/standard 解释,兼容既有合同。Node 发布时把对应事实源复制进 `adapters/<capability>/`,运行时只读取 manifest 同目录指向的合同,不存在共享目录 fallback;因此完整 Node 包与独立 adapter 包具有相同结构,更新时必须整体替换 adapter 目录并重启 Node。Node 心跳上报实际合同 SHA-256 和 Workspace 协议版本,服务端不会把 Workspace Job 派给未明确声明支持的节点。manifest 可用 `worker_timeout_minutes` 为每种软件声明 1–1440 分钟的执行上限;旧 manifest 未声明时继续使用 30 分钟。Worker 可以是受管 Python 脚本或独立 EXE。
|
||||||
|
|
||||||
两个 JSON 的职责不同:`adapter.json` 只描述本机如何启动(实际版本、runtime、入口和契约文件名),`origin.plot.v2.json` 描述云端与本机共同遵守的业务请求/输出契约。前者不能替代后者;独立打包脚本只是把根目录的同一份业务契约复制进 adapter 交付目录,不维护第二份源码。
|
两个 JSON 的职责不同:`adapter.json` 只描述本机如何启动(实际版本、runtime、入口和契约文件名),`origin.plot.v2.json` 描述云端与本机共同遵守的业务请求/输出契约。前者不能替代后者;独立打包脚本只是把根目录的同一份业务契约复制进 adapter 交付目录,不维护第二份源码。
|
||||||
|
|
||||||
只更新 Origin adapter 时,不需要编译、重装或替换 Node EXE:运行 `windows-node\package-origin-adapter.bat` 可直接把 manifest、Worker、验收脚本、依赖清单和契约打成 `dist\origin.plot@v2-adapter.zip`。在 Node 机器上先从托盘退出 zcbot Windows Node,备份并整体替换 EXE 同级的 `adapters\origin.plot@v2\`,再启动 Node。不要在任务执行中覆盖目录。若 `requirements.txt` 发生变化,在对应软件卡中“安装/更新运行环境”;仅 `adapter.json`、契约、`worker.py` 或 `acceptance.py` 变化时无需更新 runtime。Node 启动后会用 Worker 的 `--probe` 核对 manifest 与 Worker 版本,版本不一致时该能力保持不可用。
|
只更新 Origin adapter 时,不需要编译、重装或替换 Node EXE:运行 `windows-node\package-origin-adapter.bat` 可直接把 manifest、Worker、验收脚本、依赖清单和契约打成 `dist\origin.plot@v2-adapter.zip`。在 Node 机器上先从托盘退出 zcbot Windows Node,备份并整体替换 EXE 同级的 `adapters\origin.plot@v2\`,再启动 Node。不要在任务执行中覆盖目录。若 `requirements.txt` 发生变化,在对应软件卡中“安装/更新运行环境”;仅 `adapter.json`、契约、`worker.py` 或 `acceptance.py` 变化时无需更新 runtime。Node 启动后会用 Worker 的 `--probe` 核对 manifest 与 Worker 版本,版本不一致时该能力保持不可用。
|
||||||
|
|
||||||
|
中性 CAD adapter 可用 `windows-node\package-cad-adapter.bat` 独立打成 `dist\cad.geometry.prepare@v1-adapter.zip`。退出 Node 后整体替换 `adapters\cad.geometry.prepare@v1\`,再从“中性 CAD”卡安装/更新固定 `cad` runtime;probe 必须显示 CadQuery 2.8.0 / OCP 7.9.3.1 且基本体与 STEP roundtrip 可用。若默认官方 PyPI 下载失败,可通过 `ZCBOT_PIP_INDEX_URL` 指向已同步并校验 wheel 的院内源;出现 `THESE PACKAGES DO NOT MATCH THE HASHES` 时应修复镜像缓存,不能添加忽略哈希参数。
|
||||||
|
|
||||||
|
FreeCAD adapter 可用 `windows-node\package-freecad-adapter.bat` 独立打成 `dist\freecad.model.author@v1-adapter.zip`;退出 Node 后整体替换 `adapters\freecad.model.author@v1\`,在“FreeCAD”卡确认应用位置并安装/更新固定 `freecad` runtime。目标机用 `<data_root>\runtimes\freecad\Scripts\python.exe <node_program>\adapters\freecad.model.author@v1\acceptance.py --work-root <new_empty_parent>\freecad-acceptance-YYYYMMDD --repeat 3` 执行固定验收;目录必须不存在。报告 `acceptance-report.json` 覆盖基础体、参数、布尔、变换、STEP import、FCStd/STEP roundtrip、模拟 Workspace revise/rollback、取消、空格/中文路径、连续运行及 FreeCAD 进程释放;报告通过且人工检查预览后才能记录目标机通过。
|
||||||
|
|
||||||
Origin adapter 1.1.0 起随包提供固定真机 QA harness,但暂不接入 Node UI。在专用测试节点确认没有活动 Job 后,从 PowerShell 使用 Origin 卡片已安装的受管解释器运行:`<data_root>\runtimes\origin\Scripts\python.exe <node_program>\adapters\origin.plot@v2\acceptance.py --work-root <new_empty_parent>\origin-acceptance-YYYYMMDD`。`--work-root` 必须指向尚不存在的新目录;adapter 1.3.0 默认依次执行 annotations、recipe_2x2、heatmap、surface_3d、stacked、stacked_line、violin、band,也可重复传入 `--case <name>` 只跑指定用例。每个用例先通过生产 Worker 子进程生成 OPJU/PNG/SVG/PDF,再由独立 Origin 会话重开工程,核对图层、系列、工作表/矩阵和数值 oracle,复导三种图件,并等待 Origin 进程退出;必要时可用 `--release-wait <seconds>` 调整释放等待时间。最终报告位于 `<work-root>\acceptance-report.json`。输入均为脚本生成的合成数据,但 PNG/SVG/PDF 仍需人工完成视觉 gate 后才能判定目标机器通过。
|
Origin adapter 1.1.0 起随包提供固定真机 QA harness,但暂不接入 Node UI。在专用测试节点确认没有活动 Job 后,从 PowerShell 使用 Origin 卡片已安装的受管解释器运行:`<data_root>\runtimes\origin\Scripts\python.exe <node_program>\adapters\origin.plot@v2\acceptance.py --work-root <new_empty_parent>\origin-acceptance-YYYYMMDD`。`--work-root` 必须指向尚不存在的新目录;adapter 1.3.0 默认依次执行 annotations、recipe_2x2、heatmap、surface_3d、stacked、stacked_line、violin、band,也可重复传入 `--case <name>` 只跑指定用例。每个用例先通过生产 Worker 子进程生成 OPJU/PNG/SVG/PDF,再由独立 Origin 会话重开工程,核对图层、系列、工作表/矩阵和数值 oracle,复导三种图件,并等待 Origin 进程退出;必要时可用 `--release-wait <seconds>` 调整释放等待时间。最终报告位于 `<work-root>\acceptance-report.json`。输入均为脚本生成的合成数据,但 PNG/SVG/PDF 仍需人工完成视觉 gate 后才能判定目标机器通过。
|
||||||
|
|
||||||
`origin.analysis@v1` 是与绘图分离的无状态科研分析入口,首批 operation 为 `data_check`、`normalize`、`smooth`、`differentiate`、`integrate`、`linear_fit`。请求只绑定一个 CSV/XLSX/JSON artifact;除 `data_check` 可只检查整表外,其余操作必须显式提供 `input/x/y`。数据体检通过 `parameters.fail_on` 指定需要阻断后续流程的问题,命中时 Job 仍成功并在 `diagnostics.json` 返回 `passed=false`;调用方应读取诊断后再决定是否提交分析。归一化的 `max_abs` 使用最大绝对值,`area` 使用有符号梯形积分,`reference` 在数据范围内按 X 线性插值;平滑固定为完整奇数窗口的局部多项式,求导支持一/二阶且接受非等间距 X,积分输出累计值及可选 `from/to` 区间值,线性拟合输出参数、95% t 置信区间、R²、RMSE、拟合值和残差,并在 OPJU 内保留 Origin 原生线性拟合报告。所有数值操作拒绝缺失、非有限、重复或非严格递增的 X/Y 数据,不做隐式排序或删行。
|
`origin.analysis@v1` 是与绘图分离的无状态科研分析入口,首批 operation 为 `data_check`、`normalize`、`smooth`、`differentiate`、`integrate`、`linear_fit`。请求只绑定一个 CSV/XLSX/JSON artifact;除 `data_check` 可只检查整表外,其余操作必须显式提供 `input/x/y`。数据体检通过 `parameters.fail_on` 指定需要阻断后续流程的问题,命中时 Job 仍成功并在 `diagnostics.json` 返回 `passed=false`;调用方应读取诊断后再决定是否提交分析。归一化的 `max_abs` 使用最大绝对值,`area` 使用有符号梯形积分,`reference` 在数据范围内按 X 线性插值;平滑固定为完整奇数窗口的局部多项式,求导支持一/二阶且接受非等间距 X,积分输出累计值及可选 `from/to` 区间值,线性拟合输出参数、95% t 置信区间、R²、RMSE、拟合值和残差,并在 OPJU 内保留 Origin 原生线性拟合报告。所有数值操作拒绝缺失、非有限、重复或非严格递增的 X/Y 数据,不做隐式排序或删行。
|
||||||
|
|
@ -1170,7 +1200,7 @@ Blender adapter 可独立运行 `windows-node\package-blender-adapter.bat` 打
|
||||||
|
|
||||||
注册配置写入 `<data_root>\node.json`;Token 使用 DPAPI `LocalMachine` 加密,ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号配置、注册并运行 Node。当前 MVP 可由 UI 写入该账号的登录启动项,不安装 Windows Service。
|
注册配置写入 `<data_root>\node.json`;Token 使用 DPAPI `LocalMachine` 加密,ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号配置、注册并运行 Node。当前 MVP 可由 UI 写入该账号的登录启动项,不安装 Windows Service。
|
||||||
|
|
||||||
直接双击 EXE 启动托盘 UI:红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开配置窗。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless`。
|
直接双击 EXE 启动托盘 UI:红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开 WPF 主窗口,可完成首次注册、重连、清除身份、复制诊断信息和登录自启动设置。“专业软件”页可配置 Origin、ANSYS、Blender、FreeCAD 位置及五套独立 runtime;中性 CAD 无应用位置配置。该页还可运行或停止 ANSYS 固定验收、在报告通过后开启机器级执行门;“本机任务”页每秒刷新本地持久化记录;“运行设置”页可打开或安全迁移完整数据目录。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless`。
|
||||||
|
|
||||||
若执行中的 Job 遇到 Node 重连,客户端会先等待已接收的本地执行管线收尾,再建立新连接,避免同一 Worker 被新旧连接同时恢复。若云端已将 Job 判为失败或取消、但本地 Worker 随后仍生成了结果,本机任务会显示“云端已终止”并保留工作区文件,不再永久停在 90% 重传;这些本地结果不会反向覆盖云端终态。
|
若执行中的 Job 遇到 Node 重连,客户端会先等待已接收的本地执行管线收尾,再建立新连接,避免同一 Worker 被新旧连接同时恢复。若云端已将 Job 判为失败或取消、但本地 Worker 随后仍生成了结果,本机任务会显示“云端已终止”并保留工作区文件,不再永久停在 90% 重传;这些本地结果不会反向覆盖云端终态。
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ models_dir: config/models
|
||||||
model_tiers:
|
model_tiers:
|
||||||
default: # 基线:所有未分配档位的用户(= 公测期默认可用)
|
default: # 基线:所有未分配档位的用户(= 公测期默认可用)
|
||||||
- deepseek_v4.flash
|
- deepseek_v4.flash
|
||||||
- deepseek_v4.pro
|
- glm.flash53
|
||||||
- local.r1 # 内网模型(涉密任务)
|
- local.r1 # 内网模型(涉密任务)
|
||||||
- local.qwen3
|
- local.qwen3
|
||||||
- seedream_5 # 图(config/media/doubao.yaml image 段)
|
- seedream_5 # 图(config/media/doubao.yaml image 段)
|
||||||
|
|
@ -26,8 +26,7 @@ model_tiers:
|
||||||
- doubao.turbo
|
- doubao.turbo
|
||||||
- doubao.pro
|
- doubao.pro
|
||||||
- doubao.evolving
|
- doubao.evolving
|
||||||
- glm.pro
|
- glm.flash53
|
||||||
- glm.pro52
|
|
||||||
- unifyllm.fable5 # unifyllm 网关(海外旗舰,走代理出口)
|
- unifyllm.fable5 # unifyllm 网关(海外旗舰,走代理出口)
|
||||||
- unifyllm.opus48
|
- unifyllm.opus48
|
||||||
- unifyllm.sonnet46
|
- unifyllm.sonnet46
|
||||||
|
|
@ -67,14 +66,20 @@ shutdown:
|
||||||
cancel_grace_seconds: 15 # 超时转 cancel 后再给的退场宽限
|
cancel_grace_seconds: 15 # 超时转 cancel 后再给的退场宽限
|
||||||
|
|
||||||
# Sandbox 容器资源限制(docker run flag,env 可 override);改后重启 web 生效,
|
# Sandbox 容器资源限制(docker run flag,env 可 override);改后重启 web 生效,
|
||||||
# 新起的容器用新值,已 running 的不变(idle 5min 回收后下次起)。
|
# 新起的容器用新值,已 running 的不变(idle 10min 回收后下次起)。
|
||||||
sandbox:
|
sandbox:
|
||||||
memory: 2g # --memory (env: ZCBOT_SANDBOX_MEMORY)
|
memory: 4g # --memory (env: ZCBOT_SANDBOX_MEMORY)
|
||||||
cpus: 1.0 # --cpus (env: ZCBOT_SANDBOX_CPUS)
|
cpus: 2.0 # --cpus (env: ZCBOT_SANDBOX_CPUS)
|
||||||
pids_limit: 1024 # --pids-limit (env: ZCBOT_SANDBOX_PIDS_LIMIT);线程也计数,
|
pids_limit: 1024 # --pids-limit (env: ZCBOT_SANDBOX_PIDS_LIMIT);线程也计数,
|
||||||
# chromium headless 一次 ~150-200 线程 + 超时残留进程,256 会被
|
# chromium headless 一次 ~150-200 线程 + 超时残留进程,256 会被
|
||||||
# 打满致 mmdc 渲 mermaid 必崩(pthread_create EAGAIN)
|
# 打满致 mmdc 渲 mermaid 必崩(pthread_create EAGAIN)
|
||||||
shm_size: 512m # --shm-size (env: ZCBOT_SANDBOX_SHM_SIZE);chromium/mmdc 渲 mermaid 的 /dev/shm,默 64MB 不够会挂
|
shm_size: 512m # --shm-size (env: ZCBOT_SANDBOX_SHM_SIZE);chromium/mmdc 渲 mermaid 的 /dev/shm,默 64MB 不够会挂
|
||||||
|
tmp_size: 1g # /tmp tmpfs (env: ZCBOT_SANDBOX_TMP_SIZE)
|
||||||
|
idle_ttl_seconds: 600 # 普通容器无 exec 活动 10 分钟回收
|
||||||
|
max_active_execs: 6 # 宿主硬上限;前后台统一计数
|
||||||
|
max_background_execs: 4
|
||||||
|
max_active_execs_per_user: 3
|
||||||
|
min_mem_available: 1g # 低于阈值暂停新放行,不杀已运行任务
|
||||||
# 容器 DNS server 显式配置(docker run --dns,容器 /etc/resolv.conf 直接写,
|
# 容器 DNS server 显式配置(docker run --dns,容器 /etc/resolv.conf 直接写,
|
||||||
# 绕过 docker daemon 上游 DNS 探测路径;腾讯云轻量 / 部分云上 daemon 探测
|
# 绕过 docker daemon 上游 DNS 探测路径;腾讯云轻量 / 部分云上 daemon 探测
|
||||||
# systemd-resolved 上游会失败,导致 embedded DNS 127.0.0.11 forward 出去也跪)。
|
# systemd-resolved 上游会失败,导致 embedded DNS 127.0.0.11 forward 出去也跪)。
|
||||||
|
|
|
||||||
|
|
@ -16,17 +16,39 @@ variants:
|
||||||
thinking_transport: extra_body
|
thinking_transport: extra_body
|
||||||
reasoning_effort_levels: [low, high, max]
|
reasoning_effort_levels: [low, high, max]
|
||||||
default_reasoning_effort: high
|
default_reasoning_effort: high
|
||||||
reasoning_replay: tool_calls # 普通跨轮剥离;工具调用 reasoning 按官方协议完整回传
|
reasoning_replay: tool_turn # 只在当前用户轮次的工具链内原样回传 reasoning
|
||||||
code_quality: good
|
code_quality: good
|
||||||
enable_run_python: true
|
enable_run_python: true
|
||||||
max_iterations: 120 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
|
max_iterations: 120 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
|
||||||
optimal_temperature: 0.3
|
optimal_temperature: 0.3
|
||||||
prompt_caching: false
|
prompt_caching: false
|
||||||
extended_thinking: false
|
extended_thinking: false
|
||||||
# Flash-0731 官方美元价按 1 USD ~= 7.2 CNY 折算;峰谷价正式生效后再同步。
|
# 扁平价保留给旧调用方,取当前谷时折算值;正式记账使用下面的版本化 pricing。
|
||||||
input_cny_per_mtoken: 1.0
|
input_cny_per_mtoken: 1.584
|
||||||
output_cny_per_mtoken: 2.0
|
output_cny_per_mtoken: 4.752
|
||||||
cache_hit_cny_per_mtoken: 0.02 # 官方 $0.0028 / M tokens
|
cache_hit_cny_per_mtoken: 0.0504
|
||||||
|
pricing:
|
||||||
|
periods:
|
||||||
|
- revision: deepseek-v4-20260816
|
||||||
|
currency: USD
|
||||||
|
fx_to_cny: 7.2
|
||||||
|
source_url: https://api-docs.deepseek.com/quick_start/pricing/
|
||||||
|
checked_at: 2026-08-30
|
||||||
|
effective_from: 2026-08-16T16:00:00Z
|
||||||
|
default: # 谷时(官方峰时之外)
|
||||||
|
input_per_mtoken: 0.22
|
||||||
|
cache_hit_per_mtoken: 0.007
|
||||||
|
output_per_mtoken: 0.66
|
||||||
|
time_tiers:
|
||||||
|
- name: peak
|
||||||
|
timezone: UTC
|
||||||
|
weekdays: [mon, tue, wed, thu, fri]
|
||||||
|
windows:
|
||||||
|
- ["01:00", "04:00"]
|
||||||
|
- ["06:00", "10:00"]
|
||||||
|
input_per_mtoken: 0.44
|
||||||
|
cache_hit_per_mtoken: 0.014
|
||||||
|
output_per_mtoken: 1.32
|
||||||
|
|
||||||
pro:
|
pro:
|
||||||
display_name: DeepSeek V4 Pro
|
display_name: DeepSeek V4 Pro
|
||||||
|
|
@ -42,13 +64,35 @@ variants:
|
||||||
thinking_transport: extra_body
|
thinking_transport: extra_body
|
||||||
reasoning_effort_levels: [low, medium, high, max]
|
reasoning_effort_levels: [low, medium, high, max]
|
||||||
default_reasoning_effort: medium
|
default_reasoning_effort: medium
|
||||||
reasoning_replay: tool_calls
|
reasoning_replay: tool_turn
|
||||||
code_quality: excellent
|
code_quality: excellent
|
||||||
enable_run_python: true
|
enable_run_python: true
|
||||||
max_iterations: 150 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
|
max_iterations: 150 # backstop 兜底,非"轮"预算;真正的空转防护是 loop 的无进展熔断 + _RepeatGuard
|
||||||
optimal_temperature: 0.2
|
optimal_temperature: 0.2
|
||||||
prompt_caching: false
|
prompt_caching: false
|
||||||
extended_thinking: false
|
extended_thinking: false
|
||||||
input_cny_per_mtoken: 2.0
|
input_cny_per_mtoken: 4.752
|
||||||
output_cny_per_mtoken: 20.0
|
output_cny_per_mtoken: 14.256
|
||||||
cache_hit_cny_per_mtoken: 0.2 # DeepSeek 前缀缓存命中价(input 的 ~0.1x)
|
cache_hit_cny_per_mtoken: 0.1584
|
||||||
|
pricing:
|
||||||
|
periods:
|
||||||
|
- revision: deepseek-v4-20260816
|
||||||
|
currency: USD
|
||||||
|
fx_to_cny: 7.2
|
||||||
|
source_url: https://api-docs.deepseek.com/quick_start/pricing/
|
||||||
|
checked_at: 2026-08-30
|
||||||
|
effective_from: 2026-08-16T16:00:00Z
|
||||||
|
default:
|
||||||
|
input_per_mtoken: 0.66
|
||||||
|
cache_hit_per_mtoken: 0.022
|
||||||
|
output_per_mtoken: 1.98
|
||||||
|
time_tiers:
|
||||||
|
- name: peak
|
||||||
|
timezone: UTC
|
||||||
|
weekdays: [mon, tue, wed, thu, fri]
|
||||||
|
windows:
|
||||||
|
- ["01:00", "04:00"]
|
||||||
|
- ["06:00", "10:00"]
|
||||||
|
input_per_mtoken: 1.32
|
||||||
|
cache_hit_per_mtoken: 0.044
|
||||||
|
output_per_mtoken: 3.96
|
||||||
|
|
|
||||||
|
|
@ -32,6 +32,17 @@ variants:
|
||||||
input_cny_per_mtoken: 3.0
|
input_cny_per_mtoken: 3.0
|
||||||
output_cny_per_mtoken: 15.0
|
output_cny_per_mtoken: 15.0
|
||||||
cache_hit_cny_per_mtoken: 0.6
|
cache_hit_cny_per_mtoken: 0.6
|
||||||
|
pricing:
|
||||||
|
periods:
|
||||||
|
- revision: doubao-seed21-20260628
|
||||||
|
currency: CNY
|
||||||
|
source_url: https://www.volcengine.com/product/ark
|
||||||
|
checked_at: 2026-08-30
|
||||||
|
effective_from: 2026-06-28T00:00:00Z
|
||||||
|
default:
|
||||||
|
input_per_mtoken: 3.0
|
||||||
|
cache_hit_per_mtoken: 0.6
|
||||||
|
output_per_mtoken: 15.0
|
||||||
|
|
||||||
pro:
|
pro:
|
||||||
display_name: 豆包 Seed 2.1 Pro
|
display_name: 豆包 Seed 2.1 Pro
|
||||||
|
|
@ -56,6 +67,17 @@ variants:
|
||||||
input_cny_per_mtoken: 6.0
|
input_cny_per_mtoken: 6.0
|
||||||
output_cny_per_mtoken: 30.0
|
output_cny_per_mtoken: 30.0
|
||||||
cache_hit_cny_per_mtoken: 1.2
|
cache_hit_cny_per_mtoken: 1.2
|
||||||
|
pricing:
|
||||||
|
periods:
|
||||||
|
- revision: doubao-seed21-20260628
|
||||||
|
currency: CNY
|
||||||
|
source_url: https://www.volcengine.com/product/ark
|
||||||
|
checked_at: 2026-08-30
|
||||||
|
effective_from: 2026-06-28T00:00:00Z
|
||||||
|
default:
|
||||||
|
input_per_mtoken: 6.0
|
||||||
|
cache_hit_per_mtoken: 1.2
|
||||||
|
output_per_mtoken: 30.0
|
||||||
|
|
||||||
evolving:
|
evolving:
|
||||||
# 自进化版:统一 model_id `doubao-seed-evolving`,每周至少迭代一次,始终指向最新版。
|
# 自进化版:统一 model_id `doubao-seed-evolving`,每周至少迭代一次,始终指向最新版。
|
||||||
|
|
@ -83,3 +105,14 @@ variants:
|
||||||
input_cny_per_mtoken: 6.0
|
input_cny_per_mtoken: 6.0
|
||||||
output_cny_per_mtoken: 30.0
|
output_cny_per_mtoken: 30.0
|
||||||
cache_hit_cny_per_mtoken: 1.2
|
cache_hit_cny_per_mtoken: 1.2
|
||||||
|
pricing:
|
||||||
|
periods:
|
||||||
|
- revision: doubao-seed-evolving-202607
|
||||||
|
currency: CNY
|
||||||
|
source_url: https://www.volcengine.com/product/ark
|
||||||
|
checked_at: 2026-08-30
|
||||||
|
effective_from: 2026-07-01T00:00:00Z
|
||||||
|
default:
|
||||||
|
input_per_mtoken: 6.0
|
||||||
|
cache_hit_per_mtoken: 1.2
|
||||||
|
output_per_mtoken: 30.0
|
||||||
|
|
|
||||||
|
|
@ -1,55 +1,55 @@
|
||||||
# 智谱 GLM 模型档案
|
# 智谱 GLM 模型档案
|
||||||
# 走 litellm 原生 zai provider(1.83+ 内置)。litellm 默认 api_base 是国际站 api.z.ai;
|
# 走 LiteLLM 原生 zai provider。国内站 bigmodel.cn 与国际站 API key 不通用。
|
||||||
# 国内站 bigmodel.cn 通过 yaml 的 api_base 字段覆盖。两站 API key 不通用,env 也分开命名。
|
# GLM-5.3-Flash 强制 thinking;zcbot 保留同模型完整 reasoning 以维持长工具链连续性。
|
||||||
# thinking 走统一 extra_body 协议,由本档 `thinking_enabled` 决定开关。当前均 false=禁用 ——
|
|
||||||
# 因网关侧默认开 thinking,重任务上会把输出预算烧在 reasoning_content 上撞满 65536 上限
|
|
||||||
# 被截断→空响应(task 35744bea 案);禁用后线上探针实测 reasoning_content 归零、正文照常。
|
|
||||||
family: glm
|
family: glm
|
||||||
|
|
||||||
variants:
|
variants:
|
||||||
pro:
|
flash53:
|
||||||
display_name: GLM 5.1
|
display_name: GLM-5.3 Flash
|
||||||
model_id: zai/glm-5.1
|
model_id: zai/glm-5.3-flash
|
||||||
api_base: https://open.bigmodel.cn/api/paas/v4
|
api_base: https://open.bigmodel.cn/api/paas/v4
|
||||||
api_key_env: ZHIPUAI_API_KEY
|
api_key_env: ZHIPUAI_API_KEY
|
||||||
max_context: 200000
|
max_context: 1000000
|
||||||
reliable_context: 100000
|
|
||||||
max_output: 8192
|
|
||||||
parallel_tools: false
|
|
||||||
tool_calling_quality: good
|
|
||||||
thinking_enabled: false
|
|
||||||
thinking_transport: extra_body
|
|
||||||
reasoning_effort_levels: []
|
|
||||||
default_reasoning_effort: ""
|
|
||||||
code_quality: good
|
|
||||||
enable_run_python: true
|
|
||||||
max_iterations: 50
|
|
||||||
optimal_temperature: 0.3
|
|
||||||
prompt_caching: false
|
|
||||||
extended_thinking: false
|
|
||||||
|
|
||||||
# GLM 5.2:与 5.1 并存(新增 variant,不动 glm.pro,线上 task 仍引 5.1 不受影响)。
|
|
||||||
# 旗舰基座,真正可用的 1M 上下文,适合大仓库/长链路工程任务。thinking 同 pro 留 false(协议同 5.1)。
|
|
||||||
pro52:
|
|
||||||
display_name: GLM 5.2
|
|
||||||
model_id: zai/glm-5.2
|
|
||||||
api_base: https://open.bigmodel.cn/api/paas/v4
|
|
||||||
api_key_env: ZHIPUAI_API_KEY
|
|
||||||
max_context: 1000000 # 真 1M
|
|
||||||
reliable_context: 262144
|
reliable_context: 262144
|
||||||
max_output: 8192
|
max_output: 131072
|
||||||
|
input_modalities: [text, image]
|
||||||
parallel_tools: false
|
parallel_tools: false
|
||||||
tool_calling_quality: good
|
tool_calling_quality: excellent
|
||||||
thinking_enabled: false
|
thinking_enabled: true
|
||||||
thinking_transport: extra_body
|
thinking_transport: extra_body
|
||||||
reasoning_effort_levels: []
|
thinking_clear: false
|
||||||
default_reasoning_effort: ""
|
reasoning_effort_levels: [low, high, max]
|
||||||
|
default_reasoning_effort: high
|
||||||
|
reasoning_replay: conversation
|
||||||
code_quality: excellent
|
code_quality: excellent
|
||||||
enable_run_python: true
|
enable_run_python: true
|
||||||
max_iterations: 50
|
max_iterations: 120
|
||||||
optimal_temperature: 0.3
|
optimal_temperature: 1.0
|
||||||
prompt_caching: false
|
prompt_caching: false
|
||||||
extended_thinking: false
|
extended_thinking: false
|
||||||
input_cny_per_mtoken: 8.0
|
# 当前 endpoint 是国内 BigModel,使用国内按量标准价;限时活动/套餐抵扣不进入
|
||||||
output_cny_per_mtoken: 28.0
|
# 长期公开价估算,实际财务扣费仍以账号费用中心账单为准。
|
||||||
cache_hit_cny_per_mtoken: 2.0
|
input_cny_per_mtoken: 0.8
|
||||||
|
output_cny_per_mtoken: 2.8
|
||||||
|
cache_hit_cny_per_mtoken: 0.23
|
||||||
|
pricing:
|
||||||
|
periods:
|
||||||
|
- revision: bigmodel-glm53-flash-20260826
|
||||||
|
currency: CNY
|
||||||
|
source_url: https://bigmodel.cn/pricing
|
||||||
|
checked_at: 2026-08-30
|
||||||
|
effective_from: 2026-08-26T00:00:00Z
|
||||||
|
default:
|
||||||
|
input_per_mtoken: 0.8
|
||||||
|
cache_hit_per_mtoken: 0.23
|
||||||
|
output_per_mtoken: 2.8
|
||||||
|
|
||||||
|
# 隐藏兼容别名:真实调用、能力和记账身份全部归 glm.flash53。
|
||||||
|
# 待一个兼容版本确认无旧 profile 活跃后删除。
|
||||||
|
pro:
|
||||||
|
hidden: true
|
||||||
|
alias_of: flash53
|
||||||
|
|
||||||
|
pro52:
|
||||||
|
hidden: true
|
||||||
|
alias_of: flash53
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
# zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。
|
# zcbot 版本号单一事实源:web/app.py 的 FastAPI version、/healthz 返回、前端展示都引这里。
|
||||||
# 改版本只动这一行。
|
# 改版本只动这一行。
|
||||||
__version__ = "0.69.0"
|
__version__ = "0.71.0"
|
||||||
|
|
|
||||||
|
|
@ -56,10 +56,12 @@ _MEDIA_LOOK_SEG = """\
|
||||||
- **何时调**:用户消息里出现 `[用户上传的参考图] <路径>` 且需要据图内容回答(问"这图里写了啥 / 是什么 / 表格数据多少");或要基于 task 内某张图(`figures/xxx.png`)的**实际内容**做事(不是改图,改图走 seedream)。传 `image=<路径>`,并且**必须在 `question` 里一次写清完成当前任务所需的具体信息**,别分多次看。
|
- **何时调**:用户消息里出现 `[用户上传的参考图] <路径>` 且需要据图内容回答(问"这图里写了啥 / 是什么 / 表格数据多少");或要基于 task 内某张图(`figures/xxx.png`)的**实际内容**做事(不是改图,改图走 seedream)。传 `image=<路径>`,并且**必须在 `question` 里一次写清完成当前任务所需的具体信息**,别分多次看。
|
||||||
- **问题范围**:只问完成用户任务所需内容。例如问仪表读数就只读数,问某列数据就只提取该列;仅当用户明确要求"全文 OCR / 转文字 / 完整识别整张图"时,才要求逐字识别全部文字。不要无条件同时要求画面描述、全文 OCR 和图表解析。
|
- **问题范围**:只问完成用户任务所需内容。例如问仪表读数就只读数,问某列数据就只提取该列;仅当用户明确要求"全文 OCR / 转文字 / 完整识别整张图"时,才要求逐字识别全部文字。不要无条件同时要求画面描述、全文 OCR 和图表解析。
|
||||||
- **何时不调**:用户只是要改图(走 seedream i2i)/ 只要文件名不关心内容 / 图是你自己刚生成的且 prompt 已知(无需再读)/ 图的内容对当前任务可有可无。**绝不对同一张图反复看**(每看一次都是几十秒等待)。"""
|
- **何时不调**:用户只是要改图(走 seedream i2i)/ 只要文件名不关心内容 / 图是你自己刚生成的且 prompt 已知(无需再读)/ 图的内容对当前任务可有可无。**绝不对同一张图反复看**(每看一次都是几十秒等待)。"""
|
||||||
|
_MEDIA_NATIVE_LOOK_SEG = """\
|
||||||
|
- **原生看图**:你可以直接理解用户随消息上传的图片;图片已作为多模态内容随请求送入,直接据图分析,**不要调用辅助看图工具**。消息中的 `[用户上传的参考图]` 路径仅用于定位同一附件或交给改图工具。"""
|
||||||
_MEDIA_READDOC_SEG = """\
|
_MEDIA_READDOC_SEG = """\
|
||||||
- `read_document` —— 读 PDF(豆包 Seed 2.0 Lite 文档理解),**专治扫描件**:markitdown 对某 PDF 转出**空 / 近空**(纯图页无文本层)→ 用它逐页 OCR 成 markdown。每页约 1-2 厘钱,单次上限 100 页(更长先拆分卷)。
|
- `read_document` —— 读 PDF(豆包 Seed 2.0 Lite 文档理解),**专治扫描件**:markitdown 对某 PDF 转出**空 / 近空**(纯图页无文本层)→ 用它逐页 OCR 成 markdown。每页约 1-2 厘钱,单次上限 100 页(更长先拆分卷)。
|
||||||
- **多页整本 OCR 必传 `save_md`**(如 `save_md='source/xxx.md'`):全文落文件、只返回预览,后续直接 read 那个 md;只问局部内容("第3章指标是什么")传 `question` 即可不落盘。
|
- **多页整本 OCR 必传 `save_md`**(如 `save_md='source/xxx.md'`):全文落文件、只返回预览,后续直接 read 那个 md;只问局部内容("第3章指标是什么")传 `question` 即可不落盘。
|
||||||
- **何时不调**:PDF 有文本层且 markitdown 已转出正文(直接用那份 md,别重复花钱);DOCX/PPTX/XLSX(走 markitdown);单张图片(走 look_at_image)。"""
|
- **何时不调**:PDF 有文本层且 markitdown 已转出正文(直接用那份 md,别重复花钱);DOCX/PPTX/XLSX(走 markitdown);单张图片由主模型原生读取或走本轮实际提供的看图能力。"""
|
||||||
|
|
||||||
# PDF 摄取是跨 skill 的基础入口,始终注入,不能依赖 agent 是否正确加载 paper / proposal。
|
# PDF 摄取是跨 skill 的基础入口,始终注入,不能依赖 agent 是否正确加载 paper / proposal。
|
||||||
# read_document 只在配了 ARK 时挂载;没挂时扫描件明确停在“需 OCR”,不诱导现场装包或
|
# read_document 只在配了 ARK 时挂载;没挂时扫描件明确停在“需 OCR”,不诱导现场装包或
|
||||||
|
|
@ -87,15 +89,20 @@ _MEDIA_SEEDANCE_SEG = """\
|
||||||
- 兜底硬约束:用户没主动要视频就别装饰性生成(比生图更严重的红线);同一目的不满意**绝不连发**(1 次错 = ¥4+60s,连发 2 次 = ¥8+2min);固定开场图传 `image`,多张主体/产品/场景参考图传 `reference_images`;当前不支持 video-to-video。"""
|
- 兜底硬约束:用户没主动要视频就别装饰性生成(比生图更严重的红线);同一目的不满意**绝不连发**(1 次错 = ¥4+60s,连发 2 次 = ¥8+2min);固定开场图传 `image`,多张主体/产品/场景参考图传 `reference_images`;当前不支持 video-to-video。"""
|
||||||
|
|
||||||
|
|
||||||
def _media_tools_block(has_ark: bool, image_tool: str) -> str:
|
def _media_tools_block(
|
||||||
|
has_ark: bool, image_tool: str, native_image_input: bool = False
|
||||||
|
) -> str:
|
||||||
"""按本 run 实际挂载的媒体工具拼指引段;一个都没挂 → 返 ""(prompt 不出现媒体段)。
|
"""按本 run 实际挂载的媒体工具拼指引段;一个都没挂 → 返 ""(prompt 不出现媒体段)。
|
||||||
|
|
||||||
image_tool: "seedream" / "gpt_image" / ""(无图像工具)。look_at_image 与 seedance
|
image_tool: "seedream" / "gpt_image" / ""(无图像工具)。look_at_image 与 seedance
|
||||||
只跟豆包(ARK_API_KEY)走;图像工具二选一,prompt 只描述实际存在的那个。
|
只跟豆包(ARK_API_KEY)走;图像工具二选一,prompt 只描述实际存在的那个。
|
||||||
"""
|
"""
|
||||||
segs: list[str] = []
|
segs: list[str] = []
|
||||||
|
if native_image_input:
|
||||||
|
segs.append(_MEDIA_NATIVE_LOOK_SEG)
|
||||||
if has_ark:
|
if has_ark:
|
||||||
segs.append(_MEDIA_LOOK_SEG)
|
if not native_image_input:
|
||||||
|
segs.append(_MEDIA_LOOK_SEG)
|
||||||
segs.append(_MEDIA_READDOC_SEG)
|
segs.append(_MEDIA_READDOC_SEG)
|
||||||
if image_tool == "seedream":
|
if image_tool == "seedream":
|
||||||
segs.append(_MEDIA_SEEDREAM_SEG)
|
segs.append(_MEDIA_SEEDREAM_SEG)
|
||||||
|
|
@ -107,7 +114,7 @@ def _media_tools_block(has_ark: bool, image_tool: str) -> str:
|
||||||
segs.append(_MEDIA_SEEDANCE_SEG)
|
segs.append(_MEDIA_SEEDANCE_SEG)
|
||||||
if not segs:
|
if not segs:
|
||||||
return ""
|
return ""
|
||||||
return "## 媒体工具(图像生成 / 视频生成 / 看图)\n" + "\n".join(segs)
|
return "## 媒体能力(图像生成 / 视频生成 / 看图)\n" + "\n".join(segs)
|
||||||
|
|
||||||
|
|
||||||
def _choose_image_variant(
|
def _choose_image_variant(
|
||||||
|
|
@ -154,6 +161,14 @@ _HOST_ENV_BLOCK = """\
|
||||||
## 运行环境(本地 host)
|
## 运行环境(本地 host)
|
||||||
`shell` 走的是 **Windows cmd.exe**(非 bash):避免 unix-only flag,`mkdir -p` 不识别 → 用 `run_python` 的 `os.makedirs(..., exist_ok=True)` 建目录;复杂管道/重定向用 run_python 更稳。"""
|
`shell` 走的是 **Windows cmd.exe**(非 bash):避免 unix-only flag,`mkdir -p` 不识别 → 用 `run_python` 的 `os.makedirs(..., exist_ok=True)` 建目录;复杂管道/重定向用 run_python 更稳。"""
|
||||||
|
|
||||||
|
_MERMAID_STYLE_BLOCK = """\
|
||||||
|
|
||||||
|
## Mermaid 视觉规范
|
||||||
|
生成 Mermaid 图时默认采用**克制的语义配色**,不要输出全灰单色图,也不要随机彩虹色:
|
||||||
|
- 流程图 / 架构图用 `classDef` 按角色分组,通常 3–5 色足够:输入/数据用蓝色,处理/模块用青绿色,判断/风险用橙色,结果/成功用绿色,异常用红色;同类节点保持同色。
|
||||||
|
- 使用浅色填充、较深描边与高对比深色文字,线条颜色统一;分组框比节点更淡,避免大面积高饱和背景。
|
||||||
|
- 颜色服务于信息层级;若用户指定品牌色、黑白打印或已有 `classDef` / `style` 规则,优先遵循用户设置,不要覆盖。"""
|
||||||
|
|
||||||
|
|
||||||
def load_config() -> dict:
|
def load_config() -> dict:
|
||||||
return yaml.safe_load((ROOT / "config" / "agent.yaml").read_text(encoding="utf-8")) or {}
|
return yaml.safe_load((ROOT / "config" / "agent.yaml").read_text(encoding="utf-8")) or {}
|
||||||
|
|
@ -356,6 +371,7 @@ def _build_system_prompt(
|
||||||
is_docker = os.getenv("ZCBOT_SANDBOX_BACKEND", "host").lower() == "docker"
|
is_docker = os.getenv("ZCBOT_SANDBOX_BACKEND", "host").lower() == "docker"
|
||||||
# 运行环境段紧跟模板(平台/网络是基础事实,放前面);general_v1 的「平台」段指向这里。
|
# 运行环境段紧跟模板(平台/网络是基础事实,放前面);general_v1 的「平台」段指向这里。
|
||||||
prompt += _CONTAINER_ENV_BLOCK if is_docker else _HOST_ENV_BLOCK
|
prompt += _CONTAINER_ENV_BLOCK if is_docker else _HOST_ENV_BLOCK
|
||||||
|
prompt += _MERMAID_STYLE_BLOCK
|
||||||
if skills.skills:
|
if skills.skills:
|
||||||
prompt += f"\n\n## 可用 skill (用 load_skill 加载完整指引)\n{skills.discovery_block()}"
|
prompt += f"\n\n## 可用 skill (用 load_skill 加载完整指引)\n{skills.discovery_block()}"
|
||||||
# .memory/ 在 agent 视角下的可写路径:docker 给容器路径,host 给宿主绝对路径。
|
# .memory/ 在 agent 视角下的可写路径:docker 给容器路径,host 给宿主绝对路径。
|
||||||
|
|
@ -525,6 +541,7 @@ def build_agent(
|
||||||
model = cfg["default_model"]
|
model = cfg["default_model"]
|
||||||
|
|
||||||
caps = ModelCapabilities.load(model, ROOT / cfg["models_dir"])
|
caps = ModelCapabilities.load(model, ROOT / cfg["models_dir"])
|
||||||
|
model = caps.profile
|
||||||
llm = LLM(caps)
|
llm = LLM(caps)
|
||||||
|
|
||||||
workspace_dir = resolve_workspace(workspace, cfg)
|
workspace_dir = resolve_workspace(workspace, cfg)
|
||||||
|
|
@ -603,7 +620,9 @@ def build_agent(
|
||||||
system_prompt = _build_system_prompt(
|
system_prompt = _build_system_prompt(
|
||||||
cfg, skills, workspace_dir, tool_base, working_dir_path, uid,
|
cfg, skills, workspace_dir, tool_base, working_dir_path, uid,
|
||||||
task_id, task_state.name, task_state.skill,
|
task_id, task_state.name, task_state.skill,
|
||||||
media_block=_media_tools_block(ark_cfg is not None, image_tool_name),
|
media_block=_media_tools_block(
|
||||||
|
ark_cfg is not None, image_tool_name, caps.native_image_input
|
||||||
|
),
|
||||||
allow_working_dir_rename=not scheduled_run,
|
allow_working_dir_rename=not scheduled_run,
|
||||||
office_to_pdf_available=office_pdf_available,
|
office_to_pdf_available=office_pdf_available,
|
||||||
)
|
)
|
||||||
|
|
@ -671,6 +690,7 @@ def build_agent(
|
||||||
if current_profile.split(".", 1)[0] == "unifyllm":
|
if current_profile.split(".", 1)[0] == "unifyllm":
|
||||||
return None
|
return None
|
||||||
new_caps = ModelCapabilities.load(profile, ROOT / cfg["models_dir"])
|
new_caps = ModelCapabilities.load(profile, ROOT / cfg["models_dir"])
|
||||||
|
profile = new_caps.profile
|
||||||
new_llm = LLM(new_caps) # 缺 api key 会 raise → loop 捕获转 warn,不切换
|
new_llm = LLM(new_caps) # 缺 api key 会 raise → loop 捕获转 warn,不切换
|
||||||
from sqlalchemy import update as _update
|
from sqlalchemy import update as _update
|
||||||
from core.storage import session_scope as _scope
|
from core.storage import session_scope as _scope
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,6 @@ litellm 不覆盖豆包的图像/视频生成端点,这里自己用 httpx 直调
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Optional
|
from typing import Any, Optional
|
||||||
|
|
@ -35,6 +34,11 @@ class ArkConfig:
|
||||||
api_key: str
|
api_key: str
|
||||||
base_url: str
|
base_url: str
|
||||||
raw: dict # 完整 yaml 内容(便于 caller 按 image/video 子键再取)
|
raw: dict # 完整 yaml 内容(便于 caller 按 image/video 子键再取)
|
||||||
|
api_key_env: str = ""
|
||||||
|
|
||||||
|
def request_api_key(self) -> str:
|
||||||
|
from core.provider_credentials.runtime import resolve_env_secret
|
||||||
|
return resolve_env_secret(self.api_key_env) if self.api_key_env else self.api_key
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def load(cls, path: Optional[Path] = None) -> Optional["ArkConfig"]:
|
def load(cls, path: Optional[Path] = None) -> Optional["ArkConfig"]:
|
||||||
|
|
@ -50,7 +54,8 @@ class ArkConfig:
|
||||||
return None
|
return None
|
||||||
data = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
data = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
||||||
env = data.get("api_key_env") or data.get("ark_api_key_env") or "ARK_API_KEY"
|
env = data.get("api_key_env") or data.get("ark_api_key_env") or "ARK_API_KEY"
|
||||||
key = os.environ.get(env, "").strip()
|
from core.provider_credentials.runtime import resolve_env_secret
|
||||||
|
key = resolve_env_secret(env)
|
||||||
if not key:
|
if not key:
|
||||||
return None
|
return None
|
||||||
base = (
|
base = (
|
||||||
|
|
@ -58,7 +63,7 @@ class ArkConfig:
|
||||||
or data.get("ark_base_url")
|
or data.get("ark_base_url")
|
||||||
or "https://ark.cn-beijing.volces.com/api/v3"
|
or "https://ark.cn-beijing.volces.com/api/v3"
|
||||||
)
|
)
|
||||||
return cls(api_key=key, base_url=str(base).rstrip("/"), raw=data)
|
return cls(api_key=key, base_url=str(base).rstrip("/"), raw=data, api_key_env=env)
|
||||||
|
|
||||||
|
|
||||||
class ArkClient:
|
class ArkClient:
|
||||||
|
|
@ -70,10 +75,11 @@ class ArkClient:
|
||||||
def __init__(self, cfg: ArkConfig, timeout_s: float = 60.0) -> None:
|
def __init__(self, cfg: ArkConfig, timeout_s: float = 60.0) -> None:
|
||||||
self.cfg = cfg
|
self.cfg = cfg
|
||||||
self.timeout_s = timeout_s
|
self.timeout_s = timeout_s
|
||||||
|
self._api_key = cfg.request_api_key()
|
||||||
self._client = httpx.Client(
|
self._client = httpx.Client(
|
||||||
base_url=cfg.base_url,
|
base_url=cfg.base_url,
|
||||||
headers={
|
headers={
|
||||||
"Authorization": f"Bearer {cfg.api_key}",
|
"Authorization": f"Bearer {self._api_key}",
|
||||||
},
|
},
|
||||||
timeout=timeout_s,
|
timeout=timeout_s,
|
||||||
)
|
)
|
||||||
|
|
@ -118,17 +124,32 @@ class ArkClient:
|
||||||
raise ArkTimeoutError(f"network error calling GET {path}: {e}") from e
|
raise ArkTimeoutError(f"network error calling GET {path}: {e}") from e
|
||||||
return self._parse(resp, f"GET {path}")
|
return self._parse(resp, f"GET {path}")
|
||||||
|
|
||||||
@staticmethod
|
def _parse(self, resp: httpx.Response, label: str) -> dict:
|
||||||
def _parse(resp: httpx.Response, label: str) -> dict:
|
|
||||||
if resp.status_code >= 400:
|
if resp.status_code >= 400:
|
||||||
|
try:
|
||||||
|
from core.provider_credentials.registry import BY_ENV
|
||||||
|
from core.provider_credentials.service import record_business_failure
|
||||||
|
binding = BY_ENV.get(self.cfg.api_key_env)
|
||||||
|
if binding:
|
||||||
|
record_business_failure(
|
||||||
|
binding[0], status_code=resp.status_code, detail=resp.text[:300]
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
# ark 错误 body 一般是 {"error": {"code": ..., "message": ...}};能解就解
|
# ark 错误 body 一般是 {"error": {"code": ..., "message": ...}};能解就解
|
||||||
try:
|
try:
|
||||||
err = resp.json().get("error") or {}
|
err = resp.json().get("error") or {}
|
||||||
msg = err.get("message") or resp.text[:300]
|
msg = err.get("message") or resp.text[:300]
|
||||||
|
key = self._api_key
|
||||||
|
msg = str(msg).replace(key, "***") if key else str(msg)
|
||||||
code = err.get("code") or resp.status_code
|
code = err.get("code") or resp.status_code
|
||||||
raise ArkError(f"{label} → HTTP {resp.status_code} ({code}): {msg}")
|
raise ArkError(f"{label} → HTTP {resp.status_code} ({code}): {msg}")
|
||||||
except ValueError:
|
except ValueError:
|
||||||
raise ArkError(f"{label} → HTTP {resp.status_code}: {resp.text[:300]}")
|
detail = resp.text[:300]
|
||||||
|
key = self._api_key
|
||||||
|
if key:
|
||||||
|
detail = detail.replace(key, "***")
|
||||||
|
raise ArkError(f"{label} → HTTP {resp.status_code}: {detail}")
|
||||||
try:
|
try:
|
||||||
return resp.json()
|
return resp.json()
|
||||||
except ValueError as e:
|
except ValueError as e:
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,7 @@ from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from uuid import UUID, uuid4
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
from sqlalchemy import or_, select
|
from sqlalchemy import or_, select, text
|
||||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||||
|
|
||||||
from .artifacts import ARTIFACT_TRASH_DIR, ArtifactRef
|
from .artifacts import ARTIFACT_TRASH_DIR, ArtifactRef
|
||||||
|
|
@ -85,7 +85,10 @@ def register_published_artifacts(
|
||||||
content_sha256=content_sha256,
|
content_sha256=content_sha256,
|
||||||
).on_conflict_do_update(
|
).on_conflict_do_update(
|
||||||
index_elements=[Artifact.user_id, Artifact.current_path],
|
index_elements=[Artifact.user_id, Artifact.current_path],
|
||||||
index_where=Artifact.status == "active",
|
# 必须与 0028 的部分唯一索引谓词按 SQL 字面量一致。若用
|
||||||
|
# ``Artifact.status == "active"``,SQLAlchemy 会编译成绑定参数,
|
||||||
|
# PostgreSQL 无法据此推断 ON CONFLICT 对应哪个部分索引。
|
||||||
|
index_where=text("status = 'active'"),
|
||||||
set_=update_values,
|
set_=update_values,
|
||||||
).returning(Artifact.artifact_id)
|
).returning(Artifact.artifact_id)
|
||||||
artifact_id = session.execute(statement).scalar_one()
|
artifact_id = session.execute(statement).scalar_one()
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,6 @@ import base64
|
||||||
import hashlib
|
import hashlib
|
||||||
import hmac
|
import hmac
|
||||||
import json
|
import json
|
||||||
import os
|
|
||||||
import time
|
import time
|
||||||
from typing import Callable, Optional
|
from typing import Callable, Optional
|
||||||
|
|
||||||
|
|
@ -57,15 +56,15 @@ class LfasrCancelled(LfasrError):
|
||||||
|
|
||||||
def is_configured() -> bool:
|
def is_configured() -> bool:
|
||||||
"""LFASR 凭据是否齐 —— agent_builder 据此决定挂不挂 transcribe_audio tool。"""
|
"""LFASR 凭据是否齐 —— agent_builder 据此决定挂不挂 transcribe_audio tool。"""
|
||||||
return all(
|
from core.provider_credentials.runtime import provider_available
|
||||||
(os.getenv(k) or "").strip()
|
return provider_available("xfyun_lfasr")
|
||||||
for k in ("XFYUN_APPID", "XFYUN_LFASR_SECRET_KEY")
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _load_credentials() -> tuple[str, str]:
|
def _load_credentials() -> tuple[str, str]:
|
||||||
appid = (os.getenv("XFYUN_APPID") or "").strip()
|
from core.provider_credentials.runtime import resolve_credentials
|
||||||
secret = (os.getenv("XFYUN_LFASR_SECRET_KEY") or "").strip()
|
values = resolve_credentials("xfyun_lfasr").values
|
||||||
|
appid = values.get("appid", "")
|
||||||
|
secret = values.get("secret_key", "")
|
||||||
if not (appid and secret):
|
if not (appid and secret):
|
||||||
raise LfasrNotConfigured(
|
raise LfasrNotConfigured(
|
||||||
"录音转写未配置:需在 .env 设 XFYUN_APPID / XFYUN_LFASR_SECRET_KEY"
|
"录音转写未配置:需在 .env 设 XFYUN_APPID / XFYUN_LFASR_SECRET_KEY"
|
||||||
|
|
@ -99,6 +98,15 @@ def _post(path: str, params: dict, content: Optional[bytes] = None,
|
||||||
headers={"Content-Type": "application/octet-stream"} if content else None,
|
headers={"Content-Type": "application/octet-stream"} if content else None,
|
||||||
timeout=timeout_s,
|
timeout=timeout_s,
|
||||||
)
|
)
|
||||||
|
if resp.status_code >= 400:
|
||||||
|
try:
|
||||||
|
from core.provider_credentials.service import record_business_failure
|
||||||
|
record_business_failure(
|
||||||
|
"xfyun_lfasr", status_code=resp.status_code,
|
||||||
|
detail=resp.text[:200],
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
resp.raise_for_status()
|
resp.raise_for_status()
|
||||||
body = resp.json()
|
body = resp.json()
|
||||||
except LfasrError:
|
except LfasrError:
|
||||||
|
|
@ -108,6 +116,14 @@ def _post(path: str, params: dict, content: Optional[bytes] = None,
|
||||||
code = str(body.get("code") or "")
|
code = str(body.get("code") or "")
|
||||||
if code != "000000":
|
if code != "000000":
|
||||||
desc = body.get("descInfo") or "未知错误"
|
desc = body.get("descInfo") or "未知错误"
|
||||||
|
if code in {"10105", "10106", "10107", "11200"}:
|
||||||
|
try:
|
||||||
|
from core.provider_credentials.service import record_business_failure
|
||||||
|
record_business_failure(
|
||||||
|
"xfyun_lfasr", status_code=401, detail="讯飞认证失败"
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
raise LfasrError(f"讯飞录音转写失败({code}):{desc}", code=code)
|
raise LfasrError(f"讯飞录音转写失败({code}):{desc}", code=code)
|
||||||
return body.get("content") or {}
|
return body.get("content") or {}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,6 @@ import base64
|
||||||
import hashlib
|
import hashlib
|
||||||
import hmac
|
import hmac
|
||||||
import json
|
import json
|
||||||
import os
|
|
||||||
import time
|
import time
|
||||||
from contextlib import suppress
|
from contextlib import suppress
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
@ -84,9 +83,11 @@ def build_auth_url(api_key: str, api_secret: str) -> str:
|
||||||
|
|
||||||
|
|
||||||
def _load_credentials() -> tuple[str, str, str]:
|
def _load_credentials() -> tuple[str, str, str]:
|
||||||
appid = (os.getenv("XFYUN_APPID") or "").strip()
|
from core.provider_credentials.runtime import resolve_credentials
|
||||||
api_key = (os.getenv("XFYUN_API_KEY") or "").strip()
|
values = resolve_credentials("xfyun_iat").values
|
||||||
api_secret = (os.getenv("XFYUN_API_SECRET") or "").strip()
|
appid = values.get("appid", "")
|
||||||
|
api_key = values.get("api_key", "")
|
||||||
|
api_secret = values.get("api_secret", "")
|
||||||
if not (appid and api_key and api_secret):
|
if not (appid and api_key and api_secret):
|
||||||
raise XfyunASRNotConfigured(
|
raise XfyunASRNotConfigured(
|
||||||
"语音识别未配置:需在 .env 设 XFYUN_APPID / XFYUN_API_KEY / XFYUN_API_SECRET"
|
"语音识别未配置:需在 .env 设 XFYUN_APPID / XFYUN_API_KEY / XFYUN_API_SECRET"
|
||||||
|
|
@ -94,6 +95,16 @@ def _load_credentials() -> tuple[str, str, str]:
|
||||||
return appid, api_key, api_secret
|
return appid, api_key, api_secret
|
||||||
|
|
||||||
|
|
||||||
|
def _report_auth_error(code: object) -> None:
|
||||||
|
if str(code) not in {"10105", "10106", "10107", "11200"}:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
from core.provider_credentials.service import record_business_failure
|
||||||
|
record_business_failure("xfyun_iat", status_code=401, detail="讯飞认证失败")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
class XfyunStream:
|
class XfyunStream:
|
||||||
"""流式会话:实时喂 PCM 分片,partial 全文经 on_text 异步回调推出(wpgs 已合并)。
|
"""流式会话:实时喂 PCM 分片,partial 全文经 on_text 异步回调推出(wpgs 已合并)。
|
||||||
|
|
||||||
|
|
@ -151,6 +162,7 @@ class XfyunStream:
|
||||||
msg = json.loads(await self._ws.recv())
|
msg = json.loads(await self._ws.recv())
|
||||||
code = msg.get("code")
|
code = msg.get("code")
|
||||||
if code:
|
if code:
|
||||||
|
_report_auth_error(code)
|
||||||
hint = _ERR_HINTS.get(code, msg.get("message") or "未知错误")
|
hint = _ERR_HINTS.get(code, msg.get("message") or "未知错误")
|
||||||
raise XfyunASRError(f"讯飞识别失败({code}):{hint}", code=code)
|
raise XfyunASRError(f"讯飞识别失败({code}):{hint}", code=code)
|
||||||
data = msg.get("data") or {}
|
data = msg.get("data") or {}
|
||||||
|
|
@ -272,6 +284,7 @@ async def transcribe(pcm: bytes, *, language: str = "zh_cn") -> str:
|
||||||
msg = json.loads(raw)
|
msg = json.loads(raw)
|
||||||
code = msg.get("code")
|
code = msg.get("code")
|
||||||
if code:
|
if code:
|
||||||
|
_report_auth_error(code)
|
||||||
hint = _ERR_HINTS.get(code, msg.get("message") or "未知错误")
|
hint = _ERR_HINTS.get(code, msg.get("message") or "未知错误")
|
||||||
raise XfyunASRError(f"讯飞识别失败({code}):{hint}", code=code)
|
raise XfyunASRError(f"讯飞识别失败({code}):{hint}", code=code)
|
||||||
data = msg.get("data") or {}
|
data = msg.get("data") or {}
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,9 @@
|
||||||
"""Structured user-message attachments and model-context compatibility rendering."""
|
"""Structured user-message attachments and model-context compatibility rendering."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
import mimetypes
|
import mimetypes
|
||||||
|
from copy import deepcopy
|
||||||
from collections.abc import Iterable
|
from collections.abc import Iterable
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
@ -10,6 +12,14 @@ from .artifacts import ArtifactPathError, resolve_artifact_path
|
||||||
|
|
||||||
ATTACHMENT_REF_VERSION = 1
|
ATTACHMENT_REF_VERSION = 1
|
||||||
MAX_ATTACHMENTS_PER_MESSAGE = 10
|
MAX_ATTACHMENTS_PER_MESSAGE = 10
|
||||||
|
MAX_NATIVE_IMAGE_BYTES = 10 * 1024 * 1024
|
||||||
|
_NATIVE_IMAGE_MIME = {
|
||||||
|
".gif": "image/gif",
|
||||||
|
".jpeg": "image/jpeg",
|
||||||
|
".jpg": "image/jpeg",
|
||||||
|
".png": "image/png",
|
||||||
|
".webp": "image/webp",
|
||||||
|
}
|
||||||
_IMAGE_EXTENSIONS = {
|
_IMAGE_EXTENSIONS = {
|
||||||
".avif", ".bmp", ".gif", ".heic", ".heif", ".jpeg", ".jpg",
|
".avif", ".bmp", ".gif", ".heic", ".heif", ".jpeg", ".jpg",
|
||||||
".png", ".svg", ".tif", ".tiff", ".webp",
|
".png", ".svg", ".tif", ".tiff", ".webp",
|
||||||
|
|
@ -74,3 +84,60 @@ def content_for_model(content: str, refs: Iterable[dict], working_dir_name: str)
|
||||||
return text
|
return text
|
||||||
suffix = "\n".join(lines)
|
suffix = "\n".join(lines)
|
||||||
return f"{text}\n\n{suffix}" if text else suffix
|
return f"{text}\n\n{suffix}" if text else suffix
|
||||||
|
|
||||||
|
|
||||||
|
def materialize_native_images(
|
||||||
|
messages: Iterable[dict],
|
||||||
|
*,
|
||||||
|
enabled: bool,
|
||||||
|
working_dir: Path,
|
||||||
|
user_root: Path,
|
||||||
|
) -> tuple[list[dict], int]:
|
||||||
|
"""把内存附件引用按需物化为 OpenAI-compatible image_url blocks。
|
||||||
|
|
||||||
|
Base64 只存在于本次 provider-bound 副本,不写数据库。历史文件已删除、越界、
|
||||||
|
格式不受支持或超过上限时保留原有文字路径提示并跳过图片块,不能让旧附件阻断续聊。
|
||||||
|
所有 ``_`` 内部字段在返回前剥离。
|
||||||
|
"""
|
||||||
|
output: list[dict] = []
|
||||||
|
image_count = 0
|
||||||
|
for message in messages:
|
||||||
|
new_msg = deepcopy(message)
|
||||||
|
refs = list(new_msg.pop("_attachment_refs", []) or [])
|
||||||
|
new_msg.pop("_model_profile", None)
|
||||||
|
if not enabled or new_msg.get("role") != "user" or not refs:
|
||||||
|
output.append(new_msg)
|
||||||
|
continue
|
||||||
|
|
||||||
|
blocks: list[dict] = []
|
||||||
|
text = new_msg.get("content")
|
||||||
|
if isinstance(text, str) and text:
|
||||||
|
blocks.append({"type": "text", "text": text})
|
||||||
|
message_image_count = 0
|
||||||
|
for ref in refs:
|
||||||
|
if not isinstance(ref, dict) or ref.get("kind") != "image":
|
||||||
|
continue
|
||||||
|
raw_path = str(ref.get("path") or "")
|
||||||
|
try:
|
||||||
|
path, _ = resolve_artifact_path(
|
||||||
|
raw_path,
|
||||||
|
working_dir=working_dir,
|
||||||
|
user_root=user_root,
|
||||||
|
require_file=True,
|
||||||
|
)
|
||||||
|
mime = _NATIVE_IMAGE_MIME.get(path.suffix.lower())
|
||||||
|
if mime is None or path.stat().st_size > MAX_NATIVE_IMAGE_BYTES:
|
||||||
|
continue
|
||||||
|
data = base64.b64encode(path.read_bytes()).decode("ascii")
|
||||||
|
except (ArtifactPathError, OSError):
|
||||||
|
continue
|
||||||
|
blocks.append({
|
||||||
|
"type": "image_url",
|
||||||
|
"image_url": {"url": f"data:{mime};base64,{data}"},
|
||||||
|
})
|
||||||
|
image_count += 1
|
||||||
|
message_image_count += 1
|
||||||
|
if message_image_count:
|
||||||
|
new_msg["content"] = blocks
|
||||||
|
output.append(new_msg)
|
||||||
|
return output, image_count
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,6 @@
|
||||||
"""博查 (Bocha AI) Web Search API 客户端,共享给 web_search tool。"""
|
"""博查 (Bocha AI) Web Search API 客户端,共享给 web_search tool。"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
|
|
@ -22,6 +21,11 @@ class BochaError(RuntimeError):
|
||||||
class BochaConfig:
|
class BochaConfig:
|
||||||
api_key: str
|
api_key: str
|
||||||
base_url: str
|
base_url: str
|
||||||
|
api_key_env: str = ""
|
||||||
|
|
||||||
|
def request_api_key(self) -> str:
|
||||||
|
from core.provider_credentials.runtime import resolve_env_secret
|
||||||
|
return resolve_env_secret(self.api_key_env) if self.api_key_env else self.api_key
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def load(cls, path: Optional[Path] = None) -> Optional["BochaConfig"]:
|
def load(cls, path: Optional[Path] = None) -> Optional["BochaConfig"]:
|
||||||
|
|
@ -35,12 +39,14 @@ class BochaConfig:
|
||||||
return None
|
return None
|
||||||
data = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
data = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
||||||
env = data.get("bocha_api_key_env") or "BOCHA_API_KEY"
|
env = data.get("bocha_api_key_env") or "BOCHA_API_KEY"
|
||||||
key = os.environ.get(env, "").strip()
|
from core.provider_credentials.runtime import resolve_env_secret
|
||||||
|
key = resolve_env_secret(env)
|
||||||
if not key:
|
if not key:
|
||||||
return None
|
return None
|
||||||
return cls(
|
return cls(
|
||||||
api_key=key,
|
api_key=key,
|
||||||
base_url=str(data.get("bocha_base_url") or "https://api.bochaai.com/v1").rstrip("/"),
|
base_url=str(data.get("bocha_base_url") or "https://api.bochaai.com/v1").rstrip("/"),
|
||||||
|
api_key_env=env,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -49,10 +55,11 @@ class BochaClient:
|
||||||
|
|
||||||
def __init__(self, cfg: BochaConfig, timeout_s: float = 15.0) -> None:
|
def __init__(self, cfg: BochaConfig, timeout_s: float = 15.0) -> None:
|
||||||
self.cfg = cfg
|
self.cfg = cfg
|
||||||
|
self._api_key = cfg.request_api_key()
|
||||||
self._client = httpx.Client(
|
self._client = httpx.Client(
|
||||||
base_url=cfg.base_url,
|
base_url=cfg.base_url,
|
||||||
headers={
|
headers={
|
||||||
"Authorization": f"Bearer {cfg.api_key}",
|
"Authorization": f"Bearer {self._api_key}",
|
||||||
"Content-Type": "application/json",
|
"Content-Type": "application/json",
|
||||||
},
|
},
|
||||||
timeout=timeout_s,
|
timeout=timeout_s,
|
||||||
|
|
@ -73,13 +80,21 @@ class BochaClient:
|
||||||
raise BochaError(f"博查网络错误: {e}") from e
|
raise BochaError(f"博查网络错误: {e}") from e
|
||||||
return self._parse(resp)
|
return self._parse(resp)
|
||||||
|
|
||||||
@staticmethod
|
def _parse(self, resp: httpx.Response) -> dict:
|
||||||
def _parse(resp: httpx.Response) -> dict:
|
|
||||||
if resp.status_code >= 400:
|
if resp.status_code >= 400:
|
||||||
|
try:
|
||||||
|
from core.provider_credentials.service import record_business_failure
|
||||||
|
record_business_failure(
|
||||||
|
"bocha", status_code=resp.status_code, detail=resp.text[:300]
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
try:
|
try:
|
||||||
msg = resp.json().get("message", resp.text[:300])
|
msg = resp.json().get("message", resp.text[:300])
|
||||||
except ValueError:
|
except ValueError:
|
||||||
msg = resp.text[:300]
|
msg = resp.text[:300]
|
||||||
|
key = self._api_key
|
||||||
|
msg = str(msg).replace(key, "***") if key else str(msg)
|
||||||
raise BochaError(f"博查 API → HTTP {resp.status_code}: {msg}")
|
raise BochaError(f"博查 API → HTTP {resp.status_code}: {msg}")
|
||||||
try:
|
try:
|
||||||
return resp.json()
|
return resp.json()
|
||||||
|
|
|
||||||
|
|
@ -3,13 +3,26 @@ from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass, field, fields
|
from dataclasses import dataclass, field, fields
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import List
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
from .llm_params import THINKING_TRANSPORTS
|
from .llm_params import THINKING_TRANSPORTS
|
||||||
|
|
||||||
REASONING_REPLAY_POLICIES = {"none", "tool_calls", "all"}
|
REASONING_REPLAY_POLICIES = {
|
||||||
|
"none",
|
||||||
|
"tool_turn",
|
||||||
|
"conversation",
|
||||||
|
"provider_managed",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def model_profile_of(caps: object) -> str:
|
||||||
|
"""返回能力对象的 canonical profile;兼容测试和旧调用方的鸭子类型。"""
|
||||||
|
profile = getattr(caps, "profile", "")
|
||||||
|
if profile:
|
||||||
|
return str(profile)
|
||||||
|
return f"{getattr(caps, 'family', '')}.{getattr(caps, 'variant', '')}".strip(".")
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|
@ -34,10 +47,16 @@ class ModelCapabilities:
|
||||||
thinking_transport: str = "none"
|
thinking_transport: str = "none"
|
||||||
reasoning_effort_levels: List[str] = field(default_factory=list)
|
reasoning_effort_levels: List[str] = field(default_factory=list)
|
||||||
default_reasoning_effort: str = ""
|
default_reasoning_effort: str = ""
|
||||||
|
# 是否清除历史 thinking。None=不发送该 provider 可选字段;False=保留历史推理。
|
||||||
|
thinking_clear: Optional[bool] = None
|
||||||
# 历史 reasoning 发回模型的策略。持久化原始响应与 provider-bound 输入分离:
|
# 历史 reasoning 发回模型的策略。持久化原始响应与 provider-bound 输入分离:
|
||||||
# none=全部剥离;tool_calls=仅工具调用 assistant 保留;all=完整原样回传。
|
# none=剥离;tool_turn=仅当前工具链;conversation=同模型会话;
|
||||||
|
# provider_managed=同模型不透明签名/block。
|
||||||
reasoning_replay: str = "none"
|
reasoning_replay: str = "none"
|
||||||
|
|
||||||
|
# 原生输入模态。附件仍以文件引用落库,发模型前才物化成 provider content blocks。
|
||||||
|
input_modalities: List[str] = field(default_factory=lambda: ["text"])
|
||||||
|
|
||||||
# 代码 / 沙盒
|
# 代码 / 沙盒
|
||||||
code_quality: str = "good"
|
code_quality: str = "good"
|
||||||
enable_run_python: bool = False
|
enable_run_python: bool = False
|
||||||
|
|
@ -56,6 +75,9 @@ class ModelCapabilities:
|
||||||
# 前缀缓存命中价(DeepSeek 等自动缓存 prompt 前缀,命中部分按此价,通常 ~0.1x input)。
|
# 前缀缓存命中价(DeepSeek 等自动缓存 prompt 前缀,命中部分按此价,通常 ~0.1x input)。
|
||||||
# 0 = 不区分,缓存命中按 input 全价记(安全兜底,不会少记)。
|
# 0 = 不区分,缓存命中按 input 全价记(安全兜底,不会少记)。
|
||||||
cache_hit_cny_per_mtoken: float = 0.0
|
cache_hit_cny_per_mtoken: float = 0.0
|
||||||
|
# 版本化价格配置。存在时由 core.pricing 作为本地事实源;上面三个扁平字段仅作
|
||||||
|
# 旧档案/旧调用方兼容与无版本价格兜底。
|
||||||
|
pricing: Dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
# API 接入
|
# API 接入
|
||||||
api_base: str = ""
|
api_base: str = ""
|
||||||
|
|
@ -81,6 +103,16 @@ class ModelCapabilities:
|
||||||
)
|
)
|
||||||
|
|
||||||
var = dict(variants[variant])
|
var = dict(variants[variant])
|
||||||
|
# 已下架 profile 可保留为一版隐藏别名,确保存量 task / scheduled job 能续跑;
|
||||||
|
# 真正能力和记账身份统一归目标 variant,不维持旧模型双轨。
|
||||||
|
seen_aliases = {variant}
|
||||||
|
while var.get("alias_of"):
|
||||||
|
target = str(var["alias_of"]).strip()
|
||||||
|
if not target or target in seen_aliases or target not in variants:
|
||||||
|
raise ValueError(f"档案 {path} 的 variant={variant} alias_of 无效")
|
||||||
|
seen_aliases.add(target)
|
||||||
|
variant = target
|
||||||
|
var = dict(variants[target])
|
||||||
valid_keys = {f.name for f in fields(cls)}
|
valid_keys = {f.name for f in fields(cls)}
|
||||||
kwargs = {k: v for k, v in var.items() if k in valid_keys}
|
kwargs = {k: v for k, v in var.items() if k in valid_keys}
|
||||||
kwargs["family"] = data.get("family", family)
|
kwargs["family"] = data.get("family", family)
|
||||||
|
|
@ -109,3 +141,11 @@ class ModelCapabilities:
|
||||||
f"{caps.default_reasoning_effort!r} 不在 reasoning_effort_levels 中"
|
f"{caps.default_reasoning_effort!r} 不在 reasoning_effort_levels 中"
|
||||||
)
|
)
|
||||||
return caps
|
return caps
|
||||||
|
|
||||||
|
@property
|
||||||
|
def profile(self) -> str:
|
||||||
|
return f"{self.family}.{self.variant}"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def native_image_input(self) -> bool:
|
||||||
|
return "image" in self.input_modalities
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,8 @@ from typing import Any, List
|
||||||
import json
|
import json
|
||||||
import re
|
import re
|
||||||
|
|
||||||
|
from .capabilities import REASONING_REPLAY_POLICIES
|
||||||
|
|
||||||
# chars↔tokens 粗折算(CJK+代码+json 混合保守按 ~2.5 char/token)与压缩触发比例。
|
# chars↔tokens 粗折算(CJK+代码+json 混合保守按 ~2.5 char/token)与压缩触发比例。
|
||||||
# loop(50% 压缩门槛)与 context_fold(85% 折叠门槛)共用,单一事实源;压缩/折叠是
|
# loop(50% 压缩门槛)与 context_fold(85% 折叠门槛)共用,单一事实源;压缩/折叠是
|
||||||
# 成本/安全优化、非正确性关键,估算粗糙无妨。
|
# 成本/安全优化、非正确性关键,估算粗糙无妨。
|
||||||
|
|
@ -40,29 +42,41 @@ _REASONING_FIELDS = {
|
||||||
def filter_reasoning_for_replay(
|
def filter_reasoning_for_replay(
|
||||||
messages: List[dict[str, Any]],
|
messages: List[dict[str, Any]],
|
||||||
policy: str = "none",
|
policy: str = "none",
|
||||||
|
target_model_profile: str = "",
|
||||||
) -> tuple[List[dict[str, Any]], dict[str, int]]:
|
) -> tuple[List[dict[str, Any]], dict[str, int]]:
|
||||||
"""构造 provider-bound 消息副本,按能力档案裁剪历史 reasoning。
|
"""构造 provider-bound 消息副本,按能力档案裁剪历史 reasoning。
|
||||||
|
|
||||||
原始 Session/DB payload 不动,继续供 UI、导出和排障使用。工具调用需要 reasoning
|
原始 Session/DB payload 不动,继续供 UI、导出和排障使用。工具调用需要 reasoning
|
||||||
连续性的 provider 可用 ``tool_calls``;未声明能力的 provider 默认 ``none``,避免
|
``tool_turn`` 只保留当前用户轮次内工具链的状态;``conversation`` 保留同模型
|
||||||
跨模型发送私有状态。保留时把 LiteLLM 可能放在 provider_specific_fields 里的
|
会话状态;``provider_managed`` 保留同模型的不透明签名/block。未声明能力默认
|
||||||
reasoning_content 提升到 DeepSeek/OpenAI-compatible 所需的顶层字段。
|
``none``。只要历史消息的生产模型与目标模型不同,就无条件剥离私有状态。
|
||||||
"""
|
"""
|
||||||
if policy not in {"none", "tool_calls", "all"}:
|
if policy not in REASONING_REPLAY_POLICIES:
|
||||||
raise ValueError(f"unsupported reasoning replay policy: {policy!r}")
|
raise ValueError(f"unsupported reasoning replay policy: {policy!r}")
|
||||||
|
|
||||||
|
last_user = max(
|
||||||
|
(i for i, msg in enumerate(messages) if msg.get("role") == "user"),
|
||||||
|
default=-1,
|
||||||
|
)
|
||||||
|
|
||||||
filtered: List[dict[str, Any]] = []
|
filtered: List[dict[str, Any]] = []
|
||||||
stripped_messages = 0
|
stripped_messages = 0
|
||||||
stripped_chars = 0
|
stripped_chars = 0
|
||||||
for message in messages:
|
for idx, message in enumerate(messages):
|
||||||
new_msg = deepcopy(message)
|
new_msg = deepcopy(message)
|
||||||
if policy == "all":
|
source_profile = str(new_msg.pop("_model_profile", "") or "")
|
||||||
filtered.append(new_msg)
|
compatible = not target_model_profile or source_profile == target_model_profile
|
||||||
continue
|
|
||||||
preserve = (
|
preserve = (
|
||||||
policy == "tool_calls"
|
compatible
|
||||||
and new_msg.get("role") == "assistant"
|
and new_msg.get("role") == "assistant"
|
||||||
and bool(new_msg.get("tool_calls"))
|
and (
|
||||||
|
policy in {"conversation", "provider_managed"}
|
||||||
|
or (
|
||||||
|
policy == "tool_turn"
|
||||||
|
and idx > last_user
|
||||||
|
and bool(new_msg.get("tool_calls"))
|
||||||
|
)
|
||||||
|
)
|
||||||
)
|
)
|
||||||
if preserve:
|
if preserve:
|
||||||
provider_fields = new_msg.get("provider_specific_fields")
|
provider_fields = new_msg.get("provider_specific_fields")
|
||||||
|
|
@ -242,6 +256,7 @@ def prepare_messages_for_llm(
|
||||||
old_tool_chars: int = 2_000,
|
old_tool_chars: int = 2_000,
|
||||||
compact_threshold_chars: int = 0,
|
compact_threshold_chars: int = 0,
|
||||||
reasoning_replay: str = "none",
|
reasoning_replay: str = "none",
|
||||||
|
target_model_profile: str = "",
|
||||||
) -> List[dict[str, Any]]:
|
) -> List[dict[str, Any]]:
|
||||||
"""返回发给 LLM 的 messages 副本。
|
"""返回发给 LLM 的 messages 副本。
|
||||||
|
|
||||||
|
|
@ -257,6 +272,7 @@ def prepare_messages_for_llm(
|
||||||
old_tool_chars=old_tool_chars,
|
old_tool_chars=old_tool_chars,
|
||||||
compact_threshold_chars=compact_threshold_chars,
|
compact_threshold_chars=compact_threshold_chars,
|
||||||
reasoning_replay=reasoning_replay,
|
reasoning_replay=reasoning_replay,
|
||||||
|
target_model_profile=target_model_profile,
|
||||||
)
|
)
|
||||||
return prepared
|
return prepared
|
||||||
|
|
||||||
|
|
@ -268,6 +284,7 @@ def prepare_messages_with_stats(
|
||||||
old_tool_chars: int = 2_000,
|
old_tool_chars: int = 2_000,
|
||||||
compact_threshold_chars: int = 0,
|
compact_threshold_chars: int = 0,
|
||||||
reasoning_replay: str = "none",
|
reasoning_replay: str = "none",
|
||||||
|
target_model_profile: str = "",
|
||||||
) -> tuple[List[dict[str, Any]], dict[str, int]]:
|
) -> tuple[List[dict[str, Any]], dict[str, int]]:
|
||||||
"""返回发给 LLM 的 messages 副本和压缩统计。
|
"""返回发给 LLM 的 messages 副本和压缩统计。
|
||||||
|
|
||||||
|
|
@ -279,7 +296,9 @@ def prepare_messages_with_stats(
|
||||||
"""
|
"""
|
||||||
if keep_recent < 0:
|
if keep_recent < 0:
|
||||||
keep_recent = 0
|
keep_recent = 0
|
||||||
messages, reasoning_stats = filter_reasoning_for_replay(messages, reasoning_replay)
|
messages, reasoning_stats = filter_reasoning_for_replay(
|
||||||
|
messages, reasoning_replay, target_model_profile
|
||||||
|
)
|
||||||
# 先补齐被中断 run 留下的悬空 tool_calls(否则原样发给模型会被拒,见函数注释)。
|
# 先补齐被中断 run 留下的悬空 tool_calls(否则原样发给模型会被拒,见函数注释)。
|
||||||
messages, repaired_tool_calls = _repair_dangling_tool_calls(messages)
|
messages, repaired_tool_calls = _repair_dangling_tool_calls(messages)
|
||||||
original_chars = sum(_message_chars(m) for m in messages)
|
original_chars = sum(_message_chars(m) for m in messages)
|
||||||
|
|
|
||||||
|
|
@ -25,6 +25,7 @@ from uuid import UUID
|
||||||
|
|
||||||
from sqlalchemy import update
|
from sqlalchemy import update
|
||||||
|
|
||||||
|
from .capabilities import model_profile_of
|
||||||
from .context import (
|
from .context import (
|
||||||
CHARS_PER_TOKEN,
|
CHARS_PER_TOKEN,
|
||||||
COMPACT_CONTEXT_RATIO,
|
COMPACT_CONTEXT_RATIO,
|
||||||
|
|
@ -33,7 +34,8 @@ from .context import (
|
||||||
filter_reasoning_for_replay,
|
filter_reasoning_for_replay,
|
||||||
prepare_messages_with_stats,
|
prepare_messages_with_stats,
|
||||||
)
|
)
|
||||||
from .storage import session_scope
|
from .llm_transport import extract_usage_details
|
||||||
|
from .storage import record_context_fold_failure, session_scope
|
||||||
from .storage.models import Task
|
from .storage.models import Task
|
||||||
from .storage.usage import record_chat_usage
|
from .storage.usage import record_chat_usage
|
||||||
|
|
||||||
|
|
@ -144,7 +146,10 @@ def maybe_fold(
|
||||||
"""
|
"""
|
||||||
msgs = session.messages
|
msgs = session.messages
|
||||||
reasoning_replay = getattr(caps, "reasoning_replay", "none")
|
reasoning_replay = getattr(caps, "reasoning_replay", "none")
|
||||||
provider_msgs, _ = filter_reasoning_for_replay(msgs, reasoning_replay)
|
target_profile = model_profile_of(caps)
|
||||||
|
provider_msgs, _ = filter_reasoning_for_replay(
|
||||||
|
msgs, reasoning_replay, target_profile
|
||||||
|
)
|
||||||
before_chars = sum(_message_chars(m) for m in provider_msgs)
|
before_chars = sum(_message_chars(m) for m in provider_msgs)
|
||||||
est_tokens = estimate_window_tokens(provider_msgs, session.last_measured_usage())
|
est_tokens = estimate_window_tokens(provider_msgs, session.last_measured_usage())
|
||||||
if est_tokens < caps.reliable_context * FOLD_TRIGGER_RATIO:
|
if est_tokens < caps.reliable_context * FOLD_TRIGGER_RATIO:
|
||||||
|
|
@ -164,6 +169,7 @@ def maybe_fold(
|
||||||
msgs,
|
msgs,
|
||||||
compact_threshold_chars=compact_threshold,
|
compact_threshold_chars=compact_threshold,
|
||||||
reasoning_replay=reasoning_replay,
|
reasoning_replay=reasoning_replay,
|
||||||
|
target_model_profile=target_profile,
|
||||||
)
|
)
|
||||||
p_cut = _kth_user_index(prepared, _user_ordinal(msgs, cutoff))
|
p_cut = _kth_user_index(prepared, _user_ordinal(msgs, cutoff))
|
||||||
if p_cut is None: # 理论不可达,防御
|
if p_cut is None: # 理论不可达,防御
|
||||||
|
|
@ -174,20 +180,36 @@ def maybe_fold(
|
||||||
choices = getattr(response, "choices", None) or []
|
choices = getattr(response, "choices", None) or []
|
||||||
summary = ((choices[0].message.content if choices else "") or "").strip()
|
summary = ((choices[0].message.content if choices else "") or "").strip()
|
||||||
if not summary:
|
if not summary:
|
||||||
|
try:
|
||||||
|
record_context_fold_failure(
|
||||||
|
task_id=session.task_id,
|
||||||
|
user_id=user_id,
|
||||||
|
model_profile=target_profile,
|
||||||
|
reason="empty_summary",
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
emit({"type": "warn", "msg": "context fold: 摘要调用返回空,本轮跳过折叠"})
|
emit({"type": "warn", "msg": "context fold: 摘要调用返回空,本轮跳过折叠"})
|
||||||
return None
|
return None
|
||||||
|
|
||||||
usage = getattr(response, "usage", None)
|
usage = extract_usage_details(getattr(response, "usage", None))
|
||||||
try:
|
try:
|
||||||
record_chat_usage(
|
record_chat_usage(
|
||||||
task_id=session.task_id,
|
task_id=session.task_id,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
message_id=None, # 摘要不入 messages,event 挂 task 维度
|
message_id=None, # 摘要不入 messages,event 挂 task 维度
|
||||||
model_profile=f"{caps.family}.{caps.variant}",
|
model_profile=f"{caps.family}.{caps.variant}",
|
||||||
prompt_tokens=getattr(usage, "prompt_tokens", 0) or 0,
|
prompt_tokens=usage["tokens_in"],
|
||||||
completion_tokens=getattr(usage, "completion_tokens", 0) or 0,
|
completion_tokens=usage["tokens_out"],
|
||||||
input_cny_per_mtoken=caps.input_cny_per_mtoken,
|
input_cny_per_mtoken=caps.input_cny_per_mtoken,
|
||||||
output_cny_per_mtoken=caps.output_cny_per_mtoken,
|
output_cny_per_mtoken=caps.output_cny_per_mtoken,
|
||||||
|
cache_hit_tokens=usage["cache_hit_tokens"],
|
||||||
|
cache_hit_cny_per_mtoken=getattr(caps, "cache_hit_cny_per_mtoken", 0.0),
|
||||||
|
pricing=getattr(caps, "pricing", {}) or {},
|
||||||
|
extra_units={
|
||||||
|
k: v for k, v in usage.items()
|
||||||
|
if k not in ("tokens_in", "tokens_out") and v
|
||||||
|
},
|
||||||
response=response,
|
response=response,
|
||||||
# 独立 kind:按 kind 对账"折叠烧多少钱",且折叠次数 = count(该 kind)
|
# 独立 kind:按 kind 对账"折叠烧多少钱",且折叠次数 = count(该 kind)
|
||||||
# (任务详情 context_folds 字段的数据源,免加计数列)
|
# (任务详情 context_folds 字段的数据源,免加计数列)
|
||||||
|
|
@ -200,7 +222,7 @@ def maybe_fold(
|
||||||
persist_fold(session.task_id, new_base, summary)
|
persist_fold(session.task_id, new_base, summary)
|
||||||
session.apply_fold(cutoff, summary)
|
session.apply_fold(cutoff, summary)
|
||||||
after_provider_msgs, _ = filter_reasoning_for_replay(
|
after_provider_msgs, _ = filter_reasoning_for_replay(
|
||||||
session.messages, reasoning_replay
|
session.messages, reasoning_replay, target_profile
|
||||||
)
|
)
|
||||||
after_chars = sum(_message_chars(m) for m in after_provider_msgs)
|
after_chars = sum(_message_chars(m) for m in after_provider_msgs)
|
||||||
stats = {
|
stats = {
|
||||||
|
|
|
||||||
|
|
@ -35,6 +35,7 @@ class ExecCtx:
|
||||||
task_id: UUID
|
task_id: UUID
|
||||||
working_dir: Path
|
working_dir: Path
|
||||||
cancel_check: Optional[Callable[[], bool]] = None
|
cancel_check: Optional[Callable[[], bool]] = None
|
||||||
|
event_emit: Optional[Callable[[dict], None]] = None
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|
|
||||||
|
|
@ -25,7 +25,7 @@ run_python tmp .py 落 host 侧 `<user_root>/.zcbot_tmp/<task_id>/<rand>.py`(bin
|
||||||
|
|
||||||
Cancel limitation(第一版接受):
|
Cancel limitation(第一版接受):
|
||||||
- docker exec 客户端断开后,容器内 server 端进程**不会**因此终止 —— 这是 docker 设计
|
- docker exec 客户端断开后,容器内 server 端进程**不会**因此终止 —— 这是 docker 设计
|
||||||
- 第一版只杀 docker CLI(Popen.kill());容器内残留进程靠 idle 5min reaper / 下次
|
- 第一版只杀 docker CLI(Popen.kill());容器内残留进程靠 idle 10min reaper / 下次
|
||||||
ensure 时 rm -f 兜底
|
ensure 时 rm -f 兜底
|
||||||
- 升级触发(§7.5 #3 PGID 协议):用户反馈"取消了但还在烧 CPU" / 多次 cancel 后
|
- 升级触发(§7.5 #3 PGID 协议):用户反馈"取消了但还在烧 CPU" / 多次 cancel 后
|
||||||
容器内进程堆积 → 启用「ZCBOT_EXEC_ID env + PGID 写文件 + 二次 exec kill」协议
|
容器内进程堆积 → 启用「ZCBOT_EXEC_ID env + PGID 写文件 + 二次 exec kill」协议
|
||||||
|
|
@ -145,8 +145,12 @@ class DockerExecutor(Executor):
|
||||||
return ToolResult(content=f"[Error] unknown tool: {name}", exit_code=2)
|
return ToolResult(content=f"[Error] unknown tool: {name}", exit_code=2)
|
||||||
try:
|
try:
|
||||||
if name == "shell":
|
if name == "shell":
|
||||||
|
if not args.get("background"):
|
||||||
|
return self._call_heavy_foreground(name, args, ctx)
|
||||||
return self._exec_shell(args, ctx)
|
return self._exec_shell(args, ctx)
|
||||||
if name == "run_python":
|
if name == "run_python":
|
||||||
|
if not args.get("background"):
|
||||||
|
return self._call_heavy_foreground(name, args, ctx)
|
||||||
return self._exec_python(args, ctx)
|
return self._exec_python(args, ctx)
|
||||||
if name in FS_TOOLS:
|
if name in FS_TOOLS:
|
||||||
return self._exec_fs_tool(name, args, ctx)
|
return self._exec_fs_tool(name, args, ctx)
|
||||||
|
|
@ -157,6 +161,23 @@ class DockerExecutor(Executor):
|
||||||
)
|
)
|
||||||
return ToolResult(content=f"[Error] unhandled container tool: {name}", exit_code=2)
|
return ToolResult(content=f"[Error] unhandled container tool: {name}", exit_code=2)
|
||||||
|
|
||||||
|
def _call_heavy_foreground(self, name: str, args: Dict[str, Any], ctx: ExecCtx) -> ToolResult:
|
||||||
|
"""前台重型工具先排共享槽;排队不进入命令 timeout。"""
|
||||||
|
def notify(payload: dict) -> None:
|
||||||
|
if ctx.event_emit is not None:
|
||||||
|
ctx.event_emit({"type": "execution_queue", **payload})
|
||||||
|
|
||||||
|
with self.pool.capacity.foreground(
|
||||||
|
str(self.user_id), ctx.cancel_check, notify
|
||||||
|
) as admitted:
|
||||||
|
if not admitted:
|
||||||
|
return ToolResult(content="[Error] command cancelled by user while queued", exit_code=130)
|
||||||
|
self.pool.exec_started(self.user_id)
|
||||||
|
try:
|
||||||
|
return self._exec_shell(args, ctx) if name == "shell" else self._exec_python(args, ctx)
|
||||||
|
finally:
|
||||||
|
self.pool.exec_finished(self.user_id)
|
||||||
|
|
||||||
# ── shell ────────────────────────────────────────────────
|
# ── shell ────────────────────────────────────────────────
|
||||||
|
|
||||||
def _exec_shell(self, args: Dict[str, Any], ctx: ExecCtx) -> ToolResult:
|
def _exec_shell(self, args: Dict[str, Any], ctx: ExecCtx) -> ToolResult:
|
||||||
|
|
@ -248,7 +269,7 @@ class DockerExecutor(Executor):
|
||||||
def _exec_background(self, name: str, args: Dict[str, Any], ctx: ExecCtx) -> ToolResult:
|
def _exec_background(self, name: str, args: Dict[str, Any], ctx: ExecCtx) -> ToolResult:
|
||||||
"""background=true:专用容器跑长进程,立即返回 proc_id。
|
"""background=true:专用容器跑长进程,立即返回 proc_id。
|
||||||
|
|
||||||
为什么不是 `docker exec -d` 进 sandbox 容器:sandbox 有 idle 5min reaper +
|
为什么不是 `docker exec -d` 进 sandbox 容器:sandbox 有 idle 10min reaper +
|
||||||
启动时 shutdown_all,长进程会随容器陪葬。专用容器 product=proc 与这两条
|
启动时 shutdown_all,长进程会随容器陪葬。专用容器 product=proc 与这两条
|
||||||
生命周期解耦(pool.run_proc_container),状态协议同 host 模式(core/procs.py):
|
生命周期解耦(pool.run_proc_container),状态协议同 host 模式(core/procs.py):
|
||||||
runner.sh 结束时写 exit_code,check_process/sweep 负责回收容器。
|
runner.sh 结束时写 exit_code,check_process/sweep 负责回收容器。
|
||||||
|
|
@ -256,15 +277,6 @@ class DockerExecutor(Executor):
|
||||||
from core import procs
|
from core import procs
|
||||||
|
|
||||||
anchor = self.user_root
|
anchor = self.user_root
|
||||||
if procs.count_running(anchor) >= procs.MAX_RUNNING_PER_USER:
|
|
||||||
return ToolResult(
|
|
||||||
content=(
|
|
||||||
f"[Error] 已有 {procs.MAX_RUNNING_PER_USER} 个后台进程在跑(上限)。"
|
|
||||||
f"用 check_process 查看,等待完成或 kill 掉不需要的再启动。"
|
|
||||||
),
|
|
||||||
exit_code=2,
|
|
||||||
)
|
|
||||||
|
|
||||||
raw_timeout = args.get("timeout")
|
raw_timeout = args.get("timeout")
|
||||||
fg_default = 60 if name == "shell" else 120
|
fg_default = 60 if name == "shell" else 120
|
||||||
try:
|
try:
|
||||||
|
|
@ -334,29 +346,16 @@ class DockerExecutor(Executor):
|
||||||
"timeout_s": timeout_s,
|
"timeout_s": timeout_s,
|
||||||
"created_at": time.strftime("%Y-%m-%dT%H:%M:%S"),
|
"created_at": time.strftime("%Y-%m-%dT%H:%M:%S"),
|
||||||
"created_ts": time.time(),
|
"created_ts": time.time(),
|
||||||
|
"user_id": str(self.user_id),
|
||||||
|
"exec_user": self.exec_user,
|
||||||
|
"state": "queued",
|
||||||
}
|
}
|
||||||
procs.write_meta(d, meta)
|
procs.write_meta(d, meta)
|
||||||
|
|
||||||
try:
|
started = procs.start_queued_docker(meta, d, self.pool)
|
||||||
container = self.pool.run_proc_container(self.user_id, proc_id, str(d))
|
|
||||||
except Exception as e:
|
|
||||||
return ToolResult(content=f"[Error] 后台容器启动失败: {e}", exit_code=1)
|
|
||||||
meta["container"] = container
|
|
||||||
procs.write_meta(d, meta)
|
|
||||||
|
|
||||||
argv = self._docker_exec_argv(
|
|
||||||
container, extra_env=_sandbox_env({"PYTHONIOENCODING": "utf-8"}), detach=True
|
|
||||||
) + ["bash", f"{cdir}/runner.sh"]
|
|
||||||
r = subprocess.run(argv, capture_output=True, text=True, timeout=60)
|
|
||||||
if r.returncode != 0:
|
|
||||||
subprocess.run(["docker", "rm", "-f", container], capture_output=True)
|
|
||||||
return ToolResult(
|
|
||||||
content=f"[Error] 后台进程启动失败: {(r.stderr or '').strip()[:300]}",
|
|
||||||
exit_code=1,
|
|
||||||
)
|
|
||||||
return ToolResult(
|
return ToolResult(
|
||||||
content=(
|
content=(
|
||||||
f"[Background] 已启动后台进程 proc_id={proc_id}({display[:150]}),"
|
f"[Background] 已{'启动' if started else '排队'}后台进程 proc_id={proc_id}({display[:150]}),"
|
||||||
f"最长运行 {timeout_s}s。\n"
|
f"最长运行 {timeout_s}s。\n"
|
||||||
f"用 check_process(proc_id=\"{proc_id}\") 查进度和日志。进程独立于本轮对话运行,"
|
f"用 check_process(proc_id=\"{proc_id}\") 查进度和日志。进程独立于本轮对话运行,"
|
||||||
f"服务重启也不中断。现在可以继续其他工作;若无事可做,结束回合并告知用户稍后询问进度。"
|
f"服务重启也不中断。现在可以继续其他工作;若无事可做,结束回合并告知用户稍后询问进度。"
|
||||||
|
|
|
||||||
|
|
@ -30,6 +30,7 @@ from core.kb import format_index_line, kb_dir, parse_index, pending_sources
|
||||||
from core.file_store import atomic_write_text
|
from core.file_store import atomic_write_text
|
||||||
from core.kb_lock import KbBusyError, kb_is_locked, kb_mutation_lock
|
from core.kb_lock import KbBusyError, kb_is_locked, kb_mutation_lock
|
||||||
from core.llm import LLM
|
from core.llm import LLM
|
||||||
|
from core.llm_transport import extract_usage_details
|
||||||
from core.storage.usage import record_chat_usage, record_vision_usage
|
from core.storage.usage import record_chat_usage, record_vision_usage
|
||||||
|
|
||||||
# 摘要模型固定走最便宜档(与 web FALLBACK_MODEL_PROFILE 同值;单次几千 token,不随任务模型)
|
# 摘要模型固定走最便宜档(与 web FALLBACK_MODEL_PROFILE 同值;单次几千 token,不随任务模型)
|
||||||
|
|
@ -184,20 +185,30 @@ def _summarize(
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f"[kb_ingest] summary llm failed ({filename}): {type(e).__name__}: {e}", flush=True)
|
print(f"[kb_ingest] summary llm failed ({filename}): {type(e).__name__}: {e}", flush=True)
|
||||||
return fallback
|
return fallback
|
||||||
usage = getattr(response, "usage", None)
|
usage = extract_usage_details(getattr(response, "usage", None))
|
||||||
try:
|
try:
|
||||||
record_chat_usage(
|
record_chat_usage(
|
||||||
task_id=None,
|
task_id=None,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
message_id=None,
|
message_id=None,
|
||||||
model_profile=f"{caps.family}.{caps.variant}",
|
model_profile=f"{caps.family}.{caps.variant}",
|
||||||
prompt_tokens=getattr(usage, "prompt_tokens", 0) or 0,
|
prompt_tokens=usage["tokens_in"],
|
||||||
completion_tokens=getattr(usage, "completion_tokens", 0) or 0,
|
completion_tokens=usage["tokens_out"],
|
||||||
input_cny_per_mtoken=caps.input_cny_per_mtoken,
|
input_cny_per_mtoken=caps.input_cny_per_mtoken,
|
||||||
output_cny_per_mtoken=caps.output_cny_per_mtoken,
|
output_cny_per_mtoken=caps.output_cny_per_mtoken,
|
||||||
|
cache_hit_tokens=usage["cache_hit_tokens"],
|
||||||
|
cache_hit_cny_per_mtoken=getattr(caps, "cache_hit_cny_per_mtoken", 0.0),
|
||||||
|
pricing=getattr(caps, "pricing", {}) or {},
|
||||||
response=response,
|
response=response,
|
||||||
kind="kb_ingest",
|
kind="kb_ingest",
|
||||||
extra_units={"kb": kb_name, "source": filename},
|
extra_units={
|
||||||
|
"kb": kb_name,
|
||||||
|
"source": filename,
|
||||||
|
**{
|
||||||
|
k: v for k, v in usage.items()
|
||||||
|
if k not in ("tokens_in", "tokens_out") and v
|
||||||
|
},
|
||||||
|
},
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
|
||||||
68
core/llm.py
68
core/llm.py
|
|
@ -37,16 +37,39 @@ _REQUEST_TIMEOUT_S = int(os.getenv("ZCBOT_LLM_TIMEOUT_S", "600"))
|
||||||
|
|
||||||
|
|
||||||
class LLM:
|
class LLM:
|
||||||
def __init__(self, capabilities: ModelCapabilities) -> None:
|
def __init__(
|
||||||
|
self,
|
||||||
|
capabilities: ModelCapabilities,
|
||||||
|
*,
|
||||||
|
credential_resolver: Callable[[str], str] | None = None,
|
||||||
|
) -> None:
|
||||||
self.caps = capabilities
|
self.caps = capabilities
|
||||||
env_name = capabilities.api_key_env or "DEEPSEEK_API_KEY"
|
self.env_name = capabilities.api_key_env or "DEEPSEEK_API_KEY"
|
||||||
self.api_key = os.environ.get(env_name)
|
if credential_resolver is None:
|
||||||
|
from .provider_credentials.runtime import resolve_env_secret
|
||||||
|
credential_resolver = resolve_env_secret
|
||||||
|
self._credential_resolver = credential_resolver
|
||||||
|
self.api_key = self._credential_resolver(self.env_name)
|
||||||
self.api_base = capabilities.api_base or None
|
self.api_base = capabilities.api_base or None
|
||||||
if not self.api_key:
|
if not self.api_key:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f"环境变量 {env_name} 未设置,无法调用 {capabilities.model_id}"
|
f"Provider 凭据 {self.env_name} 未设置,无法调用 {capabilities.model_id}"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def _request_api_key(self) -> str:
|
||||||
|
key = self._credential_resolver(self.env_name)
|
||||||
|
if not key:
|
||||||
|
# 非注册表内的测试/私有模型仍保持构造期 env 快照兼容;正式受管
|
||||||
|
# Provider 不回退快照,删除 DB 覆盖后应立即回到当前 env 或报未配置。
|
||||||
|
from .provider_credentials.registry import BY_ENV
|
||||||
|
if self.env_name not in BY_ENV:
|
||||||
|
key = self.api_key
|
||||||
|
if not key:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"Provider 凭据 {self.env_name} 未设置,无法调用 {self.caps.model_id}"
|
||||||
|
)
|
||||||
|
return key
|
||||||
|
|
||||||
def _build_kwargs(
|
def _build_kwargs(
|
||||||
self,
|
self,
|
||||||
messages: List[dict],
|
messages: List[dict],
|
||||||
|
|
@ -54,11 +77,19 @@ class LLM:
|
||||||
parallel_tool_calls: Optional[bool],
|
parallel_tool_calls: Optional[bool],
|
||||||
reasoning_effort: Optional[str],
|
reasoning_effort: Optional[str],
|
||||||
) -> dict:
|
) -> dict:
|
||||||
|
# Session 私有元数据永不进入 provider 请求。原生图片由主 loop 在此之前
|
||||||
|
# 物化;折叠/标题等旁路则安全退化为纯文本路径提示。
|
||||||
|
provider_messages = [
|
||||||
|
{k: v for k, v in message.items() if not str(k).startswith("_")}
|
||||||
|
for message in messages
|
||||||
|
]
|
||||||
kwargs: dict = {
|
kwargs: dict = {
|
||||||
"model": self.caps.model_id,
|
"model": self.caps.model_id,
|
||||||
"messages": messages,
|
"messages": provider_messages,
|
||||||
"temperature": self.caps.optimal_temperature,
|
"temperature": self.caps.optimal_temperature,
|
||||||
"api_key": self.api_key,
|
# 请求级解析:已经创建的 LLM 在下一次请求热切新 Key;本次 kwargs
|
||||||
|
# 构建后保持局部快照,流式迭代期间不会中途换 Key。
|
||||||
|
"api_key": self._request_api_key(),
|
||||||
"timeout": _REQUEST_TIMEOUT_S,
|
"timeout": _REQUEST_TIMEOUT_S,
|
||||||
}
|
}
|
||||||
if self.api_base:
|
if self.api_base:
|
||||||
|
|
@ -72,12 +103,29 @@ class LLM:
|
||||||
enabled=self.caps.thinking_enabled,
|
enabled=self.caps.thinking_enabled,
|
||||||
transport=self.caps.thinking_transport,
|
transport=self.caps.thinking_transport,
|
||||||
reasoning_effort=reasoning_effort,
|
reasoning_effort=reasoning_effort,
|
||||||
|
clear_thinking=self.caps.thinking_clear,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
if self.caps.prompt_caching:
|
if self.caps.prompt_caching:
|
||||||
kwargs["extra_headers"] = {"anthropic-beta": "prompt-caching-2024-07-31"}
|
kwargs["extra_headers"] = {"anthropic-beta": "prompt-caching-2024-07-31"}
|
||||||
return kwargs
|
return kwargs
|
||||||
|
|
||||||
|
def _report_provider_error(self, error: Exception) -> None:
|
||||||
|
try:
|
||||||
|
from .provider_credentials.registry import BY_ENV
|
||||||
|
from .provider_credentials.service import record_business_failure
|
||||||
|
binding = BY_ENV.get(self.env_name)
|
||||||
|
if binding is None:
|
||||||
|
return
|
||||||
|
raw_status = getattr(error, "status_code", 0)
|
||||||
|
if hasattr(raw_status, "value"):
|
||||||
|
raw_status = raw_status.value
|
||||||
|
record_business_failure(
|
||||||
|
binding[0], status_code=int(raw_status or 0), detail=str(error)
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
def chat(
|
def chat(
|
||||||
self,
|
self,
|
||||||
messages: List[dict],
|
messages: List[dict],
|
||||||
|
|
@ -94,6 +142,7 @@ class LLM:
|
||||||
return response
|
return response
|
||||||
except (RateLimitError, APIConnectionError, ServiceUnavailableError, Timeout, APIError) as e:
|
except (RateLimitError, APIConnectionError, ServiceUnavailableError, Timeout, APIError) as e:
|
||||||
last_err = e
|
last_err = e
|
||||||
|
self._report_provider_error(e)
|
||||||
if attempt == max_retries - 1:
|
if attempt == max_retries - 1:
|
||||||
break
|
break
|
||||||
time.sleep(2 ** attempt)
|
time.sleep(2 ** attempt)
|
||||||
|
|
@ -130,8 +179,7 @@ class LLM:
|
||||||
|
|
||||||
yield from self._chat_stream_direct(kwargs, max_retries=max_retries)
|
yield from self._chat_stream_direct(kwargs, max_retries=max_retries)
|
||||||
|
|
||||||
@staticmethod
|
def _chat_stream_direct(self, kwargs: dict, *, max_retries: int) -> Iterator[Any]:
|
||||||
def _chat_stream_direct(kwargs: dict, *, max_retries: int) -> Iterator[Any]:
|
|
||||||
"""无取消调用方的直连路径(probe 等保持原有同步语义)。"""
|
"""无取消调用方的直连路径(probe 等保持原有同步语义)。"""
|
||||||
|
|
||||||
last_err: Optional[Exception] = None
|
last_err: Optional[Exception] = None
|
||||||
|
|
@ -141,6 +189,7 @@ class LLM:
|
||||||
break
|
break
|
||||||
except (RateLimitError, APIConnectionError, ServiceUnavailableError, Timeout, APIError) as e:
|
except (RateLimitError, APIConnectionError, ServiceUnavailableError, Timeout, APIError) as e:
|
||||||
last_err = e
|
last_err = e
|
||||||
|
self._report_provider_error(e)
|
||||||
if attempt == max_retries - 1:
|
if attempt == max_retries - 1:
|
||||||
raise
|
raise
|
||||||
time.sleep(2 ** attempt)
|
time.sleep(2 ** attempt)
|
||||||
|
|
@ -159,8 +208,8 @@ class LLM:
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _chat_stream_interruptible(
|
def _chat_stream_interruptible(
|
||||||
|
self,
|
||||||
kwargs: dict,
|
kwargs: dict,
|
||||||
*,
|
*,
|
||||||
max_retries: int,
|
max_retries: int,
|
||||||
|
|
@ -199,6 +248,7 @@ class LLM:
|
||||||
APIError,
|
APIError,
|
||||||
) as e:
|
) as e:
|
||||||
last_err = e
|
last_err = e
|
||||||
|
self._report_provider_error(e)
|
||||||
if attempt == max_retries - 1:
|
if attempt == max_retries - 1:
|
||||||
raise
|
raise
|
||||||
if stop.wait(2 ** attempt):
|
if stop.wait(2 ** attempt):
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,11 @@ THINKING_TRANSPORTS = {"none", "extra_body"}
|
||||||
|
|
||||||
|
|
||||||
def build_thinking_kwargs(
|
def build_thinking_kwargs(
|
||||||
*, enabled: bool, transport: str, reasoning_effort: str | None
|
*,
|
||||||
|
enabled: bool,
|
||||||
|
transport: str,
|
||||||
|
reasoning_effort: str | None,
|
||||||
|
clear_thinking: bool | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""把统一的 thinking 配置转换为 LiteLLM 调用参数。
|
"""把统一的 thinking 配置转换为 LiteLLM 调用参数。
|
||||||
|
|
||||||
|
|
@ -22,9 +26,12 @@ def build_thinking_kwargs(
|
||||||
f"不支持的 thinking_transport={transport!r};可选: {sorted(THINKING_TRANSPORTS)}"
|
f"不支持的 thinking_transport={transport!r};可选: {sorted(THINKING_TRANSPORTS)}"
|
||||||
)
|
)
|
||||||
|
|
||||||
body: dict[str, Any] = {
|
thinking: dict[str, Any] = {
|
||||||
"thinking": {"type": "enabled" if enabled else "disabled"}
|
"type": "enabled" if enabled else "disabled"
|
||||||
}
|
}
|
||||||
|
if enabled and clear_thinking is not None:
|
||||||
|
thinking["clear_thinking"] = clear_thinking
|
||||||
|
body: dict[str, Any] = {"thinking": thinking}
|
||||||
if enabled and reasoning_effort:
|
if enabled and reasoning_effort:
|
||||||
body["reasoning_effort"] = reasoning_effort
|
body["reasoning_effort"] = reasoning_effort
|
||||||
return {"extra_body": body}
|
return {"extra_body": body}
|
||||||
|
|
|
||||||
130
core/loop.py
130
core/loop.py
|
|
@ -22,7 +22,8 @@ import litellm
|
||||||
|
|
||||||
from . import pptx_guard
|
from . import pptx_guard
|
||||||
from .artifacts import MAX_ARTIFACTS_PER_MESSAGE
|
from .artifacts import MAX_ARTIFACTS_PER_MESSAGE
|
||||||
from .capabilities import ModelCapabilities
|
from .attachments import materialize_native_images
|
||||||
|
from .capabilities import ModelCapabilities, model_profile_of
|
||||||
from .context import (
|
from .context import (
|
||||||
CHARS_PER_TOKEN,
|
CHARS_PER_TOKEN,
|
||||||
COMPACT_CONTEXT_RATIO,
|
COMPACT_CONTEXT_RATIO,
|
||||||
|
|
@ -45,8 +46,13 @@ from .llm_transport import (
|
||||||
from .salvage import salvage_tool_arguments
|
from .salvage import salvage_tool_arguments
|
||||||
from .session import Session
|
from .session import Session
|
||||||
from .storage import (
|
from .storage import (
|
||||||
|
record_agent_guard,
|
||||||
record_chat_usage,
|
record_chat_usage,
|
||||||
|
record_context_fold_failure,
|
||||||
|
record_quality_gate,
|
||||||
|
record_run_stopped,
|
||||||
record_salvaged_tool_call,
|
record_salvaged_tool_call,
|
||||||
|
record_tool_failure,
|
||||||
)
|
)
|
||||||
from .task_actions import DeferredTaskActions
|
from .task_actions import DeferredTaskActions
|
||||||
|
|
||||||
|
|
@ -312,20 +318,21 @@ class AgentLoop:
|
||||||
assert response is not None
|
assert response is not None
|
||||||
msg = response.choices[0].message
|
msg = response.choices[0].message
|
||||||
tool_calls = getattr(msg, "tool_calls", None) or []
|
tool_calls = getattr(msg, "tool_calls", None) or []
|
||||||
|
model_profile = model_profile_of(self.caps)
|
||||||
asst_msg_id = self.session.append(
|
asst_msg_id = self.session.append(
|
||||||
msg,
|
msg,
|
||||||
artifact_refs=(list(self._pending_artifact_refs) if not tool_calls else None),
|
artifact_refs=(list(self._pending_artifact_refs) if not tool_calls else None),
|
||||||
|
source_model_profile=model_profile,
|
||||||
)
|
)
|
||||||
|
|
||||||
usage_details = extract_usage_details(getattr(response, "usage", None))
|
usage_details = extract_usage_details(getattr(response, "usage", None))
|
||||||
pt, ct = usage_details["tokens_in"], usage_details["tokens_out"]
|
pt, ct = usage_details["tokens_in"], usage_details["tokens_out"]
|
||||||
# 用本轮实报 prompt_tokens 刷新 chars/token 校准比值(下一轮门槛/占用环即用)。
|
# 用本轮实报 prompt_tokens 刷新 chars/token 校准比值(下一轮门槛/占用环即用)。
|
||||||
if pt > 0 and self._last_sent_chars > 0:
|
if pt > 0 and self._last_sent_chars > 0 and not self._last_had_native_images:
|
||||||
self._ctx_chars_per_token = clamp_ratio(self._last_sent_chars / pt)
|
self._ctx_chars_per_token = clamp_ratio(self._last_sent_chars / pt)
|
||||||
# 记账(0006):一行 usage_event + 回填 messages.tokens_in/out + model_profile。
|
# 记账(0006):一行 usage_event + 回填 messages.tokens_in/out + model_profile。
|
||||||
# 任何失败都吞掉(litellm cost map miss / DB 异常),不阻塞主 loop;
|
# 任何失败都吞掉(litellm cost map miss / DB 异常),不阻塞主 loop;
|
||||||
# message 仍在 session/DB 里,后续重启不影响。
|
# message 仍在 session/DB 里,后续重启不影响。
|
||||||
model_profile = f"{self.caps.family}.{self.caps.variant}"
|
|
||||||
try:
|
try:
|
||||||
record_chat_usage(
|
record_chat_usage(
|
||||||
task_id=self.session.task_id,
|
task_id=self.session.task_id,
|
||||||
|
|
@ -338,6 +345,7 @@ class AgentLoop:
|
||||||
output_cny_per_mtoken=self.caps.output_cny_per_mtoken,
|
output_cny_per_mtoken=self.caps.output_cny_per_mtoken,
|
||||||
cache_hit_tokens=usage_details["cache_hit_tokens"],
|
cache_hit_tokens=usage_details["cache_hit_tokens"],
|
||||||
cache_hit_cny_per_mtoken=self.caps.cache_hit_cny_per_mtoken,
|
cache_hit_cny_per_mtoken=self.caps.cache_hit_cny_per_mtoken,
|
||||||
|
pricing=self.caps.pricing,
|
||||||
extra_units={
|
extra_units={
|
||||||
k: v for k, v in usage_details.items()
|
k: v for k, v in usage_details.items()
|
||||||
if k not in ("tokens_in", "tokens_out") and v
|
if k not in ("tokens_in", "tokens_out") and v
|
||||||
|
|
@ -381,7 +389,7 @@ class AgentLoop:
|
||||||
result, productive, artifacts = self._execute_tool_call(tc)
|
result, productive, artifacts = self._execute_tool_call(tc)
|
||||||
self._remember_artifacts(artifacts)
|
self._remember_artifacts(artifacts)
|
||||||
step_productive = step_productive or productive
|
step_productive = step_productive or productive
|
||||||
self.session.append(
|
message_id = self.session.append(
|
||||||
{
|
{
|
||||||
"role": "tool",
|
"role": "tool",
|
||||||
"tool_call_id": tc.id,
|
"tool_call_id": tc.id,
|
||||||
|
|
@ -389,6 +397,20 @@ class AgentLoop:
|
||||||
"content": result,
|
"content": result,
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
if message_id is not None:
|
||||||
|
try:
|
||||||
|
record_tool_failure(
|
||||||
|
task_id=self.session.task_id,
|
||||||
|
user_id=self.user_id,
|
||||||
|
message_id=message_id,
|
||||||
|
model_profile=model_profile_of(self.caps),
|
||||||
|
tool=tc.function.name,
|
||||||
|
content=result,
|
||||||
|
arguments=tc.function.arguments,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
# 可观测性留痕失败不能打断主对话。
|
||||||
|
pass
|
||||||
|
|
||||||
# ask_user:本步调用了人工选择工具 → 提前结束本轮,等用户点选项 / 文字讨论,
|
# ask_user:本步调用了人工选择工具 → 提前结束本轮,等用户点选项 / 文字讨论,
|
||||||
# 不回灌 LLM。选项已随该 tool_call 的 arguments 流给前端渲染成选项卡;tool 结果
|
# 不回灌 LLM。选项已随该 tool_call 的 arguments 流给前端渲染成选项卡;tool 结果
|
||||||
|
|
@ -404,6 +426,15 @@ class AgentLoop:
|
||||||
else:
|
else:
|
||||||
self._stall += 1
|
self._stall += 1
|
||||||
if self._stall >= self._STALL_LIMIT:
|
if self._stall >= self._STALL_LIMIT:
|
||||||
|
try:
|
||||||
|
record_run_stopped(
|
||||||
|
task_id=self.session.task_id,
|
||||||
|
user_id=self.user_id,
|
||||||
|
model_profile=model_profile_of(self.caps),
|
||||||
|
reason="no_progress",
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
self._emit({
|
self._emit({
|
||||||
"type": "warn",
|
"type": "warn",
|
||||||
"msg": (
|
"msg": (
|
||||||
|
|
@ -415,6 +446,15 @@ class AgentLoop:
|
||||||
return "[stopped: no progress]"
|
return "[stopped: no progress]"
|
||||||
|
|
||||||
# 跑满 backstop:不是出错,是单轮自主步数到顶。明确提示可续跑,别静默停。
|
# 跑满 backstop:不是出错,是单轮自主步数到顶。明确提示可续跑,别静默停。
|
||||||
|
try:
|
||||||
|
record_run_stopped(
|
||||||
|
task_id=self.session.task_id,
|
||||||
|
user_id=self.user_id,
|
||||||
|
model_profile=model_profile_of(self.caps),
|
||||||
|
reason="max_iterations",
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
self._emit({
|
self._emit({
|
||||||
"type": "warn",
|
"type": "warn",
|
||||||
"msg": (
|
"msg": (
|
||||||
|
|
@ -456,6 +496,7 @@ class AgentLoop:
|
||||||
# 无实测时的回退(详 context.py 常数注释)。
|
# 无实测时的回退(详 context.py 常数注释)。
|
||||||
_ctx_chars_per_token: Optional[float] = None
|
_ctx_chars_per_token: Optional[float] = None
|
||||||
_last_sent_chars: int = 0
|
_last_sent_chars: int = 0
|
||||||
|
_last_had_native_images: bool = False
|
||||||
|
|
||||||
def _context_ratio(self) -> float:
|
def _context_ratio(self) -> float:
|
||||||
"""当前窗口的 chars/token 换算比值(校准态)。
|
"""当前窗口的 chars/token 换算比值(校准态)。
|
||||||
|
|
@ -470,6 +511,7 @@ class AgentLoop:
|
||||||
provider_messages, _ = filter_reasoning_for_replay(
|
provider_messages, _ = filter_reasoning_for_replay(
|
||||||
self.session.messages,
|
self.session.messages,
|
||||||
getattr(self.caps, "reasoning_replay", "none"),
|
getattr(self.caps, "reasoning_replay", "none"),
|
||||||
|
model_profile_of(self.caps),
|
||||||
)
|
)
|
||||||
self._ctx_chars_per_token = calibrated_chars_per_token(
|
self._ctx_chars_per_token = calibrated_chars_per_token(
|
||||||
provider_messages, self.session.last_measured_usage()
|
provider_messages, self.session.last_measured_usage()
|
||||||
|
|
@ -492,6 +534,16 @@ class AgentLoop:
|
||||||
user_id=self.user_id, emit=self._emit,
|
user_id=self.user_id, emit=self._emit,
|
||||||
)
|
)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
try:
|
||||||
|
record_context_fold_failure(
|
||||||
|
task_id=self.session.task_id,
|
||||||
|
user_id=self.user_id,
|
||||||
|
model_profile=model_profile_of(self.caps),
|
||||||
|
reason="exception",
|
||||||
|
detail=f"{type(e).__name__}: {e}",
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
self._emit({
|
self._emit({
|
||||||
"type": "warn",
|
"type": "warn",
|
||||||
"msg": f"context fold failed: {type(e).__name__}: {e};本轮跳过折叠",
|
"msg": f"context fold failed: {type(e).__name__}: {e};本轮跳过折叠",
|
||||||
|
|
@ -514,7 +566,18 @@ class AgentLoop:
|
||||||
self.session.messages,
|
self.session.messages,
|
||||||
compact_threshold_chars=compact_threshold,
|
compact_threshold_chars=compact_threshold,
|
||||||
reasoning_replay=getattr(self.caps, "reasoning_replay", "none"),
|
reasoning_replay=getattr(self.caps, "reasoning_replay", "none"),
|
||||||
|
target_model_profile=model_profile_of(self.caps),
|
||||||
)
|
)
|
||||||
|
native_images = 0
|
||||||
|
if self.user_root is not None:
|
||||||
|
llm_messages, native_images = materialize_native_images(
|
||||||
|
llm_messages,
|
||||||
|
enabled=self.caps.native_image_input,
|
||||||
|
working_dir=self.working_dir,
|
||||||
|
user_root=self.user_root,
|
||||||
|
)
|
||||||
|
self._last_had_native_images = native_images > 0
|
||||||
|
context_stats["native_images"] = native_images
|
||||||
self._last_sent_chars = context_stats.get("sent_chars", 0)
|
self._last_sent_chars = context_stats.get("sent_chars", 0)
|
||||||
llm_start_event = {
|
llm_start_event = {
|
||||||
"type": "llm_start",
|
"type": "llm_start",
|
||||||
|
|
@ -623,9 +686,8 @@ class AgentLoop:
|
||||||
cancel_check=self._is_cancelled,
|
cancel_check=self._is_cancelled,
|
||||||
)
|
)
|
||||||
cancelled = False
|
cancelled = False
|
||||||
pending_events: List[dict] = []
|
|
||||||
may_reroute = self.caps.family == "deepseek_v4"
|
may_reroute = self.caps.family == "deepseek_v4"
|
||||||
output_route_known = not may_reroute
|
reasoning_emitted = False
|
||||||
try:
|
try:
|
||||||
for chunk in stream:
|
for chunk in stream:
|
||||||
if self._is_cancelled():
|
if self._is_cancelled():
|
||||||
|
|
@ -637,7 +699,10 @@ class AgentLoop:
|
||||||
may_reroute
|
may_reroute
|
||||||
and any(name in self._DEEPSEEK_NONSTREAM_TOOLS for name in tool_names)
|
and any(name in self._DEEPSEEK_NONSTREAM_TOOLS for name in tool_names)
|
||||||
):
|
):
|
||||||
# 推理 delta 先暂存,避免切换后非流式完整 reasoning 再发一次造成重复。
|
# 当前流会被放弃并以非流式完整重发。已实时展示的推理属于被放弃
|
||||||
|
# 的响应,先通知前端清掉,避免随后展示最终响应时重复或与历史不一致。
|
||||||
|
if reasoning_emitted:
|
||||||
|
self._emit({"type": "reasoning_reset"})
|
||||||
raise PreferNonstreamToolCall(next(
|
raise PreferNonstreamToolCall(next(
|
||||||
name for name in tool_names
|
name for name in tool_names
|
||||||
if name in self._DEEPSEEK_NONSTREAM_TOOLS
|
if name in self._DEEPSEEK_NONSTREAM_TOOLS
|
||||||
|
|
@ -647,20 +712,13 @@ class AgentLoop:
|
||||||
# _execute_tool_call 时机发更直观)。
|
# _execute_tool_call 时机发更直观)。
|
||||||
delta_text = extract_delta_content(chunk)
|
delta_text = extract_delta_content(chunk)
|
||||||
if delta_text:
|
if delta_text:
|
||||||
pending_events.append({"type": "text", "delta": delta_text})
|
self._emit({"type": "text", "delta": delta_text})
|
||||||
# thinking 模型的推理 delta 也实时流出(reasoning 事件):深度推理可达
|
# thinking 模型的推理 delta 也实时流出(reasoning 事件):深度推理可达
|
||||||
# 分钟级,不发的话前端全程静止"思考中",用户以为卡死。
|
# 分钟级,不发的话前端全程静止"思考中",用户以为卡死。
|
||||||
delta_reasoning = extract_delta_reasoning(chunk)
|
delta_reasoning = extract_delta_reasoning(chunk)
|
||||||
if delta_reasoning:
|
if delta_reasoning:
|
||||||
pending_events.append({"type": "reasoning", "delta": delta_reasoning})
|
self._emit({"type": "reasoning", "delta": delta_reasoning})
|
||||||
# 首个正文或任意工具名已确定本轮不会再切换;释放之前暂存的推理片段,
|
reasoning_emitted = True
|
||||||
# 后续 chunk 也继续即时释放。只有尚未看到输出类型时才短暂缓冲。
|
|
||||||
if delta_text or tool_names:
|
|
||||||
output_route_known = True
|
|
||||||
if output_route_known and pending_events:
|
|
||||||
for event in pending_events:
|
|
||||||
self._emit(event)
|
|
||||||
pending_events.clear()
|
|
||||||
# interruptible stream 会在无新 chunk 的等待期直接因 cancel 结束迭代;
|
# interruptible stream 会在无新 chunk 的等待期直接因 cancel 结束迭代;
|
||||||
# 循环体没有机会执行上面的检查,故在正常耗尽处再判一次。
|
# 循环体没有机会执行上面的检查,故在正常耗尽处再判一次。
|
||||||
if self._is_cancelled():
|
if self._is_cancelled():
|
||||||
|
|
@ -674,10 +732,6 @@ class AgentLoop:
|
||||||
if cancelled:
|
if cancelled:
|
||||||
return None, True
|
return None, True
|
||||||
|
|
||||||
# 极少数 provider 只有 reasoning/空收尾、始终没有正文或工具名,不能吞掉已收内容。
|
|
||||||
for event in pending_events:
|
|
||||||
self._emit(event)
|
|
||||||
|
|
||||||
# 用 litellm 官方 helper 拼回完整 response(包括 tool_calls 拼接 + usage)。
|
# 用 litellm 官方 helper 拼回完整 response(包括 tool_calls 拼接 + usage)。
|
||||||
# messages 参数仅用于失败时回填 prompt token 估算,正常路径 stream_options.include_usage
|
# messages 参数仅用于失败时回填 prompt token 估算,正常路径 stream_options.include_usage
|
||||||
# 已让最后一个 chunk 带准确 usage。
|
# 已让最后一个 chunk 带准确 usage。
|
||||||
|
|
@ -765,6 +819,7 @@ class AgentLoop:
|
||||||
task_id=self.session.task_id,
|
task_id=self.session.task_id,
|
||||||
working_dir=self.working_dir,
|
working_dir=self.working_dir,
|
||||||
cancel_check=self.cancel_check,
|
cancel_check=self.cancel_check,
|
||||||
|
event_emit=self._emit,
|
||||||
)
|
)
|
||||||
tool_started_at = time.time()
|
tool_started_at = time.time()
|
||||||
tool_result = self.executor.call_tool(name, args, ctx)
|
tool_result = self.executor.call_tool(name, args, ctx)
|
||||||
|
|
@ -834,6 +889,17 @@ class AgentLoop:
|
||||||
"""
|
"""
|
||||||
if self._repeat_guard.should_block(name, args):
|
if self._repeat_guard.should_block(name, args):
|
||||||
n, _blocked = self._repeat_guard.register_block(name, args)
|
n, _blocked = self._repeat_guard.register_block(name, args)
|
||||||
|
try:
|
||||||
|
record_agent_guard(
|
||||||
|
task_id=self.session.task_id,
|
||||||
|
user_id=self.user_id,
|
||||||
|
model_profile=model_profile_of(self.caps),
|
||||||
|
tool=name,
|
||||||
|
guard="same_arguments",
|
||||||
|
count=n,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
result = (
|
result = (
|
||||||
f"[已拦截重复调用] {name} 用完全相同的参数已调用 {n} 次且结果始终未变,本次未执行。"
|
f"[已拦截重复调用] {name} 用完全相同的参数已调用 {n} 次且结果始终未变,本次未执行。"
|
||||||
"这通常意味着思路卡死:① 换不同的参数或方法;② 读一下相关文件/报错重新定位;"
|
"这通常意味着思路卡死:① 换不同的参数或方法;② 读一下相关文件/报错重新定位;"
|
||||||
|
|
@ -845,6 +911,18 @@ class AgentLoop:
|
||||||
|
|
||||||
if self._repeat_guard.should_block_err(name):
|
if self._repeat_guard.should_block_err(name):
|
||||||
cnt, esig = self._repeat_guard.register_err_block(name)
|
cnt, esig = self._repeat_guard.register_err_block(name)
|
||||||
|
try:
|
||||||
|
record_agent_guard(
|
||||||
|
task_id=self.session.task_id,
|
||||||
|
user_id=self.user_id,
|
||||||
|
model_profile=model_profile_of(self.caps),
|
||||||
|
tool=name,
|
||||||
|
guard="same_error",
|
||||||
|
count=cnt,
|
||||||
|
signature=esig,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
result = (
|
result = (
|
||||||
f"[已拦截重复调用] {name} 已连续 {cnt} 次撞同一个错误「{esig}」(每次只微调了参数)。"
|
f"[已拦截重复调用] {name} 已连续 {cnt} 次撞同一个错误「{esig}」(每次只微调了参数)。"
|
||||||
"再这么试下去不会有新结果。换个做法:① 先 read 目标文件/用 grep 看确切内容"
|
"再这么试下去不会有新结果。换个做法:① 先 read 目标文件/用 grep 看确切内容"
|
||||||
|
|
@ -937,6 +1015,16 @@ class AgentLoop:
|
||||||
except Exception:
|
except Exception:
|
||||||
guard_msg = None # 机检自身故障绝不拖垮工具链路
|
guard_msg = None # 机检自身故障绝不拖垮工具链路
|
||||||
if guard_msg:
|
if guard_msg:
|
||||||
|
try:
|
||||||
|
record_quality_gate(
|
||||||
|
task_id=self.session.task_id,
|
||||||
|
user_id=self.user_id,
|
||||||
|
model_profile=model_profile_of(self.caps),
|
||||||
|
tool=name,
|
||||||
|
gate="pptx_full_page_image",
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
result += "\n\n" + guard_msg
|
result += "\n\n" + guard_msg
|
||||||
self._emit({
|
self._emit({
|
||||||
"type": "warn",
|
"type": "warn",
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@
|
||||||
- 某档成员里出现 `"*"` → 该档全开(含未来新增模型),给内部档用。
|
- 某档成员里出现 `"*"` → 该档全开(含未来新增模型),给内部档用。
|
||||||
|
|
||||||
模型 id 约定(与 list 端点 / resolve 校验一致):
|
模型 id 约定(与 list 端点 / resolve 校验一致):
|
||||||
- 文本模型 = `family.variant`(config/models/<family>.yaml),如 `doubao.pro`、`glm.pro52`
|
- 文本模型 = `family.variant`(config/models/<family>.yaml),如 `doubao.pro`、`glm.flash53`
|
||||||
- 图 / 视频模型 = variant key(config/media/doubao.yaml),如 `seedream_5`、`seedance_2_fast`
|
- 图 / 视频模型 = variant key(config/media/doubao.yaml),如 `seedream_5`、`seedance_2_fast`
|
||||||
两者命名不冲突(文本带点、媒体 variant 不带点),同一档集合里混放即可。
|
两者命名不冲突(文本带点、媒体 variant 不带点),同一档集合里混放即可。
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,209 @@
|
||||||
|
"""版本化 chat 价格解析与成本计算。
|
||||||
|
|
||||||
|
模型 API 通常只返回 token usage,不返回实际金额。这里以模型 YAML 中经过核对的
|
||||||
|
provider 价格为事实源,并把命中的价格版本、时段和汇率快照交给 usage_events 留痕。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, time, timezone
|
||||||
|
from decimal import Decimal
|
||||||
|
from typing import Any, Mapping, Optional
|
||||||
|
|
||||||
|
|
||||||
|
_MTOKEN = Decimal("1000000")
|
||||||
|
_WEEKDAYS = {
|
||||||
|
"mon": 0,
|
||||||
|
"tue": 1,
|
||||||
|
"wed": 2,
|
||||||
|
"thu": 3,
|
||||||
|
"fri": 4,
|
||||||
|
"sat": 5,
|
||||||
|
"sun": 6,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _decimal(value: Any, default: str = "0") -> Decimal:
|
||||||
|
if value is None or value == "":
|
||||||
|
return Decimal(default)
|
||||||
|
return Decimal(str(value))
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_datetime(value: Any) -> Optional[datetime]:
|
||||||
|
if not value:
|
||||||
|
return None
|
||||||
|
if isinstance(value, datetime):
|
||||||
|
dt = value
|
||||||
|
else:
|
||||||
|
raw = str(value).strip().replace("Z", "+00:00")
|
||||||
|
dt = datetime.fromisoformat(raw)
|
||||||
|
if dt.tzinfo is None:
|
||||||
|
dt = dt.replace(tzinfo=timezone.utc)
|
||||||
|
return dt.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_time(value: str) -> time:
|
||||||
|
hour, minute = (int(part) for part in value.split(":", 1))
|
||||||
|
return time(hour=hour, minute=minute)
|
||||||
|
|
||||||
|
|
||||||
|
def _in_window(now: time, start: time, end: time) -> bool:
|
||||||
|
if start <= end:
|
||||||
|
return start <= now < end
|
||||||
|
return now >= start or now < end
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PriceQuote:
|
||||||
|
revision: str
|
||||||
|
source_url: str
|
||||||
|
currency: str
|
||||||
|
fx_to_cny: Decimal
|
||||||
|
tier: str
|
||||||
|
input_per_mtoken: Decimal
|
||||||
|
output_per_mtoken: Decimal
|
||||||
|
cache_hit_per_mtoken: Decimal
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CostBreakdown:
|
||||||
|
total_cny: Decimal
|
||||||
|
input_cny: Decimal
|
||||||
|
cache_hit_cny: Decimal
|
||||||
|
output_cny: Decimal
|
||||||
|
cache_hit_tokens: int
|
||||||
|
cache_miss_tokens: int
|
||||||
|
|
||||||
|
|
||||||
|
def _periods(pricing: Mapping[str, Any]) -> list[Mapping[str, Any]]:
|
||||||
|
periods = pricing.get("periods")
|
||||||
|
if isinstance(periods, list):
|
||||||
|
return [p for p in periods if isinstance(p, Mapping)]
|
||||||
|
# 兼容一个价格版本直接写在 pricing 顶层的简写。
|
||||||
|
return [pricing]
|
||||||
|
|
||||||
|
|
||||||
|
def _select_period(
|
||||||
|
pricing: Mapping[str, Any], occurred_at: datetime
|
||||||
|
) -> Optional[Mapping[str, Any]]:
|
||||||
|
candidates: list[tuple[datetime, Mapping[str, Any]]] = []
|
||||||
|
for period in _periods(pricing):
|
||||||
|
start = _parse_datetime(period.get("effective_from")) or datetime.min.replace(
|
||||||
|
tzinfo=timezone.utc
|
||||||
|
)
|
||||||
|
end = _parse_datetime(period.get("effective_to"))
|
||||||
|
if start <= occurred_at and (end is None or occurred_at < end):
|
||||||
|
candidates.append((start, period))
|
||||||
|
if not candidates:
|
||||||
|
return None
|
||||||
|
return max(candidates, key=lambda item: item[0])[1]
|
||||||
|
|
||||||
|
|
||||||
|
def _tier_matches(tier: Mapping[str, Any], occurred_at: datetime) -> bool:
|
||||||
|
tz_name = str(tier.get("timezone") or "UTC").upper()
|
||||||
|
if tz_name != "UTC":
|
||||||
|
raise ValueError(f"pricing time tier 暂只支持 UTC,收到 {tz_name!r}")
|
||||||
|
now = occurred_at.astimezone(timezone.utc)
|
||||||
|
weekdays = tier.get("weekdays") or []
|
||||||
|
if weekdays:
|
||||||
|
allowed = {_WEEKDAYS[str(day).lower()] for day in weekdays}
|
||||||
|
if now.weekday() not in allowed:
|
||||||
|
return False
|
||||||
|
windows = tier.get("windows") or []
|
||||||
|
if not windows:
|
||||||
|
return True
|
||||||
|
current = now.time().replace(tzinfo=None)
|
||||||
|
return any(
|
||||||
|
_in_window(current, _parse_time(window[0]), _parse_time(window[1]))
|
||||||
|
for window in windows
|
||||||
|
if isinstance(window, (list, tuple)) and len(window) == 2
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_chat_price(
|
||||||
|
pricing: Mapping[str, Any] | None,
|
||||||
|
*,
|
||||||
|
occurred_at: Optional[datetime] = None,
|
||||||
|
) -> Optional[PriceQuote]:
|
||||||
|
"""解析调用时刻适用的价格;无配置或尚未生效时返回 ``None``。"""
|
||||||
|
if not pricing:
|
||||||
|
return None
|
||||||
|
when = occurred_at or datetime.now(timezone.utc)
|
||||||
|
if when.tzinfo is None:
|
||||||
|
when = when.replace(tzinfo=timezone.utc)
|
||||||
|
when = when.astimezone(timezone.utc)
|
||||||
|
period = _select_period(pricing, when)
|
||||||
|
if period is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
rates = period.get("default") or {}
|
||||||
|
tier_name = "default"
|
||||||
|
for tier in period.get("time_tiers") or []:
|
||||||
|
if isinstance(tier, Mapping) and _tier_matches(tier, when):
|
||||||
|
rates = tier
|
||||||
|
tier_name = str(tier.get("name") or "time_tier")
|
||||||
|
break
|
||||||
|
|
||||||
|
currency = str(period.get("currency") or "CNY").upper()
|
||||||
|
if currency not in {"CNY", "USD"}:
|
||||||
|
raise ValueError(f"不支持的 pricing currency: {currency!r}")
|
||||||
|
fx = _decimal(period.get("fx_to_cny"), "1") if currency == "USD" else Decimal("1")
|
||||||
|
return PriceQuote(
|
||||||
|
revision=str(period.get("revision") or "unversioned"),
|
||||||
|
source_url=str(period.get("source_url") or ""),
|
||||||
|
currency=currency,
|
||||||
|
fx_to_cny=fx,
|
||||||
|
tier=tier_name,
|
||||||
|
input_per_mtoken=_decimal(rates.get("input_per_mtoken")),
|
||||||
|
output_per_mtoken=_decimal(rates.get("output_per_mtoken")),
|
||||||
|
cache_hit_per_mtoken=_decimal(
|
||||||
|
rates.get("cache_hit_per_mtoken"),
|
||||||
|
str(rates.get("input_per_mtoken") or 0),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def calculate_chat_cost(
|
||||||
|
quote: PriceQuote,
|
||||||
|
*,
|
||||||
|
prompt_tokens: int,
|
||||||
|
completion_tokens: int,
|
||||||
|
cache_hit_tokens: int = 0,
|
||||||
|
) -> CostBreakdown:
|
||||||
|
"""按价格快照拆分缓存命中、未命中和输出成本,返回人民币金额。"""
|
||||||
|
tokens_in = max(0, int(prompt_tokens))
|
||||||
|
hit = max(0, min(int(cache_hit_tokens), tokens_in))
|
||||||
|
miss = tokens_in - hit
|
||||||
|
tokens_out = max(0, int(completion_tokens))
|
||||||
|
input_cost = Decimal(miss) * quote.input_per_mtoken * quote.fx_to_cny / _MTOKEN
|
||||||
|
hit_cost = Decimal(hit) * quote.cache_hit_per_mtoken * quote.fx_to_cny / _MTOKEN
|
||||||
|
output_cost = Decimal(tokens_out) * quote.output_per_mtoken * quote.fx_to_cny / _MTOKEN
|
||||||
|
quant = Decimal("0.000001")
|
||||||
|
return CostBreakdown(
|
||||||
|
total_cny=(input_cost + hit_cost + output_cost).quantize(quant),
|
||||||
|
input_cny=input_cost.quantize(quant),
|
||||||
|
cache_hit_cny=hit_cost.quantize(quant),
|
||||||
|
output_cny=output_cost.quantize(quant),
|
||||||
|
cache_hit_tokens=hit,
|
||||||
|
cache_miss_tokens=miss,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def pricing_snapshot(quote: PriceQuote, breakdown: CostBreakdown) -> dict[str, Any]:
|
||||||
|
"""生成可直接合并进 usage_events.units 的价格快照。"""
|
||||||
|
return {
|
||||||
|
"pricing_revision": quote.revision,
|
||||||
|
"pricing_source": "local_catalog",
|
||||||
|
"pricing_source_url": quote.source_url,
|
||||||
|
"pricing_currency": quote.currency,
|
||||||
|
"price_tier": quote.tier,
|
||||||
|
"fx_to_cny": float(quote.fx_to_cny),
|
||||||
|
"input_price_per_mtoken": float(quote.input_per_mtoken),
|
||||||
|
"cache_hit_price_per_mtoken": float(quote.cache_hit_per_mtoken),
|
||||||
|
"output_price_per_mtoken": float(quote.output_per_mtoken),
|
||||||
|
"cache_hit_tokens": breakdown.cache_hit_tokens,
|
||||||
|
"cache_miss_tokens": breakdown.cache_miss_tokens,
|
||||||
|
"input_cost_cny": float(breakdown.input_cny),
|
||||||
|
"cache_hit_cost_cny": float(breakdown.cache_hit_cny),
|
||||||
|
"output_cost_cny": float(breakdown.output_cny),
|
||||||
|
}
|
||||||
111
core/procs.py
111
core/procs.py
|
|
@ -45,9 +45,6 @@ PROCS_SUBDIR = ".zcbot_procs"
|
||||||
DEFAULT_TIMEOUT_S = 7200
|
DEFAULT_TIMEOUT_S = 7200
|
||||||
MAX_TIMEOUT_S = 86400
|
MAX_TIMEOUT_S = 86400
|
||||||
|
|
||||||
# 每用户并发 bg 进程上限(跨 task 合计);防模型失控起一堆
|
|
||||||
MAX_RUNNING_PER_USER = int(os.getenv("ZCBOT_MAX_BG_PROCS", "3"))
|
|
||||||
|
|
||||||
# 终态 proc 目录保留时长,sweep 超期删除
|
# 终态 proc 目录保留时长,sweep 超期删除
|
||||||
FINISHED_TTL_S = 7 * 86400
|
FINISHED_TTL_S = 7 * 86400
|
||||||
|
|
||||||
|
|
@ -151,12 +148,14 @@ def _container_running(name: str) -> bool:
|
||||||
|
|
||||||
|
|
||||||
def status_of(meta: Dict[str, Any], d: Path) -> Tuple[str, Optional[int]]:
|
def status_of(meta: Dict[str, Any], d: Path) -> Tuple[str, Optional[int]]:
|
||||||
"""→ ("running"|"finished"|"lost", exit_code|None)。exit_code 文件是唯一终态锚。"""
|
"""→ queued/running/finished/lost。exit_code 文件是唯一终态锚。"""
|
||||||
try:
|
try:
|
||||||
raw = (d / "exit_code").read_text(encoding="utf-8").strip()
|
raw = (d / "exit_code").read_text(encoding="utf-8").strip()
|
||||||
return "finished", int(raw)
|
return "finished", int(raw)
|
||||||
except (OSError, ValueError):
|
except (OSError, ValueError):
|
||||||
pass
|
pass
|
||||||
|
if meta.get("state") == "queued":
|
||||||
|
return "queued", None
|
||||||
if meta.get("backend") == "docker":
|
if meta.get("backend") == "docker":
|
||||||
if _container_running(str(meta.get("container") or "")):
|
if _container_running(str(meta.get("container") or "")):
|
||||||
return "running", None
|
return "running", None
|
||||||
|
|
@ -304,10 +303,24 @@ def kill_proc(meta: Dict[str, Any], d: Path) -> str:
|
||||||
st, _ = status_of(meta, d)
|
st, _ = status_of(meta, d)
|
||||||
if st == "finished":
|
if st == "finished":
|
||||||
return "进程已结束,无需终止"
|
return "进程已结束,无需终止"
|
||||||
|
if st == "queued":
|
||||||
|
meta["state"] = "finished"
|
||||||
|
meta["killed"] = True
|
||||||
|
write_meta(d, meta)
|
||||||
|
(d / "exit_code").write_text("137", encoding="utf-8")
|
||||||
|
return "已取消排队"
|
||||||
if meta.get("backend") == "docker":
|
if meta.get("backend") == "docker":
|
||||||
name = str(meta.get("container") or "")
|
name = str(meta.get("container") or "")
|
||||||
if name:
|
if name:
|
||||||
|
_scan_proc_container(meta, d, name)
|
||||||
subprocess.run(["docker", "rm", "-f", name], capture_output=True, timeout=30)
|
subprocess.run(["docker", "rm", "-f", name], capture_output=True, timeout=30)
|
||||||
|
try:
|
||||||
|
from core.sandbox import get_pool
|
||||||
|
pool = get_pool()
|
||||||
|
if pool is not None:
|
||||||
|
pool.capacity.release(meta.get("capacity_lease_id"))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
else:
|
else:
|
||||||
wrapper_pid = int(meta.get("wrapper_pid") or 0)
|
wrapper_pid = int(meta.get("wrapper_pid") or 0)
|
||||||
child_pid = int(meta.get("child_pid") or 0)
|
child_pid = int(meta.get("child_pid") or 0)
|
||||||
|
|
@ -338,6 +351,69 @@ def kill_proc(meta: Dict[str, Any], d: Path) -> str:
|
||||||
return "已终止"
|
return "已终止"
|
||||||
|
|
||||||
|
|
||||||
|
def start_queued_docker(meta: Dict[str, Any], d: Path, pool) -> bool:
|
||||||
|
"""尝试为 queued proc 获取共享槽并启动;无槽返回 False,保持排队。"""
|
||||||
|
from core.file_store import interprocess_file_lock
|
||||||
|
with interprocess_file_lock(d / "dispatch.lock", timeout_seconds=None):
|
||||||
|
meta = read_meta(d) or meta
|
||||||
|
if status_of(meta, d)[0] != "queued":
|
||||||
|
return False
|
||||||
|
uid = str(meta.get("user_id") or "")
|
||||||
|
lease_id = f"bg-{meta.get('proc_id')}"
|
||||||
|
lease = pool.capacity.try_acquire(uid, "background", lease_id=lease_id,
|
||||||
|
proc_id=str(meta.get("proc_id")))
|
||||||
|
if not lease:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
from uuid import UUID
|
||||||
|
container = pool.run_proc_container(UUID(uid), str(meta["proc_id"]), str(d))
|
||||||
|
cdir = f"/workspace/{PROCS_SUBDIR}/{meta['task_id']}/{meta['proc_id']}"
|
||||||
|
argv = ["docker", "exec", "--user", str(meta.get("exec_user") or "zcbot"),
|
||||||
|
"--workdir", str(meta.get("cwd") or "/workspace"), "-d",
|
||||||
|
"-e", "PYTHONPATH=/sandbox:/workspace", "-e", "HOME=/tmp",
|
||||||
|
"-e", "PYTHONIOENCODING=utf-8", container, "bash", f"{cdir}/runner.sh"]
|
||||||
|
r = subprocess.run(argv, capture_output=True, text=True, timeout=60)
|
||||||
|
if r.returncode != 0:
|
||||||
|
subprocess.run(["docker", "rm", "-f", container], capture_output=True)
|
||||||
|
raise RuntimeError((r.stderr or "").strip()[:300])
|
||||||
|
meta.update({"container": container, "capacity_lease_id": lease,
|
||||||
|
"state": "running", "started_ts": time.time()})
|
||||||
|
write_meta(d, meta)
|
||||||
|
return True
|
||||||
|
except Exception as exc:
|
||||||
|
pool.capacity.release(lease)
|
||||||
|
meta["last_start_error"] = f"{type(exc).__name__}: {exc}"
|
||||||
|
write_meta(d, meta)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def dispatch_queued(user_root_base: Path, pool) -> int:
|
||||||
|
started = 0
|
||||||
|
base = Path(user_root_base)
|
||||||
|
if not base.is_dir():
|
||||||
|
return 0
|
||||||
|
candidates = []
|
||||||
|
running_ids: set[str] = set()
|
||||||
|
for uroot in base.iterdir():
|
||||||
|
root = uroot / PROCS_SUBDIR
|
||||||
|
if not root.is_dir():
|
||||||
|
continue
|
||||||
|
for meta_file in root.glob("*/*/proc.json"):
|
||||||
|
d = meta_file.parent
|
||||||
|
meta = read_meta(d)
|
||||||
|
if meta:
|
||||||
|
status = status_of(meta, d)[0]
|
||||||
|
if status == "queued":
|
||||||
|
candidates.append((float(meta.get("created_ts") or 0), meta, d))
|
||||||
|
elif status == "running":
|
||||||
|
running_ids.add(str(meta.get("proc_id") or d.name))
|
||||||
|
pool.capacity.reconcile_background(running_ids)
|
||||||
|
for _, meta, d in sorted(candidates, key=lambda x: x[0]):
|
||||||
|
if start_queued_docker(meta, d, pool):
|
||||||
|
started += 1
|
||||||
|
return started
|
||||||
|
|
||||||
|
|
||||||
# ───────────── 清扫(web lifespan 周期调用) ─────────────
|
# ───────────── 清扫(web lifespan 周期调用) ─────────────
|
||||||
|
|
||||||
def sweep(user_root_base: Path, ttl_s: int = FINISHED_TTL_S) -> Dict[str, int]:
|
def sweep(user_root_base: Path, ttl_s: int = FINISHED_TTL_S) -> Dict[str, int]:
|
||||||
|
|
@ -365,16 +441,27 @@ def sweep(user_root_base: Path, ttl_s: int = FINISHED_TTL_S) -> Dict[str, int]:
|
||||||
if not meta:
|
if not meta:
|
||||||
continue
|
continue
|
||||||
st, _ = status_of(meta, d)
|
st, _ = status_of(meta, d)
|
||||||
if st != "running" and meta.get("backend") == "docker":
|
if st in {"finished", "lost"} and meta.get("capacity_lease_id"):
|
||||||
|
try:
|
||||||
|
from core.sandbox import get_pool
|
||||||
|
pool = get_pool()
|
||||||
|
if pool is not None:
|
||||||
|
pool.capacity.release(meta.get("capacity_lease_id"))
|
||||||
|
meta.pop("capacity_lease_id", None)
|
||||||
|
write_meta(d, meta)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if st not in {"running", "queued"} and meta.get("backend") == "docker":
|
||||||
name = str(meta.get("container") or "")
|
name = str(meta.get("container") or "")
|
||||||
if name and _container_exists(name):
|
if name and _container_exists(name):
|
||||||
|
_scan_proc_container(meta, d, name)
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
["docker", "rm", "-f", name],
|
["docker", "rm", "-f", name],
|
||||||
capture_output=True, timeout=30,
|
capture_output=True, timeout=30,
|
||||||
)
|
)
|
||||||
reaped_containers += 1
|
reaped_containers += 1
|
||||||
age = now - float(meta.get("created_ts") or now)
|
age = now - float(meta.get("created_ts") or now)
|
||||||
if st != "running" and age > ttl_s:
|
if st not in {"running", "queued"} and age > ttl_s:
|
||||||
shutil.rmtree(d, ignore_errors=True)
|
shutil.rmtree(d, ignore_errors=True)
|
||||||
removed_dirs += 1
|
removed_dirs += 1
|
||||||
try:
|
try:
|
||||||
|
|
@ -410,6 +497,16 @@ def _container_exists(name: str) -> bool:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _scan_proc_container(meta: Dict[str, Any], d: Path, name: str) -> None:
|
||||||
|
try:
|
||||||
|
from uuid import UUID
|
||||||
|
from core.sandbox.package_scans import scan_and_persist
|
||||||
|
uid = UUID(str(meta.get("user_id")))
|
||||||
|
scan_and_persist(name, uid, "background", str(meta.get("proc_id") or d.name))
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"[sandbox-package-scan] proc finalizer failed container={name}: {type(exc).__name__}: {exc}")
|
||||||
|
|
||||||
|
|
||||||
# ───────────── LLM 面向的描述格式化(check_process / 启动返回共用) ─────────────
|
# ───────────── LLM 面向的描述格式化(check_process / 启动返回共用) ─────────────
|
||||||
|
|
||||||
def format_proc_line(meta: Dict[str, Any]) -> str:
|
def format_proc_line(meta: Dict[str, Any]) -> str:
|
||||||
|
|
@ -418,7 +515,7 @@ def format_proc_line(meta: Dict[str, Any]) -> str:
|
||||||
elapsed = ""
|
elapsed = ""
|
||||||
ts = meta.get("created_ts")
|
ts = meta.get("created_ts")
|
||||||
if ts:
|
if ts:
|
||||||
elapsed = f" · 已运行 {_fmt_elapsed(time.time() - float(ts))}" if st == "running" else ""
|
elapsed = f" · 已运行 {_fmt_elapsed(time.time() - float(meta.get('started_ts') or ts))}" if st == "running" else ""
|
||||||
tail = f"(exit {ec})" if st == "finished" else ""
|
tail = f"(exit {ec})" if st == "finished" else ""
|
||||||
return (
|
return (
|
||||||
f"- {meta.get('proc_id')} [{st}{tail}] {meta.get('kind')}: "
|
f"- {meta.get('proc_id')} [{st}{tail}] {meta.get('kind')}: "
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,5 @@
|
||||||
|
"""Admin 管理的平台 Provider 凭据控制面。"""
|
||||||
|
|
||||||
|
from .runtime import resolve_credentials, resolve_secret
|
||||||
|
|
||||||
|
__all__ = ["resolve_credentials", "resolve_secret"]
|
||||||
|
|
@ -0,0 +1,50 @@
|
||||||
|
"""蓝绿单轮选主的 Provider 巡检与开发者邮件。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from core.storage import get_engine
|
||||||
|
|
||||||
|
from .registry import get_provider
|
||||||
|
from .service import due_provider_ids, test_current_credentials
|
||||||
|
from .testing import TestResult
|
||||||
|
|
||||||
|
_LOCK_SQL = "SELECT pg_try_advisory_lock(31331, 2)"
|
||||||
|
_UNLOCK_SQL = "SELECT pg_advisory_unlock(31331, 2)"
|
||||||
|
|
||||||
|
|
||||||
|
def send_notification(provider_id: str, event: str, result: TestResult) -> None:
|
||||||
|
provider = get_provider(provider_id)
|
||||||
|
label = "恢复正常" if event == "recovered" else result.detail
|
||||||
|
print(f"[provider] {provider_id} {event}: {label}")
|
||||||
|
email = os.getenv("ZCBOT_DEVELOPER_EMAIL", "").strip()
|
||||||
|
if not email:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
from tools.send_email import send_email_smtp, smtp_configured
|
||||||
|
if smtp_configured():
|
||||||
|
send_email_smtp(
|
||||||
|
email, f"[zcbot] {provider.display_name} {label}",
|
||||||
|
f"Provider: {provider.display_name}\n状态: {event}\n详情: {result.detail}",
|
||||||
|
)
|
||||||
|
except Exception as exc: # noqa: BLE001 - 告警失败不得影响业务状态
|
||||||
|
print(f"[provider] notification failed: {type(exc).__name__}")
|
||||||
|
|
||||||
|
|
||||||
|
def run_due_checks() -> int:
|
||||||
|
engine = get_engine()
|
||||||
|
with engine.connect() as connection:
|
||||||
|
claimed = bool(connection.exec_driver_sql(_LOCK_SQL).scalar())
|
||||||
|
if not claimed:
|
||||||
|
return 0
|
||||||
|
try:
|
||||||
|
count = 0
|
||||||
|
for provider_id in due_provider_ids():
|
||||||
|
try:
|
||||||
|
test_current_credentials(provider_id, notify=send_notification)
|
||||||
|
count += 1
|
||||||
|
except Exception as exc: # noqa: BLE001 - 单 Provider 隔离
|
||||||
|
print(f"[provider] {provider_id} check failed: {type(exc).__name__}")
|
||||||
|
return count
|
||||||
|
finally:
|
||||||
|
connection.exec_driver_sql(_UNLOCK_SQL)
|
||||||
|
|
@ -0,0 +1,83 @@
|
||||||
|
"""静态可信 Provider 注册表;网络目标和协议不可由 Admin 修改。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CredentialField:
|
||||||
|
name: str
|
||||||
|
label: str
|
||||||
|
env: str
|
||||||
|
secret: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProviderDefinition:
|
||||||
|
provider_id: str
|
||||||
|
display_name: str
|
||||||
|
category: str
|
||||||
|
fields: tuple[CredentialField, ...]
|
||||||
|
test_kind: str
|
||||||
|
test_url: str = ""
|
||||||
|
test_url_env: str = ""
|
||||||
|
billable: bool = False
|
||||||
|
balance_supported: bool = False
|
||||||
|
check_interval_seconds: int = 86400
|
||||||
|
low_balance_threshold: float | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def _key(env: str, label: str = "API Key") -> CredentialField:
|
||||||
|
return CredentialField("api_key", label, env)
|
||||||
|
|
||||||
|
|
||||||
|
PROVIDERS: tuple[ProviderDefinition, ...] = (
|
||||||
|
ProviderDefinition("deepseek", "DeepSeek", "模型", (_key("DEEPSEEK_API_KEY"),),
|
||||||
|
"deepseek_balance", "https://api.deepseek.com/user/balance",
|
||||||
|
balance_supported=True, check_interval_seconds=1800,
|
||||||
|
low_balance_threshold=30.0),
|
||||||
|
ProviderDefinition("zhipuai", "智谱开放平台", "模型", (_key("ZHIPUAI_API_KEY"),),
|
||||||
|
"bearer_get", "https://open.bigmodel.cn/api/paas/v4/models"),
|
||||||
|
ProviderDefinition("ark", "火山方舟", "模型与媒体", (_key("ARK_API_KEY"),),
|
||||||
|
"bearer_get", "https://ark.cn-beijing.volces.com/api/v3/models"),
|
||||||
|
ProviderDefinition("unifyllm", "国际旗舰模型网关", "模型与媒体", (_key("UNIFYLLM_API_KEY"),),
|
||||||
|
"bearer_get", "https://unifyllm.ai/v1/models"),
|
||||||
|
ProviderDefinition("local_llm", "内网本地模型", "模型", (_key("LOCAL_LLM_API_KEY"),),
|
||||||
|
"bearer_get", "http://182.54.21.126:9000/v1/models"),
|
||||||
|
ProviderDefinition("bocha", "博查搜索", "搜索与平台数据", (_key("BOCHA_API_KEY"),),
|
||||||
|
"bocha_search", "https://api.bochaai.com/v1/web-search", billable=True),
|
||||||
|
ProviderDefinition("document_search", "内部材料库", "搜索与平台数据", (_key("DOCUMENT_SEARCH_API_KEY"),),
|
||||||
|
"bearer_get", "https://ai.ctc-zc.com:8100/api/document_search/list_knowledge_bases",
|
||||||
|
test_url_env="DOCUMENT_SEARCH_URL"),
|
||||||
|
ProviderDefinition("paper_server", "论文服务", "搜索与平台数据", (_key("PAPER_SERVER_API_KEY"),),
|
||||||
|
"query_get", "http://paper.xxhhcty.xyz:8080/api/resm/paper/",
|
||||||
|
test_url_env="PAPER_SERVER_URL"),
|
||||||
|
ProviderDefinition("materials_project", "Materials Project", "搜索与平台数据", (_key("MP_API_KEY"),),
|
||||||
|
"mp_get", "https://api.materialsproject.org/materials/summary/?_limit=1"),
|
||||||
|
ProviderDefinition(
|
||||||
|
"xfyun_iat", "讯飞语音听写 IAT", "语音",
|
||||||
|
(CredentialField("appid", "APPID", "XFYUN_APPID"),
|
||||||
|
CredentialField("api_key", "API Key", "XFYUN_API_KEY"),
|
||||||
|
CredentialField("api_secret", "API Secret", "XFYUN_API_SECRET")),
|
||||||
|
"xfyun_iat", billable=True,
|
||||||
|
),
|
||||||
|
ProviderDefinition(
|
||||||
|
"xfyun_lfasr", "讯飞录音转写 LFASR", "语音",
|
||||||
|
(CredentialField("appid", "APPID", "XFYUN_APPID"),
|
||||||
|
CredentialField("secret_key", "Secret Key", "XFYUN_LFASR_SECRET_KEY")),
|
||||||
|
"xfyun_lfasr", "https://raasr.xfyun.cn/v2/api/getResult", billable=False,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
BY_ID = {provider.provider_id: provider for provider in PROVIDERS}
|
||||||
|
BY_ENV = {
|
||||||
|
field.env: (provider.provider_id, field.name)
|
||||||
|
for provider in PROVIDERS for field in provider.fields
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_provider(provider_id: str) -> ProviderDefinition:
|
||||||
|
try:
|
||||||
|
return BY_ID[provider_id]
|
||||||
|
except KeyError as exc:
|
||||||
|
raise ValueError("unknown provider") from exc
|
||||||
|
|
@ -0,0 +1,73 @@
|
||||||
|
"""每个新外部请求调用一次的数据库优先/env fallback 凭据解析器。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.exc import SQLAlchemyError
|
||||||
|
|
||||||
|
from core.external_systems.crypto import decrypt_secret
|
||||||
|
|
||||||
|
from .registry import BY_ENV, get_provider
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ResolvedCredentials:
|
||||||
|
provider_id: str
|
||||||
|
source: str
|
||||||
|
values: dict[str, str]
|
||||||
|
|
||||||
|
|
||||||
|
def _env_values(provider_id: str) -> dict[str, str]:
|
||||||
|
provider = get_provider(provider_id)
|
||||||
|
return {
|
||||||
|
field.name: value
|
||||||
|
for field in provider.fields
|
||||||
|
if (value := (os.getenv(field.env) or "").strip())
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_credentials(provider_id: str) -> ResolvedCredentials:
|
||||||
|
provider = get_provider(provider_id)
|
||||||
|
row = None
|
||||||
|
try:
|
||||||
|
from core.storage import session_scope
|
||||||
|
from core.storage.models import ProviderCredential
|
||||||
|
with session_scope() as session:
|
||||||
|
row = session.execute(
|
||||||
|
select(ProviderCredential).where(
|
||||||
|
ProviderCredential.provider_id == provider_id
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
except (RuntimeError, SQLAlchemyError):
|
||||||
|
# CLI、migration 前部署或 DB 短暂不可用时保持历史 env 行为。
|
||||||
|
pass
|
||||||
|
if row is not None and all(field.name in row.credentials for field in provider.fields):
|
||||||
|
# 已存在完整 DB 覆盖时,master key/AAD 错误必须显式失败,不能静默绕回 env。
|
||||||
|
values = {
|
||||||
|
field.name: decrypt_secret(
|
||||||
|
row.credentials[field.name], aad=f"provider:{provider_id}:{field.name}"
|
||||||
|
)
|
||||||
|
for field in provider.fields
|
||||||
|
}
|
||||||
|
return ResolvedCredentials(provider_id, "database", values)
|
||||||
|
values = _env_values(provider_id)
|
||||||
|
source = "env" if len(values) == len(provider.fields) else "missing"
|
||||||
|
return ResolvedCredentials(provider_id, source, values)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_secret(provider_id: str, field: str = "api_key") -> str:
|
||||||
|
return resolve_credentials(provider_id).values.get(field, "")
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_env_secret(env_name: str) -> str:
|
||||||
|
binding = BY_ENV.get(env_name)
|
||||||
|
if binding is None:
|
||||||
|
return (os.getenv(env_name) or "").strip()
|
||||||
|
return resolve_secret(*binding)
|
||||||
|
|
||||||
|
|
||||||
|
def provider_available(provider_id: str) -> bool:
|
||||||
|
provider = get_provider(provider_id)
|
||||||
|
return len(resolve_credentials(provider_id).values) == len(provider.fields)
|
||||||
|
|
@ -0,0 +1,263 @@
|
||||||
|
"""Provider 凭据 CRUD、乐观替换、状态持久化与定时检查。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Callable
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from sqlalchemy import delete, select, update
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from core.external_systems.crypto import configured as crypto_configured
|
||||||
|
from core.external_systems.crypto import encrypt_secret
|
||||||
|
from core.storage import session_scope
|
||||||
|
from core.storage.models import ProviderCredential
|
||||||
|
|
||||||
|
from .registry import PROVIDERS, get_provider
|
||||||
|
from .runtime import _env_values, resolve_credentials
|
||||||
|
from .testing import TestResult, classify_response, test_provider
|
||||||
|
|
||||||
|
_BAD = {"low_balance", "exhausted", "auth_error", "unreachable"}
|
||||||
|
_NOTIFY_COOLDOWN = timedelta(hours=24)
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderCredentialError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class RevisionConflict(ProviderCredentialError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _hint(value: str) -> str:
|
||||||
|
value = str(value or "")
|
||||||
|
return f"***{value[-4:]}" if len(value) >= 4 else "***"
|
||||||
|
|
||||||
|
|
||||||
|
def _row_payload(provider, row: ProviderCredential | None) -> dict:
|
||||||
|
database = bool(row and all(field.name in row.credentials for field in provider.fields))
|
||||||
|
env_values = _env_values(provider.provider_id)
|
||||||
|
env_source = "env" if len(env_values) == len(provider.fields) else "missing"
|
||||||
|
source = "database" if database else env_source
|
||||||
|
hints = row.credential_hint if database else {
|
||||||
|
field.name: _hint(env_values.get(field.name, ""))
|
||||||
|
for field in provider.fields if env_values.get(field.name)
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"provider_id": provider.provider_id,
|
||||||
|
"display_name": provider.display_name,
|
||||||
|
"category": provider.category,
|
||||||
|
"fields": [
|
||||||
|
{"name": field.name, "label": field.label, "required": True,
|
||||||
|
"hint": hints.get(field.name, "")}
|
||||||
|
for field in provider.fields
|
||||||
|
],
|
||||||
|
"configured": source != "missing",
|
||||||
|
"source": source,
|
||||||
|
"revision": row.revision if row else 0,
|
||||||
|
"test_status": row.test_status if row else "untested",
|
||||||
|
"test_detail": row.test_detail if row else None,
|
||||||
|
"last_tested_at": row.last_tested_at.isoformat() if row and row.last_tested_at else None,
|
||||||
|
"balance": (
|
||||||
|
{"amount": str(row.balance_amount), "currency": row.balance_currency}
|
||||||
|
if row and row.balance_amount is not None else None
|
||||||
|
),
|
||||||
|
"balance_supported": provider.balance_supported,
|
||||||
|
"billable_test": provider.billable,
|
||||||
|
"alerting": bool(row and row.test_status in _BAD),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def list_providers() -> list[dict]:
|
||||||
|
with session_scope() as session:
|
||||||
|
rows = {
|
||||||
|
row.provider_id: row
|
||||||
|
for row in session.execute(select(ProviderCredential)).scalars()
|
||||||
|
}
|
||||||
|
return [_row_payload(provider, rows.get(provider.provider_id)) for provider in PROVIDERS]
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_values(provider_id: str, values: dict[str, str]) -> dict[str, str]:
|
||||||
|
provider = get_provider(provider_id)
|
||||||
|
expected = {field.name for field in provider.fields}
|
||||||
|
cleaned = {str(k): str(v).strip() for k, v in values.items()}
|
||||||
|
if set(cleaned) != expected or any(not value for value in cleaned.values()):
|
||||||
|
raise ProviderCredentialError("凭据字段不完整或包含未知字段")
|
||||||
|
return cleaned
|
||||||
|
|
||||||
|
|
||||||
|
def _notification_transition(
|
||||||
|
old_status: str, old_notified_at: datetime | None, result: TestResult, now: datetime
|
||||||
|
) -> str | None:
|
||||||
|
if result.status == "normal" and old_status in _BAD:
|
||||||
|
return "recovered"
|
||||||
|
if result.status not in _BAD:
|
||||||
|
return None
|
||||||
|
if old_status not in _BAD or old_status != result.status:
|
||||||
|
return result.status
|
||||||
|
if old_notified_at is None or now - old_notified_at >= _NOTIFY_COOLDOWN:
|
||||||
|
return result.status
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_result(
|
||||||
|
provider_id: str,
|
||||||
|
result: TestResult,
|
||||||
|
*,
|
||||||
|
notify: Callable[[str, str, TestResult], None] | None = None,
|
||||||
|
) -> None:
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
event: str | None = None
|
||||||
|
with session_scope() as session:
|
||||||
|
row = session.execute(
|
||||||
|
select(ProviderCredential).where(ProviderCredential.provider_id == provider_id)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if row is None:
|
||||||
|
row = ProviderCredential(
|
||||||
|
provider_id=provider_id, credentials={}, credential_hint={}, revision=1
|
||||||
|
)
|
||||||
|
session.add(row)
|
||||||
|
session.flush()
|
||||||
|
old_status = row.test_status
|
||||||
|
event = _notification_transition(old_status, row.last_notified_at, result, now)
|
||||||
|
row.test_status = result.status
|
||||||
|
row.test_detail = result.detail[:500]
|
||||||
|
row.balance_amount = result.balance_amount
|
||||||
|
row.balance_currency = result.balance_currency
|
||||||
|
row.last_tested_at = now
|
||||||
|
if event == "recovered":
|
||||||
|
row.last_notified_at = None
|
||||||
|
row.last_notified_status = None
|
||||||
|
elif event:
|
||||||
|
row.last_notified_at = now
|
||||||
|
row.last_notified_status = result.status
|
||||||
|
if event and notify:
|
||||||
|
notify(provider_id, event, result)
|
||||||
|
|
||||||
|
|
||||||
|
def replace_credentials(
|
||||||
|
provider_id: str,
|
||||||
|
values: dict[str, str],
|
||||||
|
*,
|
||||||
|
expected_revision: int,
|
||||||
|
updated_by: UUID,
|
||||||
|
request=None,
|
||||||
|
notify: Callable[[str, str, TestResult], None] | None = None,
|
||||||
|
) -> dict:
|
||||||
|
if not crypto_configured():
|
||||||
|
raise ProviderCredentialError(
|
||||||
|
"未配置 ZCBOT_CREDENTIAL_MASTER_KEY,禁止保存数据库凭据"
|
||||||
|
)
|
||||||
|
provider = get_provider(provider_id)
|
||||||
|
cleaned = _validate_values(provider_id, values)
|
||||||
|
kwargs = {"request": request} if request is not None else {}
|
||||||
|
result = test_provider(provider_id, cleaned, **kwargs)
|
||||||
|
if not result.accepted:
|
||||||
|
raise ProviderCredentialError(f"候选凭据测试失败:{result.detail}")
|
||||||
|
encrypted = {
|
||||||
|
field.name: encrypt_secret(
|
||||||
|
cleaned[field.name], aad=f"provider:{provider_id}:{field.name}"
|
||||||
|
)
|
||||||
|
for field in provider.fields
|
||||||
|
}
|
||||||
|
hints = {field.name: _hint(cleaned[field.name]) for field in provider.fields}
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
try:
|
||||||
|
with session_scope() as session:
|
||||||
|
if expected_revision == 0:
|
||||||
|
session.add(ProviderCredential(
|
||||||
|
provider_id=provider_id, credentials=encrypted,
|
||||||
|
credential_hint=hints, revision=1, test_status=result.status,
|
||||||
|
test_detail=result.detail, balance_amount=result.balance_amount,
|
||||||
|
balance_currency=result.balance_currency, last_tested_at=now,
|
||||||
|
updated_by=updated_by,
|
||||||
|
))
|
||||||
|
revision = 1
|
||||||
|
else:
|
||||||
|
changed = session.execute(
|
||||||
|
update(ProviderCredential)
|
||||||
|
.where(ProviderCredential.provider_id == provider_id)
|
||||||
|
.where(ProviderCredential.revision == expected_revision)
|
||||||
|
.values(
|
||||||
|
credentials=encrypted, credential_hint=hints,
|
||||||
|
revision=expected_revision + 1, test_status=result.status,
|
||||||
|
test_detail=result.detail, balance_amount=result.balance_amount,
|
||||||
|
balance_currency=result.balance_currency, last_tested_at=now,
|
||||||
|
updated_by=updated_by, updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if int(changed.rowcount or 0) != 1:
|
||||||
|
raise RevisionConflict("凭据已被其他管理员更新,请刷新后重试")
|
||||||
|
revision = expected_revision + 1
|
||||||
|
except IntegrityError as exc:
|
||||||
|
raise RevisionConflict("凭据已被其他管理员更新,请刷新后重试") from exc
|
||||||
|
if result.status == "low_balance" and notify:
|
||||||
|
notify(provider_id, result.status, result)
|
||||||
|
with session_scope() as session:
|
||||||
|
session.execute(
|
||||||
|
update(ProviderCredential)
|
||||||
|
.where(ProviderCredential.provider_id == provider_id)
|
||||||
|
.where(ProviderCredential.revision == revision)
|
||||||
|
.values(last_notified_at=now, last_notified_status=result.status)
|
||||||
|
)
|
||||||
|
return {"provider_id": provider_id, "revision": revision,
|
||||||
|
"test_status": result.status, "test_detail": result.detail}
|
||||||
|
|
||||||
|
|
||||||
|
def test_current_credentials(
|
||||||
|
provider_id: str, *, request=None,
|
||||||
|
notify: Callable[[str, str, TestResult], None] | None = None,
|
||||||
|
) -> TestResult:
|
||||||
|
resolved = resolve_credentials(provider_id)
|
||||||
|
provider = get_provider(provider_id)
|
||||||
|
if len(resolved.values) != len(provider.fields):
|
||||||
|
raise ProviderCredentialError("Provider 凭据未完整配置")
|
||||||
|
kwargs = {"request": request} if request is not None else {}
|
||||||
|
result = test_provider(provider_id, resolved.values, **kwargs)
|
||||||
|
_apply_result(provider_id, result, notify=notify)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def delete_override(provider_id: str, *, expected_revision: int) -> None:
|
||||||
|
get_provider(provider_id)
|
||||||
|
with session_scope() as session:
|
||||||
|
changed = session.execute(
|
||||||
|
delete(ProviderCredential)
|
||||||
|
.where(ProviderCredential.provider_id == provider_id)
|
||||||
|
.where(ProviderCredential.revision == expected_revision)
|
||||||
|
)
|
||||||
|
if int(changed.rowcount or 0) != 1:
|
||||||
|
raise RevisionConflict("凭据已被更新或不存在,请刷新后重试")
|
||||||
|
|
||||||
|
|
||||||
|
def record_business_failure(
|
||||||
|
provider_id: str, *, status_code: int = 0, detail: str = "",
|
||||||
|
notify: Callable[[str, str, TestResult], None] | None = None,
|
||||||
|
) -> str | None:
|
||||||
|
result = classify_response(status_code, detail)
|
||||||
|
if result.status not in {"auth_error", "exhausted"}:
|
||||||
|
return None
|
||||||
|
if notify is None:
|
||||||
|
from .monitor import send_notification
|
||||||
|
notify = send_notification
|
||||||
|
_apply_result(provider_id, result, notify=notify)
|
||||||
|
return result.status
|
||||||
|
|
||||||
|
|
||||||
|
def due_provider_ids(now: datetime | None = None) -> list[str]:
|
||||||
|
now = now or datetime.now(timezone.utc)
|
||||||
|
with session_scope() as session:
|
||||||
|
rows = {
|
||||||
|
row.provider_id: row for row in session.execute(select(ProviderCredential)).scalars()
|
||||||
|
}
|
||||||
|
due = []
|
||||||
|
for provider in PROVIDERS:
|
||||||
|
resolved = resolve_credentials(provider.provider_id)
|
||||||
|
if len(resolved.values) != len(provider.fields):
|
||||||
|
continue
|
||||||
|
row = rows.get(provider.provider_id)
|
||||||
|
if row is None or row.last_tested_at is None or (
|
||||||
|
now - row.last_tested_at >= timedelta(seconds=provider.check_interval_seconds)
|
||||||
|
):
|
||||||
|
due.append(provider.provider_id)
|
||||||
|
return due
|
||||||
|
|
@ -0,0 +1,172 @@
|
||||||
|
"""Provider 候选凭据测试与统一、保守的错误分类。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import hmac
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import time
|
||||||
|
from collections.abc import Callable
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from decimal import Decimal, InvalidOperation
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from .registry import ProviderDefinition, get_provider
|
||||||
|
|
||||||
|
_EXHAUSTED_RE = re.compile(
|
||||||
|
r"insufficient[ _-]?(?:balance|quota|credit)|balance[ _-]?not[ _-]?enough|余额不足|额度不足|quota exhausted",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_AUTH_RE = re.compile(
|
||||||
|
r"invalid.*(?:key|token)|authentication|unauthori[sz]ed|鉴权|认证失败",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TestResult:
|
||||||
|
status: str
|
||||||
|
detail: str
|
||||||
|
balance_amount: Decimal | None = None
|
||||||
|
balance_currency: str | None = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def accepted(self) -> bool:
|
||||||
|
return self.status in {"normal", "low_balance"}
|
||||||
|
|
||||||
|
|
||||||
|
def classify_response(status_code: int, text: str = "") -> TestResult:
|
||||||
|
safe = re.sub(r"\s+", " ", text or "").strip()[:300]
|
||||||
|
if status_code in {401, 403} or _AUTH_RE.search(safe):
|
||||||
|
return TestResult("auth_error", f"认证失败(HTTP {status_code})")
|
||||||
|
if status_code == 402 or _EXHAUSTED_RE.search(safe):
|
||||||
|
return TestResult("exhausted", f"余额或额度已耗尽(HTTP {status_code})")
|
||||||
|
if status_code == 429:
|
||||||
|
return TestResult("unreachable", "服务限流(HTTP 429),未判定为余额耗尽")
|
||||||
|
if status_code >= 400:
|
||||||
|
return TestResult("unreachable", f"服务返回 HTTP {status_code}")
|
||||||
|
return TestResult("normal", "认证与连通性正常")
|
||||||
|
|
||||||
|
|
||||||
|
def _deepseek(response: httpx.Response, provider: ProviderDefinition) -> TestResult:
|
||||||
|
base = classify_response(response.status_code, response.text)
|
||||||
|
if base.status != "normal":
|
||||||
|
return base
|
||||||
|
try:
|
||||||
|
body = response.json()
|
||||||
|
infos = body.get("balance_infos") or []
|
||||||
|
cny = next(x for x in infos if str(x.get("currency", "")).upper() == "CNY")
|
||||||
|
amount = Decimal(str(cny.get("total_balance")))
|
||||||
|
except (ValueError, KeyError, StopIteration, InvalidOperation, TypeError):
|
||||||
|
return TestResult("unreachable", "余额响应格式异常")
|
||||||
|
if body.get("is_available") is False or amount <= 0:
|
||||||
|
return TestResult("exhausted", "余额已耗尽", amount, "CNY")
|
||||||
|
threshold = Decimal(str(provider.low_balance_threshold or 0))
|
||||||
|
status = "low_balance" if amount < threshold else "normal"
|
||||||
|
detail = f"CNY 可用余额 ¥{amount:.2f}"
|
||||||
|
return TestResult(status, detail, amount, "CNY")
|
||||||
|
|
||||||
|
|
||||||
|
def _xfyun_sign(credentials: dict[str, str]) -> dict[str, str]:
|
||||||
|
ts = str(int(time.time()))
|
||||||
|
md5hex = hashlib.md5(
|
||||||
|
(credentials["appid"] + ts).encode(), usedforsecurity=False
|
||||||
|
).hexdigest()
|
||||||
|
signa = base64.b64encode(
|
||||||
|
hmac.new(credentials["secret_key"].encode(), md5hex.encode(), hashlib.sha1).digest()
|
||||||
|
).decode()
|
||||||
|
return {"appId": credentials["appid"], "ts": ts, "signa": signa,
|
||||||
|
"orderId": "zcbot-credential-check", "resultType": "transfer"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_provider(
|
||||||
|
provider_id: str,
|
||||||
|
credentials: dict[str, str],
|
||||||
|
*,
|
||||||
|
request: Callable[..., httpx.Response] = httpx.request,
|
||||||
|
) -> TestResult:
|
||||||
|
provider = get_provider(provider_id)
|
||||||
|
expected = {field.name for field in provider.fields}
|
||||||
|
if set(credentials) != expected or any(not str(v).strip() for v in credentials.values()):
|
||||||
|
return TestResult("auth_error", "凭据字段不完整")
|
||||||
|
headers: dict[str, str] = {}
|
||||||
|
params: dict[str, str] = {}
|
||||||
|
json_body: dict[str, Any] | None = None
|
||||||
|
method = "GET"
|
||||||
|
test_url = provider.test_url
|
||||||
|
if provider.test_url_env:
|
||||||
|
base = (os.getenv(provider.test_url_env) or "").strip().rstrip("/")
|
||||||
|
if base and provider_id == "document_search":
|
||||||
|
test_url = f"{base}/document_search/list_knowledge_bases"
|
||||||
|
elif base and provider_id == "paper_server":
|
||||||
|
test_url = f"{base}/api/resm/paper/"
|
||||||
|
if provider.test_kind in {"deepseek_balance", "bearer_get", "bocha_search"}:
|
||||||
|
headers["Authorization"] = f"Bearer {credentials['api_key']}"
|
||||||
|
if provider.test_kind == "query_get":
|
||||||
|
params["api_key"] = credentials["api_key"]
|
||||||
|
params["page_size"] = "1"
|
||||||
|
elif provider.test_kind == "mp_get":
|
||||||
|
headers["X-API-KEY"] = credentials["api_key"]
|
||||||
|
elif provider.test_kind == "bocha_search":
|
||||||
|
method = "POST"
|
||||||
|
json_body = {"query": "水泥", "count": 1, "freshness": "noLimit"}
|
||||||
|
elif provider.test_kind == "xfyun_lfasr":
|
||||||
|
method = "POST"
|
||||||
|
params.update(_xfyun_sign(credentials))
|
||||||
|
elif provider.test_kind == "xfyun_iat":
|
||||||
|
# IAT 鉴权只存在于 WebSocket upgrade;复用官方签名函数并允许测试替换
|
||||||
|
# requester。200/101 均视为握手成功,真实 handler 不发送音频、不计费。
|
||||||
|
from core.asr_xfyun import build_auth_url
|
||||||
|
url = build_auth_url(credentials["api_key"], credentials["api_secret"])
|
||||||
|
if request is httpx.request:
|
||||||
|
try:
|
||||||
|
from websockets.sync.client import connect
|
||||||
|
with connect(url, open_timeout=8) as websocket:
|
||||||
|
websocket.send(json.dumps({
|
||||||
|
"common": {"app_id": credentials["appid"]},
|
||||||
|
"business": {"language": "zh_cn", "domain": "iat", "accent": "mandarin"},
|
||||||
|
"data": {"status": 2, "format": "audio/L16;rate=16000",
|
||||||
|
"encoding": "raw", "audio": ""},
|
||||||
|
}))
|
||||||
|
body = json.loads(websocket.recv(timeout=8))
|
||||||
|
if int(body.get("code") or 0) == 0:
|
||||||
|
return TestResult("normal", "WebSocket 凭据组鉴权正常")
|
||||||
|
return TestResult("auth_error", "WebSocket 凭据组认证失败")
|
||||||
|
except Exception as exc: # noqa: BLE001 - WebSocket 库异常族随版本变化
|
||||||
|
text = str(exc)
|
||||||
|
status = int(getattr(exc, "status_code", 0) or 0)
|
||||||
|
return classify_response(status, text) if status else TestResult(
|
||||||
|
"unreachable", "WebSocket 连接失败"
|
||||||
|
)
|
||||||
|
response = request("GET", url, headers={"X-Appid": credentials["appid"]}, timeout=8)
|
||||||
|
return classify_response(response.status_code, response.text)
|
||||||
|
try:
|
||||||
|
response = request(
|
||||||
|
method, test_url, headers=headers, params=params,
|
||||||
|
json=json_body, timeout=8,
|
||||||
|
)
|
||||||
|
except httpx.HTTPError:
|
||||||
|
return TestResult("unreachable", "网络连接失败")
|
||||||
|
if provider.test_kind == "deepseek_balance":
|
||||||
|
return _deepseek(response, provider)
|
||||||
|
if provider.test_kind == "xfyun_lfasr" and response.status_code < 400:
|
||||||
|
try:
|
||||||
|
body = response.json()
|
||||||
|
code = str(body.get("code") or "")
|
||||||
|
description = str(body.get("descInfo") or "")
|
||||||
|
if code in {"10105", "10106", "10107"}:
|
||||||
|
return TestResult("auth_error", "认证失败")
|
||||||
|
# 虚拟订单的“订单不存在”说明签名已通过。
|
||||||
|
if code and re.search(
|
||||||
|
r"order|订单.*(?:不存在|无效)", description, re.IGNORECASE
|
||||||
|
):
|
||||||
|
return TestResult("normal", "认证正常(未创建计费订单)")
|
||||||
|
if code:
|
||||||
|
return TestResult("unreachable", "服务未确认凭据有效")
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
pass
|
||||||
|
return classify_response(response.status_code, response.text)
|
||||||
|
|
@ -22,22 +22,45 @@ def salvage_tool_arguments(raw: str, allowed_keys: set) -> Optional[dict]:
|
||||||
(a) parse-to-end 成功(json.loads 要求整个后缀是合法 JSON,尾部有残渣就失败);
|
(a) parse-to-end 成功(json.loads 要求整个后缀是合法 JSON,尾部有残渣就失败);
|
||||||
(b) 结果是非空 dict;
|
(b) 结果是非空 dict;
|
||||||
(c) 顶层 key ⊆ 该工具 schema 的参数名。
|
(c) 顶层 key ⊆ 该工具 schema 的参数名。
|
||||||
双护栏缺一不可:(a) 挡尾部被截断的半个 JSON(截断则拒绝、回落既有重试,不冒险执行
|
双护栏缺一不可:(a) 挡尾部被截断的半个 JSON;唯一例外是 wire 已给出至少两个
|
||||||
残缺写入);(c) 挡垃圾前缀里恰好自洽的旁支 JSON(如模型正文里引用的 `{...}` 片段)。
|
完全一致的完整参数对象、随后又粘了截断副本,此时一致副本可作为相互校验;
|
||||||
|
(c) 挡垃圾前缀里恰好自洽的旁支 JSON(如模型正文里引用的 `{...}` 片段)。
|
||||||
都不满足返回 None。纯函数、无副作用,便于单测(线上真前缀样本当夹具)。
|
都不满足返回 None。纯函数、无副作用,便于单测(线上真前缀样本当夹具)。
|
||||||
"""
|
"""
|
||||||
# 只可能出现在 '{' 处;非 '{' 直接跳过,省下大部分 json.loads 尝试。
|
# 只可能出现在 '{' 处;非 '{' 直接跳过,省下大部分 json.loads 尝试。
|
||||||
start = 0
|
start = 0
|
||||||
|
decoder = json.JSONDecoder()
|
||||||
|
repeated_candidates: list[dict] = []
|
||||||
while True:
|
while True:
|
||||||
i = raw.find("{", start)
|
i = raw.find("{", start)
|
||||||
if i < 0:
|
if i < 0:
|
||||||
return None
|
break
|
||||||
try:
|
try:
|
||||||
obj = json.loads(raw[i:])
|
obj = json.loads(raw[i:])
|
||||||
except (json.JSONDecodeError, ValueError):
|
except (json.JSONDecodeError, ValueError):
|
||||||
|
# 某些 wire 抖动会形成 ``{完整参数}{完整参数}{截断副本``。
|
||||||
|
# parse-to-end 无法处理,但 raw_decode 仍能取出前面的完整对象。
|
||||||
|
# 这里只收集候选,最终必须至少两个且完全一致才可执行。
|
||||||
|
try:
|
||||||
|
partial_obj, _end = decoder.raw_decode(raw, i)
|
||||||
|
except (json.JSONDecodeError, ValueError):
|
||||||
|
start = i + 1
|
||||||
|
continue
|
||||||
|
if (
|
||||||
|
isinstance(partial_obj, dict)
|
||||||
|
and partial_obj
|
||||||
|
and set(partial_obj.keys()) <= allowed_keys
|
||||||
|
):
|
||||||
|
repeated_candidates.append(partial_obj)
|
||||||
start = i + 1
|
start = i + 1
|
||||||
continue
|
continue
|
||||||
if isinstance(obj, dict) and obj and set(obj.keys()) <= allowed_keys:
|
if isinstance(obj, dict) and obj and set(obj.keys()) <= allowed_keys:
|
||||||
return obj
|
return obj
|
||||||
# 解析成功但不是想要的 dict(如前缀里一段自洽 JSON):继续往后找,真 JSON 可能在更后面。
|
# 解析成功但不是想要的 dict(如前缀里一段自洽 JSON):继续往后找,真 JSON 可能在更后面。
|
||||||
start = i + 1
|
start = i + 1
|
||||||
|
if (
|
||||||
|
len(repeated_candidates) >= 2
|
||||||
|
and all(obj == repeated_candidates[0] for obj in repeated_candidates[1:])
|
||||||
|
):
|
||||||
|
return repeated_candidates[0]
|
||||||
|
return None
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,268 @@
|
||||||
|
"""宿主级 Sandbox 重型执行容量。
|
||||||
|
|
||||||
|
状态保存在 workspace/.sandbox 下并由 advisory file lock 串行化,因此蓝绿和多
|
||||||
|
Web 进程共享同一组槽位。业务 DB 不承载实时状态;后台排队事实源仍是
|
||||||
|
``.zcbot_procs``,这里只记录已经获得槽位的租约和短暂的前台候选。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from contextlib import contextmanager, suppress
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Callable, Dict, Iterator, Optional
|
||||||
|
|
||||||
|
from core.file_store import atomic_write_text, interprocess_file_lock
|
||||||
|
|
||||||
|
DEFAULT_MAX_ACTIVE_EXECS = 6
|
||||||
|
DEFAULT_MAX_BACKGROUND_EXECS = 4
|
||||||
|
DEFAULT_MAX_ACTIVE_EXECS_PER_USER = 3
|
||||||
|
DEFAULT_MIN_MEM_AVAILABLE_BYTES = 1024 ** 3
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_bytes(value: object, default: int) -> int:
|
||||||
|
text = str(value or "").strip().lower()
|
||||||
|
if not text:
|
||||||
|
return default
|
||||||
|
units = {"k": 1024, "kb": 1024, "m": 1024**2, "mb": 1024**2,
|
||||||
|
"g": 1024**3, "gb": 1024**3}
|
||||||
|
for suffix, factor in sorted(units.items(), key=lambda x: -len(x[0])):
|
||||||
|
if text.endswith(suffix):
|
||||||
|
return int(float(text[:-len(suffix)]) * factor)
|
||||||
|
return int(text)
|
||||||
|
|
||||||
|
|
||||||
|
def mem_available_bytes() -> Optional[int]:
|
||||||
|
try:
|
||||||
|
for line in Path("/proc/meminfo").read_text(encoding="ascii").splitlines():
|
||||||
|
if line.startswith("MemAvailable:"):
|
||||||
|
return int(line.split()[1]) * 1024
|
||||||
|
except (OSError, ValueError, IndexError):
|
||||||
|
return None
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class ExecCapacity:
|
||||||
|
def __init__(self, state_dir: Path, cfg: Optional[dict] = None) -> None:
|
||||||
|
cfg = cfg or {}
|
||||||
|
self.state_dir = Path(state_dir)
|
||||||
|
self.state_path = self.state_dir / "exec-capacity.json"
|
||||||
|
self.lock_path = self.state_dir / "exec-capacity.lock"
|
||||||
|
self.max_active = max(1, min(DEFAULT_MAX_ACTIVE_EXECS, int(os.getenv("ZCBOT_MAX_ACTIVE_EXECS") or cfg.get("max_active_execs") or DEFAULT_MAX_ACTIVE_EXECS)))
|
||||||
|
self.max_background = max(1, min(DEFAULT_MAX_BACKGROUND_EXECS, int(os.getenv("ZCBOT_MAX_BACKGROUND_EXECS") or cfg.get("max_background_execs") or DEFAULT_MAX_BACKGROUND_EXECS)))
|
||||||
|
self.max_per_user = max(1, min(DEFAULT_MAX_ACTIVE_EXECS_PER_USER, int(os.getenv("ZCBOT_MAX_ACTIVE_EXECS_PER_USER") or cfg.get("max_active_execs_per_user") or DEFAULT_MAX_ACTIVE_EXECS_PER_USER)))
|
||||||
|
self.min_mem_available = _parse_bytes(
|
||||||
|
os.getenv("ZCBOT_MIN_MEM_AVAILABLE") or cfg.get("min_mem_available"),
|
||||||
|
DEFAULT_MIN_MEM_AVAILABLE_BYTES,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _read(self) -> dict:
|
||||||
|
try:
|
||||||
|
data = json.loads(self.state_path.read_text(encoding="utf-8"))
|
||||||
|
if isinstance(data, dict):
|
||||||
|
data.setdefault("leases", {})
|
||||||
|
data.setdefault("foreground_queue", [])
|
||||||
|
data.setdefault("containers", {})
|
||||||
|
return data
|
||||||
|
except (OSError, ValueError):
|
||||||
|
pass
|
||||||
|
return {"leases": {}, "foreground_queue": [], "containers": {}}
|
||||||
|
|
||||||
|
def _write(self, state: dict) -> None:
|
||||||
|
atomic_write_text(self.state_path, json.dumps(state, ensure_ascii=False, sort_keys=True))
|
||||||
|
|
||||||
|
def _prune(self, state: dict) -> None:
|
||||||
|
leases = state["leases"]
|
||||||
|
for key, lease in list(leases.items()):
|
||||||
|
if lease.get("kind") == "foreground":
|
||||||
|
pid = int(lease.get("owner_pid") or 0)
|
||||||
|
if pid and not self._pid_alive(pid):
|
||||||
|
leases.pop(key, None)
|
||||||
|
state["foreground_queue"] = [
|
||||||
|
q for q in state["foreground_queue"]
|
||||||
|
if self._pid_alive(int(q.get("owner_pid") or 0))
|
||||||
|
]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _pid_alive(pid: int) -> bool:
|
||||||
|
try:
|
||||||
|
os.kill(pid, 0)
|
||||||
|
return True
|
||||||
|
except PermissionError:
|
||||||
|
return True
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _can_admit(self, state: dict, user_id: str, kind: str) -> bool:
|
||||||
|
leases = list(state["leases"].values())
|
||||||
|
if len(leases) >= self.max_active:
|
||||||
|
return False
|
||||||
|
if sum(1 for x in leases if x.get("user_id") == user_id) >= self.max_per_user:
|
||||||
|
return False
|
||||||
|
if kind == "background" and sum(1 for x in leases if x.get("kind") == "background") >= self.max_background:
|
||||||
|
return False
|
||||||
|
available = mem_available_bytes()
|
||||||
|
return available is None or available >= self.min_mem_available
|
||||||
|
|
||||||
|
def try_acquire(self, user_id: str, kind: str, *, lease_id: Optional[str] = None, proc_id: Optional[str] = None) -> Optional[str]:
|
||||||
|
lease_id = lease_id or uuid.uuid4().hex
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read()
|
||||||
|
self._prune(state)
|
||||||
|
if lease_id in state["leases"]:
|
||||||
|
return None
|
||||||
|
if not self._can_admit(state, str(user_id), kind):
|
||||||
|
self._write(state)
|
||||||
|
return None
|
||||||
|
state["leases"][lease_id] = {
|
||||||
|
"lease_id": lease_id, "user_id": str(user_id), "kind": kind,
|
||||||
|
"proc_id": proc_id, "owner_pid": os.getpid(), "started_ts": time.time(),
|
||||||
|
}
|
||||||
|
self._write(state)
|
||||||
|
return lease_id
|
||||||
|
|
||||||
|
def acquire_foreground(
|
||||||
|
self,
|
||||||
|
user_id: str,
|
||||||
|
cancel_check: Optional[Callable[[], bool]] = None,
|
||||||
|
wait_notify: Optional[Callable[[dict], None]] = None,
|
||||||
|
) -> Optional[str]:
|
||||||
|
ticket = uuid.uuid4().hex
|
||||||
|
wait_notified = False
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read(); self._prune(state)
|
||||||
|
state["foreground_queue"].append({"ticket": ticket, "user_id": str(user_id), "created_ts": time.time(), "owner_pid": os.getpid()})
|
||||||
|
self._write(state)
|
||||||
|
while True:
|
||||||
|
admitted = False
|
||||||
|
notification = None
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read(); self._prune(state)
|
||||||
|
queue = state["foreground_queue"]
|
||||||
|
eligible = next((q for q in queue if q.get("user_id") == str(user_id)), None)
|
||||||
|
if eligible and eligible.get("ticket") == ticket and self._can_admit(state, str(user_id), "foreground"):
|
||||||
|
queue[:] = [q for q in queue if q.get("ticket") != ticket]
|
||||||
|
state["leases"][ticket] = {"lease_id": ticket, "user_id": str(user_id), "kind": "foreground", "owner_pid": os.getpid(), "started_ts": time.time()}
|
||||||
|
self._write(state)
|
||||||
|
admitted = True
|
||||||
|
else:
|
||||||
|
self._write(state)
|
||||||
|
if not admitted and not wait_notified and wait_notify is not None:
|
||||||
|
leases = list(state["leases"].values())
|
||||||
|
user_running = sum(
|
||||||
|
1 for lease in leases
|
||||||
|
if lease.get("user_id") == str(user_id)
|
||||||
|
)
|
||||||
|
available = mem_available_bytes()
|
||||||
|
if user_running >= self.max_per_user:
|
||||||
|
reason = "per_user_limit"
|
||||||
|
elif len(leases) >= self.max_active:
|
||||||
|
reason = "global_limit"
|
||||||
|
elif available is not None and available < self.min_mem_available:
|
||||||
|
reason = "memory_pressure"
|
||||||
|
else:
|
||||||
|
reason = "queue_order"
|
||||||
|
notification = {
|
||||||
|
"state": "waiting",
|
||||||
|
"reason": reason,
|
||||||
|
"user_running": user_running,
|
||||||
|
"user_limit": self.max_per_user,
|
||||||
|
"global_running": len(leases),
|
||||||
|
"global_limit": self.max_active,
|
||||||
|
}
|
||||||
|
wait_notified = True
|
||||||
|
if admitted:
|
||||||
|
if wait_notified and wait_notify is not None:
|
||||||
|
with suppress(Exception):
|
||||||
|
wait_notify({"state": "admitted"})
|
||||||
|
return ticket
|
||||||
|
if notification is not None and wait_notify is not None:
|
||||||
|
with suppress(Exception):
|
||||||
|
wait_notify(notification)
|
||||||
|
if cancel_check is not None and cancel_check():
|
||||||
|
self.cancel_waiter(ticket)
|
||||||
|
return None
|
||||||
|
time.sleep(0.1)
|
||||||
|
|
||||||
|
def cancel_waiter(self, ticket: str) -> None:
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read()
|
||||||
|
state["foreground_queue"] = [q for q in state["foreground_queue"] if q.get("ticket") != ticket]
|
||||||
|
self._write(state)
|
||||||
|
|
||||||
|
def release(self, lease_id: Optional[str]) -> None:
|
||||||
|
if not lease_id:
|
||||||
|
return
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read(); state["leases"].pop(lease_id, None); self._write(state)
|
||||||
|
|
||||||
|
def reconcile_background(self, running_proc_ids: set[str]) -> None:
|
||||||
|
"""服务重启后保留 Docker 仍在跑的租约,清启动窗口遗留的孤儿租约。"""
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read()
|
||||||
|
for key, lease in list(state["leases"].items()):
|
||||||
|
if (lease.get("kind") == "background"
|
||||||
|
and str(lease.get("proc_id") or "") not in running_proc_ids
|
||||||
|
and not self._pid_alive(int(lease.get("owner_pid") or 0))):
|
||||||
|
state["leases"].pop(key, None)
|
||||||
|
self._write(state)
|
||||||
|
|
||||||
|
def touch_container(self, name: str, *, active_delta: int = 0) -> None:
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read()
|
||||||
|
row = state["containers"].setdefault(name, {"active_execs": 0})
|
||||||
|
row["active_execs"] = max(0, int(row.get("active_execs") or 0) + active_delta)
|
||||||
|
row["last_active_ts"] = time.time()
|
||||||
|
self._write(state)
|
||||||
|
|
||||||
|
def remove_container(self, name: str) -> None:
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read(); state["containers"].pop(name, None); self._write(state)
|
||||||
|
|
||||||
|
def reconcile_containers(self, running_names: set[str]) -> None:
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read()
|
||||||
|
for name in list(state["containers"]):
|
||||||
|
if name not in running_names:
|
||||||
|
state["containers"].pop(name, None)
|
||||||
|
self._write(state)
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def foreground(
|
||||||
|
self,
|
||||||
|
user_id: str,
|
||||||
|
cancel_check: Optional[Callable[[], bool]] = None,
|
||||||
|
wait_notify: Optional[Callable[[dict], None]] = None,
|
||||||
|
) -> Iterator[bool]:
|
||||||
|
lease = self.acquire_foreground(user_id, cancel_check, wait_notify)
|
||||||
|
try:
|
||||||
|
yield lease is not None
|
||||||
|
finally:
|
||||||
|
self.release(lease)
|
||||||
|
|
||||||
|
def snapshot(self) -> dict:
|
||||||
|
with interprocess_file_lock(self.lock_path, timeout_seconds=None):
|
||||||
|
state = self._read(); self._prune(state); self._write(state)
|
||||||
|
leases = list(state["leases"].values())
|
||||||
|
fg = sum(x.get("kind") == "foreground" for x in leases)
|
||||||
|
bg = sum(x.get("kind") == "background" for x in leases)
|
||||||
|
by_user: Dict[str, int] = {}
|
||||||
|
for x in leases:
|
||||||
|
by_user[x["user_id"]] = by_user.get(x["user_id"], 0) + 1
|
||||||
|
available_mem = mem_available_bytes()
|
||||||
|
memory_paused = available_mem is not None and available_mem < self.min_mem_available
|
||||||
|
containers = state.get("containers", {})
|
||||||
|
now = time.time()
|
||||||
|
return {
|
||||||
|
"limits": {"active": self.max_active, "background": self.max_background, "per_user": self.max_per_user},
|
||||||
|
"foreground_running": fg, "foreground_queued": len(state["foreground_queue"]),
|
||||||
|
"background_running": bg, "per_user": by_user,
|
||||||
|
"admit_available": 0 if memory_paused else max(0, self.max_active - len(leases)),
|
||||||
|
"memory_paused": memory_paused,
|
||||||
|
"mem_available_bytes": available_mem,
|
||||||
|
"cpu_load": list(os.getloadavg()) if hasattr(os, "getloadavg") else None,
|
||||||
|
"active_sandbox_containers": len(containers),
|
||||||
|
"idle_reap_candidates": sum(int(x.get("active_execs") or 0) == 0 and float(x.get("last_active_ts") or now) < now - getattr(self, "idle_ttl_seconds", 600) for x in containers.values()),
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,57 @@
|
||||||
|
"""Sandbox 临时 Python 包扫描与 best-effort 持久化。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Optional
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
|
||||||
|
def scan_container(container: str) -> Optional[dict]:
|
||||||
|
try:
|
||||||
|
scan = subprocess.run(["docker", "exec", "--user", "zcbot", "--workdir", "/sandbox", container,
|
||||||
|
"python", "-I", "-S", "/sandbox/package_scan.py"], capture_output=True,
|
||||||
|
text=True, timeout=30)
|
||||||
|
if scan.returncode != 0:
|
||||||
|
raise RuntimeError(scan.stderr.strip())
|
||||||
|
packages = json.loads(scan.stdout)
|
||||||
|
if not isinstance(packages, list):
|
||||||
|
raise ValueError("scanner output is not a package list")
|
||||||
|
inspect = subprocess.run(["docker", "inspect", "--format={{.Image}}", container],
|
||||||
|
capture_output=True, text=True, timeout=15)
|
||||||
|
py = subprocess.run(["docker", "exec", "--workdir", "/sandbox", container, "python", "-I", "-S", "-c",
|
||||||
|
"import sys;print('.'.join(map(str,sys.version_info[:3])))"],
|
||||||
|
capture_output=True, text=True, timeout=15)
|
||||||
|
if inspect.returncode != 0 or py.returncode != 0:
|
||||||
|
raise RuntimeError("failed to inspect image or Python version")
|
||||||
|
return {"packages": packages, "image_digest": inspect.stdout.strip(),
|
||||||
|
"python_version": py.stdout.strip(),
|
||||||
|
"total_installed_bytes": sum(int(x.get("installed_bytes") or 0) for x in packages)}
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"[sandbox-package-scan] failed container={container}: {type(exc).__name__}: {exc}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def persist_scan(*, container_session_id: str, user_id: UUID, execution_kind: str,
|
||||||
|
result: Optional[dict]) -> None:
|
||||||
|
if not result or not result.get("packages"):
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
from core.storage import session_scope
|
||||||
|
from core.storage.models import SandboxPackageScan
|
||||||
|
with session_scope() as s:
|
||||||
|
s.merge(SandboxPackageScan(
|
||||||
|
container_session_id=container_session_id, user_id=user_id,
|
||||||
|
execution_kind=execution_kind, image_digest=result["image_digest"],
|
||||||
|
python_version=result["python_version"], packages=result["packages"],
|
||||||
|
package_count=len(result["packages"]),
|
||||||
|
total_installed_bytes=result["total_installed_bytes"],
|
||||||
|
finished_at=datetime.now(timezone.utc),
|
||||||
|
))
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"[sandbox-package-scan] persist failed session={container_session_id}: {type(exc).__name__}: {exc}")
|
||||||
|
|
||||||
|
|
||||||
|
def scan_and_persist(container: str, user_id: UUID, kind: str, session_id: str) -> None:
|
||||||
|
persist_scan(container_session_id=session_id, user_id=user_id,
|
||||||
|
execution_kind=kind, result=scan_container(container))
|
||||||
|
|
@ -56,11 +56,11 @@ LABEL_INSTANCE_KEY = "zcbot.instance"
|
||||||
INSTANCE = os.getenv("ZCBOT_INSTANCE", "").strip()
|
INSTANCE = os.getenv("ZCBOT_INSTANCE", "").strip()
|
||||||
|
|
||||||
DEFAULT_IMAGE = "zcbot-sandbox:latest"
|
DEFAULT_IMAGE = "zcbot-sandbox:latest"
|
||||||
DEFAULT_IDLE_TTL_SECONDS = 300
|
DEFAULT_IDLE_TTL_SECONDS = 600
|
||||||
|
|
||||||
# 容器资源限制默认值(可被 yaml `sandbox.*` / env override,详 SandboxPool ctor)
|
# 容器资源限制默认值(可被 yaml `sandbox.*` / env override,详 SandboxPool ctor)
|
||||||
DEFAULT_MEMORY = "2g"
|
DEFAULT_MEMORY = "4g"
|
||||||
DEFAULT_CPUS = "1.0"
|
DEFAULT_CPUS = "2.0"
|
||||||
# pids-limit 把**线程**也计数:chromium headless 一次启动(browser+GPU+network+renderer
|
# pids-limit 把**线程**也计数:chromium headless 一次启动(browser+GPU+network+renderer
|
||||||
# 多进程)就 ~150-200 线程,256 本来就贴边;shell 超时又只杀 host 侧 docker CLI,容器内
|
# 多进程)就 ~150-200 线程,256 本来就贴边;shell 超时又只杀 host 侧 docker CLI,容器内
|
||||||
# mmdc+chromium 树留着(见 executor_docker.py 头注 Cancel limitation),残留几棵就把配额
|
# mmdc+chromium 树留着(见 executor_docker.py 头注 Cancel limitation),残留几棵就把配额
|
||||||
|
|
@ -70,8 +70,9 @@ DEFAULT_PIDS_LIMIT = 1024
|
||||||
# chromium(mmdc 渲 mermaid / puppeteer)默认走 /dev/shm,docker 不传 --shm-size 时
|
# chromium(mmdc 渲 mermaid / puppeteer)默认走 /dev/shm,docker 不传 --shm-size 时
|
||||||
# 只给 64MB,起不来就一直挂到 timeout。镜像备的 puppeteer-config 有 --disable-dev-shm-usage,
|
# 只给 64MB,起不来就一直挂到 timeout。镜像备的 puppeteer-config 有 --disable-dev-shm-usage,
|
||||||
# 但模型不一定用那份;这里从根上把 /dev/shm 撑到够用,任何 chromium 路径都不再挂。
|
# 但模型不一定用那份;这里从根上把 /dev/shm 撑到够用,任何 chromium 路径都不再挂。
|
||||||
# 从 --memory(默 2g)里切,512m 是上限非占用(tmpfs 按需用)。
|
# 从 --memory(默 4g)里切,512m 是上限非占用(tmpfs 按需用)。
|
||||||
DEFAULT_SHM_SIZE = "512m"
|
DEFAULT_SHM_SIZE = "512m"
|
||||||
|
DEFAULT_TMP_SIZE = "1g"
|
||||||
|
|
||||||
|
|
||||||
def container_name(user_id: UUID) -> str:
|
def container_name(user_id: UUID) -> str:
|
||||||
|
|
@ -102,6 +103,11 @@ def _container_running(name: str) -> bool:
|
||||||
return r.returncode == 0 and r.stdout.strip() == "true"
|
return r.returncode == 0 and r.stdout.strip() == "true"
|
||||||
|
|
||||||
|
|
||||||
|
def _container_session_id(name: str) -> str:
|
||||||
|
r = subprocess.run(["docker", "inspect", "--format={{.Id}}", name], capture_output=True, text=True)
|
||||||
|
return r.stdout.strip() if r.returncode == 0 else name
|
||||||
|
|
||||||
|
|
||||||
class SandboxPool:
|
class SandboxPool:
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
|
|
@ -115,7 +121,9 @@ class SandboxPool:
|
||||||
cpus: Optional[str] = None,
|
cpus: Optional[str] = None,
|
||||||
pids_limit: Optional[int] = None,
|
pids_limit: Optional[int] = None,
|
||||||
shm_size: Optional[str] = None,
|
shm_size: Optional[str] = None,
|
||||||
|
tmp_size: Optional[str] = None,
|
||||||
dns: Optional[List[str]] = None,
|
dns: Optional[List[str]] = None,
|
||||||
|
capacity_cfg: Optional[Dict[str, object]] = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""
|
"""
|
||||||
user_root_base: per-user 子树父目录,典型 `<workspace>/users`。bind mount 源
|
user_root_base: per-user 子树父目录,典型 `<workspace>/users`。bind mount 源
|
||||||
|
|
@ -134,9 +142,9 @@ class SandboxPool:
|
||||||
pg_ips: 逗号分隔的 PG IP 串,塞容器 `ZCBOT_PG_IPS` env,init.sh 加 DROP 规则
|
pg_ips: 逗号分隔的 PG IP 串,塞容器 `ZCBOT_PG_IPS` env,init.sh 加 DROP 规则
|
||||||
(env `ZCBOT_PG_IPS`)。defense-in-depth ── 即便落内网三段。
|
(env `ZCBOT_PG_IPS`)。defense-in-depth ── 即便落内网三段。
|
||||||
memory/cpus/pids_limit/shm_size:
|
memory/cpus/pids_limit/shm_size:
|
||||||
容器资源限制,默 2g/1.0/256/512m;env(`ZCBOT_SANDBOX_MEMORY` 等)
|
容器资源限制,默 4g/2.0/1024/512m;env(`ZCBOT_SANDBOX_MEMORY` 等)
|
||||||
override caller 参数 override 默认。改后重启 web 生效,新起的
|
override caller 参数 override 默认。改后重启 web 生效,新起的
|
||||||
容器用新值;已 running 不变(idle 5min 回收后下次起按新值)。
|
容器用新值;已 running 不变(idle 10min 回收后下次起按新值)。
|
||||||
shm_size 撑 chromium 的 /dev/shm(默 64MB 不够,mmdc 渲图会挂)。
|
shm_size 撑 chromium 的 /dev/shm(默 64MB 不够,mmdc 渲图会挂)。
|
||||||
"""
|
"""
|
||||||
self.user_root_base = user_root_base
|
self.user_root_base = user_root_base
|
||||||
|
|
@ -153,6 +161,7 @@ class SandboxPool:
|
||||||
self.memory = os.getenv("ZCBOT_SANDBOX_MEMORY") or memory or DEFAULT_MEMORY
|
self.memory = os.getenv("ZCBOT_SANDBOX_MEMORY") or memory or DEFAULT_MEMORY
|
||||||
self.cpus = os.getenv("ZCBOT_SANDBOX_CPUS") or cpus or DEFAULT_CPUS
|
self.cpus = os.getenv("ZCBOT_SANDBOX_CPUS") or cpus or DEFAULT_CPUS
|
||||||
self.shm_size = os.getenv("ZCBOT_SANDBOX_SHM_SIZE") or shm_size or DEFAULT_SHM_SIZE
|
self.shm_size = os.getenv("ZCBOT_SANDBOX_SHM_SIZE") or shm_size or DEFAULT_SHM_SIZE
|
||||||
|
self.tmp_size = os.getenv("ZCBOT_SANDBOX_TMP_SIZE") or tmp_size or DEFAULT_TMP_SIZE
|
||||||
self.pids_limit = int(
|
self.pids_limit = int(
|
||||||
os.getenv("ZCBOT_SANDBOX_PIDS_LIMIT")
|
os.getenv("ZCBOT_SANDBOX_PIDS_LIMIT")
|
||||||
or (pids_limit if pids_limit is not None else DEFAULT_PIDS_LIMIT)
|
or (pids_limit if pids_limit is not None else DEFAULT_PIDS_LIMIT)
|
||||||
|
|
@ -166,6 +175,10 @@ class SandboxPool:
|
||||||
self._dict_lock = threading.Lock() # 保护 _locks / _last_active 的字典级 race
|
self._dict_lock = threading.Lock() # 保护 _locks / _last_active 的字典级 race
|
||||||
self._locks: Dict[UUID, threading.Lock] = {}
|
self._locks: Dict[UUID, threading.Lock] = {}
|
||||||
self._last_active: Dict[UUID, int] = {}
|
self._last_active: Dict[UUID, int] = {}
|
||||||
|
self._active_execs: Dict[UUID, int] = {}
|
||||||
|
from .capacity import ExecCapacity
|
||||||
|
self.capacity = ExecCapacity(self.user_root_base.parent / ".sandbox", capacity_cfg)
|
||||||
|
self.capacity.idle_ttl_seconds = self.idle_ttl
|
||||||
|
|
||||||
def _lock_for(self, user_id: UUID) -> threading.Lock:
|
def _lock_for(self, user_id: UUID) -> threading.Lock:
|
||||||
with self._dict_lock:
|
with self._dict_lock:
|
||||||
|
|
@ -179,15 +192,18 @@ class SandboxPool:
|
||||||
name = container_name(user_id)
|
name = container_name(user_id)
|
||||||
if _container_running(name):
|
if _container_running(name):
|
||||||
self._last_active[user_id] = _now()
|
self._last_active[user_id] = _now()
|
||||||
|
self.capacity.touch_container(name)
|
||||||
return name
|
return name
|
||||||
if _container_exists(name):
|
if _container_exists(name):
|
||||||
# stopped / crashed ── rm 重起。iptables 规则随容器生命周期重新 apply。
|
# stopped / crashed ── rm 重起。iptables 规则随容器生命周期重新 apply。
|
||||||
|
self._scan_before_remove(name, user_id, "foreground")
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
["docker", "rm", "-f", name],
|
["docker", "rm", "-f", name],
|
||||||
capture_output=True, check=False,
|
capture_output=True, check=False,
|
||||||
)
|
)
|
||||||
self._docker_run(user_id, name)
|
self._docker_run(user_id, name)
|
||||||
self._last_active[user_id] = _now()
|
self._last_active[user_id] = _now()
|
||||||
|
self.capacity.touch_container(name)
|
||||||
return name
|
return name
|
||||||
|
|
||||||
def _ensure_resolv_conf_file(self) -> Optional[Path]:
|
def _ensure_resolv_conf_file(self) -> Optional[Path]:
|
||||||
|
|
@ -255,7 +271,7 @@ class SandboxPool:
|
||||||
"--network", NETWORK_NAME,
|
"--network", NETWORK_NAME,
|
||||||
# §7.5 硬限制(任一缺失视为 hardening 未完成)
|
# §7.5 硬限制(任一缺失视为 hardening 未完成)
|
||||||
"--read-only", # rootfs read-only
|
"--read-only", # rootfs read-only
|
||||||
"--tmpfs", "/tmp:exec,size=512m,mode=1777", # 可写临时区,exec 允许 (run_python 写脚本)
|
"--tmpfs", f"/tmp:exec,size={self.tmp_size},mode=1777",
|
||||||
f"--shm-size={self.shm_size}", # chromium/mmdc 的 /dev/shm,默 64MB 不够会挂(DEFAULT_SHM_SIZE)
|
f"--shm-size={self.shm_size}", # chromium/mmdc 的 /dev/shm,默 64MB 不够会挂(DEFAULT_SHM_SIZE)
|
||||||
"--cap-drop=ALL", # 默全丢
|
"--cap-drop=ALL", # 默全丢
|
||||||
"--cap-add=NET_ADMIN", # init.sh 配 iptables 需要;exec 进来的 uid 1000 拿不到
|
"--cap-add=NET_ADMIN", # init.sh 配 iptables 需要;exec 进来的 uid 1000 拿不到
|
||||||
|
|
@ -305,20 +321,48 @@ class SandboxPool:
|
||||||
def mark_active(self, user_id: UUID) -> None:
|
def mark_active(self, user_id: UUID) -> None:
|
||||||
"""每次 `docker exec` 完调一次,刷新 idle 计时。"""
|
"""每次 `docker exec` 完调一次,刷新 idle 计时。"""
|
||||||
self._last_active[user_id] = _now()
|
self._last_active[user_id] = _now()
|
||||||
|
self.capacity.touch_container(container_name(user_id))
|
||||||
|
|
||||||
|
def exec_started(self, user_id: UUID) -> None:
|
||||||
|
"""普通容器内 exec 开始;显式计数防止 idle reaper 误删长命令。"""
|
||||||
|
with self._dict_lock:
|
||||||
|
self._active_execs[user_id] = self._active_execs.get(user_id, 0) + 1
|
||||||
|
self._last_active[user_id] = _now()
|
||||||
|
self.capacity.touch_container(container_name(user_id), active_delta=1)
|
||||||
|
|
||||||
|
def exec_finished(self, user_id: UUID) -> None:
|
||||||
|
with self._dict_lock:
|
||||||
|
n = max(0, self._active_execs.get(user_id, 0) - 1)
|
||||||
|
if n:
|
||||||
|
self._active_execs[user_id] = n
|
||||||
|
else:
|
||||||
|
self._active_execs.pop(user_id, None)
|
||||||
|
self._last_active[user_id] = _now()
|
||||||
|
self.capacity.touch_container(container_name(user_id), active_delta=-1)
|
||||||
|
|
||||||
|
def _scan_before_remove(self, name: str, user_id: UUID, kind: str, session_id: Optional[str] = None) -> None:
|
||||||
|
"""扫描/落库均 best-effort,调用方无论如何继续删除容器。"""
|
||||||
|
try:
|
||||||
|
from .package_scans import scan_and_persist
|
||||||
|
scan_and_persist(name, user_id, kind, session_id or _container_session_id(name))
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"[sandbox-package-scan] finalizer failed container={name}: {type(exc).__name__}: {exc}")
|
||||||
|
|
||||||
def reap_idle(self) -> List[str]:
|
def reap_idle(self) -> List[str]:
|
||||||
"""杀超过 idle_ttl 没活跃的容器。返回已杀容器名列表(供日志 / 审计)。"""
|
"""杀超过 idle_ttl 没活跃的容器。返回已杀容器名列表(供日志 / 审计)。"""
|
||||||
removed: List[str] = []
|
removed: List[str] = []
|
||||||
cutoff = _now() - self.idle_ttl
|
cutoff = _now() - self.idle_ttl
|
||||||
for uid, ts in list(self._last_active.items()):
|
for uid, ts in list(self._last_active.items()):
|
||||||
if ts < cutoff:
|
if ts < cutoff and self._active_execs.get(uid, 0) == 0:
|
||||||
name = container_name(uid)
|
name = container_name(uid)
|
||||||
|
self._scan_before_remove(name, uid, "foreground")
|
||||||
r = subprocess.run(
|
r = subprocess.run(
|
||||||
["docker", "rm", "-f", name],
|
["docker", "rm", "-f", name],
|
||||||
capture_output=True, text=True,
|
capture_output=True, text=True,
|
||||||
)
|
)
|
||||||
if r.returncode == 0:
|
if r.returncode == 0:
|
||||||
removed.append(name)
|
removed.append(name)
|
||||||
|
self.capacity.remove_container(name)
|
||||||
# 无论 rm 成功与否,从 dict 移除 ── 失败则下次启动靠 shutdown_all 兜底
|
# 无论 rm 成功与否,从 dict 移除 ── 失败则下次启动靠 shutdown_all 兜底
|
||||||
del self._last_active[uid]
|
del self._last_active[uid]
|
||||||
return removed
|
return removed
|
||||||
|
|
@ -341,6 +385,21 @@ class SandboxPool:
|
||||||
if list_r.returncode != 0 or not list_r.stdout.strip():
|
if list_r.returncode != 0 or not list_r.stdout.strip():
|
||||||
return []
|
return []
|
||||||
ids = list_r.stdout.strip().splitlines()
|
ids = list_r.stdout.strip().splitlines()
|
||||||
|
for cid in ids:
|
||||||
|
info = subprocess.run(
|
||||||
|
["docker", "inspect", "--format={{index .Config.Labels \"zcbot.user_id\"}}", cid],
|
||||||
|
capture_output=True, text=True,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
uid = UUID(info.stdout.strip())
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
continue
|
||||||
|
self._scan_before_remove(cid, uid, "foreground", cid)
|
||||||
|
try:
|
||||||
|
name_r = subprocess.run(["docker", "inspect", "--format={{.Name}}", cid], capture_output=True, text=True)
|
||||||
|
self.capacity.remove_container(name_r.stdout.strip().lstrip("/") or cid)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
["docker", "rm", "-f", *ids],
|
["docker", "rm", "-f", *ids],
|
||||||
capture_output=True, text=True,
|
capture_output=True, text=True,
|
||||||
|
|
@ -349,6 +408,31 @@ class SandboxPool:
|
||||||
self._last_active.clear()
|
self._last_active.clear()
|
||||||
return ids
|
return ids
|
||||||
|
|
||||||
|
def runtime_snapshot(self) -> dict:
|
||||||
|
try:
|
||||||
|
listed = subprocess.run(
|
||||||
|
["docker", "ps", "--format={{.Names}}", "--filter", f"label={LABEL_PRODUCT_KEY}={LABEL_PRODUCT_VALUE}"],
|
||||||
|
capture_output=True, text=True, timeout=15,
|
||||||
|
)
|
||||||
|
if listed.returncode == 0:
|
||||||
|
self.capacity.reconcile_containers(set(listed.stdout.split()))
|
||||||
|
except (OSError, subprocess.TimeoutExpired):
|
||||||
|
pass
|
||||||
|
snap = self.capacity.snapshot()
|
||||||
|
now = _now()
|
||||||
|
snap["idle_ttl_seconds"] = self.idle_ttl
|
||||||
|
try:
|
||||||
|
from core import procs
|
||||||
|
items = []
|
||||||
|
for uroot in self.user_root_base.iterdir():
|
||||||
|
items.extend(procs.list_all_procs(uroot))
|
||||||
|
snap["background_queued"] = sum(x.get("_status") == "queued" for x in items)
|
||||||
|
snap["running_proc_containers"] = sum(x.get("_status") == "running" and x.get("backend") == "docker" for x in items)
|
||||||
|
except OSError:
|
||||||
|
snap["background_queued"] = 0
|
||||||
|
snap["running_proc_containers"] = 0
|
||||||
|
return snap
|
||||||
|
|
||||||
|
|
||||||
def setup_pool(
|
def setup_pool(
|
||||||
user_root_base: Path,
|
user_root_base: Path,
|
||||||
|
|
@ -376,12 +460,16 @@ def setup_pool(
|
||||||
cpus = cfg.get("cpus")
|
cpus = cfg.get("cpus")
|
||||||
pids_limit = cfg.get("pids_limit")
|
pids_limit = cfg.get("pids_limit")
|
||||||
shm_size = cfg.get("shm_size")
|
shm_size = cfg.get("shm_size")
|
||||||
|
tmp_size = cfg.get("tmp_size")
|
||||||
return SandboxPool(
|
return SandboxPool(
|
||||||
user_root_base=user_root_base,
|
user_root_base=user_root_base,
|
||||||
repo_root=repo_root,
|
repo_root=repo_root,
|
||||||
|
idle_ttl=int(str(cfg.get("idle_ttl_seconds"))) if cfg.get("idle_ttl_seconds") is not None else None,
|
||||||
memory=memory if isinstance(memory, str) else None,
|
memory=memory if isinstance(memory, str) else None,
|
||||||
cpus=str(cpus) if cpus is not None else None,
|
cpus=str(cpus) if cpus is not None else None,
|
||||||
pids_limit=int(str(pids_limit)) if pids_limit is not None else None,
|
pids_limit=int(str(pids_limit)) if pids_limit is not None else None,
|
||||||
shm_size=shm_size if isinstance(shm_size, str) else None,
|
shm_size=shm_size if isinstance(shm_size, str) else None,
|
||||||
|
tmp_size=tmp_size if isinstance(tmp_size, str) else None,
|
||||||
dns=[str(x) for x in dns_cfg],
|
dns=[str(x) for x in dns_cfg],
|
||||||
|
capacity_cfg=cfg,
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -65,6 +65,7 @@ class Session:
|
||||||
msg: Any,
|
msg: Any,
|
||||||
*,
|
*,
|
||||||
artifact_refs: Optional[list[dict]] = None,
|
artifact_refs: Optional[list[dict]] = None,
|
||||||
|
source_model_profile: str = "",
|
||||||
) -> Optional[UUID]:
|
) -> Optional[UUID]:
|
||||||
"""追加消息;非 system 落 DB,system 仅内存。返回新落库行的 message_id。
|
"""追加消息;非 system 落 DB,system 仅内存。返回新落库行的 message_id。
|
||||||
|
|
||||||
|
|
@ -91,7 +92,10 @@ class Session:
|
||||||
f"[markdown:fence-warning] task={self.task_id} unclosed=1",
|
f"[markdown:fence-warning] task={self.task_id} unclosed=1",
|
||||||
flush=True,
|
flush=True,
|
||||||
)
|
)
|
||||||
self.messages.append(msg_dict)
|
memory_msg = dict(msg_dict)
|
||||||
|
if source_model_profile and memory_msg.get("role") == "assistant":
|
||||||
|
memory_msg["_model_profile"] = source_model_profile
|
||||||
|
self.messages.append(memory_msg)
|
||||||
if msg_dict.get("role") == "system":
|
if msg_dict.get("role") == "system":
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
@ -236,6 +240,9 @@ class Session:
|
||||||
attachment_refs,
|
attachment_refs,
|
||||||
wd_name,
|
wd_name,
|
||||||
)
|
)
|
||||||
|
payload["_attachment_refs"] = list(attachment_refs)
|
||||||
|
if payload.get("role") == "assistant" and row.model_profile:
|
||||||
|
payload["_model_profile"] = row.model_profile
|
||||||
if payload.get("role") == "assistant" and isinstance(payload.get("content"), str):
|
if payload.get("role") == "assistant" and isinstance(payload.get("content"), str):
|
||||||
# 历史行不回写生产库;只在重建 LLM 上下文时应用同一窄修复,
|
# 历史行不回写生产库;只在重建 LLM 上下文时应用同一窄修复,
|
||||||
# 与 Web 展示层保持一致,避免旧坏围栏继续污染后续轮次。
|
# 与 Web 展示层保持一致,避免旧坏围栏继续污染后续轮次。
|
||||||
|
|
|
||||||
|
|
@ -7,6 +7,7 @@ Loop 不直接 print,改 emit({type, ...})。Sink 决定怎么呈现。
|
||||||
llm_end {type, prompt_tokens, completion_tokens, elapsed}
|
llm_end {type, prompt_tokens, completion_tokens, elapsed}
|
||||||
text {type, content} —— assistant 文字段(整段,非流式)
|
text {type, content} —— assistant 文字段(整段,非流式)
|
||||||
tool_call {type, name, args, args_preview}
|
tool_call {type, name, args, args_preview}
|
||||||
|
execution_queue {type, state, reason, ...} —— 前台重型工具等待/放行
|
||||||
tool_result {type, name, result, preview, truncated}
|
tool_result {type, name, result, preview, truncated}
|
||||||
done {type} —— 一次 run 全部结束
|
done {type} —— 一次 run 全部结束
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,7 @@ from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
import math
|
||||||
import re
|
import re
|
||||||
import threading
|
import threading
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
|
|
@ -38,6 +39,14 @@ class WorkspaceSpec:
|
||||||
required_adapter_version: str
|
required_adapter_version: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CapabilityLifecycle:
|
||||||
|
status: str = "active"
|
||||||
|
visibility: str = "standard"
|
||||||
|
replacement: str | None = None
|
||||||
|
detail: str = ""
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class CapabilityContract:
|
class CapabilityContract:
|
||||||
capability: str
|
capability: str
|
||||||
|
|
@ -52,8 +61,20 @@ class CapabilityContract:
|
||||||
summary: dict[str, Any]
|
summary: dict[str, Any]
|
||||||
legacy_runtime: dict[str, Any] | None
|
legacy_runtime: dict[str, Any] | None
|
||||||
workspace: WorkspaceSpec | None
|
workspace: WorkspaceSpec | None
|
||||||
|
lifecycle: CapabilityLifecycle
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ordinarily_visible(self) -> bool:
|
||||||
|
return (
|
||||||
|
self.lifecycle.status == "active"
|
||||||
|
and self.lifecycle.visibility == "standard"
|
||||||
|
)
|
||||||
|
|
||||||
def normalize_request(self, request: object) -> tuple[dict[str, Any], str]:
|
def normalize_request(self, request: object) -> tuple[dict[str, Any], str]:
|
||||||
|
if not _all_numbers_finite(request):
|
||||||
|
raise SoftwareContractError(
|
||||||
|
f"invalid {self.capability} request: non-finite numbers are forbidden"
|
||||||
|
)
|
||||||
errors = sorted(
|
errors = sorted(
|
||||||
Draft202012Validator(
|
Draft202012Validator(
|
||||||
self.request_schema, format_checker=FormatChecker()
|
self.request_schema, format_checker=FormatChecker()
|
||||||
|
|
@ -65,7 +86,10 @@ class CapabilityContract:
|
||||||
location = ".".join(str(item) for item in first.absolute_path)
|
location = ".".join(str(item) for item in first.absolute_path)
|
||||||
prefix = f"{location}: " if location else ""
|
prefix = f"{location}: " if location else ""
|
||||||
raise SoftwareContractError(f"invalid {self.capability} request: {prefix}{first.message}")
|
raise SoftwareContractError(f"invalid {self.capability} request: {prefix}{first.message}")
|
||||||
encoded = json.dumps(request, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
encoded = json.dumps(
|
||||||
|
request, ensure_ascii=False, sort_keys=True, separators=(",", ":"),
|
||||||
|
allow_nan=False,
|
||||||
|
)
|
||||||
max_bytes = int(self.request_schema.get("x-maxBytes") or 256 * 1024)
|
max_bytes = int(self.request_schema.get("x-maxBytes") or 256 * 1024)
|
||||||
if len(encoded.encode("utf-8")) > max_bytes:
|
if len(encoded.encode("utf-8")) > max_bytes:
|
||||||
raise SoftwareContractError(f"{self.capability} request is too large")
|
raise SoftwareContractError(f"{self.capability} request is too large")
|
||||||
|
|
@ -145,6 +169,20 @@ def _value_at(value: object, path: list[str]) -> object:
|
||||||
return current
|
return current
|
||||||
|
|
||||||
|
|
||||||
|
def _all_numbers_finite(value: object) -> bool:
|
||||||
|
if isinstance(value, bool) or value is None or isinstance(value, str):
|
||||||
|
return True
|
||||||
|
if isinstance(value, float):
|
||||||
|
return math.isfinite(value)
|
||||||
|
if isinstance(value, int):
|
||||||
|
return True
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return all(_all_numbers_finite(item) for item in value.values())
|
||||||
|
if isinstance(value, (list, tuple)):
|
||||||
|
return all(_all_numbers_finite(item) for item in value)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
def _resolve_local_refs(value: Any, definitions: dict[str, Any]) -> Any:
|
def _resolve_local_refs(value: Any, definitions: dict[str, Any]) -> Any:
|
||||||
if isinstance(value, list):
|
if isinstance(value, list):
|
||||||
return [_resolve_local_refs(item, definitions) for item in value]
|
return [_resolve_local_refs(item, definitions) for item in value]
|
||||||
|
|
@ -171,7 +209,8 @@ def _load_contract(path: Path) -> CapabilityContract:
|
||||||
"input_policy", "outputs", "feature_path", "features", "summary", "legacy_runtime",
|
"input_policy", "outputs", "feature_path", "features", "summary", "legacy_runtime",
|
||||||
"workspace",
|
"workspace",
|
||||||
}
|
}
|
||||||
if not isinstance(raw, dict) or set(raw) != required:
|
optional = {"lifecycle"}
|
||||||
|
if not isinstance(raw, dict) or not required.issubset(raw) or set(raw) - required - optional:
|
||||||
raise RuntimeError(f"invalid software contract fields: {path.name}")
|
raise RuntimeError(f"invalid software contract fields: {path.name}")
|
||||||
capability = raw["capability"]
|
capability = raw["capability"]
|
||||||
namespace = raw["output_namespace"]
|
namespace = raw["output_namespace"]
|
||||||
|
|
@ -234,6 +273,32 @@ def _load_contract(path: Path) -> CapabilityContract:
|
||||||
workspace = WorkspaceSpec(
|
workspace = WorkspaceSpec(
|
||||||
state_output, tuple(preview_outputs), required_adapter_version
|
state_output, tuple(preview_outputs), required_adapter_version
|
||||||
)
|
)
|
||||||
|
lifecycle_raw = raw.get("lifecycle") or {}
|
||||||
|
if not isinstance(lifecycle_raw, dict) or set(lifecycle_raw) - {
|
||||||
|
"status", "visibility", "replacement", "detail"
|
||||||
|
}:
|
||||||
|
raise RuntimeError(f"invalid lifecycle spec: {path.name}")
|
||||||
|
lifecycle = CapabilityLifecycle(
|
||||||
|
status=lifecycle_raw.get("status", "active"),
|
||||||
|
visibility=lifecycle_raw.get("visibility", "standard"),
|
||||||
|
replacement=lifecycle_raw.get("replacement"),
|
||||||
|
detail=lifecycle_raw.get("detail", ""),
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
lifecycle.status not in {"active", "deprecated"}
|
||||||
|
or lifecycle.visibility not in {"standard", "diagnostic_only"}
|
||||||
|
or (
|
||||||
|
lifecycle.replacement is not None
|
||||||
|
and (
|
||||||
|
not isinstance(lifecycle.replacement, str)
|
||||||
|
or not re.fullmatch(
|
||||||
|
r"[a-z][a-z0-9_.-]+@v[1-9][0-9]*", lifecycle.replacement
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
or not isinstance(lifecycle.detail, str)
|
||||||
|
):
|
||||||
|
raise RuntimeError(f"unsafe lifecycle spec: {path.name}")
|
||||||
Draft202012Validator.check_schema(raw["request_schema"])
|
Draft202012Validator.check_schema(raw["request_schema"])
|
||||||
return CapabilityContract(
|
return CapabilityContract(
|
||||||
capability=capability,
|
capability=capability,
|
||||||
|
|
@ -248,6 +313,7 @@ def _load_contract(path: Path) -> CapabilityContract:
|
||||||
summary=raw["summary"],
|
summary=raw["summary"],
|
||||||
legacy_runtime=raw["legacy_runtime"],
|
legacy_runtime=raw["legacy_runtime"],
|
||||||
workspace=workspace,
|
workspace=workspace,
|
||||||
|
lifecycle=lifecycle,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -283,6 +349,15 @@ class _ContractRegistry:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
"software contracts are missing or contain duplicate capabilities"
|
"software contracts are missing or contain duplicate capabilities"
|
||||||
)
|
)
|
||||||
|
for item in loaded:
|
||||||
|
replacement = item.lifecycle.replacement
|
||||||
|
if replacement is not None and (
|
||||||
|
replacement == item.capability or replacement not in contracts
|
||||||
|
):
|
||||||
|
raise RuntimeError(
|
||||||
|
f"software contract has invalid lifecycle replacement: "
|
||||||
|
f"{item.capability}"
|
||||||
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
if initial:
|
if initial:
|
||||||
raise
|
raise
|
||||||
|
|
|
||||||
|
|
@ -841,6 +841,9 @@ def offer_next_job(node_ids: set[UUID]) -> dict | None:
|
||||||
"capability": job.capability,
|
"capability": job.capability,
|
||||||
"request_digest": job.request_digest,
|
"request_digest": job.request_digest,
|
||||||
"request": job.request,
|
"request": job.request,
|
||||||
|
# Node 的本机任务列表只消费合同生成的通用摘要,不解析各软件
|
||||||
|
# 私有 request 结构。旧 Node 会忽略这个新增字段。
|
||||||
|
"request_summary": _request_summary(job),
|
||||||
"workspace": (
|
"workspace": (
|
||||||
{
|
{
|
||||||
"workspace_id": str(job.workspace_id),
|
"workspace_id": str(job.workspace_id),
|
||||||
|
|
|
||||||
|
|
@ -8,13 +8,19 @@ from hashlib import sha256
|
||||||
from uuid import UUID, uuid4
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
import bcrypt
|
import bcrypt
|
||||||
from sqlalchemy import select, update
|
from sqlalchemy import func, select, update
|
||||||
|
|
||||||
from core.software_contracts import supported_capabilities
|
from core.software_contracts import supported_capabilities
|
||||||
from core.storage.engine import session_scope
|
from core.storage.engine import session_scope
|
||||||
from core.storage.models import SoftwareNode, SoftwareNodeEnrollment, SoftwareWorkspace
|
from core.storage.models import (
|
||||||
|
SoftwareJob,
|
||||||
|
SoftwareNode,
|
||||||
|
SoftwareNodeEnrollment,
|
||||||
|
SoftwareWorkspace,
|
||||||
|
)
|
||||||
|
|
||||||
MAX_ENROLLMENT_FAILURES = 5
|
MAX_ENROLLMENT_FAILURES = 5
|
||||||
|
ACTIVE_JOB_STATUSES = {"offered", "dispatched", "running", "disconnected", "cancelling"}
|
||||||
|
|
||||||
|
|
||||||
class SoftwareNodeError(Exception):
|
class SoftwareNodeError(Exception):
|
||||||
|
|
@ -208,6 +214,16 @@ def list_nodes() -> list[dict]:
|
||||||
.scalars()
|
.scalars()
|
||||||
.all()
|
.all()
|
||||||
)
|
)
|
||||||
|
active_job_counts = dict(
|
||||||
|
session.execute(
|
||||||
|
select(SoftwareJob.node_id, func.count(SoftwareJob.job_id))
|
||||||
|
.where(
|
||||||
|
SoftwareJob.node_id.is_not(None),
|
||||||
|
SoftwareJob.status.in_(ACTIVE_JOB_STATUSES),
|
||||||
|
)
|
||||||
|
.group_by(SoftwareJob.node_id)
|
||||||
|
).all()
|
||||||
|
)
|
||||||
return [
|
return [
|
||||||
{
|
{
|
||||||
"node_id": str(row.node_id),
|
"node_id": str(row.node_id),
|
||||||
|
|
@ -217,6 +233,7 @@ def list_nodes() -> list[dict]:
|
||||||
"node_version": row.node_version,
|
"node_version": row.node_version,
|
||||||
"os_version": row.os_version,
|
"os_version": row.os_version,
|
||||||
"runtime": row.runtime,
|
"runtime": row.runtime,
|
||||||
|
"active_job_count": int(active_job_counts.get(row.node_id, 0)),
|
||||||
"last_seen_at": row.last_seen_at.isoformat()
|
"last_seen_at": row.last_seen_at.isoformat()
|
||||||
if row.last_seen_at
|
if row.last_seen_at
|
||||||
else None,
|
else None,
|
||||||
|
|
|
||||||
|
|
@ -14,9 +14,14 @@ from .engine import (
|
||||||
)
|
)
|
||||||
from .telemetry import (
|
from .telemetry import (
|
||||||
record_empty_response,
|
record_empty_response,
|
||||||
|
record_agent_guard,
|
||||||
|
record_context_fold_failure,
|
||||||
record_malformed_tool_call,
|
record_malformed_tool_call,
|
||||||
|
record_quality_gate,
|
||||||
record_run_error,
|
record_run_error,
|
||||||
|
record_run_stopped,
|
||||||
record_salvaged_tool_call,
|
record_salvaged_tool_call,
|
||||||
|
record_tool_failure,
|
||||||
)
|
)
|
||||||
from .usage import record_chat_usage
|
from .usage import record_chat_usage
|
||||||
from .utils import (
|
from .utils import (
|
||||||
|
|
@ -35,10 +40,15 @@ __all__ = [
|
||||||
"get_engine",
|
"get_engine",
|
||||||
"get_task",
|
"get_task",
|
||||||
"record_chat_usage",
|
"record_chat_usage",
|
||||||
|
"record_agent_guard",
|
||||||
|
"record_context_fold_failure",
|
||||||
"record_empty_response",
|
"record_empty_response",
|
||||||
"record_malformed_tool_call",
|
"record_malformed_tool_call",
|
||||||
|
"record_quality_gate",
|
||||||
"record_run_error",
|
"record_run_error",
|
||||||
|
"record_run_stopped",
|
||||||
"record_salvaged_tool_call",
|
"record_salvaged_tool_call",
|
||||||
|
"record_tool_failure",
|
||||||
"session_scope",
|
"session_scope",
|
||||||
"update_task",
|
"update_task",
|
||||||
"upsert_task",
|
"upsert_task",
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,9 @@
|
||||||
"""Per-user 工作目录配额(§7.5 #4 软配额,应用层 gate)。
|
"""Per-user 工作目录配额(§7.5 #4 软配额,应用层 gate)。
|
||||||
|
|
||||||
调用入口:
|
调用入口:
|
||||||
- `scan_user_dir(user_root) -> (bytes, count)` ── os.walk 累加,跳 dotfile / 损坏 stat
|
- `scan_user_dir(user_root) -> (bytes, count)` ── 累加计入用户配额的文件
|
||||||
- `upsert_user_usage(user_id, bytes, count)` ── 落 user_disk_usage 表
|
- `scan_user_trash(user_root) -> (bytes, count)` ── 单独统计隐藏回收站物理占用
|
||||||
|
- `upsert_user_usage(...)` ── 同时落配额占用与回收站快照
|
||||||
- `check_disk_quota(user_id, limit_bytes) -> Optional[str]` ── 写前查,返 None=放行 /
|
- `check_disk_quota(user_id, limit_bytes) -> Optional[str]` ── 写前查,返 None=放行 /
|
||||||
str=拒绝原因。`limit_bytes <= 0` 短路放行(不限)
|
str=拒绝原因。`limit_bytes <= 0` 短路放行(不限)
|
||||||
- `scan_all_users(user_root_base, limit_bytes)` ── lifespan 后台 task 周期跑,
|
- `scan_all_users(user_root_base, limit_bytes)` ── lifespan 后台 task 周期跑,
|
||||||
|
|
@ -64,15 +65,18 @@ def parse_bytes(value) -> Optional[int]:
|
||||||
_SKIP_TOPLEVEL = frozenset({
|
_SKIP_TOPLEVEL = frozenset({
|
||||||
".zcbot_cache",
|
".zcbot_cache",
|
||||||
".zcbot_external_results",
|
".zcbot_external_results",
|
||||||
|
".zcbot_artifact_trash",
|
||||||
".zcbot_tmp",
|
".zcbot_tmp",
|
||||||
".memory",
|
".memory",
|
||||||
})
|
})
|
||||||
|
_ARTIFACT_TRASH_DIR = ".zcbot_artifact_trash"
|
||||||
|
|
||||||
|
|
||||||
def scan_user_dir(user_root: Path) -> Tuple[int, int]:
|
def scan_user_dir(user_root: Path) -> Tuple[int, int]:
|
||||||
"""os.walk 累加 user_root 下所有文件大小,返 (bytes, count)。
|
"""os.walk 累加 user_root 下所有文件大小,返 (bytes, count)。
|
||||||
|
|
||||||
跳过顶层平台缓存/临时区与 .memory(均有独立生命周期,不算入产品配额);
|
跳过顶层平台缓存/临时区、隐藏回收站与 .memory(均有独立生命周期,
|
||||||
|
不算入产品配额);
|
||||||
follow_symlinks=False 防 symlink 循环爆。
|
follow_symlinks=False 防 symlink 循环爆。
|
||||||
"""
|
"""
|
||||||
if not user_root.exists() or not user_root.is_dir():
|
if not user_root.exists() or not user_root.is_dir():
|
||||||
|
|
@ -102,6 +106,14 @@ def scan_user_dir(user_root: Path) -> Tuple[int, int]:
|
||||||
return total_bytes, total_count
|
return total_bytes, total_count
|
||||||
|
|
||||||
|
|
||||||
|
def scan_user_trash(user_root: Path) -> Tuple[int, int]:
|
||||||
|
"""统计隐藏 artifact 回收站的物理占用,不计入用户配额。"""
|
||||||
|
trash = user_root / _ARTIFACT_TRASH_DIR
|
||||||
|
if not trash.is_dir():
|
||||||
|
return 0, 0
|
||||||
|
return _walk_dir(trash)
|
||||||
|
|
||||||
|
|
||||||
def _walk_dir(d: Path) -> Tuple[int, int]:
|
def _walk_dir(d: Path) -> Tuple[int, int]:
|
||||||
total_b, total_c = 0, 0
|
total_b, total_c = 0, 0
|
||||||
for root, dirs, files in os.walk(d, followlinks=False, onerror=lambda _e: None):
|
for root, dirs, files in os.walk(d, followlinks=False, onerror=lambda _e: None):
|
||||||
|
|
@ -115,19 +127,29 @@ def _walk_dir(d: Path) -> Tuple[int, int]:
|
||||||
return total_b, total_c
|
return total_b, total_c
|
||||||
|
|
||||||
|
|
||||||
def upsert_user_usage(user_id: UUID, bytes_used: int, file_count: int) -> None:
|
def upsert_user_usage(
|
||||||
"""落 user_disk_usage 单行;首次 INSERT,后续 UPDATE。"""
|
user_id: UUID,
|
||||||
|
bytes_used: int,
|
||||||
|
file_count: int,
|
||||||
|
trash_bytes: int = 0,
|
||||||
|
trash_file_count: int = 0,
|
||||||
|
) -> None:
|
||||||
|
"""落 user_disk_usage 单行;首次 INSERT,后续 UPDATE。"""
|
||||||
from sqlalchemy import func
|
from sqlalchemy import func
|
||||||
with session_scope() as s:
|
with session_scope() as s:
|
||||||
stmt = pg_insert(UserDiskUsage).values(
|
stmt = pg_insert(UserDiskUsage).values(
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
bytes_used=bytes_used,
|
bytes_used=bytes_used,
|
||||||
file_count=file_count,
|
file_count=file_count,
|
||||||
|
trash_bytes=trash_bytes,
|
||||||
|
trash_file_count=trash_file_count,
|
||||||
).on_conflict_do_update(
|
).on_conflict_do_update(
|
||||||
index_elements=["user_id"],
|
index_elements=["user_id"],
|
||||||
set_={
|
set_={
|
||||||
"bytes_used": bytes_used,
|
"bytes_used": bytes_used,
|
||||||
"file_count": file_count,
|
"file_count": file_count,
|
||||||
|
"trash_bytes": trash_bytes,
|
||||||
|
"trash_file_count": trash_file_count,
|
||||||
"scanned_at": func.now(),
|
"scanned_at": func.now(),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
@ -202,8 +224,10 @@ def scan_all_users(user_root_base: Path) -> int:
|
||||||
count = 0
|
count = 0
|
||||||
for uid in list_user_ids_with_root(user_root_base):
|
for uid in list_user_ids_with_root(user_root_base):
|
||||||
try:
|
try:
|
||||||
b, c = scan_user_dir(user_root_base / str(uid))
|
user_root = user_root_base / str(uid)
|
||||||
upsert_user_usage(uid, b, c)
|
b, c = scan_user_dir(user_root)
|
||||||
|
trash_b, trash_c = scan_user_trash(user_root)
|
||||||
|
upsert_user_usage(uid, b, c, trash_b, trash_c)
|
||||||
count += 1
|
count += 1
|
||||||
except Exception:
|
except Exception:
|
||||||
# 单 user 扫挂不阻塞其他 user;下次周期重试。日志靠 caller 注入。
|
# 单 user 扫挂不阻塞其他 user;下次周期重试。日志靠 caller 注入。
|
||||||
|
|
|
||||||
|
|
@ -258,6 +258,48 @@ class WebPreview(Base):
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderCredential(Base):
|
||||||
|
"""平台级可信 Provider 当前凭据与健康状态;credentials 只保存字段密文。"""
|
||||||
|
|
||||||
|
__tablename__ = "provider_credentials"
|
||||||
|
|
||||||
|
provider_id: Mapped[str] = mapped_column(Text, primary_key=True)
|
||||||
|
credentials: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
||||||
|
credential_hint: Mapped[dict[str, Any]] = mapped_column(
|
||||||
|
JSONB, nullable=False, default=dict
|
||||||
|
)
|
||||||
|
revision: Mapped[int] = mapped_column(
|
||||||
|
Integer, nullable=False, default=1, server_default="1"
|
||||||
|
)
|
||||||
|
test_status: Mapped[str] = mapped_column(
|
||||||
|
Text, nullable=False, default="untested", server_default="untested"
|
||||||
|
)
|
||||||
|
test_detail: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
balance_amount: Mapped[Optional[Decimal]] = mapped_column(
|
||||||
|
Numeric(18, 6), nullable=True
|
||||||
|
)
|
||||||
|
balance_currency: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
last_tested_at: Mapped[Optional[datetime]] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=True
|
||||||
|
)
|
||||||
|
last_notified_at: Mapped[Optional[datetime]] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=True
|
||||||
|
)
|
||||||
|
last_notified_status: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
updated_by: Mapped[Optional[UUID]] = mapped_column(
|
||||||
|
PG_UUID(as_uuid=True),
|
||||||
|
ForeignKey("users.user_id", ondelete="SET NULL"),
|
||||||
|
nullable=True,
|
||||||
|
)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||||
|
)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), server_default=func.now(),
|
||||||
|
onupdate=func.now(), nullable=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class Artifact(Base):
|
class Artifact(Base):
|
||||||
"""Stable identity and lifecycle metadata for a published workspace file."""
|
"""Stable identity and lifecycle metadata for a published workspace file."""
|
||||||
|
|
||||||
|
|
@ -324,6 +366,22 @@ class UsageEvent(Base):
|
||||||
__tablename__ = "usage_events"
|
__tablename__ = "usage_events"
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
Index("ix_usage_created_brin", "created_at", postgresql_using="brin"),
|
Index("ix_usage_created_brin", "created_at", postgresql_using="brin"),
|
||||||
|
Index(
|
||||||
|
"ix_usage_health_kind_created",
|
||||||
|
"kind",
|
||||||
|
"created_at",
|
||||||
|
postgresql_where=text(
|
||||||
|
"kind IN ('tool_failure', 'tool_malformed', 'tool_salvaged', "
|
||||||
|
"'run_error', 'empty_response', 'run_stopped', 'agent_guard', "
|
||||||
|
"'context_fold_failure', 'quality_gate')"
|
||||||
|
),
|
||||||
|
),
|
||||||
|
Index(
|
||||||
|
"ux_usage_tool_failure_message",
|
||||||
|
"message_id",
|
||||||
|
unique=True,
|
||||||
|
postgresql_where=text("kind IN ('tool_failure', 'quality_gate')"),
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
event_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
event_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
||||||
|
|
@ -355,8 +413,9 @@ class UserDiskUsage(Base):
|
||||||
"""per-user 工作目录字节使用快照(0008,§7.5 #4 软配额表)。
|
"""per-user 工作目录字节使用快照(0008,§7.5 #4 软配额表)。
|
||||||
|
|
||||||
每个 user_id 单行 upsert,lifespan 后台 task 周期(默 15min)扫描 user_root 落库;
|
每个 user_id 单行 upsert,lifespan 后台 task 周期(默 15min)扫描 user_root 落库;
|
||||||
write 前 gate(DockerExecutor / /v1/files/upload)查这表对比 yaml `quotas.disk_bytes_per_user`,
|
write 前 gate(DockerExecutor / /v1/files/upload)用 bytes_used 对比 yaml
|
||||||
超额返 [Error] 硬阻。
|
`quotas.disk_bytes_per_user`,超额返 [Error] 硬阻。trash_* 只记录隐藏回收站
|
||||||
|
的物理占用,不参与用户配额判断。
|
||||||
|
|
||||||
扫描间隙写入会突破上限一点(race-tolerant,跟 image/video 配额一致接受);外部用户
|
扫描间隙写入会突破上限一点(race-tolerant,跟 image/video 配额一致接受);外部用户
|
||||||
开放前 OS 层 xfs prjquota 兜底真上限。详 DESIGN §7.5 #4 / PROGRESS。
|
开放前 OS 层 xfs prjquota 兜底真上限。详 DESIGN §7.5 #4 / PROGRESS。
|
||||||
|
|
@ -371,6 +430,8 @@ class UserDiskUsage(Base):
|
||||||
)
|
)
|
||||||
bytes_used: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
|
bytes_used: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
|
||||||
file_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
file_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
trash_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
|
||||||
|
trash_file_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
scanned_at: Mapped[datetime] = mapped_column(
|
scanned_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||||
)
|
)
|
||||||
|
|
@ -801,3 +862,27 @@ class ExternalSystemAudit(Base):
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class SandboxPackageScan(Base):
|
||||||
|
"""一个发现临时 Python 包的容器会话一行(空扫描不入库)。"""
|
||||||
|
|
||||||
|
__tablename__ = "sandbox_package_scans"
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ix_sandbox_package_scans_finished_at", "finished_at"),
|
||||||
|
Index("ix_sandbox_package_scans_user_finished", "user_id", "finished_at"),
|
||||||
|
)
|
||||||
|
container_session_id: Mapped[str] = mapped_column(Text, primary_key=True)
|
||||||
|
user_id: Mapped[UUID] = mapped_column(
|
||||||
|
PG_UUID(as_uuid=True), ForeignKey("users.user_id", ondelete="CASCADE"), nullable=False
|
||||||
|
)
|
||||||
|
execution_kind: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
image_digest: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
python_version: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
packages: Mapped[list[Any]] = mapped_column(JSONB, nullable=False)
|
||||||
|
package_count: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
total_installed_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False)
|
||||||
|
finished_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||||
|
)
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
"""失败埋点(telemetry)—— 与计费(usage.py)分家的另一类 usage_events 写入。
|
"""失败埋点(telemetry)—— 与计费(usage.py)分家的另一类 usage_events 写入。
|
||||||
|
|
||||||
judged by kind:这里的四类 cost 恒 0、tokens 不入 kind=chat 汇总,语义是**可观测性
|
这里的异常类 kind 均 cost 恒 0、tokens 不入 kind=chat 汇总,语义是**可观测性
|
||||||
留痕**而非用户花销 —— 它们是 admin「工具失败聚集」面板与巡检邮件(core/toolfail.py)
|
留痕**而非用户花销 —— 它们是 admin「工具失败聚集」面板与巡检邮件(core/toolfail.py)
|
||||||
的唯一持久数据源。此前与真计费(chat/image/video/vision)混住 usage.py,污染
|
的唯一持久数据源。此前与真计费(chat/image/video/vision)混住 usage.py,污染
|
||||||
「usage=计费」的心智模型(架构审查数据层 Top4),2026-07-23 拆出。
|
「usage=计费」的心智模型(架构审查数据层 Top4),2026-07-23 拆出。
|
||||||
|
|
@ -11,6 +11,7 @@ kind 常量是 loop/llm_transport(写)与 toolfail(读 SQL)之间的契约单一
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
from typing import Any, Optional
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
||||||
from .engine import session_scope
|
from .engine import session_scope
|
||||||
|
|
@ -19,8 +20,136 @@ from .models import UsageEvent
|
||||||
# ── kind 契约常量(写侧本模块 / 读侧 core/toolfail.py 共用)──
|
# ── kind 契约常量(写侧本模块 / 读侧 core/toolfail.py 共用)──
|
||||||
KIND_TOOL_MALFORMED = "tool_malformed"
|
KIND_TOOL_MALFORMED = "tool_malformed"
|
||||||
KIND_TOOL_SALVAGED = "tool_salvaged"
|
KIND_TOOL_SALVAGED = "tool_salvaged"
|
||||||
|
KIND_TOOL_FAILURE = "tool_failure"
|
||||||
KIND_EMPTY_RESPONSE = "empty_response"
|
KIND_EMPTY_RESPONSE = "empty_response"
|
||||||
KIND_RUN_ERROR = "run_error"
|
KIND_RUN_ERROR = "run_error"
|
||||||
|
KIND_RUN_STOPPED = "run_stopped"
|
||||||
|
KIND_AGENT_GUARD = "agent_guard"
|
||||||
|
KIND_CONTEXT_FOLD_FAILURE = "context_fold_failure"
|
||||||
|
KIND_QUALITY_GATE = "quality_gate"
|
||||||
|
|
||||||
|
|
||||||
|
def _record_health_event(
|
||||||
|
*,
|
||||||
|
kind: str,
|
||||||
|
task_id: UUID,
|
||||||
|
user_id: UUID,
|
||||||
|
model_profile: str,
|
||||||
|
units: dict[str, Any],
|
||||||
|
message_id: Optional[UUID] = None,
|
||||||
|
) -> None:
|
||||||
|
with session_scope() as s:
|
||||||
|
s.add(UsageEvent(
|
||||||
|
user_id=user_id,
|
||||||
|
task_id=task_id,
|
||||||
|
message_id=message_id,
|
||||||
|
kind=kind,
|
||||||
|
model_profile=model_profile,
|
||||||
|
units=units,
|
||||||
|
cost_cny=Decimal("0"),
|
||||||
|
))
|
||||||
|
|
||||||
|
|
||||||
|
def record_tool_failure(
|
||||||
|
*,
|
||||||
|
task_id: UUID,
|
||||||
|
user_id: UUID,
|
||||||
|
message_id: UUID,
|
||||||
|
model_profile: str,
|
||||||
|
tool: str,
|
||||||
|
content: str,
|
||||||
|
arguments=None,
|
||||||
|
) -> bool:
|
||||||
|
"""结构化记录普通工具失败;正常结果不写入并返回 False。"""
|
||||||
|
from core.tool_failure import structured_failure
|
||||||
|
|
||||||
|
units = structured_failure(tool, content, arguments=arguments)
|
||||||
|
if units is None:
|
||||||
|
return False
|
||||||
|
event_kind = KIND_QUALITY_GATE if units["category"] == "quality_gate" else KIND_TOOL_FAILURE
|
||||||
|
_record_health_event(
|
||||||
|
kind=event_kind,
|
||||||
|
task_id=task_id,
|
||||||
|
user_id=user_id,
|
||||||
|
message_id=message_id,
|
||||||
|
model_profile=model_profile,
|
||||||
|
units=units,
|
||||||
|
)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def record_run_stopped(
|
||||||
|
*, task_id: UUID, user_id: UUID, model_profile: str, reason: str,
|
||||||
|
) -> None:
|
||||||
|
_record_health_event(
|
||||||
|
kind=KIND_RUN_STOPPED,
|
||||||
|
task_id=task_id,
|
||||||
|
user_id=user_id,
|
||||||
|
model_profile=model_profile,
|
||||||
|
units={
|
||||||
|
"tool": "(run)", "failure_kind": "stopped", "signature": reason,
|
||||||
|
"category": "agent_control", "sample": reason,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def record_agent_guard(
|
||||||
|
*,
|
||||||
|
task_id: UUID,
|
||||||
|
user_id: UUID,
|
||||||
|
model_profile: str,
|
||||||
|
tool: str,
|
||||||
|
guard: str,
|
||||||
|
count: int,
|
||||||
|
signature: str = "",
|
||||||
|
) -> None:
|
||||||
|
from core.tool_failure import normalize_failure_signature
|
||||||
|
|
||||||
|
stable = normalize_failure_signature(f"{guard}: {signature}" if signature else guard)
|
||||||
|
_record_health_event(
|
||||||
|
kind=KIND_AGENT_GUARD,
|
||||||
|
task_id=task_id,
|
||||||
|
user_id=user_id,
|
||||||
|
model_profile=model_profile,
|
||||||
|
units={
|
||||||
|
"tool": tool, "failure_kind": "guard", "signature": stable,
|
||||||
|
"category": "agent_control", "sample": f"{guard}, count={count}",
|
||||||
|
"count": int(count),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def record_context_fold_failure(
|
||||||
|
*, task_id: UUID, user_id: UUID, model_profile: str, reason: str, detail: str = "",
|
||||||
|
) -> None:
|
||||||
|
from core.tool_failure import normalize_failure_signature
|
||||||
|
|
||||||
|
_record_health_event(
|
||||||
|
kind=KIND_CONTEXT_FOLD_FAILURE,
|
||||||
|
task_id=task_id,
|
||||||
|
user_id=user_id,
|
||||||
|
model_profile=model_profile,
|
||||||
|
units={
|
||||||
|
"tool": "(context)", "failure_kind": "context", "signature": reason,
|
||||||
|
"category": "agent_control",
|
||||||
|
"sample": normalize_failure_signature(detail) if detail else reason,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def record_quality_gate(
|
||||||
|
*, task_id: UUID, user_id: UUID, model_profile: str, tool: str, gate: str,
|
||||||
|
) -> None:
|
||||||
|
_record_health_event(
|
||||||
|
kind=KIND_QUALITY_GATE,
|
||||||
|
task_id=task_id,
|
||||||
|
user_id=user_id,
|
||||||
|
model_profile=model_profile,
|
||||||
|
units={
|
||||||
|
"tool": tool, "failure_kind": "gate", "signature": gate,
|
||||||
|
"category": "quality_gate", "sample": gate,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def record_malformed_tool_call(
|
def record_malformed_tool_call(
|
||||||
|
|
|
||||||
|
|
@ -3,19 +3,20 @@
|
||||||
chat 类型的入口由 loop.py 在 assistant message 入库后调用;媒体工具(image/video/audio)
|
chat 类型的入口由 loop.py 在 assistant message 入库后调用;媒体工具(image/video/audio)
|
||||||
在 tool execute 完后由 tool 直接调用对应入口(record_image_usage 等)。
|
在 tool execute 完后由 tool 直接调用对应入口(record_image_usage 等)。
|
||||||
|
|
||||||
币种(0007):全表统一 CNY(`cost_cny` 列)。chat 路径走 litellm 的 USD cost_map → 内部
|
币种(0007):全表统一 CNY(`cost_cny` 列)。有版本化本地价格的直连模型以该配置为
|
||||||
×USD_TO_CNY 折算落库;媒体路径价格本身就是 CNY,直接落。units jsonb 里 snapshot 当时
|
事实源;未配置的网关模型才走 LiteLLM USD cost map,再退到扁平人民币价格。
|
||||||
的关键价格参数(chat 没有,media 存 price_cny_per_image 等),便于跨调价对账。
|
units jsonb 快照当时价格版本、时段、币种、汇率和分项成本,便于跨调价对账。
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime, timezone
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
from typing import Any, Mapping, Optional
|
from typing import Any, Mapping, Optional
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
||||||
from sqlalchemy import func, select, update
|
from sqlalchemy import func, select, update
|
||||||
|
|
||||||
|
from ..pricing import calculate_chat_cost, pricing_snapshot, resolve_chat_price
|
||||||
from .engine import session_scope
|
from .engine import session_scope
|
||||||
from .models import Message, UsageEvent
|
from .models import Message, UsageEvent
|
||||||
|
|
||||||
|
|
@ -88,6 +89,8 @@ def record_chat_usage(
|
||||||
output_cny_per_mtoken: float = 0.0,
|
output_cny_per_mtoken: float = 0.0,
|
||||||
cache_hit_tokens: int = 0,
|
cache_hit_tokens: int = 0,
|
||||||
cache_hit_cny_per_mtoken: float = 0.0,
|
cache_hit_cny_per_mtoken: float = 0.0,
|
||||||
|
pricing: Optional[Mapping[str, Any]] = None,
|
||||||
|
occurred_at: Optional[datetime] = None,
|
||||||
extra_units: Optional[Mapping[str, Any]] = None,
|
extra_units: Optional[Mapping[str, Any]] = None,
|
||||||
response: Any = None,
|
response: Any = None,
|
||||||
kind: str = "chat",
|
kind: str = "chat",
|
||||||
|
|
@ -103,9 +106,32 @@ def record_chat_usage(
|
||||||
可挂,溯源靠 kind + extra_units(kb_ingest 带 {"kb": 库名, "source": 原件名})。
|
可挂,溯源靠 kind + extra_units(kb_ingest 带 {"kb": 库名, "source": 原件名})。
|
||||||
返回算出的 cost_cny(已落库),调用方可用作 SSE 显示。
|
返回算出的 cost_cny(已落库),调用方可用作 SSE 显示。
|
||||||
"""
|
"""
|
||||||
cost_usd = _safe_chat_cost_usd(response)
|
# 直连 provider 的版本化本地价格是事实源,避免 LiteLLM 内置价目滞后覆盖已核对
|
||||||
cost_cny = (cost_usd * USD_TO_CNY).quantize(Decimal("0.000001"))
|
# 的厂商价格。无本地适用价格的旧档案/网关模型才沿用 LiteLLM → 扁平价兜底。
|
||||||
if cost_cny == 0 and (input_cny_per_mtoken or output_cny_per_mtoken):
|
when = occurred_at
|
||||||
|
if when is None:
|
||||||
|
created = getattr(response, "created", None)
|
||||||
|
try:
|
||||||
|
when = datetime.fromtimestamp(float(created), tz=timezone.utc) if created else None
|
||||||
|
except (TypeError, ValueError, OSError):
|
||||||
|
when = None
|
||||||
|
when = when or datetime.now(timezone.utc)
|
||||||
|
quote = resolve_chat_price(pricing, occurred_at=when)
|
||||||
|
price_units: dict[str, Any] = {}
|
||||||
|
if quote is not None:
|
||||||
|
breakdown = calculate_chat_cost(
|
||||||
|
quote,
|
||||||
|
prompt_tokens=prompt_tokens,
|
||||||
|
completion_tokens=completion_tokens,
|
||||||
|
cache_hit_tokens=cache_hit_tokens,
|
||||||
|
)
|
||||||
|
cost_cny = breakdown.total_cny
|
||||||
|
price_units = pricing_snapshot(quote, breakdown)
|
||||||
|
price_units["pricing_at"] = when.astimezone(timezone.utc).isoformat()
|
||||||
|
else:
|
||||||
|
cost_usd = _safe_chat_cost_usd(response)
|
||||||
|
cost_cny = (cost_usd * USD_TO_CNY).quantize(Decimal("0.000001"))
|
||||||
|
if quote is None and cost_cny == 0 and (input_cny_per_mtoken or output_cny_per_mtoken):
|
||||||
cost_cny = _fallback_chat_cost_cny(
|
cost_cny = _fallback_chat_cost_cny(
|
||||||
prompt_tokens=prompt_tokens,
|
prompt_tokens=prompt_tokens,
|
||||||
completion_tokens=completion_tokens,
|
completion_tokens=completion_tokens,
|
||||||
|
|
@ -114,19 +140,23 @@ def record_chat_usage(
|
||||||
cache_hit_tokens=cache_hit_tokens,
|
cache_hit_tokens=cache_hit_tokens,
|
||||||
cache_hit_cny_per_mtoken=cache_hit_cny_per_mtoken,
|
cache_hit_cny_per_mtoken=cache_hit_cny_per_mtoken,
|
||||||
)
|
)
|
||||||
units = {
|
units: dict[str, Any] = {
|
||||||
"tokens_in": int(prompt_tokens),
|
"tokens_in": int(prompt_tokens),
|
||||||
"tokens_out": int(completion_tokens),
|
"tokens_out": int(completion_tokens),
|
||||||
# snapshot 折算系数,便于历史对账(汇率/价格涨跌后仍能还原当时折算逻辑)
|
"cache_hit_tokens": max(0, min(int(cache_hit_tokens), int(prompt_tokens))),
|
||||||
"usd_to_cny": float(USD_TO_CNY),
|
|
||||||
}
|
}
|
||||||
|
if quote is None:
|
||||||
|
# LiteLLM cost map 是 USD;快照折算系数供旧路径历史对账。
|
||||||
|
units["usd_to_cny"] = float(USD_TO_CNY)
|
||||||
if input_cny_per_mtoken or output_cny_per_mtoken:
|
if input_cny_per_mtoken or output_cny_per_mtoken:
|
||||||
units.update({
|
units.update({
|
||||||
"input_cny_per_mtoken": float(input_cny_per_mtoken or 0),
|
"input_cny_per_mtoken": float(input_cny_per_mtoken or 0),
|
||||||
"output_cny_per_mtoken": float(output_cny_per_mtoken or 0),
|
"output_cny_per_mtoken": float(output_cny_per_mtoken or 0),
|
||||||
|
"cache_hit_cny_per_mtoken": float(cache_hit_cny_per_mtoken or 0),
|
||||||
})
|
})
|
||||||
if extra_units:
|
if extra_units:
|
||||||
units.update(extra_units)
|
units.update(extra_units)
|
||||||
|
units.update(price_units)
|
||||||
|
|
||||||
with session_scope() as s:
|
with session_scope() as s:
|
||||||
s.add(UsageEvent(
|
s.add(UsageEvent(
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,7 @@ from sqlalchemy import select, update
|
||||||
from .agent_builder import ROOT, load_config
|
from .agent_builder import ROOT, load_config
|
||||||
from .capabilities import ModelCapabilities
|
from .capabilities import ModelCapabilities
|
||||||
from .llm import LLM
|
from .llm import LLM
|
||||||
|
from .llm_transport import extract_usage_details
|
||||||
from .storage import session_scope
|
from .storage import session_scope
|
||||||
from .storage.models import Task
|
from .storage.models import Task
|
||||||
from .storage.usage import record_chat_usage
|
from .storage.usage import record_chat_usage
|
||||||
|
|
@ -127,17 +128,24 @@ def generate_task_title(
|
||||||
)
|
)
|
||||||
applied = bool(getattr(result, "rowcount", 0))
|
applied = bool(getattr(result, "rowcount", 0))
|
||||||
if response is not None and caps is not None:
|
if response is not None and caps is not None:
|
||||||
usage = getattr(response, "usage", None)
|
usage = extract_usage_details(getattr(response, "usage", None))
|
||||||
try:
|
try:
|
||||||
record_chat_usage(
|
record_chat_usage(
|
||||||
task_id=task_id,
|
task_id=task_id,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
message_id=None,
|
message_id=None,
|
||||||
model_profile=f"{caps.family}.{caps.variant}",
|
model_profile=f"{caps.family}.{caps.variant}",
|
||||||
prompt_tokens=getattr(usage, "prompt_tokens", 0) or 0,
|
prompt_tokens=usage["tokens_in"],
|
||||||
completion_tokens=getattr(usage, "completion_tokens", 0) or 0,
|
completion_tokens=usage["tokens_out"],
|
||||||
input_cny_per_mtoken=caps.input_cny_per_mtoken,
|
input_cny_per_mtoken=caps.input_cny_per_mtoken,
|
||||||
output_cny_per_mtoken=caps.output_cny_per_mtoken,
|
output_cny_per_mtoken=caps.output_cny_per_mtoken,
|
||||||
|
cache_hit_tokens=usage["cache_hit_tokens"],
|
||||||
|
cache_hit_cny_per_mtoken=getattr(caps, "cache_hit_cny_per_mtoken", 0.0),
|
||||||
|
pricing=getattr(caps, "pricing", {}) or {},
|
||||||
|
extra_units={
|
||||||
|
k: v for k, v in usage.items()
|
||||||
|
if k not in ("tokens_in", "tokens_out") and v
|
||||||
|
},
|
||||||
response=response,
|
response=response,
|
||||||
kind="task_title",
|
kind="task_title",
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,139 @@
|
||||||
|
"""工具结果失败分类的单一事实源。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
from typing import Any, Optional, Tuple
|
||||||
|
|
||||||
|
|
||||||
|
_RE_UUID = re.compile(
|
||||||
|
r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-"
|
||||||
|
r"[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"
|
||||||
|
)
|
||||||
|
_RE_HEX = re.compile(r"0x[0-9a-fA-F]+")
|
||||||
|
_RE_PATH = re.compile(r"(?:[A-Za-z]:)?(?:[/\\][\w.\-一-鿿*]+){2,}")
|
||||||
|
_RE_NUM = re.compile(r"\d+")
|
||||||
|
_RE_WS = re.compile(r"\s+")
|
||||||
|
_EXIT_TAIL = re.compile(r"\[exit (\d+)\]\s*$")
|
||||||
|
_STREAM_MARKS = ("[stdout]", "[stderr]")
|
||||||
|
_BARE_EXIT_RE = re.compile(r"^exit \d+$")
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_failure_signature(value: str) -> str:
|
||||||
|
value = _RE_UUID.sub("<id>", value)
|
||||||
|
value = _RE_HEX.sub("<hex>", value)
|
||||||
|
value = _RE_PATH.sub("<path>", value)
|
||||||
|
value = _RE_NUM.sub("N", value)
|
||||||
|
return _RE_WS.sub(" ", value).strip()[:120]
|
||||||
|
|
||||||
|
|
||||||
|
def classify_failure(content: str) -> Optional[Tuple[str, str]]:
|
||||||
|
"""返回 ``(kind, 原始签名行)``;正常工具结果返回 None。"""
|
||||||
|
head = content.lstrip()
|
||||||
|
if "command timed out" in content:
|
||||||
|
return "timeout", "command timed out"
|
||||||
|
if head.startswith("[Error"):
|
||||||
|
return "error", head.splitlines()[0]
|
||||||
|
match = _EXIT_TAIL.search(content)
|
||||||
|
if match and match.group(1) != "0":
|
||||||
|
lines = [
|
||||||
|
line.strip()
|
||||||
|
for line in content.splitlines()[:-1]
|
||||||
|
if line.strip() and line.strip() not in _STREAM_MARKS
|
||||||
|
]
|
||||||
|
return "exit", (lines[-1] if lines else f"exit {match.group(1)}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def is_bare_exit_signature(value: str) -> bool:
|
||||||
|
return _BARE_EXIT_RE.match(value) is not None
|
||||||
|
|
||||||
|
|
||||||
|
def _command_hint(command: str) -> str:
|
||||||
|
command = command.strip().lower()
|
||||||
|
if not command:
|
||||||
|
return ""
|
||||||
|
if re.search(r"\b(?:apt|dpkg|pip)\b.*\b(?:list|show)\b", command):
|
||||||
|
return "dependency probe"
|
||||||
|
if re.search(r"(?:^|[;&|]\s*)(?:which|whereis|command\s+-v)\b", command):
|
||||||
|
return "dependency probe"
|
||||||
|
if re.search(r"(?:^|[;&|]\s*)(?:grep|rg)\b", command):
|
||||||
|
return "search/no match"
|
||||||
|
match = re.search(r"(?:^|&&|;|\|)\s*([a-z0-9_.-]+)", command)
|
||||||
|
return match.group(1) if match else ""
|
||||||
|
|
||||||
|
|
||||||
|
def shell_command_hint(prior_payload: Any, tool_call_id: str) -> str:
|
||||||
|
"""从历史 assistant tool_call 中提取稳定的 shell 命令类别。"""
|
||||||
|
if not isinstance(prior_payload, dict):
|
||||||
|
return ""
|
||||||
|
calls = prior_payload.get("tool_calls") or []
|
||||||
|
candidates = []
|
||||||
|
for call in calls:
|
||||||
|
if not isinstance(call, dict):
|
||||||
|
continue
|
||||||
|
function = call.get("function") or {}
|
||||||
|
if function.get("name") != "shell":
|
||||||
|
continue
|
||||||
|
if tool_call_id and call.get("id") == tool_call_id:
|
||||||
|
candidates = [call]
|
||||||
|
break
|
||||||
|
candidates.append(call)
|
||||||
|
if len(candidates) != 1:
|
||||||
|
return ""
|
||||||
|
return shell_command_hint_from_arguments(
|
||||||
|
(candidates[0].get("function") or {}).get("arguments")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def shell_command_hint_from_arguments(arguments: Any) -> str:
|
||||||
|
try:
|
||||||
|
args = json.loads(arguments) if isinstance(arguments, str) else arguments
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return ""
|
||||||
|
if not isinstance(args, dict):
|
||||||
|
return ""
|
||||||
|
return _command_hint(str(args.get("command") or ""))
|
||||||
|
|
||||||
|
|
||||||
|
def failure_category(kind: str, signature_line: str, sample: str) -> str:
|
||||||
|
if kind != "exit":
|
||||||
|
return "failure"
|
||||||
|
# 组合导出脚本会在 gate 行之后继续打印修复提示/命令,因此 gate 不一定是
|
||||||
|
# classify_failure 选中的最后一条签名行。按完整输出中的行首标记识别。
|
||||||
|
if signature_line.startswith("[GATE FAIL]") or re.search(
|
||||||
|
r"(?m)^\[GATE FAIL\]", sample
|
||||||
|
):
|
||||||
|
return "quality_gate"
|
||||||
|
if (
|
||||||
|
("[篇幅核算]" in sample or "[字数核算]" in sample)
|
||||||
|
and re.search(r"\[WARN\]\s*\d+\s*项超出\s*/\s*\d+\s*项不足", sample)
|
||||||
|
):
|
||||||
|
return "quality_gate"
|
||||||
|
if "[质量检查]" in sample and "[WARN] 共发现" in sample:
|
||||||
|
return "quality_gate"
|
||||||
|
return "failure"
|
||||||
|
|
||||||
|
|
||||||
|
def structured_failure(
|
||||||
|
tool: str,
|
||||||
|
content: str,
|
||||||
|
*,
|
||||||
|
arguments: Any = None,
|
||||||
|
) -> Optional[dict[str, str]]:
|
||||||
|
"""把工具结果转成可直接持久化的稳定失败事件。"""
|
||||||
|
hit = classify_failure(content)
|
||||||
|
if hit is None:
|
||||||
|
return None
|
||||||
|
kind, signature_line = hit
|
||||||
|
if tool == "shell" and kind == "exit" and _BARE_EXIT_RE.match(signature_line):
|
||||||
|
hint = shell_command_hint_from_arguments(arguments)
|
||||||
|
if hint:
|
||||||
|
signature_line = f"{signature_line} ({hint})"
|
||||||
|
return {
|
||||||
|
"tool": tool or "?",
|
||||||
|
"failure_kind": kind,
|
||||||
|
"signature": normalize_failure_signature(signature_line),
|
||||||
|
"category": failure_category(kind, signature_line, content),
|
||||||
|
"sample": content[:300],
|
||||||
|
}
|
||||||
|
|
@ -269,17 +269,24 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
|
||||||
cancel_check=ctx.cancel_check, daily_limit=videos_per_day, **base,
|
cancel_check=ctx.cancel_check, daily_limit=videos_per_day, **base,
|
||||||
)]
|
)]
|
||||||
|
|
||||||
def _vision() -> list:
|
def _vision_kwargs() -> Optional[dict]:
|
||||||
# 看图 + 文档理解共用 vision variant(读图不分档,固定第一个)。
|
# 看图 + 文档理解共用 vision variant(读图不分档,固定第一个)。
|
||||||
key, v = _pick_variant(ctx.ark_cfg.raw.get("vision") or {})
|
key, v = _pick_variant(ctx.ark_cfg.raw.get("vision") or {})
|
||||||
if v is None:
|
if v is None:
|
||||||
return []
|
return None
|
||||||
kw = dict(
|
return dict(
|
||||||
ark_cfg=ctx.ark_cfg, vision_variant_cfg=v, variant_key=key,
|
ark_cfg=ctx.ark_cfg, vision_variant_cfg=v, variant_key=key,
|
||||||
working_dir=ctx.working_dir_path, task_id=ctx.task_id, user_id=ctx.uid,
|
working_dir=ctx.working_dir_path, task_id=ctx.task_id, user_id=ctx.uid,
|
||||||
**base,
|
**base,
|
||||||
)
|
)
|
||||||
return [LookAtImageTool(**kw), ReadDocumentTool(**kw)]
|
|
||||||
|
def _look_at_image() -> list:
|
||||||
|
kw = _vision_kwargs()
|
||||||
|
return [LookAtImageTool(**kw)] if kw is not None else []
|
||||||
|
|
||||||
|
def _read_document() -> list:
|
||||||
|
kw = _vision_kwargs()
|
||||||
|
return [ReadDocumentTool(**kw)] if kw is not None else []
|
||||||
|
|
||||||
def _transcribe() -> list:
|
def _transcribe() -> list:
|
||||||
# 录音文件转写(讯飞 LFASR,与 IAT 听写是两个服务两套 key)。
|
# 录音文件转写(讯飞 LFASR,与 IAT 听写是两个服务两套 key)。
|
||||||
|
|
@ -316,7 +323,13 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
|
||||||
("office_to_pdf", lambda: ctx.office_to_pdf_available, _office_to_pdf),
|
("office_to_pdf", lambda: ctx.office_to_pdf_available, _office_to_pdf),
|
||||||
("image", lambda: ctx.img_cfg is not None, _image),
|
("image", lambda: ctx.img_cfg is not None, _image),
|
||||||
("video", lambda: ctx.ark_cfg is not None, _video),
|
("video", lambda: ctx.ark_cfg is not None, _video),
|
||||||
("vision", lambda: ctx.ark_cfg is not None, _vision),
|
(
|
||||||
|
"look_at_image",
|
||||||
|
lambda: ctx.ark_cfg is not None
|
||||||
|
and not getattr(ctx.caps, "native_image_input", False),
|
||||||
|
_look_at_image,
|
||||||
|
),
|
||||||
|
("read_document", lambda: ctx.ark_cfg is not None, _read_document),
|
||||||
("transcribe_audio", lfasr_configured, _transcribe),
|
("transcribe_audio", lfasr_configured, _transcribe),
|
||||||
("web_search", lambda: BochaConfig.load() is not None, _web_search),
|
("web_search", lambda: BochaConfig.load() is not None, _web_search),
|
||||||
]
|
]
|
||||||
|
|
|
||||||
307
core/toolfail.py
307
core/toolfail.py
|
|
@ -2,11 +2,12 @@
|
||||||
|
|
||||||
背景(2026-07,task 9dcae061 终案的结构性教训):mermaid 渲染在生产挂了 90 天
|
背景(2026-07,task 9dcae061 终案的结构性教训):mermaid 渲染在生产挂了 90 天
|
||||||
0 成功(67 次超时 + 26 次 launch fail、烧掉数十万 token),没有任何机制发现,
|
0 成功(67 次超时 + 26 次 launch fail、烧掉数十万 token),没有任何机制发现,
|
||||||
靠人工扫 DB 才挖出来。本模块把「失败聚集」变成信号:扫 messages 里 role=tool
|
靠人工扫 DB 才挖出来。本模块把「失败聚集」变成信号:统一扫描 usage_events
|
||||||
的错误结果,按 (工具名 + 归一化错误签名) 聚合,超阈值即算聚集。
|
中的结构化异常事件,按 (工具名 + 归一化错误签名) 聚合,超阈值即算聚集。
|
||||||
|
|
||||||
纯只读查询、无新表无状态;告警通道由调用方决定(web/app.py 的巡检 loop 发
|
普通工具失败从 2026-09-03 起在 tool result 落 messages 后同步写 tool_failure;
|
||||||
开发者邮箱,admin API 直接返给前端表格)。
|
切换前只存在于 messages 的历史故障不再回扫。纯只读查询、无派生状态;告警通道
|
||||||
|
由调用方决定(web/app.py 的巡检 loop 发开发者邮箱,admin API 直接返给前端表格)。
|
||||||
|
|
||||||
第二数据源(0.58.19):被丢弃的畸形 tool_call 参数(kind=malformed)——这类失败
|
第二数据源(0.58.19):被丢弃的畸形 tool_call 参数(kind=malformed)——这类失败
|
||||||
整轮不入 messages(防投毒级联),loop 落 usage_events(kind=tool_malformed),
|
整轮不入 messages(防投毒级联),loop 落 usage_events(kind=tool_malformed),
|
||||||
|
|
@ -31,9 +32,10 @@ tool_use 漏成正文后丢空)。loop 落 usage_events(kind=empty_response),在
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
|
||||||
import math
|
import math
|
||||||
import re
|
import re
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import datetime, timedelta, timezone
|
||||||
from typing import Any, Dict, List, Optional, Tuple
|
from typing import Any, Dict, List, Optional, Tuple
|
||||||
|
|
||||||
|
|
@ -41,117 +43,42 @@ from sqlalchemy import text
|
||||||
|
|
||||||
from core.storage import session_scope
|
from core.storage import session_scope
|
||||||
from core.storage.telemetry import (
|
from core.storage.telemetry import (
|
||||||
|
KIND_AGENT_GUARD,
|
||||||
|
KIND_CONTEXT_FOLD_FAILURE,
|
||||||
KIND_EMPTY_RESPONSE,
|
KIND_EMPTY_RESPONSE,
|
||||||
|
KIND_QUALITY_GATE,
|
||||||
KIND_RUN_ERROR,
|
KIND_RUN_ERROR,
|
||||||
|
KIND_RUN_STOPPED,
|
||||||
|
KIND_TOOL_FAILURE,
|
||||||
KIND_TOOL_MALFORMED,
|
KIND_TOOL_MALFORMED,
|
||||||
KIND_TOOL_SALVAGED,
|
KIND_TOOL_SALVAGED,
|
||||||
)
|
)
|
||||||
|
from core.tool_failure import (
|
||||||
# 签名归一:同一类错误在不同 task/参数下的差异(路径/数字/uuid/十六进制)抹平,
|
failure_category as _failure_category,
|
||||||
# 让 "figures/a.png doesn't exist" 和 "figures/b.png doesn't exist" 聚成一条。
|
normalize_failure_signature as _normalize,
|
||||||
_RE_UUID = re.compile(r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}")
|
)
|
||||||
_RE_HEX = re.compile(r"0x[0-9a-fA-F]+")
|
|
||||||
_RE_PATH = re.compile(r"(?:[A-Za-z]:)?(?:[/\\][\w.\-一-鿿*]+){2,}")
|
|
||||||
_RE_NUM = re.compile(r"\d+")
|
|
||||||
_RE_WS = re.compile(r"\s+")
|
|
||||||
|
|
||||||
_EXIT_TAIL = re.compile(r"\[exit (\d+)\]\s*$")
|
|
||||||
_STREAM_MARKS = ("[stdout]", "[stderr]")
|
|
||||||
_BARE_EXIT_RE = re.compile(r"^exit \d+$")
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize(s: str) -> str:
|
_CACHE_TTL_SECONDS = 90.0
|
||||||
s = _RE_UUID.sub("<id>", s)
|
_failure_cache_lock = threading.Lock()
|
||||||
s = _RE_HEX.sub("<hex>", s)
|
_wire_cache_lock = threading.Lock()
|
||||||
s = _RE_PATH.sub("<path>", s)
|
_failure_cache: dict[tuple[float, int, int], tuple[float, List[Dict[str, Any]]]] = {}
|
||||||
s = _RE_NUM.sub("N", s)
|
_wire_cache: dict[int, tuple[float, Dict[str, Any]]] = {}
|
||||||
s = _RE_WS.sub(" ", s).strip()
|
|
||||||
return s[:120]
|
|
||||||
|
|
||||||
|
|
||||||
def _classify(content: str) -> Optional[Tuple[str, str]]:
|
def _clear_tool_health_cache() -> None:
|
||||||
"""返回 (kind, 原始签名行) 或 None(不算失败)。"""
|
"""测试/运维进程内失效入口;不触碰数据库。"""
|
||||||
head = content.lstrip()
|
with _failure_cache_lock:
|
||||||
# 超时判定在前:超时结果形如 "[Error] command timed out after 30s",
|
_failure_cache.clear()
|
||||||
# 让它归 timeout 而不是被 [Error 前缀截走(kind 对排查方向有指示意义)
|
with _wire_cache_lock:
|
||||||
if "command timed out" in content:
|
_wire_cache.clear()
|
||||||
return "timeout", "command timed out"
|
|
||||||
if head.startswith("[Error"):
|
|
||||||
return "error", head.splitlines()[0]
|
|
||||||
m = _EXIT_TAIL.search(content)
|
|
||||||
if m and m.group(1) != "0":
|
|
||||||
# 签名取 [exit N] 前最后一行有实际内容的输出(通常是真正的报错行)
|
|
||||||
lines = [
|
|
||||||
ln.strip() for ln in content.splitlines()[:-1]
|
|
||||||
if ln.strip() and ln.strip() not in _STREAM_MARKS
|
|
||||||
]
|
|
||||||
return "exit", (lines[-1] if lines else f"exit {m.group(1)}")
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _shell_command_hint(prior_payload: Any, tool_call_id: str) -> str:
|
|
||||||
"""从相邻 assistant tool_call 提取空输出 shell 的命令类别。
|
|
||||||
|
|
||||||
shell 仅返回 ``[exit 1]`` 时,原签名会把 grep 未命中、依赖探测和真正的命令失败
|
|
||||||
全揉成 ``exit N``。messages 已保存 tool_call_id,可关联最近 assistant 精确找到
|
|
||||||
arguments;这里只返回稳定类别,不把完整命令或路径带进聚集签名。
|
|
||||||
"""
|
|
||||||
if not isinstance(prior_payload, dict):
|
|
||||||
return ""
|
|
||||||
calls = prior_payload.get("tool_calls") or []
|
|
||||||
candidates = []
|
|
||||||
for call in calls:
|
|
||||||
if not isinstance(call, dict):
|
|
||||||
continue
|
|
||||||
fn = call.get("function") or {}
|
|
||||||
if fn.get("name") != "shell":
|
|
||||||
continue
|
|
||||||
if tool_call_id and call.get("id") == tool_call_id:
|
|
||||||
candidates = [call]
|
|
||||||
break
|
|
||||||
candidates.append(call)
|
|
||||||
if len(candidates) != 1:
|
|
||||||
return ""
|
|
||||||
raw = (candidates[0].get("function") or {}).get("arguments") or {}
|
|
||||||
try:
|
|
||||||
args = json.loads(raw) if isinstance(raw, str) else raw
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return ""
|
|
||||||
command = str((args or {}).get("command") or "").strip().lower()
|
|
||||||
if not command:
|
|
||||||
return ""
|
|
||||||
if re.search(r"\b(?:apt|dpkg|pip)\b.*\b(?:list|show)\b", command):
|
|
||||||
return "dependency probe"
|
|
||||||
if re.search(r"(?:^|[;&|]\s*)(?:which|whereis|command\s+-v)\b", command):
|
|
||||||
return "dependency probe"
|
|
||||||
if re.search(r"(?:^|[;&|]\s*)(?:grep|rg)\b", command):
|
|
||||||
return "search/no match"
|
|
||||||
m = re.search(r"(?:^|&&|;|\|)\s*([a-z0-9_.-]+)", command)
|
|
||||||
return m.group(1) if m else ""
|
|
||||||
|
|
||||||
|
|
||||||
def _failure_category(kind: str, sig_line: str, sample: str) -> str:
|
|
||||||
"""区分产品/内容质量门与平台工具故障;返回值作为 API 加法字段。"""
|
|
||||||
if kind != "exit":
|
|
||||||
return "failure"
|
|
||||||
if sig_line.startswith("[GATE FAIL]"):
|
|
||||||
return "quality_gate"
|
|
||||||
if (
|
|
||||||
("[篇幅核算]" in sample or "[字数核算]" in sample)
|
|
||||||
and re.search(r"\[WARN\]\s*\d+\s*项超出\s*/\s*\d+\s*项不足", sample)
|
|
||||||
):
|
|
||||||
return "quality_gate"
|
|
||||||
if "[质量检查]" in sample and "[WARN] 共发现" in sample:
|
|
||||||
return "quality_gate"
|
|
||||||
return "failure"
|
|
||||||
|
|
||||||
|
|
||||||
def scan_tool_failures(
|
def scan_tool_failures(
|
||||||
days: float = 7,
|
days: float = 7,
|
||||||
min_count: int = 5,
|
min_count: int = 5,
|
||||||
min_tasks: int = 2,
|
min_tasks: int = 2,
|
||||||
) -> List[Dict[str, Any]]:
|
) -> List[Dict[str, Any]]:
|
||||||
"""扫近 `days` 天的 tool 错误消息,返回超阈值的聚集。
|
"""扫近 `days` 天的结构化异常事件,返回超阈值的聚集。
|
||||||
|
|
||||||
阈值语义:同签名 >= min_count 次 且 跨 >= min_tasks 个 task —— 单 task 内
|
阈值语义:同签名 >= min_count 次 且 跨 >= min_tasks 个 task —— 单 task 内
|
||||||
模型试错几次就自愈的正常噪音不触发;跨 task 复现的才是平台性问题。
|
模型试错几次就自愈的正常噪音不触发;跨 task 复现的才是平台性问题。
|
||||||
|
|
@ -166,62 +93,21 @@ def scan_tool_failures(
|
||||||
cutoff = now - timedelta(days=days)
|
cutoff = now - timedelta(days=days)
|
||||||
n_buckets = max(1, math.ceil(days))
|
n_buckets = max(1, math.ceil(days))
|
||||||
with session_scope() as s:
|
with session_scope() as s:
|
||||||
rows = s.execute(
|
event_rows = s.execute(
|
||||||
text(
|
text(
|
||||||
"select m.task_id, t.user_id, m.created_at, "
|
"select kind, task_id, user_id, created_at, model_profile, "
|
||||||
" m.payload->>'name' as tool_name, "
|
" units->>'tool' as tool_name, units->>'err' as err, "
|
||||||
" m.payload->>'content' as content, "
|
" units->>'head' as head, units->>'tail' as tail, "
|
||||||
" m.payload->>'tool_call_id' as tool_call_id, "
|
" units->>'failure_kind' as failure_kind, "
|
||||||
" p.payload as prior_payload "
|
" units->>'signature' as signature, "
|
||||||
"from messages m join tasks t on t.task_id = m.task_id "
|
" units->>'category' as category, "
|
||||||
"left join lateral ("
|
" units->>'sample' as sample "
|
||||||
" select pm.payload from messages pm "
|
|
||||||
" where pm.task_id = m.task_id and pm.idx < m.idx "
|
|
||||||
" and pm.payload->>'role' = 'assistant' "
|
|
||||||
" order by pm.idx desc limit 1"
|
|
||||||
") p on true "
|
|
||||||
"where m.created_at >= :cutoff "
|
|
||||||
" and m.payload->>'role' = 'tool' "
|
|
||||||
" and (m.payload->>'content' like '[Error%' "
|
|
||||||
" or m.payload->>'content' like '%command timed out%' "
|
|
||||||
" or m.payload->>'content' like '%[exit %')"
|
|
||||||
),
|
|
||||||
{"cutoff": cutoff},
|
|
||||||
).fetchall()
|
|
||||||
# 第二段:被丢弃的畸形 tool_call 参数(kind=tool_malformed)。这类失败整轮
|
|
||||||
# 不入 messages(防投毒),llm_transport.log_malformed_args 落在 usage_events,
|
|
||||||
# 是它们进面板/巡检邮件的唯一路径。
|
|
||||||
mrows = s.execute(
|
|
||||||
text(
|
|
||||||
"select task_id, user_id, created_at, "
|
|
||||||
" units->>'tool' as tool_name, "
|
|
||||||
" units->>'err' as err, "
|
|
||||||
" units->>'head' as head, "
|
|
||||||
" units->>'tail' as tail "
|
|
||||||
"from usage_events "
|
"from usage_events "
|
||||||
f"where kind = '{KIND_TOOL_MALFORMED}' and created_at >= :cutoff"
|
f"where kind in ('{KIND_TOOL_FAILURE}', '{KIND_TOOL_MALFORMED}', "
|
||||||
),
|
f"'{KIND_RUN_ERROR}', '{KIND_EMPTY_RESPONSE}', '{KIND_RUN_STOPPED}', "
|
||||||
{"cutoff": cutoff},
|
f"'{KIND_AGENT_GUARD}', '{KIND_CONTEXT_FOLD_FAILURE}', "
|
||||||
).fetchall()
|
f"'{KIND_QUALITY_GATE}') "
|
||||||
# 第三段:run 级终态错误(kind=run)。LLM 层抛异常时整轮无 tool 消息,
|
" and created_at >= :cutoff"
|
||||||
# tasks.run_error 只留最后一次,usage_events(kind=run_error)才是完整留痕。
|
|
||||||
rrows = s.execute(
|
|
||||||
text(
|
|
||||||
"select task_id, user_id, created_at, "
|
|
||||||
" units->>'err' as err "
|
|
||||||
"from usage_events "
|
|
||||||
f"where kind = '{KIND_RUN_ERROR}' and created_at >= :cutoff"
|
|
||||||
),
|
|
||||||
{"cutoff": cutoff},
|
|
||||||
).fetchall()
|
|
||||||
# 第四段:provider 吐空(kind=empty_response)。assistant 轮既无 tool_calls 又无正文,
|
|
||||||
# 会被 run loop 当正常收尾静默 done;loop 落 usage_events(kind=empty_response)是唯一
|
|
||||||
# 留痕。tool 名固定 "(empty)",签名固定,sample=model_profile —— 看是哪个网关档在吐空。
|
|
||||||
erows = s.execute(
|
|
||||||
text(
|
|
||||||
"select task_id, user_id, created_at, model_profile "
|
|
||||||
"from usage_events "
|
|
||||||
f"where kind = '{KIND_EMPTY_RESPONSE}' and created_at >= :cutoff"
|
|
||||||
),
|
),
|
||||||
{"cutoff": cutoff},
|
{"cutoff": cutoff},
|
||||||
).fetchall()
|
).fetchall()
|
||||||
|
|
@ -231,10 +117,15 @@ def scan_tool_failures(
|
||||||
def _add(
|
def _add(
|
||||||
tool_name: str, kind: str, sig_line: str, sample: str,
|
tool_name: str, kind: str, sig_line: str, sample: str,
|
||||||
task_id: Any, user_id: Any, created_at: datetime,
|
task_id: Any, user_id: Any, created_at: datetime,
|
||||||
|
*, category: Optional[str] = None, signature_normalized: bool = False,
|
||||||
) -> None:
|
) -> None:
|
||||||
# DB 列若是 naive timestamp(存 UTC),补 tzinfo 才能和 now 做减法
|
# DB 列若是 naive timestamp(存 UTC),补 tzinfo 才能和 now 做减法
|
||||||
ts = created_at if created_at.tzinfo else created_at.replace(tzinfo=timezone.utc)
|
ts = created_at if created_at.tzinfo else created_at.replace(tzinfo=timezone.utc)
|
||||||
key = (tool_name or "?", _normalize(sig_line))
|
key = (
|
||||||
|
tool_name or "?",
|
||||||
|
sig_line if signature_normalized else _normalize(sig_line),
|
||||||
|
)
|
||||||
|
resolved_category = category or _failure_category(kind, sig_line, sample)
|
||||||
c = agg.get(key)
|
c = agg.get(key)
|
||||||
if c is None:
|
if c is None:
|
||||||
c = agg[key] = {
|
c = agg[key] = {
|
||||||
|
|
@ -242,7 +133,7 @@ def scan_tool_failures(
|
||||||
"signature": key[1],
|
"signature": key[1],
|
||||||
"kind": kind,
|
"kind": kind,
|
||||||
# 分类必须看未截断的完整结果;sample 只保留 300 字给前端悬浮。
|
# 分类必须看未截断的完整结果;sample 只保留 300 字给前端悬浮。
|
||||||
"category": _failure_category(kind, sig_line, sample),
|
"category": resolved_category,
|
||||||
"count": 0,
|
"count": 0,
|
||||||
"tasks": set(),
|
"tasks": set(),
|
||||||
"users": set(),
|
"users": set(),
|
||||||
|
|
@ -251,7 +142,7 @@ def scan_tool_failures(
|
||||||
"sample": sample[:300],
|
"sample": sample[:300],
|
||||||
"daily": [0] * n_buckets,
|
"daily": [0] * n_buckets,
|
||||||
}
|
}
|
||||||
elif _failure_category(kind, sig_line, sample) == "quality_gate":
|
elif resolved_category == "quality_gate":
|
||||||
c["category"] = "quality_gate"
|
c["category"] = "quality_gate"
|
||||||
c["count"] += 1
|
c["count"] += 1
|
||||||
c["tasks"].add(task_id)
|
c["tasks"].add(task_id)
|
||||||
|
|
@ -265,38 +156,34 @@ def scan_tool_failures(
|
||||||
c["last_at"] = ts
|
c["last_at"] = ts
|
||||||
c["sample"] = sample[:300]
|
c["sample"] = sample[:300]
|
||||||
|
|
||||||
for row in rows:
|
for (
|
||||||
# 兼容旧测试夹具的 5 列消息行;生产查询额外带 tool_call_id/prior_payload。
|
event_kind, task_id, user_id, created_at, model_profile, tool_name,
|
||||||
task_id, user_id, created_at, tool_name, content = row[:5]
|
err, head, tail, failure_kind, signature, category, sample,
|
||||||
tool_call_id = row[5] if len(row) > 5 else ""
|
) in event_rows:
|
||||||
prior_payload = row[6] if len(row) > 6 else None
|
if event_kind in {
|
||||||
if not content:
|
KIND_TOOL_FAILURE,
|
||||||
continue
|
KIND_RUN_STOPPED,
|
||||||
hit = _classify(content)
|
KIND_AGENT_GUARD,
|
||||||
if hit is None:
|
KIND_CONTEXT_FOLD_FAILURE,
|
||||||
continue
|
KIND_QUALITY_GATE,
|
||||||
kind, sig_line = hit
|
}:
|
||||||
if tool_name == "shell" and kind == "exit" and _BARE_EXIT_RE.match(sig_line):
|
_add(
|
||||||
hint = _shell_command_hint(prior_payload, tool_call_id or "")
|
tool_name, failure_kind or "failure", signature or "?", sample or "",
|
||||||
if hint:
|
task_id, user_id, created_at,
|
||||||
sig_line = f"{sig_line} ({hint})"
|
category=category or "failure", signature_normalized=True,
|
||||||
_add(tool_name, kind, sig_line, content, task_id, user_id, created_at)
|
)
|
||||||
|
elif event_kind == KIND_TOOL_MALFORMED:
|
||||||
for task_id, user_id, created_at, tool_name, err, head, tail in mrows:
|
_add(
|
||||||
_add(
|
tool_name, "malformed", err or "?", f"{head or ''} … {tail or ''}",
|
||||||
tool_name, "malformed", err or "?",
|
task_id, user_id, created_at,
|
||||||
f"{head or ''} … {tail or ''}",
|
)
|
||||||
task_id, user_id, created_at,
|
elif event_kind == KIND_RUN_ERROR:
|
||||||
)
|
_add("(run)", "run", err or "?", err or "", task_id, user_id, created_at)
|
||||||
|
elif event_kind == KIND_EMPTY_RESPONSE:
|
||||||
for task_id, user_id, created_at, err in rrows:
|
_add(
|
||||||
_add("(run)", "run", err or "?", err or "", task_id, user_id, created_at)
|
"(empty)", "empty", "provider returned empty response",
|
||||||
|
model_profile or "?", task_id, user_id, created_at,
|
||||||
for task_id, user_id, created_at, model_profile in erows:
|
)
|
||||||
_add(
|
|
||||||
"(empty)", "empty", "provider returned empty response",
|
|
||||||
model_profile or "?", task_id, user_id, created_at,
|
|
||||||
)
|
|
||||||
|
|
||||||
out = []
|
out = []
|
||||||
for c in agg.values():
|
for c in agg.values():
|
||||||
|
|
@ -321,6 +208,30 @@ def scan_tool_failures(
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def scan_tool_failures_cached(
|
||||||
|
days: float = 7,
|
||||||
|
min_count: int = 5,
|
||||||
|
min_tasks: int = 2,
|
||||||
|
) -> List[Dict[str, Any]]:
|
||||||
|
"""管理页短时缓存;锁覆盖扫描,使同参数并发请求只扫库一次。"""
|
||||||
|
key = (float(days), int(min_count), int(min_tasks))
|
||||||
|
now = time.monotonic()
|
||||||
|
with _failure_cache_lock:
|
||||||
|
cached = _failure_cache.get(key)
|
||||||
|
if cached and cached[0] > now:
|
||||||
|
return cached[1]
|
||||||
|
for stale_key, (expires_at, _value) in list(_failure_cache.items()):
|
||||||
|
if expires_at <= now:
|
||||||
|
_failure_cache.pop(stale_key, None)
|
||||||
|
value = scan_tool_failures(
|
||||||
|
days=days,
|
||||||
|
min_count=min_count,
|
||||||
|
min_tasks=min_tasks,
|
||||||
|
)
|
||||||
|
_failure_cache[key] = (time.monotonic() + _CACHE_TTL_SECONDS, value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
def scan_tool_wire_health(days: int = 7) -> Dict[str, Any]:
|
def scan_tool_wire_health(days: int = 7) -> Dict[str, Any]:
|
||||||
"""聚合 provider 工具参数损坏的抢救/残余比例;纯只读、无派生状态。
|
"""聚合 provider 工具参数损坏的抢救/残余比例;纯只读、无派生状态。
|
||||||
|
|
||||||
|
|
@ -407,6 +318,22 @@ def scan_tool_wire_health(days: int = 7) -> Dict[str, Any]:
|
||||||
return {"days": days, "rows": out, "total": totals}
|
return {"days": days, "rows": out, "total": totals}
|
||||||
|
|
||||||
|
|
||||||
|
def scan_tool_wire_health_cached(days: int = 7) -> Dict[str, Any]:
|
||||||
|
"""管理页链路健康短时缓存;同窗口并发请求共享一次聚合。"""
|
||||||
|
key = min(90, max(1, int(days)))
|
||||||
|
now = time.monotonic()
|
||||||
|
with _wire_cache_lock:
|
||||||
|
cached = _wire_cache.get(key)
|
||||||
|
if cached and cached[0] > now:
|
||||||
|
return cached[1]
|
||||||
|
for stale_key, (expires_at, _value) in list(_wire_cache.items()):
|
||||||
|
if expires_at <= now:
|
||||||
|
_wire_cache.pop(stale_key, None)
|
||||||
|
value = scan_tool_wire_health(days=key)
|
||||||
|
_wire_cache[key] = (time.monotonic() + _CACHE_TTL_SECONDS, value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
# ── provider 级致命错误即时告警 ──
|
# ── provider 级致命错误即时告警 ──
|
||||||
# 命中判据:错误文案含余额/配额/认证类关键词 —— 这类错误不是单任务偶发,而是该
|
# 命中判据:错误文案含余额/配额/认证类关键词 —— 这类错误不是单任务偶发,而是该
|
||||||
# provider 上所有 run 全挂(如 Zai 余额不足),等日巡检的 5 次/2 task 阈值太慢。
|
# provider 上所有 run 全挂(如 Zai 余额不足),等日巡检的 5 次/2 task 阈值太慢。
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,31 @@
|
||||||
|
"""Track artifact trash separately from quota usage.
|
||||||
|
|
||||||
|
Revision ID: 0037
|
||||||
|
Revises: 0036
|
||||||
|
Create Date: 2026-08-28
|
||||||
|
"""
|
||||||
|
from collections.abc import Sequence
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
revision: str = "0037"
|
||||||
|
down_revision: str | None = "0036"
|
||||||
|
branch_labels: str | Sequence[str] | None = None
|
||||||
|
depends_on: str | Sequence[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.add_column(
|
||||||
|
"user_disk_usage",
|
||||||
|
sa.Column("trash_bytes", sa.BigInteger(), nullable=False, server_default="0"),
|
||||||
|
)
|
||||||
|
op.add_column(
|
||||||
|
"user_disk_usage",
|
||||||
|
sa.Column("trash_file_count", sa.Integer(), nullable=False, server_default="0"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_column("user_disk_usage", "trash_file_count")
|
||||||
|
op.drop_column("user_disk_usage", "trash_bytes")
|
||||||
|
|
@ -0,0 +1,37 @@
|
||||||
|
"""Add sandbox package scan sessions.
|
||||||
|
|
||||||
|
Revision ID: 0038
|
||||||
|
Revises: 0037
|
||||||
|
Create Date: 2026-09-01
|
||||||
|
"""
|
||||||
|
from collections.abc import Sequence
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
from sqlalchemy.dialects import postgresql
|
||||||
|
|
||||||
|
revision: str = "0038"
|
||||||
|
down_revision: str | None = "0037"
|
||||||
|
branch_labels: str | Sequence[str] | None = None
|
||||||
|
depends_on: str | Sequence[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"sandbox_package_scans",
|
||||||
|
sa.Column("container_session_id", sa.Text(), primary_key=True),
|
||||||
|
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.user_id", ondelete="CASCADE"), nullable=False),
|
||||||
|
sa.Column("execution_kind", sa.Text(), nullable=False),
|
||||||
|
sa.Column("image_digest", sa.Text(), nullable=False),
|
||||||
|
sa.Column("python_version", sa.Text(), nullable=False),
|
||||||
|
sa.Column("packages", postgresql.JSONB(), nullable=False),
|
||||||
|
sa.Column("package_count", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("total_installed_bytes", sa.BigInteger(), nullable=False),
|
||||||
|
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
|
||||||
|
)
|
||||||
|
op.create_index("ix_sandbox_package_scans_finished_at", "sandbox_package_scans", ["finished_at"])
|
||||||
|
op.create_index("ix_sandbox_package_scans_user_finished", "sandbox_package_scans", ["user_id", "finished_at"])
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_table("sandbox_package_scans")
|
||||||
|
|
@ -0,0 +1,46 @@
|
||||||
|
"""Add encrypted provider credentials control plane.
|
||||||
|
|
||||||
|
Revision ID: 0039
|
||||||
|
Revises: 0038
|
||||||
|
Create Date: 2026-09-02
|
||||||
|
"""
|
||||||
|
from collections.abc import Sequence
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
from sqlalchemy.dialects import postgresql
|
||||||
|
|
||||||
|
revision: str = "0039"
|
||||||
|
down_revision: str | None = "0038"
|
||||||
|
branch_labels: str | Sequence[str] | None = None
|
||||||
|
depends_on: str | Sequence[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"provider_credentials",
|
||||||
|
sa.Column("provider_id", sa.Text(), primary_key=True),
|
||||||
|
sa.Column("credentials", postgresql.JSONB(), nullable=False),
|
||||||
|
sa.Column("credential_hint", postgresql.JSONB(), nullable=False),
|
||||||
|
sa.Column("revision", sa.Integer(), server_default="1", nullable=False),
|
||||||
|
sa.Column("test_status", sa.Text(), server_default="untested", nullable=False),
|
||||||
|
sa.Column("test_detail", sa.Text(), nullable=True),
|
||||||
|
sa.Column("balance_amount", sa.Numeric(18, 6), nullable=True),
|
||||||
|
sa.Column("balance_currency", sa.Text(), nullable=True),
|
||||||
|
sa.Column("last_tested_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("last_notified_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("last_notified_status", sa.Text(), nullable=True),
|
||||||
|
sa.Column(
|
||||||
|
"updated_by",
|
||||||
|
postgresql.UUID(as_uuid=True),
|
||||||
|
sa.ForeignKey("users.user_id", ondelete="SET NULL"),
|
||||||
|
nullable=True,
|
||||||
|
),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.CheckConstraint("revision > 0", name="ck_provider_credentials_revision_positive"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_table("provider_credentials")
|
||||||
|
|
@ -0,0 +1,41 @@
|
||||||
|
"""Add indexes for structured tool health events.
|
||||||
|
|
||||||
|
Revision ID: 0040
|
||||||
|
Revises: 0039
|
||||||
|
Create Date: 2026-09-03
|
||||||
|
"""
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
|
||||||
|
revision: str = "0040"
|
||||||
|
down_revision: Union[str, None] = "0039"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_index(
|
||||||
|
"ix_usage_health_kind_created",
|
||||||
|
"usage_events",
|
||||||
|
["kind", "created_at"],
|
||||||
|
postgresql_where=sa.text(
|
||||||
|
"kind IN ('tool_failure', 'tool_malformed', 'tool_salvaged', "
|
||||||
|
"'run_error', 'empty_response', 'run_stopped', 'agent_guard', "
|
||||||
|
"'context_fold_failure', 'quality_gate')"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ux_usage_tool_failure_message",
|
||||||
|
"usage_events",
|
||||||
|
["message_id"],
|
||||||
|
unique=True,
|
||||||
|
postgresql_where=sa.text("kind IN ('tool_failure', 'quality_gate')"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_index("ux_usage_tool_failure_message", table_name="usage_events")
|
||||||
|
op.drop_index("ix_usage_health_kind_created", table_name="usage_events")
|
||||||
|
|
@ -64,6 +64,9 @@ RUN grep -v '\[host-only\]' /tmp/requirements.txt > /tmp/requirements.sandbox.tx
|
||||||
-r /tmp/requirements.sandbox.txt \
|
-r /tmp/requirements.sandbox.txt \
|
||||||
&& rm /tmp/requirements.txt /tmp/requirements.sandbox.txt
|
&& rm /tmp/requirements.txt /tmp/requirements.sandbox.txt
|
||||||
|
|
||||||
|
# 构建期基础包清单;运行时扫描只与它比较,不 import/执行用户包。
|
||||||
|
RUN mkdir -p /sandbox && python -c "import importlib.metadata as m,json; c=lambda s:s.lower().replace('_','-').replace('.','-'); print(json.dumps({c(d.metadata['Name']):d.version for d in m.distributions() if d.metadata.get('Name')},sort_keys=True))" > /sandbox/base-python-packages.json
|
||||||
|
|
||||||
# 持久化 pip 源到 /etc/pip.conf ── 让运行时模型用 `pip install foo` 也走 mirror,
|
# 持久化 pip 源到 /etc/pip.conf ── 让运行时模型用 `pip install foo` 也走 mirror,
|
||||||
# 不只 build 时。zcbot user / root 都吃这个 global 配置。
|
# 不只 build 时。zcbot user / root 都吃这个 global 配置。
|
||||||
RUN printf '[global]\nindex-url = %s\ntimeout = 60\n%s\n' \
|
RUN printf '[global]\nindex-url = %s\ntimeout = 60\n%s\n' \
|
||||||
|
|
@ -176,6 +179,7 @@ COPY core/__init__.py core/file_store.py core/kb_lock.py /sandbox/core/
|
||||||
COPY core/sandbox/tool_runner.py /sandbox/tool_runner.py
|
COPY core/sandbox/tool_runner.py /sandbox/tool_runner.py
|
||||||
|
|
||||||
COPY deploy/sandbox/init.sh /init.sh
|
COPY deploy/sandbox/init.sh /init.sh
|
||||||
|
COPY deploy/sandbox/package_scan.py /sandbox/package_scan.py
|
||||||
RUN chmod +x /init.sh
|
RUN chmod +x /init.sh
|
||||||
|
|
||||||
# 默认 cwd /workspace ── 但每次 `docker exec --workdir /workspace/<wd>` 会覆盖
|
# 默认 cwd /workspace ── 但每次 `docker exec --workdir /workspace/<wd>` 会覆盖
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,84 @@
|
||||||
|
"""可信临时包扫描器;只解析 dist-info 元数据,不 import 用户代码。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
import csv
|
||||||
|
import email.parser
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
BASE = Path("/sandbox/base-python-packages.json")
|
||||||
|
ROOT = Path("/tmp/.local/lib")
|
||||||
|
|
||||||
|
|
||||||
|
def safe_file(root: Path, path: Path) -> bool:
|
||||||
|
try:
|
||||||
|
path.relative_to(root)
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
current = root
|
||||||
|
for part in path.relative_to(root).parts:
|
||||||
|
current = current / part
|
||||||
|
if current.is_symlink():
|
||||||
|
return False
|
||||||
|
return path.is_file()
|
||||||
|
|
||||||
|
|
||||||
|
def safe_dir(root: Path, path: Path) -> bool:
|
||||||
|
try:
|
||||||
|
rel = path.relative_to(root)
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
current = root
|
||||||
|
for part in rel.parts:
|
||||||
|
current = current / part
|
||||||
|
if current.is_symlink():
|
||||||
|
return False
|
||||||
|
return path.is_dir()
|
||||||
|
|
||||||
|
|
||||||
|
def canonical(name: str) -> str:
|
||||||
|
return name.lower().replace("_", "-").replace(".", "-")
|
||||||
|
|
||||||
|
|
||||||
|
def scan_packages(root: Path, base: dict) -> list[dict]:
|
||||||
|
out = []
|
||||||
|
if root.is_dir():
|
||||||
|
for site in sorted(root.glob("python*/site-packages")):
|
||||||
|
if not safe_dir(root, site):
|
||||||
|
continue
|
||||||
|
for dist in sorted(site.glob("*.dist-info")):
|
||||||
|
if len(out) >= 5000:
|
||||||
|
return out
|
||||||
|
meta_path = dist / "METADATA"
|
||||||
|
if not safe_file(site, meta_path) or meta_path.stat().st_size > 1024 * 1024:
|
||||||
|
continue
|
||||||
|
msg = email.parser.Parser().parsestr(meta_path.read_text(encoding="utf-8", errors="replace"), headersonly=True)
|
||||||
|
name, version = msg.get("Name", "").strip(), msg.get("Version", "").strip()
|
||||||
|
if not name:
|
||||||
|
continue
|
||||||
|
total = 0
|
||||||
|
record = dist / "RECORD"
|
||||||
|
if safe_file(site, record) and record.stat().st_size <= 32 * 1024 * 1024:
|
||||||
|
for index, row in enumerate(csv.reader(record.read_text(encoding="utf-8", errors="replace").splitlines())):
|
||||||
|
if index >= 200_000:
|
||||||
|
break
|
||||||
|
if not row:
|
||||||
|
continue
|
||||||
|
candidate = site / row[0]
|
||||||
|
if safe_file(site, candidate):
|
||||||
|
try: total += candidate.stat().st_size
|
||||||
|
except OSError: pass
|
||||||
|
key = canonical(name)
|
||||||
|
old = base.get(key)
|
||||||
|
change = "added" if old is None else ("reinstall" if old == version else "override")
|
||||||
|
out.append({"name": name, "version": version, "base_version": old,
|
||||||
|
"change": change, "direct_requested": safe_file(site, dist / "REQUESTED") or safe_file(site, dist / "direct_url.json"),
|
||||||
|
"installed_bytes": total})
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
base = json.loads(BASE.read_text(encoding="utf-8"))
|
||||||
|
print(json.dumps(scan_packages(ROOT, base), ensure_ascii=False))
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
|
@ -39,7 +39,8 @@ _LIST_FIELDS = (
|
||||||
|
|
||||||
|
|
||||||
def _api_key() -> str:
|
def _api_key() -> str:
|
||||||
key = os.environ.get("DOCUMENT_SEARCH_API_KEY", "").strip()
|
from core.provider_credentials.runtime import resolve_secret
|
||||||
|
key = resolve_secret("document_search")
|
||||||
if not key:
|
if not key:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
"DOCUMENT_SEARCH_API_KEY env 未设置 —— 配置后再使用 literature skill 的内部材料库来源"
|
"DOCUMENT_SEARCH_API_KEY env 未设置 —— 配置后再使用 literature skill 的内部材料库来源"
|
||||||
|
|
@ -54,6 +55,19 @@ def _auth_headers(extra: Optional[dict] = None) -> dict:
|
||||||
return h
|
return h
|
||||||
|
|
||||||
|
|
||||||
|
def _check_response(response: httpx.Response) -> None:
|
||||||
|
if response.status_code >= 400:
|
||||||
|
try:
|
||||||
|
from core.provider_credentials.service import record_business_failure
|
||||||
|
record_business_failure(
|
||||||
|
"document_search", status_code=response.status_code,
|
||||||
|
detail=response.text[:300],
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
response.raise_for_status()
|
||||||
|
|
||||||
|
|
||||||
def _safe_name(name: str) -> str:
|
def _safe_name(name: str) -> str:
|
||||||
# 防目录穿越;保留扩展名
|
# 防目录穿越;保留扩展名
|
||||||
return name.replace("/", "_").replace("\\", "_").replace("..", "_")
|
return name.replace("/", "_").replace("\\", "_").replace("..", "_")
|
||||||
|
|
@ -66,7 +80,7 @@ def list_kb() -> list[dict]:
|
||||||
/ create_time / file_count。只返回 ID 映射里有效的(分类 1-7)。
|
/ create_time / file_count。只返回 ID 映射里有效的(分类 1-7)。
|
||||||
"""
|
"""
|
||||||
r = httpx.get(f"{_API}/list_knowledge_bases", headers=_auth_headers(), timeout=_TIMEOUT)
|
r = httpx.get(f"{_API}/list_knowledge_bases", headers=_auth_headers(), timeout=_TIMEOUT)
|
||||||
r.raise_for_status()
|
_check_response(r)
|
||||||
payload = r.json()
|
payload = r.json()
|
||||||
data = payload.get("data") or {}
|
data = payload.get("data") or {}
|
||||||
return list(data.get("knowledge_bases") or [])
|
return list(data.get("knowledge_bases") or [])
|
||||||
|
|
@ -104,7 +118,7 @@ def search(
|
||||||
json=body,
|
json=body,
|
||||||
timeout=_TIMEOUT,
|
timeout=_TIMEOUT,
|
||||||
)
|
)
|
||||||
r.raise_for_status()
|
_check_response(r)
|
||||||
payload = r.json()
|
payload = r.json()
|
||||||
data = payload.get("data") or {}
|
data = payload.get("data") or {}
|
||||||
docs = data.get("documents") or []
|
docs = data.get("documents") or []
|
||||||
|
|
@ -144,7 +158,7 @@ def download(
|
||||||
params=params,
|
params=params,
|
||||||
timeout=_DOWNLOAD_TIMEOUT,
|
timeout=_DOWNLOAD_TIMEOUT,
|
||||||
) as resp:
|
) as resp:
|
||||||
resp.raise_for_status()
|
_check_response(resp)
|
||||||
with open(dest, "wb") as f:
|
with open(dest, "wb") as f:
|
||||||
for chunk in resp.iter_bytes(chunk_size=64 * 1024):
|
for chunk in resp.iter_bytes(chunk_size=64 * 1024):
|
||||||
f.write(chunk)
|
f.write(chunk)
|
||||||
|
|
|
||||||
|
|
@ -30,7 +30,8 @@ _DEFAULT_SUMMARY_FIELDS = [
|
||||||
|
|
||||||
|
|
||||||
def _mp_key() -> str:
|
def _mp_key() -> str:
|
||||||
key = os.environ.get("MP_API_KEY", "").strip()
|
from core.provider_credentials.runtime import resolve_secret
|
||||||
|
key = resolve_secret("materials_project")
|
||||||
if not key:
|
if not key:
|
||||||
raise RuntimeError("MP_API_KEY env 未设置,无法查询 Materials Project")
|
raise RuntimeError("MP_API_KEY env 未设置,无法查询 Materials Project")
|
||||||
return key
|
return key
|
||||||
|
|
@ -61,6 +62,17 @@ def _mpr():
|
||||||
return MPRester(_mp_key())
|
return MPRester(_mp_key())
|
||||||
|
|
||||||
|
|
||||||
|
def _report_provider_error(error: Exception) -> None:
|
||||||
|
try:
|
||||||
|
from core.provider_credentials.service import record_business_failure
|
||||||
|
status = int(getattr(error, "status_code", 0) or 0)
|
||||||
|
record_business_failure(
|
||||||
|
"materials_project", status_code=status, detail=str(error)
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
class MaterialsProjectSearchTool(Tool):
|
class MaterialsProjectSearchTool(Tool):
|
||||||
name = "materials_project_search"
|
name = "materials_project_search"
|
||||||
description = (
|
description = (
|
||||||
|
|
@ -128,6 +140,7 @@ class MaterialsProjectSearchTool(Tool):
|
||||||
num_chunks=1, chunk_size=limit, **kwargs
|
num_chunks=1, chunk_size=limit, **kwargs
|
||||||
)
|
)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
_report_provider_error(e)
|
||||||
detail = safe_error_text(e, ("MP_API_KEY",))
|
detail = safe_error_text(e, ("MP_API_KEY",))
|
||||||
return f"[Error] materials_project_search failed: {type(e).__name__}: {detail}"
|
return f"[Error] materials_project_search failed: {type(e).__name__}: {detail}"
|
||||||
plain = [_to_plain(d) for d in list(docs)[:limit]]
|
plain = [_to_plain(d) for d in list(docs)[:limit]]
|
||||||
|
|
@ -152,6 +165,7 @@ class MaterialsProjectSearchTool(Tool):
|
||||||
try:
|
try:
|
||||||
session = _mpr()
|
session = _mpr()
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
_report_provider_error(e)
|
||||||
detail = safe_error_text(e, ("MP_API_KEY",))
|
detail = safe_error_text(e, ("MP_API_KEY",))
|
||||||
return f"[Error] materials_project_search batch failed: {type(e).__name__}: {detail}"
|
return f"[Error] materials_project_search batch failed: {type(e).__name__}: {detail}"
|
||||||
agg: list[dict[str, Any]] = []
|
agg: list[dict[str, Any]] = []
|
||||||
|
|
@ -169,6 +183,7 @@ class MaterialsProjectSearchTool(Tool):
|
||||||
n_ok += 1
|
n_ok += 1
|
||||||
agg.append({"formula": f, "n": len(results), "results": results})
|
agg.append({"formula": f, "n": len(results), "results": results})
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
_report_provider_error(e)
|
||||||
detail = safe_error_text(e, ("MP_API_KEY",))
|
detail = safe_error_text(e, ("MP_API_KEY",))
|
||||||
agg.append({"formula": f, "error": f"{type(e).__name__}: {detail}"})
|
agg.append({"formula": f, "error": f"{type(e).__name__}: {detail}"})
|
||||||
return self._render_batch(agg, fields, n_ok)
|
return self._render_batch(agg, fields, n_ok)
|
||||||
|
|
@ -251,6 +266,7 @@ class MaterialsProjectGetStructureTool(Tool):
|
||||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||||
struct.to(filename=str(dest))
|
struct.to(filename=str(dest))
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
_report_provider_error(e)
|
||||||
detail = safe_error_text(e, ("MP_API_KEY",))
|
detail = safe_error_text(e, ("MP_API_KEY",))
|
||||||
return f"[Error] materials_project_get_structure failed: {type(e).__name__}: {detail}"
|
return f"[Error] materials_project_get_structure failed: {type(e).__name__}: {detail}"
|
||||||
return f"saved: {self._display(dest)}"
|
return f"saved: {self._display(dest)}"
|
||||||
|
|
|
||||||
|
|
@ -47,7 +47,8 @@ _LIST_FIELDS = (
|
||||||
|
|
||||||
def _config() -> tuple[str, str, str]:
|
def _config() -> tuple[str, str, str]:
|
||||||
base_url = os.environ.get("PAPER_SERVER_URL", _DEFAULT_BASE_URL).strip().rstrip("/")
|
base_url = os.environ.get("PAPER_SERVER_URL", _DEFAULT_BASE_URL).strip().rstrip("/")
|
||||||
api_key = os.environ.get("PAPER_SERVER_API_KEY", "").strip()
|
from core.provider_credentials.runtime import resolve_secret
|
||||||
|
api_key = resolve_secret("paper_server")
|
||||||
if not api_key:
|
if not api_key:
|
||||||
raise RuntimeError("PAPER_SERVER_API_KEY env 未设置,无法查询 paper_server")
|
raise RuntimeError("PAPER_SERVER_API_KEY env 未设置,无法查询 paper_server")
|
||||||
return base_url, f"{base_url}/api/resm/paper", api_key
|
return base_url, f"{base_url}/api/resm/paper", api_key
|
||||||
|
|
@ -70,6 +71,14 @@ def _raise_response_error(response: httpx.Response) -> None:
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
if response.status_code in (401, 403) or err_code in _AUTH_ERR_CODES:
|
if response.status_code in (401, 403) or err_code in _AUTH_ERR_CODES:
|
||||||
|
try:
|
||||||
|
from core.provider_credentials.service import record_business_failure
|
||||||
|
record_business_failure(
|
||||||
|
"paper_server", status_code=response.status_code or 401,
|
||||||
|
detail=err_code,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f"paper_server auth failed (HTTP {response.status_code}, "
|
f"paper_server auth failed (HTTP {response.status_code}, "
|
||||||
f"{err_code or 'no err_code'}):请管理员检查平台 PAPER_SERVER_API_KEY"
|
f"{err_code or 'no err_code'}):请管理员检查平台 PAPER_SERVER_API_KEY"
|
||||||
|
|
|
||||||
|
|
@ -54,7 +54,8 @@ class PlatformSourceProvider(Protocol):
|
||||||
|
|
||||||
|
|
||||||
def _env_set(name: str) -> bool:
|
def _env_set(name: str) -> bool:
|
||||||
return bool(os.environ.get(name, "").strip())
|
from core.provider_credentials.runtime import resolve_env_secret
|
||||||
|
return bool(resolve_env_secret(name))
|
||||||
|
|
||||||
|
|
||||||
def _require_valid_http_url(name: str, default: str) -> None:
|
def _require_valid_http_url(name: str, default: str) -> None:
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,225 @@
|
||||||
|
"""按版本化模型价格重算历史 usage_events,默认只输出 dry-run 汇总。
|
||||||
|
|
||||||
|
示例:
|
||||||
|
.venv/Scripts/python.exe scripts/reprice_usage.py --from 2026-08-16T16:00:00Z
|
||||||
|
.venv/Scripts/python.exe scripts/reprice_usage.py --from 2026-08-16T16:00:00Z \
|
||||||
|
--apply --confirm APPLY_DEEPSEEK_REPRICE
|
||||||
|
|
||||||
|
脚本不加载 .env;调用方必须显式提供 ZCBOT_DB_URL。生产写入前先核对 dry-run 输出和
|
||||||
|
打印的脱敏目标。缺少 cache_hit_tokens 的旧辅助调用按全部未命中估算并留标记。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import os
|
||||||
|
from collections import defaultdict
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from decimal import Decimal
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.engine import make_url
|
||||||
|
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
if str(ROOT) not in sys.path:
|
||||||
|
sys.path.insert(0, str(ROOT))
|
||||||
|
|
||||||
|
from core.capabilities import ModelCapabilities # noqa: E402
|
||||||
|
from core.pricing import calculate_chat_cost, pricing_snapshot, resolve_chat_price # noqa: E402
|
||||||
|
from core.storage import session_scope # noqa: E402
|
||||||
|
from core.storage.models import UsageEvent # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_PROFILES = ("deepseek_v4.flash", "deepseek_v4.pro")
|
||||||
|
COST_KINDS = {"chat", "prompt_optimize", "context_fold", "task_title", "kb_ingest"}
|
||||||
|
CONFIRM_TEXT = "APPLY_DEEPSEEK_REPRICE"
|
||||||
|
PRICE_SNAPSHOT_KEYS = {
|
||||||
|
"pricing_revision",
|
||||||
|
"pricing_source",
|
||||||
|
"pricing_source_url",
|
||||||
|
"pricing_currency",
|
||||||
|
"pricing_at",
|
||||||
|
"price_tier",
|
||||||
|
"fx_to_cny",
|
||||||
|
"input_price_per_mtoken",
|
||||||
|
"cache_hit_price_per_mtoken",
|
||||||
|
"output_price_per_mtoken",
|
||||||
|
"input_cost_cny",
|
||||||
|
"cache_hit_cost_cny",
|
||||||
|
"output_cost_cny",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_datetime(value: str) -> datetime:
|
||||||
|
dt = datetime.fromisoformat(value.strip().replace("Z", "+00:00"))
|
||||||
|
if dt.tzinfo is None:
|
||||||
|
dt = dt.replace(tzinfo=timezone.utc)
|
||||||
|
return dt.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _args() -> argparse.Namespace:
|
||||||
|
p = argparse.ArgumentParser(description="重算历史模型成本(默认 dry-run)")
|
||||||
|
p.add_argument("--from", dest="from_at", required=True, type=_parse_datetime)
|
||||||
|
p.add_argument("--profile", action="append", dest="profiles")
|
||||||
|
p.add_argument("--apply", action="store_true", help="事务内写回;默认只预览")
|
||||||
|
p.add_argument("--rollback", action="store_true", help="恢复本 revision 保存的旧成本")
|
||||||
|
p.add_argument("--confirm", default="", help=f"写入确认串:{CONFIRM_TEXT}")
|
||||||
|
return p.parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
def _target() -> str:
|
||||||
|
raw = os.environ.get("ZCBOT_DB_URL", "").strip()
|
||||||
|
if not raw:
|
||||||
|
raise RuntimeError("ZCBOT_DB_URL is not set")
|
||||||
|
return make_url(raw).render_as_string(hide_password=True)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
args = _args()
|
||||||
|
if args.apply and args.rollback:
|
||||||
|
raise SystemExit("[ERR] --apply 与 --rollback 不能同时使用")
|
||||||
|
if (args.apply or args.rollback) and args.confirm != CONFIRM_TEXT:
|
||||||
|
raise SystemExit(f"[ERR] 写入必须传 --confirm {CONFIRM_TEXT}")
|
||||||
|
profiles = tuple(args.profiles or DEFAULT_PROFILES)
|
||||||
|
caps_by_profile = {
|
||||||
|
profile: ModelCapabilities.load(profile, ROOT / "config" / "models")
|
||||||
|
for profile in profiles
|
||||||
|
}
|
||||||
|
revisions = {
|
||||||
|
period.get("revision")
|
||||||
|
for caps in caps_by_profile.values()
|
||||||
|
for period in caps.pricing.get("periods", [])
|
||||||
|
if period.get("revision")
|
||||||
|
}
|
||||||
|
target = _target()
|
||||||
|
mode = "ROLLBACK" if args.rollback else ("APPLY" if args.apply else "DRY-RUN")
|
||||||
|
print(f"[INFO] mode={mode} target={target} from={args.from_at.isoformat()}")
|
||||||
|
print(f"[INFO] profiles={','.join(profiles)} revisions={','.join(sorted(revisions))}")
|
||||||
|
|
||||||
|
summary: dict[tuple[str, str], dict[str, Any]] = defaultdict(
|
||||||
|
lambda: {
|
||||||
|
"rows": 0,
|
||||||
|
"estimated": 0,
|
||||||
|
"old": Decimal("0"),
|
||||||
|
"new": Decimal("0"),
|
||||||
|
"estimated_old": Decimal("0"),
|
||||||
|
"estimated_new": Decimal("0"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
estimated_by_kind: dict[tuple[str, str], dict[str, Any]] = defaultdict(
|
||||||
|
lambda: {"rows": 0, "old": Decimal("0"), "new": Decimal("0")}
|
||||||
|
)
|
||||||
|
changed = 0
|
||||||
|
with session_scope() as s:
|
||||||
|
events = s.execute(
|
||||||
|
select(UsageEvent).where(
|
||||||
|
UsageEvent.model_profile.in_(profiles),
|
||||||
|
UsageEvent.kind.in_(COST_KINDS),
|
||||||
|
UsageEvent.created_at >= args.from_at,
|
||||||
|
).order_by(UsageEvent.created_at, UsageEvent.event_id)
|
||||||
|
).scalars().all()
|
||||||
|
for event in events:
|
||||||
|
units = dict(event.units or {})
|
||||||
|
caps = caps_by_profile[event.model_profile]
|
||||||
|
if args.rollback:
|
||||||
|
if units.get("repricing_revision") not in revisions:
|
||||||
|
continue
|
||||||
|
previous = units.get("previous_cost_cny")
|
||||||
|
if previous is None:
|
||||||
|
continue
|
||||||
|
old = Decimal(str(event.cost_cny or 0))
|
||||||
|
new = Decimal(str(previous))
|
||||||
|
previous_pricing = units.get("previous_pricing_units") or {}
|
||||||
|
for price_key in PRICE_SNAPSHOT_KEYS:
|
||||||
|
units.pop(price_key, None)
|
||||||
|
if isinstance(previous_pricing, dict):
|
||||||
|
units.update(previous_pricing)
|
||||||
|
for key in list(units):
|
||||||
|
if (
|
||||||
|
key.startswith("repricing_")
|
||||||
|
or key in {"previous_cost_cny", "previous_pricing_units"}
|
||||||
|
):
|
||||||
|
units.pop(key, None)
|
||||||
|
key = (event.model_profile, "rollback")
|
||||||
|
else:
|
||||||
|
quote = resolve_chat_price(caps.pricing, occurred_at=event.created_at)
|
||||||
|
if quote is None or units.get("repricing_revision") == quote.revision:
|
||||||
|
continue
|
||||||
|
tokens_in = int(units.get("tokens_in") or 0)
|
||||||
|
tokens_out = int(units.get("tokens_out") or 0)
|
||||||
|
# 旧主循环一直提取 DeepSeek cache usage,但 record_chat_usage 的
|
||||||
|
# extra_units 会省略值为 0 的字段,所以 chat 缺 key 可判定为零命中;
|
||||||
|
# 旧辅助调用没有提取缓存明细,缺 key 才是真正的不确定。
|
||||||
|
has_cache = "cache_hit_tokens" in units or event.kind == "chat"
|
||||||
|
cache_hit = int(units.get("cache_hit_tokens") or 0)
|
||||||
|
breakdown = calculate_chat_cost(
|
||||||
|
quote,
|
||||||
|
prompt_tokens=tokens_in,
|
||||||
|
completion_tokens=tokens_out,
|
||||||
|
cache_hit_tokens=cache_hit,
|
||||||
|
)
|
||||||
|
old = Decimal(str(event.cost_cny or 0))
|
||||||
|
new = breakdown.total_cny
|
||||||
|
previous_pricing = {
|
||||||
|
key: units[key] for key in PRICE_SNAPSHOT_KEYS if key in units
|
||||||
|
}
|
||||||
|
units.update(pricing_snapshot(quote, breakdown))
|
||||||
|
units.update({
|
||||||
|
"repricing_revision": quote.revision,
|
||||||
|
"previous_cost_cny": float(old),
|
||||||
|
"repriced_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"repricing_time_basis": "usage_event.created_at",
|
||||||
|
"repricing_estimated": not has_cache,
|
||||||
|
"previous_pricing_units": previous_pricing,
|
||||||
|
})
|
||||||
|
key = (event.model_profile, quote.tier)
|
||||||
|
if not has_cache:
|
||||||
|
summary[key]["estimated"] += 1
|
||||||
|
summary[key]["estimated_old"] += old
|
||||||
|
summary[key]["estimated_new"] += new
|
||||||
|
uncertain = estimated_by_kind[(event.model_profile, event.kind)]
|
||||||
|
uncertain["rows"] += 1
|
||||||
|
uncertain["old"] += old
|
||||||
|
uncertain["new"] += new
|
||||||
|
|
||||||
|
bucket = summary[key]
|
||||||
|
bucket["rows"] += 1
|
||||||
|
bucket["old"] += old
|
||||||
|
bucket["new"] += new
|
||||||
|
changed += 1
|
||||||
|
if args.apply or args.rollback:
|
||||||
|
event.cost_cny = new.quantize(Decimal("0.000001"))
|
||||||
|
event.units = units
|
||||||
|
|
||||||
|
if not (args.apply or args.rollback):
|
||||||
|
s.rollback()
|
||||||
|
|
||||||
|
total_old = Decimal("0")
|
||||||
|
total_new = Decimal("0")
|
||||||
|
for (profile, tier), item in sorted(summary.items()):
|
||||||
|
total_old += item["old"]
|
||||||
|
total_new += item["new"]
|
||||||
|
print(
|
||||||
|
f"[INFO] {profile} tier={tier} rows={item['rows']} "
|
||||||
|
f"estimated={item['estimated']} old={item['old']:.6f} "
|
||||||
|
f"new={item['new']:.6f} delta={(item['new'] - item['old']):.6f} "
|
||||||
|
f"estimated_delta={(item['estimated_new'] - item['estimated_old']):.6f}"
|
||||||
|
)
|
||||||
|
for (profile, kind), item in sorted(estimated_by_kind.items()):
|
||||||
|
print(
|
||||||
|
f"[WARN] estimated profile={profile} kind={kind} rows={item['rows']} "
|
||||||
|
f"old={item['old']:.6f} new={item['new']:.6f} "
|
||||||
|
f"delta={(item['new'] - item['old']):.6f}"
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
f"[OK] rows={changed} old={total_old:.6f} new={total_new:.6f} "
|
||||||
|
f"delta={(total_new - total_old):.6f} mode={mode}"
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
|
|
@ -2,6 +2,12 @@
|
||||||
"capability": "ansys.geometry.inspect@v1",
|
"capability": "ansys.geometry.inspect@v1",
|
||||||
"display_name": "ANSYS Mechanical 几何检查",
|
"display_name": "ANSYS Mechanical 几何检查",
|
||||||
"default_enrollment": false,
|
"default_enrollment": false,
|
||||||
|
"lifecycle": {
|
||||||
|
"status": "deprecated",
|
||||||
|
"visibility": "diagnostic_only",
|
||||||
|
"replacement": "cad.geometry.prepare@v1",
|
||||||
|
"detail": "保留用于 ANSYS Mechanical 真实导入专项诊断;普通几何准备请使用中性 CAD 能力。"
|
||||||
|
},
|
||||||
"output_namespace": "ansys",
|
"output_namespace": "ansys",
|
||||||
"input_policy": {
|
"input_policy": {
|
||||||
"suffixes": [".step", ".stp", ".x_t", ".x_b", ".iges", ".igs"],
|
"suffixes": [".step", ".stp", ".x_t", ".x_b", ".iges", ".igs"],
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,345 @@
|
||||||
|
{
|
||||||
|
"capability": "cad.geometry.prepare@v1",
|
||||||
|
"display_name": "中性 CAD 几何准备",
|
||||||
|
"default_enrollment": false,
|
||||||
|
"lifecycle": {
|
||||||
|
"status": "active",
|
||||||
|
"visibility": "standard",
|
||||||
|
"replacement": null,
|
||||||
|
"detail": "生成供 ANSYS、Abaqus 等后续任务复用的中性 STEP。"
|
||||||
|
},
|
||||||
|
"output_namespace": "cad",
|
||||||
|
"input_policy": {
|
||||||
|
"suffixes": [".step", ".stp"],
|
||||||
|
"max_count": 1,
|
||||||
|
"max_bytes": 268435456,
|
||||||
|
"max_total_bytes": 268435456
|
||||||
|
},
|
||||||
|
"outputs": {
|
||||||
|
"geometry": {
|
||||||
|
"filename": "geometry.step",
|
||||||
|
"media_type": "model/step",
|
||||||
|
"relative_path": "geometry.step",
|
||||||
|
"publish": true,
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
"geometry_recipe": {
|
||||||
|
"filename": "geometry-recipe.json",
|
||||||
|
"media_type": "application/json",
|
||||||
|
"relative_path": "geometry-recipe.json",
|
||||||
|
"publish": true,
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
"geometry_manifest": {
|
||||||
|
"filename": "geometry-manifest.json",
|
||||||
|
"media_type": "application/json",
|
||||||
|
"relative_path": "geometry-manifest.json",
|
||||||
|
"publish": true,
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
"geometry_preview": {
|
||||||
|
"filename": "geometry-preview.png",
|
||||||
|
"media_type": "image/png",
|
||||||
|
"relative_path": "geometry-preview.png",
|
||||||
|
"publish": true,
|
||||||
|
"required": true
|
||||||
|
},
|
||||||
|
"provenance": {
|
||||||
|
"filename": "provenance.json",
|
||||||
|
"media_type": "application/json",
|
||||||
|
"relative_path": ".meta/provenance.json",
|
||||||
|
"publish": false,
|
||||||
|
"required": true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"feature_path": ["operation", "prepare", "type"],
|
||||||
|
"features": {
|
||||||
|
"inspect": "1.0.0",
|
||||||
|
"primitive": "1.0.0",
|
||||||
|
"recipe": "1.0.0"
|
||||||
|
},
|
||||||
|
"summary": {
|
||||||
|
"title_path": ["operation", "prepare", "title"]
|
||||||
|
},
|
||||||
|
"legacy_runtime": null,
|
||||||
|
"workspace": null,
|
||||||
|
"request_schema": {
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"type": "object",
|
||||||
|
"x-maxBytes": 262144,
|
||||||
|
"required": ["schema_version", "inputs", "operation", "outputs"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"schema_version": {"const": 1},
|
||||||
|
"inputs": {
|
||||||
|
"type": "array",
|
||||||
|
"maxItems": 1,
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["key", "artifact_id"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"key": {"const": "geometry"},
|
||||||
|
"artifact_id": {"type": "string", "format": "uuid"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"operation": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["prepare"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"prepare": {
|
||||||
|
"oneOf": [
|
||||||
|
{"$ref": "#/$defs/inspect"},
|
||||||
|
{"$ref": "#/$defs/primitive_prepare"},
|
||||||
|
{"$ref": "#/$defs/recipe_prepare"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"outputs": {"type": "array", "maxItems": 0}
|
||||||
|
},
|
||||||
|
"$defs": {
|
||||||
|
"finite": {"type": "number", "minimum": -1000000, "maximum": 1000000},
|
||||||
|
"positive": {"type": "number", "exclusiveMinimum": 0, "maximum": 1000000},
|
||||||
|
"nonnegative": {"type": "number", "minimum": 0, "maximum": 1000000},
|
||||||
|
"vector3": {
|
||||||
|
"type": "array", "minItems": 3, "maxItems": 3,
|
||||||
|
"items": {"$ref": "#/$defs/finite"}
|
||||||
|
},
|
||||||
|
"direction3": {
|
||||||
|
"type": "array", "minItems": 3, "maxItems": 3,
|
||||||
|
"items": {"type": "number", "minimum": -1000000000000, "maximum": 1000000000000}
|
||||||
|
},
|
||||||
|
"point3": {
|
||||||
|
"type": "array", "minItems": 3, "maxItems": 3,
|
||||||
|
"items": {"$ref": "#/$defs/finite"}
|
||||||
|
},
|
||||||
|
"expected_count": {
|
||||||
|
"type": "integer", "minimum": 1, "maximum": 128,
|
||||||
|
"description": "Exact selected topology count assertion."
|
||||||
|
},
|
||||||
|
"extreme_face": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type", "axis", "side"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"type": {"const": "extreme_face"},
|
||||||
|
"axis": {"enum": ["x", "y", "z"]},
|
||||||
|
"side": {"enum": ["min", "max"]},
|
||||||
|
"tolerance": {"$ref": "#/$defs/nonnegative"},
|
||||||
|
"angle_tolerance_degrees": {"type": "number", "minimum": 0, "maximum": 15, "default": 5},
|
||||||
|
"expected_count": {"$ref": "#/$defs/expected_count"}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"planar_faces": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type", "normal", "offset", "tolerance"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"type": {"const": "planar_faces"},
|
||||||
|
"normal": {"$ref": "#/$defs/direction3"},
|
||||||
|
"offset": {"$ref": "#/$defs/finite"},
|
||||||
|
"tolerance": {"$ref": "#/$defs/positive"},
|
||||||
|
"angle_tolerance_degrees": {"type": "number", "minimum": 0, "maximum": 15, "default": 5},
|
||||||
|
"expected_count": {"$ref": "#/$defs/expected_count"}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"surface_faces": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type", "surface"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"type": {"const": "surface_faces"},
|
||||||
|
"surface": {"enum": ["plane", "cylinder", "cone", "sphere", "torus", "other"]},
|
||||||
|
"expected_count": {"$ref": "#/$defs/expected_count"}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"face_selector": {
|
||||||
|
"oneOf": [
|
||||||
|
{"$ref": "#/$defs/extreme_face"},
|
||||||
|
{"$ref": "#/$defs/planar_faces"},
|
||||||
|
{"$ref": "#/$defs/surface_faces"}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"edge_selector": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type", "curve"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"type": {"const": "curve_edges"},
|
||||||
|
"curve": {"enum": ["line", "circle", "ellipse", "spline", "other"]},
|
||||||
|
"expected_count": {"$ref": "#/$defs/expected_count"}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"solid_selector": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"type": {"const": "all_solids"},
|
||||||
|
"expected_count": {"$ref": "#/$defs/expected_count"}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"region": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["name", "topology", "selector"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{"if": {"properties": {"topology": {"const": "face"}}, "required": ["topology"]}, "then": {"properties": {"selector": {"$ref": "#/$defs/face_selector"}}}},
|
||||||
|
{"if": {"properties": {"topology": {"const": "edge"}}, "required": ["topology"]}, "then": {"properties": {"selector": {"$ref": "#/$defs/edge_selector"}}}},
|
||||||
|
{"if": {"properties": {"topology": {"const": "solid"}}, "required": ["topology"]}, "then": {"properties": {"selector": {"$ref": "#/$defs/solid_selector"}}}}
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"name": {"type": "string", "minLength": 1, "maxLength": 64, "pattern": "^[A-Za-z][A-Za-z0-9_-]*$"},
|
||||||
|
"topology": {"enum": ["face", "edge", "solid"]},
|
||||||
|
"selector": {
|
||||||
|
"oneOf": [
|
||||||
|
{"$ref": "#/$defs/face_selector"},
|
||||||
|
{"$ref": "#/$defs/edge_selector"},
|
||||||
|
{"$ref": "#/$defs/solid_selector"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"primitive": {
|
||||||
|
"oneOf": [
|
||||||
|
{
|
||||||
|
"type": "object", "required": ["shape", "length", "width", "height"], "additionalProperties": false,
|
||||||
|
"properties": {"shape": {"const": "box"}, "length": {"$ref": "#/$defs/positive"}, "width": {"$ref": "#/$defs/positive"}, "height": {"$ref": "#/$defs/positive"}}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object", "required": ["shape", "radius", "height"], "additionalProperties": false,
|
||||||
|
"properties": {"shape": {"const": "cylinder"}, "radius": {"$ref": "#/$defs/positive"}, "height": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object", "required": ["shape", "radius", "inner_radius", "height"], "additionalProperties": false,
|
||||||
|
"properties": {"shape": {"const": "tube"}, "radius": {"$ref": "#/$defs/positive"}, "inner_radius": {"$ref": "#/$defs/positive"}, "height": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object", "required": ["shape", "radius", "height"], "additionalProperties": false,
|
||||||
|
"properties": {"shape": {"const": "cone"}, "radius": {"$ref": "#/$defs/positive"}, "height": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object", "required": ["shape", "radius1", "radius2", "height"], "additionalProperties": false,
|
||||||
|
"properties": {"shape": {"const": "frustum"}, "radius1": {"$ref": "#/$defs/nonnegative"}, "radius2": {"$ref": "#/$defs/nonnegative"}, "height": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object", "required": ["shape", "radius"], "additionalProperties": false,
|
||||||
|
"properties": {"shape": {"const": "sphere"}, "radius": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object", "required": ["shape", "x_min", "x_max", "y_min", "y_max", "z_min", "z_max"], "additionalProperties": false,
|
||||||
|
"properties": {"shape": {"const": "wedge"}, "x_min": {"$ref": "#/$defs/finite"}, "x_max": {"$ref": "#/$defs/finite"}, "y_min": {"$ref": "#/$defs/finite"}, "y_max": {"$ref": "#/$defs/finite"}, "z_min": {"$ref": "#/$defs/finite"}, "z_max": {"$ref": "#/$defs/finite"}}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "object", "required": ["shape", "major_radius", "minor_radius"], "additionalProperties": false,
|
||||||
|
"properties": {"shape": {"const": "torus"}, "major_radius": {"$ref": "#/$defs/positive"}, "minor_radius": {"$ref": "#/$defs/positive"}, "angle_degrees": {"type": "number", "exclusiveMinimum": 0, "maximum": 360}}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"profile": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["id", "points"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"id": {"$ref": "#/$defs/name"},
|
||||||
|
"points": {
|
||||||
|
"type": "array", "minItems": 3, "maxItems": 512,
|
||||||
|
"items": {"$ref": "#/$defs/point3"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"name": {"type": "string", "minLength": 1, "maxLength": 64, "pattern": "^[A-Za-z][A-Za-z0-9_-]*$"},
|
||||||
|
"step": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["id", "op"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"id": {"$ref": "#/$defs/name"},
|
||||||
|
"op": {"enum": ["box", "cylinder", "tube", "cone", "frustum", "sphere", "wedge", "torus", "import_step", "extrude", "revolve", "sweep", "loft", "union", "cut", "intersect", "fillet", "chamfer", "shell", "translate", "rotate", "mirror", "linear_pattern", "circular_pattern", "combine"]},
|
||||||
|
"primitive": {"$ref": "#/$defs/primitive"},
|
||||||
|
"input": {"const": "geometry"},
|
||||||
|
"profile": {"$ref": "#/$defs/name"},
|
||||||
|
"profiles": {"type": "array", "minItems": 2, "maxItems": 32, "uniqueItems": true, "items": {"$ref": "#/$defs/name"}},
|
||||||
|
"target": {"$ref": "#/$defs/name"},
|
||||||
|
"targets": {"type": "array", "minItems": 1, "maxItems": 64, "uniqueItems": true, "items": {"$ref": "#/$defs/name"}},
|
||||||
|
"tools": {"type": "array", "minItems": 1, "maxItems": 64, "uniqueItems": true, "items": {"$ref": "#/$defs/name"}},
|
||||||
|
"vector": {"$ref": "#/$defs/vector3"},
|
||||||
|
"axis_start": {"$ref": "#/$defs/point3"},
|
||||||
|
"axis_end": {"$ref": "#/$defs/point3"},
|
||||||
|
"angle_degrees": {"type": "number", "exclusiveMinimum": -360, "maximum": 360},
|
||||||
|
"path": {"type": "array", "minItems": 2, "maxItems": 512, "items": {"$ref": "#/$defs/point3"}},
|
||||||
|
"selector": {"oneOf": [{"$ref": "#/$defs/face_selector"}, {"$ref": "#/$defs/edge_selector"}]},
|
||||||
|
"radius": {"$ref": "#/$defs/positive"},
|
||||||
|
"distance": {"$ref": "#/$defs/positive"},
|
||||||
|
"thickness": {"$ref": "#/$defs/positive"},
|
||||||
|
"plane": {"enum": ["XY", "XZ", "YZ"]},
|
||||||
|
"count": {"type": "integer", "minimum": 2, "maximum": 128},
|
||||||
|
"spacing": {"$ref": "#/$defs/positive"},
|
||||||
|
"center": {"$ref": "#/$defs/point3"}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"base_prepare": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type", "title", "canonical_unit", "regions"],
|
||||||
|
"properties": {
|
||||||
|
"type": {"enum": ["inspect", "primitive", "recipe"]},
|
||||||
|
"title": {"type": "string", "minLength": 1, "maxLength": 160},
|
||||||
|
"canonical_unit": {"enum": ["mm", "m"]},
|
||||||
|
"regions": {"type": "array", "maxItems": 64, "items": {"$ref": "#/$defs/region"}}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"inspect": {
|
||||||
|
"allOf": [
|
||||||
|
{"$ref": "#/$defs/base_prepare"},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type", "title", "canonical_unit", "regions", "geometry"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"type": {"const": "inspect"},
|
||||||
|
"title": true, "canonical_unit": true, "regions": true,
|
||||||
|
"geometry": {"type": "object", "required": ["input"], "additionalProperties": false, "properties": {"input": {"const": "geometry"}}}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"primitive_prepare": {
|
||||||
|
"allOf": [
|
||||||
|
{"$ref": "#/$defs/base_prepare"},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type", "title", "canonical_unit", "regions", "primitive"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {"type": {"const": "primitive"}, "title": true, "canonical_unit": true, "regions": true, "primitive": {"$ref": "#/$defs/primitive"}}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"recipe_prepare": {
|
||||||
|
"allOf": [
|
||||||
|
{"$ref": "#/$defs/base_prepare"},
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"required": ["type", "title", "canonical_unit", "regions", "recipe"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"type": {"const": "recipe"}, "title": true, "canonical_unit": true, "regions": true,
|
||||||
|
"recipe": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["profiles", "steps", "result"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"profiles": {"type": "array", "maxItems": 64, "items": {"$ref": "#/$defs/profile"}},
|
||||||
|
"steps": {"type": "array", "minItems": 1, "maxItems": 128, "items": {"$ref": "#/$defs/step"}},
|
||||||
|
"result": {"$ref": "#/$defs/name"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,121 @@
|
||||||
|
{
|
||||||
|
"capability": "freecad.model.author@v1",
|
||||||
|
"display_name": "FreeCAD 受控建模",
|
||||||
|
"default_enrollment": false,
|
||||||
|
"output_namespace": "freecad",
|
||||||
|
"input_policy": {
|
||||||
|
"suffixes": [".step", ".stp"],
|
||||||
|
"max_count": 1,
|
||||||
|
"max_bytes": 268435456,
|
||||||
|
"max_total_bytes": 268435456
|
||||||
|
},
|
||||||
|
"outputs": {
|
||||||
|
"project": {"filename": "model.FCStd", "media_type": "application/vnd.freecad", "relative_path": "model.FCStd", "publish": true, "required": true},
|
||||||
|
"geometry_step": {"filename": "model.step", "media_type": "model/step", "relative_path": "model.step", "publish": true, "required": true},
|
||||||
|
"preview": {"filename": "model-preview.png", "media_type": "image/png", "relative_path": "model-preview.png", "publish": true, "required": true},
|
||||||
|
"model_recipe": {"filename": "model-recipe.json", "media_type": "application/json", "relative_path": "model-recipe.json", "publish": true, "required": true},
|
||||||
|
"model_manifest": {"filename": "model-manifest.json", "media_type": "application/json", "relative_path": "model-manifest.json", "publish": true, "required": true},
|
||||||
|
"provenance": {"filename": "provenance.json", "media_type": "application/json", "relative_path": ".meta/provenance.json", "publish": false, "required": true}
|
||||||
|
},
|
||||||
|
"feature_path": ["operation", "model", "type"],
|
||||||
|
"features": {"recipe": "1.0.0"},
|
||||||
|
"summary": {"title_path": ["operation", "model", "title"]},
|
||||||
|
"legacy_runtime": null,
|
||||||
|
"workspace": {
|
||||||
|
"state_output": "project",
|
||||||
|
"preview_outputs": ["preview"],
|
||||||
|
"required_adapter_version": "1.0.0"
|
||||||
|
},
|
||||||
|
"request_schema": {
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"type": "object",
|
||||||
|
"x-maxBytes": 262144,
|
||||||
|
"required": ["schema_version", "inputs", "operation", "outputs"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"schema_version": {"const": 1},
|
||||||
|
"inputs": {
|
||||||
|
"type": "array", "maxItems": 1,
|
||||||
|
"items": {
|
||||||
|
"type": "object", "required": ["key", "artifact_id"], "additionalProperties": false,
|
||||||
|
"properties": {"key": {"const": "geometry"}, "artifact_id": {"type": "string", "format": "uuid"}}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"operation": {
|
||||||
|
"type": "object", "required": ["model"], "additionalProperties": false,
|
||||||
|
"properties": {"model": {"$ref": "#/$defs/model"}}
|
||||||
|
},
|
||||||
|
"outputs": {"type": "array", "maxItems": 0}
|
||||||
|
},
|
||||||
|
"$defs": {
|
||||||
|
"identifier": {"type": "string", "pattern": "^[A-Za-z][A-Za-z0-9_-]{0,63}$"},
|
||||||
|
"finite": {"type": "number", "minimum": -1000000, "maximum": 1000000},
|
||||||
|
"positive": {"type": "number", "exclusiveMinimum": 0, "maximum": 1000000},
|
||||||
|
"parameterRef": {
|
||||||
|
"type": "object", "required": ["parameter"], "additionalProperties": false,
|
||||||
|
"properties": {"parameter": {"$ref": "#/$defs/identifier"}}
|
||||||
|
},
|
||||||
|
"numberValue": {"oneOf": [{"$ref": "#/$defs/finite"}, {"$ref": "#/$defs/parameterRef"}]},
|
||||||
|
"positiveValue": {"oneOf": [{"$ref": "#/$defs/positive"}, {"$ref": "#/$defs/parameterRef"}]},
|
||||||
|
"vector3": {"type": "array", "minItems": 3, "maxItems": 3, "items": {"$ref": "#/$defs/numberValue"}},
|
||||||
|
"resultRef": {"$ref": "#/$defs/identifier"},
|
||||||
|
"box": {
|
||||||
|
"type": "object", "required": ["id", "op", "length", "width", "height"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "box"}, "length": {"$ref": "#/$defs/positiveValue"}, "width": {"$ref": "#/$defs/positiveValue"}, "height": {"$ref": "#/$defs/positiveValue"}}
|
||||||
|
},
|
||||||
|
"cylinder": {
|
||||||
|
"type": "object", "required": ["id", "op", "radius", "height"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "cylinder"}, "radius": {"$ref": "#/$defs/positiveValue"}, "height": {"$ref": "#/$defs/positiveValue"}, "angle_degrees": {"$ref": "#/$defs/positiveValue"}}
|
||||||
|
},
|
||||||
|
"cone": {
|
||||||
|
"type": "object", "required": ["id", "op", "radius1", "radius2", "height"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "cone"}, "radius1": {"$ref": "#/$defs/numberValue"}, "radius2": {"$ref": "#/$defs/numberValue"}, "height": {"$ref": "#/$defs/positiveValue"}, "angle_degrees": {"$ref": "#/$defs/positiveValue"}}
|
||||||
|
},
|
||||||
|
"sphere": {
|
||||||
|
"type": "object", "required": ["id", "op", "radius"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "sphere"}, "radius": {"$ref": "#/$defs/positiveValue"}, "angle1_degrees": {"$ref": "#/$defs/numberValue"}, "angle2_degrees": {"$ref": "#/$defs/numberValue"}, "angle3_degrees": {"$ref": "#/$defs/positiveValue"}}
|
||||||
|
},
|
||||||
|
"torus": {
|
||||||
|
"type": "object", "required": ["id", "op", "major_radius", "minor_radius"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "torus"}, "major_radius": {"$ref": "#/$defs/positiveValue"}, "minor_radius": {"$ref": "#/$defs/positiveValue"}, "angle1_degrees": {"$ref": "#/$defs/numberValue"}, "angle2_degrees": {"$ref": "#/$defs/numberValue"}, "angle3_degrees": {"$ref": "#/$defs/positiveValue"}}
|
||||||
|
},
|
||||||
|
"importStep": {
|
||||||
|
"type": "object", "required": ["id", "op", "input"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "import_step"}, "input": {"const": "geometry"}}
|
||||||
|
},
|
||||||
|
"boolean": {
|
||||||
|
"type": "object", "required": ["id", "op", "target", "tools"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"enum": ["union", "cut", "intersect"]}, "target": {"$ref": "#/$defs/resultRef"}, "tools": {"type": "array", "minItems": 1, "maxItems": 32, "uniqueItems": true, "items": {"$ref": "#/$defs/resultRef"}}}
|
||||||
|
},
|
||||||
|
"translate": {
|
||||||
|
"type": "object", "required": ["id", "op", "target", "vector"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "translate"}, "target": {"$ref": "#/$defs/resultRef"}, "vector": {"$ref": "#/$defs/vector3"}}
|
||||||
|
},
|
||||||
|
"rotate": {
|
||||||
|
"type": "object", "required": ["id", "op", "target", "axis_point", "axis_direction", "angle_degrees"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "rotate"}, "target": {"$ref": "#/$defs/resultRef"}, "axis_point": {"$ref": "#/$defs/vector3"}, "axis_direction": {"$ref": "#/$defs/vector3"}, "angle_degrees": {"$ref": "#/$defs/numberValue"}}
|
||||||
|
},
|
||||||
|
"combine": {
|
||||||
|
"type": "object", "required": ["id", "op", "targets"], "additionalProperties": false,
|
||||||
|
"properties": {"id": {"$ref": "#/$defs/identifier"}, "op": {"const": "combine"}, "targets": {"type": "array", "minItems": 1, "maxItems": 64, "uniqueItems": true, "items": {"$ref": "#/$defs/resultRef"}}}
|
||||||
|
},
|
||||||
|
"step": {"oneOf": [{"$ref": "#/$defs/box"}, {"$ref": "#/$defs/cylinder"}, {"$ref": "#/$defs/cone"}, {"$ref": "#/$defs/sphere"}, {"$ref": "#/$defs/torus"}, {"$ref": "#/$defs/importStep"}, {"$ref": "#/$defs/boolean"}, {"$ref": "#/$defs/translate"}, {"$ref": "#/$defs/rotate"}, {"$ref": "#/$defs/combine"}]},
|
||||||
|
"preview": {
|
||||||
|
"type": "object", "additionalProperties": false,
|
||||||
|
"properties": {"width": {"type": "integer", "minimum": 320, "maximum": 1600}, "height": {"type": "integer", "minimum": 240, "maximum": 1600}, "background": {"type": "string", "pattern": "^#[0-9A-Fa-f]{6}$"}}
|
||||||
|
},
|
||||||
|
"model": {
|
||||||
|
"type": "object", "required": ["type", "recipe_version", "title", "unit", "steps", "result"], "additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"type": {"const": "recipe"}, "recipe_version": {"const": 1},
|
||||||
|
"title": {"type": "string", "minLength": 1, "maxLength": 160},
|
||||||
|
"unit": {"const": "mm"},
|
||||||
|
"parameters": {"type": "object", "maxProperties": 64, "propertyNames": {"$ref": "#/$defs/identifier"}, "additionalProperties": {"$ref": "#/$defs/finite"}},
|
||||||
|
"steps": {"type": "array", "minItems": 1, "maxItems": 128, "items": {"$ref": "#/$defs/step"}},
|
||||||
|
"result": {"$ref": "#/$defs/resultRef"},
|
||||||
|
"preview": {"$ref": "#/$defs/preview"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -55,18 +55,30 @@ test("composer uses one Codex-style action button for stop and queued send", ()
|
||||||
assert.match(html, /\.msg\.queued/);
|
assert.match(html, /\.msg\.queued/);
|
||||||
assert.match(chat, /card\.className = "msg user queued"/);
|
assert.match(chat, /card\.className = "msg user queued"/);
|
||||||
assert.match(chat, /function composerHasPayload\(\)/);
|
assert.match(chat, /function composerHasPayload\(\)/);
|
||||||
|
assert.match(chat, /function isCurrentTaskRunActive\(\)/);
|
||||||
assert.match(chat, /btn\.textContent = "停止"/);
|
assert.match(chat, /btn\.textContent = "停止"/);
|
||||||
assert.match(chat, /if \(composerHasPayload\(\)\)[\s\S]*queueCurrentMessage\(\)/);
|
assert.match(chat, /if \(composerHasPayload\(\)\)[\s\S]*queueCurrentMessage\(\)/);
|
||||||
assert.match(chat, /_actionMode === "loading" \|\| _actionMode === "submitting"[\s\S]*btn\.disabled = true/);
|
assert.match(chat, /_actionMode === "loading" \|\| _actionMode === "submitting"[\s\S]*btn\.disabled = true/);
|
||||||
assert.match(chat, /setActionMode\("submitting"\)/);
|
assert.match(chat, /setActionMode\("submitting"\)/);
|
||||||
assert.match(chat, /if \(state\.taskId === taskId\) \$\("chat-input"\)\.value = ""/);
|
assert.match(chat, /if \(state\.taskId === taskId\) \$\("chat-input"\)\.value = ""/);
|
||||||
assert.match(chat, /if \(state\.taskId === taskId\) setActionMode\("streaming"\)/);
|
assert.match(chat, /if \(state\.taskId === taskId\) setActionMode\("streaming"\)/);
|
||||||
|
assert.match(chat, /const activeRun = isCurrentTaskRunActive\(\)/);
|
||||||
|
assert.match(chat, /else if \(activeRun\)[\s\S]*cancelCurrentTask\(\)/);
|
||||||
assert.match(chat, /function queueCurrentMessage\(\)/);
|
assert.match(chat, /function queueCurrentMessage\(\)/);
|
||||||
assert.match(chat, /async function dispatchNextQueuedMessage\(taskId\)/);
|
assert.match(chat, /async function dispatchNextQueuedMessage\(taskId\)/);
|
||||||
assert.match(chat, /void dispatchNextQueuedMessage\(ctx\.taskId\)/);
|
assert.match(chat, /void dispatchNextQueuedMessage\(ctx\.taskId\)/);
|
||||||
assert.match(chat, /removeQueuedMessage\([\s\S]*streamSse\(r\.events_url, run\)/);
|
assert.match(chat, /removeQueuedMessage\([\s\S]*streamSse\(r\.events_url, run\)/);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("heavy execution queue exposes the blocking reason", () => {
|
||||||
|
const chat = readFileSync(new URL("../web/static/js/chat.js", import.meta.url), "utf8");
|
||||||
|
assert.match(chat, /t === "execution_queue"/);
|
||||||
|
assert.match(chat, /等待执行容量(当前用户/);
|
||||||
|
assert.match(chat, /等待执行容量(整机/);
|
||||||
|
assert.match(chat, /等待执行容量(宿主内存压力)/);
|
||||||
|
assert.match(chat, /dataset\.runningLabel/);
|
||||||
|
});
|
||||||
|
|
||||||
test("automatic title polling starts before a long-running response finishes", () => {
|
test("automatic title polling starts before a long-running response finishes", () => {
|
||||||
const chat = readFileSync(new URL("../web/static/js/chat.js", import.meta.url), "utf8");
|
const chat = readFileSync(new URL("../web/static/js/chat.js", import.meta.url), "utf8");
|
||||||
const sendMessage = chat.indexOf("async function sendMessage(");
|
const sendMessage = chat.indexOf("async function sendMessage(");
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,7 @@ import test from "node:test";
|
||||||
const filesJs = fs.readFileSync("web/static/js/files.js", "utf8");
|
const filesJs = fs.readFileSync("web/static/js/files.js", "utf8");
|
||||||
const layoutJs = fs.readFileSync("web/static/js/layout.js", "utf8");
|
const layoutJs = fs.readFileSync("web/static/js/layout.js", "utf8");
|
||||||
const jobsJs = fs.readFileSync("web/static/js/software_jobs.js", "utf8");
|
const jobsJs = fs.readFileSync("web/static/js/software_jobs.js", "utf8");
|
||||||
|
const previewJs = fs.readFileSync("web/static/js/preview.js", "utf8");
|
||||||
const pageHtml = fs.readFileSync("web/static/dev.html", "utf8");
|
const pageHtml = fs.readFileSync("web/static/dev.html", "utf8");
|
||||||
|
|
||||||
test("file menu exposes an explicit confirmed replacement action", () => {
|
test("file menu exposes an explicit confirmed replacement action", () => {
|
||||||
|
|
@ -48,6 +49,22 @@ test("software jobs share the embed-safe right workspace", () => {
|
||||||
assert.match(jobsJs, /作业列表加载失败,请点击刷新重试/);
|
assert.match(jobsJs, /作业列表加载失败,请点击刷新重试/);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("current task software jobs render in a dedicated result rail", () => {
|
||||||
|
assert.match(pageHtml, /id="software-result-rail"/);
|
||||||
|
assert.match(pageHtml, /id="mobile-software-results"/);
|
||||||
|
assert.match(jobsJs, /task_id: taskJobsTaskId/);
|
||||||
|
assert.match(jobsJs, /renderTaskResultRail/);
|
||||||
|
assert.match(jobsJs, /task-selection-changed/);
|
||||||
|
assert.match(jobsJs, /openSoftwarePreviewGallery/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("software preview gallery uses authenticated fetch and hides artifact download", () => {
|
||||||
|
assert.match(previewJs, /export async function openSoftwarePreviewGallery/);
|
||||||
|
assert.match(previewJs, /"Authorization": "Bearer " \+ state\.token/);
|
||||||
|
assert.match(previewJs, /\$\("fp-download"\)\.hidden = true/);
|
||||||
|
assert.match(pageHtml, /software-preview-gallery/);
|
||||||
|
});
|
||||||
|
|
||||||
test("mobile navigation distinguishes list, files, and software jobs", () => {
|
test("mobile navigation distinguishes list, files, and software jobs", () => {
|
||||||
assert.match(pageHtml, /id="mv-tab-left"[^>]*>\s*列表/);
|
assert.match(pageHtml, /id="mv-tab-left"[^>]*>\s*列表/);
|
||||||
assert.match(pageHtml, /id="mv-tab-right"[^>]*data-right-tab="files"/);
|
assert.match(pageHtml, /id="mv-tab-right"[^>]*data-right-tab="files"/);
|
||||||
|
|
|
||||||
|
|
@ -78,6 +78,11 @@ test("renders completed mermaid blocks with strict local configuration", async (
|
||||||
assert.equal(calls.initialize.length, 1);
|
assert.equal(calls.initialize.length, 1);
|
||||||
assert.equal(calls.initialize[0].securityLevel, "strict");
|
assert.equal(calls.initialize[0].securityLevel, "strict");
|
||||||
assert.equal(calls.initialize[0].startOnLoad, false);
|
assert.equal(calls.initialize[0].startOnLoad, false);
|
||||||
|
assert.equal(calls.initialize[0].theme, "base");
|
||||||
|
assert.equal(calls.initialize[0].themeVariables.primaryColor, "#eaf2ff");
|
||||||
|
assert.equal(calls.initialize[0].themeVariables.secondaryColor, "#e8f7f3");
|
||||||
|
assert.equal(calls.initialize[0].themeVariables.tertiaryColor, "#fff4df");
|
||||||
|
assert.equal(calls.initialize[0].themeVariables.pie5, "#d96c75");
|
||||||
assert.equal(calls.render[0].source, "flowchart LR\nA --> B");
|
assert.equal(calls.render[0].source, "flowchart LR\nA --> B");
|
||||||
assert.equal(fixture.pre.replacement.className, "mermaid-diagram");
|
assert.equal(fixture.pre.replacement.className, "mermaid-diagram");
|
||||||
assert.equal(fixture.pre.replacement.children[0].innerHTML, "<svg viewBox='0 0 10 10'></svg>");
|
assert.equal(fixture.pre.replacement.children[0].innerHTML, "<svg viewBox='0 0 10 10'></svg>");
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,48 @@
|
||||||
|
"""Admin 存储快照的配额、回收站与物理合计口径。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import unittest
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from web.admin import _storage_page
|
||||||
|
|
||||||
|
|
||||||
|
class AdminStorageTests(unittest.TestCase):
|
||||||
|
def test_storage_page_exposes_trash_and_physical_total(self) -> None:
|
||||||
|
uid = uuid4()
|
||||||
|
scanned = datetime(2026, 8, 28, tzinfo=timezone.utc)
|
||||||
|
totals_result = MagicMock()
|
||||||
|
totals_result.one.return_value = (1, 100, 40, 2, 1)
|
||||||
|
rows_result = MagicMock()
|
||||||
|
rows_result.all.return_value = [
|
||||||
|
(uid, "user@example.com", "测试用户", "tester", 100, 2, 40, 1, scanned)
|
||||||
|
]
|
||||||
|
session = MagicMock()
|
||||||
|
session.execute.side_effect = [totals_result, rows_result]
|
||||||
|
|
||||||
|
with patch("core.agent_builder.load_config", return_value={"quotas": {}}):
|
||||||
|
page = _storage_page(session, 0, 20)
|
||||||
|
|
||||||
|
row = page["rows"][0]
|
||||||
|
self.assertEqual(row["bytes_used"], 100)
|
||||||
|
self.assertEqual(row["trash_bytes"], 40)
|
||||||
|
self.assertEqual(row["trash_file_count"], 1)
|
||||||
|
self.assertEqual(row["physical_bytes"], 140)
|
||||||
|
self.assertEqual(page["total"], 1)
|
||||||
|
self.assertEqual(page["totals"], {
|
||||||
|
"bytes_used": 100,
|
||||||
|
"trash_bytes": 40,
|
||||||
|
"physical_bytes": 140,
|
||||||
|
"file_count": 2,
|
||||||
|
"trash_file_count": 1,
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -0,0 +1,21 @@
|
||||||
|
from sqlalchemy.dialects.postgresql import dialect, insert as pg_insert
|
||||||
|
from sqlalchemy import text
|
||||||
|
|
||||||
|
from core.storage.models import Artifact
|
||||||
|
|
||||||
|
|
||||||
|
def test_artifact_upsert_partial_index_predicate_is_literal():
|
||||||
|
statement = pg_insert(Artifact).values(
|
||||||
|
user_id="00000000-0000-0000-0000-000000000001",
|
||||||
|
origin_task_id="00000000-0000-0000-0000-000000000002",
|
||||||
|
current_path="task/report.docx",
|
||||||
|
label="report",
|
||||||
|
).on_conflict_do_update(
|
||||||
|
index_elements=[Artifact.user_id, Artifact.current_path],
|
||||||
|
index_where=text("status = 'active'"),
|
||||||
|
set_={"label": "report"},
|
||||||
|
)
|
||||||
|
|
||||||
|
sql = str(statement.compile(dialect=dialect()))
|
||||||
|
assert "WHERE status = 'active'" in sql
|
||||||
|
assert "WHERE status = %(" not in sql
|
||||||
|
|
@ -3,7 +3,11 @@ import unittest
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from core.artifacts import ArtifactPathError
|
from core.artifacts import ArtifactPathError
|
||||||
from core.attachments import content_for_model, normalize_attachment_refs
|
from core.attachments import (
|
||||||
|
content_for_model,
|
||||||
|
materialize_native_images,
|
||||||
|
normalize_attachment_refs,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class AttachmentRefTests(unittest.TestCase):
|
class AttachmentRefTests(unittest.TestCase):
|
||||||
|
|
@ -47,3 +51,27 @@ class AttachmentRefTests(unittest.TestCase):
|
||||||
content_for_model("", refs, "实验分析"),
|
content_for_model("", refs, "实验分析"),
|
||||||
"[用户上传的参考图] 实验分析/figures/a.png",
|
"[用户上传的参考图] 实验分析/figures/a.png",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_materializes_native_image_without_persisting_internal_fields(self):
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
wd = root / "任务"
|
||||||
|
wd.mkdir()
|
||||||
|
(wd / "a.png").write_bytes(b"png")
|
||||||
|
messages = [{
|
||||||
|
"role": "user",
|
||||||
|
"content": "请分析\n\n[用户上传的参考图] 任务/a.png",
|
||||||
|
"_attachment_refs": [{"path": "a.png", "kind": "image"}],
|
||||||
|
}]
|
||||||
|
prepared, count = materialize_native_images(
|
||||||
|
messages, enabled=True, working_dir=wd, user_root=root,
|
||||||
|
)
|
||||||
|
self.assertEqual(count, 1)
|
||||||
|
self.assertNotIn("_attachment_refs", prepared[0])
|
||||||
|
self.assertEqual(prepared[0]["content"][0]["type"], "text")
|
||||||
|
self.assertEqual(prepared[0]["content"][1]["type"], "image_url")
|
||||||
|
self.assertTrue(
|
||||||
|
prepared[0]["content"][1]["image_url"]["url"].startswith(
|
||||||
|
"data:image/png;base64,"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
|
||||||
|
|
@ -138,6 +138,10 @@ class BlenderContractTests(unittest.TestCase):
|
||||||
tuple(f"preview_{index:02d}" for index in range(1, 9)),
|
tuple(f"preview_{index:02d}" for index in range(1, 9)),
|
||||||
)
|
)
|
||||||
self.assertEqual(contract.required_adapter_version(normalized), "1.0.0")
|
self.assertEqual(contract.required_adapter_version(normalized), "1.0.0")
|
||||||
|
self.assertEqual(
|
||||||
|
contract.summarize(normalized)["title"],
|
||||||
|
"水泥实验室设备布置",
|
||||||
|
)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
set(contract.expected_outputs(normalized)),
|
set(contract.expected_outputs(normalized)),
|
||||||
{
|
{
|
||||||
|
|
|
||||||
|
|
@ -10,6 +10,7 @@
|
||||||
用两个 broker 实例共享同一 FakeServer 模拟蓝绿两进程。
|
用两个 broker 实例共享同一 FakeServer 模拟蓝绿两进程。
|
||||||
"""
|
"""
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import threading
|
||||||
import unittest
|
import unittest
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
|
|
@ -134,8 +135,36 @@ class TestRedisBroker(unittest.TestCase):
|
||||||
b.emit(tid, {"type": "text", "delta": "x"}) # 不抛
|
b.emit(tid, {"type": "text", "delta": "x"}) # 不抛
|
||||||
self.assertFalse(b.is_cancelled(tid)) # 降级 False
|
self.assertFalse(b.is_cancelled(tid)) # 降级 False
|
||||||
b.request_cancel(tid) # 不抛
|
b.request_cancel(tid) # 不抛
|
||||||
|
self.assertFalse(b.is_cancelled(tid))
|
||||||
b.close(tid) # 不抛
|
b.close(tid) # 不抛
|
||||||
|
|
||||||
|
def test_cancel_uses_local_signal_while_redis_write_is_blocked(self):
|
||||||
|
"""Redis 卡顿时,同实例 run 无需等待网络超时即可看到停止。"""
|
||||||
|
class SlowBroken:
|
||||||
|
def __init__(self):
|
||||||
|
self.writing = threading.Event()
|
||||||
|
self.release = threading.Event()
|
||||||
|
|
||||||
|
def set(self, *args, **kwargs):
|
||||||
|
self.writing.set()
|
||||||
|
self.release.wait(2)
|
||||||
|
raise ConnectionError("redis down")
|
||||||
|
|
||||||
|
def get(self, *args, **kwargs):
|
||||||
|
raise ConnectionError("redis down")
|
||||||
|
|
||||||
|
redis = SlowBroken()
|
||||||
|
b = RedisRunBroker("redis://fake:6379/0", sync_client=redis,
|
||||||
|
async_client=fakeredis.aioredis.FakeRedis(decode_responses=True))
|
||||||
|
tid = uuid4()
|
||||||
|
worker = threading.Thread(target=b.request_cancel, args=(tid,), daemon=True)
|
||||||
|
worker.start()
|
||||||
|
self.assertTrue(redis.writing.wait(1))
|
||||||
|
self.assertTrue(b.is_cancelled(tid))
|
||||||
|
redis.release.set()
|
||||||
|
worker.join(1)
|
||||||
|
self.assertFalse(b.is_cancelled(tid))
|
||||||
|
|
||||||
def test_bind_loop_fail_fast_when_unreachable(self):
|
def test_bind_loop_fail_fast_when_unreachable(self):
|
||||||
class Boom:
|
class Boom:
|
||||||
def ping(self):
|
def ping(self):
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,389 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from core.software_contracts import SoftwareContractError, get_contract
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
WORKER_PATH = (
|
||||||
|
ROOT / "windows-node" / "adapters" / "cad.geometry.prepare@v1" / "worker.py"
|
||||||
|
)
|
||||||
|
SPEC = importlib.util.spec_from_file_location("zcbot_cad_worker", WORKER_PATH)
|
||||||
|
assert SPEC and SPEC.loader
|
||||||
|
worker = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(worker)
|
||||||
|
HAS_CADQUERY = importlib.util.find_spec("cadquery") is not None
|
||||||
|
|
||||||
|
|
||||||
|
def _request(feature: str, definition: dict, *, regions: list[dict] | None = None) -> dict:
|
||||||
|
prepare = {
|
||||||
|
"type": feature,
|
||||||
|
"title": f"CAD {feature}",
|
||||||
|
"canonical_unit": "mm",
|
||||||
|
"regions": regions or [],
|
||||||
|
}
|
||||||
|
prepare.update(definition)
|
||||||
|
return {
|
||||||
|
"schema_version": 1,
|
||||||
|
"inputs": [],
|
||||||
|
"operation": {"prepare": prepare},
|
||||||
|
"outputs": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _primitive(shape: str, **parameters) -> dict:
|
||||||
|
return _request("primitive", {"primitive": {"shape": shape, **parameters}})
|
||||||
|
|
||||||
|
|
||||||
|
class CadContractTests(unittest.TestCase):
|
||||||
|
def test_contract_exposes_fixed_features_outputs_and_quotas(self) -> None:
|
||||||
|
contract = get_contract("cad.geometry.prepare@v1")
|
||||||
|
self.assertTrue(contract.ordinarily_visible)
|
||||||
|
self.assertEqual(set(contract.features), {"inspect", "primitive", "recipe"})
|
||||||
|
self.assertEqual(contract.input_policy["suffixes"], [".step", ".stp"])
|
||||||
|
request = _primitive("box", length=10, width=20, height=30)
|
||||||
|
normalized, digest = contract.normalize_request(request)
|
||||||
|
self.assertEqual(contract.feature(normalized), "primitive")
|
||||||
|
self.assertEqual(len(digest), 64)
|
||||||
|
self.assertEqual(
|
||||||
|
set(contract.expected_outputs(normalized)),
|
||||||
|
{"geometry", "geometry_recipe", "geometry_manifest", "geometry_preview", "provenance"},
|
||||||
|
)
|
||||||
|
geometry_output = contract.outputs["geometry"]
|
||||||
|
self.assertEqual(geometry_output.filename, "geometry.step")
|
||||||
|
ansys = get_contract("ansys.mechanical.static_structural@v2")
|
||||||
|
self.assertIn(Path(geometry_output.filename).suffix, ansys.input_policy["suffixes"])
|
||||||
|
|
||||||
|
def test_contract_rejects_scripts_paths_nonfinite_and_missing_units(self) -> None:
|
||||||
|
contract = get_contract("cad.geometry.prepare@v1")
|
||||||
|
for key, value in (("script", "anything"), ("path", "C:/secret.step")):
|
||||||
|
request = _primitive("box", length=10, width=20, height=30)
|
||||||
|
request["operation"]["prepare"][key] = value
|
||||||
|
with self.assertRaises(SoftwareContractError):
|
||||||
|
contract.normalize_request(request)
|
||||||
|
request = _primitive("box", length=math.inf, width=20, height=30)
|
||||||
|
with self.assertRaises(SoftwareContractError):
|
||||||
|
contract.normalize_request(request)
|
||||||
|
request = _primitive("box", length=math.nan, width=20, height=30)
|
||||||
|
with self.assertRaisesRegex(SoftwareContractError, "non-finite"):
|
||||||
|
contract.normalize_request(request)
|
||||||
|
request = _primitive("box", length=10, width=20, height=30)
|
||||||
|
del request["operation"]["prepare"]["canonical_unit"]
|
||||||
|
with self.assertRaises(SoftwareContractError):
|
||||||
|
contract.normalize_request(request)
|
||||||
|
for primitive in (
|
||||||
|
{"shape": "box", "length": 10, "width": 20},
|
||||||
|
{"shape": "box", "length": 10, "width": 20, "height": 30, "radius": 2},
|
||||||
|
):
|
||||||
|
with self.assertRaises(SoftwareContractError):
|
||||||
|
contract.normalize_request(_request("primitive", {"primitive": primitive}))
|
||||||
|
|
||||||
|
def test_ansys_and_cad_planar_selector_contracts_stay_consistent(self) -> None:
|
||||||
|
cad = get_contract("cad.geometry.prepare@v1").request_schema["$defs"]
|
||||||
|
ansys = get_contract("ansys.mechanical.static_structural@v2").request_schema["$defs"]
|
||||||
|
|
||||||
|
def resolved(value: object, definitions: dict) -> object:
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [resolved(item, definitions) for item in value]
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
return value
|
||||||
|
if set(value) == {"$ref"}:
|
||||||
|
return resolved(definitions[value["$ref"].split("/")[-1]], definitions)
|
||||||
|
return {
|
||||||
|
key: resolved(item, definitions)
|
||||||
|
for key, item in value.items()
|
||||||
|
if key != "description"
|
||||||
|
}
|
||||||
|
|
||||||
|
for name in ("extreme_face", "planar_faces"):
|
||||||
|
ansys_name = f"{name}_scope"
|
||||||
|
self.assertEqual(
|
||||||
|
resolved(cad[name], cad),
|
||||||
|
resolved(ansys[ansys_name], ansys),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_ansys_inspect_is_hidden_diagnostic_with_replacement(self) -> None:
|
||||||
|
contract = get_contract("ansys.geometry.inspect@v1")
|
||||||
|
self.assertFalse(contract.ordinarily_visible)
|
||||||
|
self.assertEqual(contract.lifecycle.status, "deprecated")
|
||||||
|
self.assertEqual(contract.lifecycle.visibility, "diagnostic_only")
|
||||||
|
self.assertEqual(contract.lifecycle.replacement, "cad.geometry.prepare@v1")
|
||||||
|
|
||||||
|
|
||||||
|
@unittest.skipUnless(HAS_CADQUERY, "fixed CAD runtime is not installed in project test environment")
|
||||||
|
class CadKernelTests(unittest.TestCase):
|
||||||
|
def test_all_required_primitives_have_numeric_oracles(self) -> None:
|
||||||
|
cases = [
|
||||||
|
({"shape": "box", "length": 2, "width": 3, "height": 5}, 30.0),
|
||||||
|
({"shape": "cylinder", "radius": 2, "height": 5}, 20 * math.pi),
|
||||||
|
({"shape": "tube", "radius": 3, "inner_radius": 2, "height": 5}, 25 * math.pi),
|
||||||
|
({"shape": "cone", "radius": 3, "height": 4}, 12 * math.pi),
|
||||||
|
({"shape": "frustum", "radius1": 3, "radius2": 1, "height": 4}, 52 * math.pi / 3),
|
||||||
|
({"shape": "sphere", "radius": 3}, 36 * math.pi),
|
||||||
|
({"shape": "wedge", "x_min": 0, "x_max": 4, "y_min": 0, "y_max": 3, "z_min": 0, "z_max": 2}, 12.0),
|
||||||
|
({"shape": "torus", "major_radius": 5, "minor_radius": 2}, 40 * math.pi**2),
|
||||||
|
]
|
||||||
|
for spec, expected in cases:
|
||||||
|
with self.subTest(spec["shape"]):
|
||||||
|
shape = worker._primitive(spec)
|
||||||
|
self.assertAlmostEqual(shape.Volume(), expected, delta=max(1e-8, expected * 1e-8))
|
||||||
|
|
||||||
|
def test_recipe_builds_plate_hole_notch_i_beam_and_multi_body(self) -> None:
|
||||||
|
plate = {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "plate", "op": "box", "primitive": {"shape": "box", "length": 100, "width": 60, "height": 10}},
|
||||||
|
{"id": "hole", "op": "cylinder", "primitive": {"shape": "cylinder", "radius": 10, "height": 10}},
|
||||||
|
{"id": "centered_hole", "op": "translate", "target": "hole", "vector": [50, 30, 0]},
|
||||||
|
{"id": "perforated", "op": "cut", "target": "plate", "tools": ["centered_hole"]},
|
||||||
|
{"id": "notch", "op": "box", "primitive": {"shape": "box", "length": 20, "width": 20, "height": 10}},
|
||||||
|
{"id": "placed_notch", "op": "translate", "target": "notch", "vector": [80, 40, 0]},
|
||||||
|
{"id": "result", "op": "cut", "target": "perforated", "tools": ["placed_notch"]},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
}
|
||||||
|
shape = worker._recipe(plate, None, "mm")
|
||||||
|
self.assertAlmostEqual(
|
||||||
|
shape.Volume(),
|
||||||
|
100 * 60 * 10 - math.pi * 10**2 * 10 - 20 * 20 * 10,
|
||||||
|
delta=1e-6,
|
||||||
|
)
|
||||||
|
|
||||||
|
i_beam = {
|
||||||
|
"profiles": [{"id": "i", "points": [[0, 0, 0], [60, 0, 0], [60, 10, 0], [35, 10, 0], [35, 90, 0], [60, 90, 0], [60, 100, 0], [0, 100, 0], [0, 90, 0], [25, 90, 0], [25, 10, 0], [0, 10, 0]]}],
|
||||||
|
"steps": [{"id": "beam", "op": "extrude", "profile": "i", "vector": [0, 0, 200]}],
|
||||||
|
"result": "beam",
|
||||||
|
}
|
||||||
|
beam = worker._recipe(i_beam, None, "mm")
|
||||||
|
self.assertAlmostEqual(beam.Volume(), 2000 * 200, delta=1e-6)
|
||||||
|
|
||||||
|
multi = {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "coupon", "op": "box", "primitive": {"shape": "box", "length": 10, "width": 10, "height": 20}},
|
||||||
|
{"id": "top", "op": "translate", "target": "coupon", "vector": [0, 0, 30]},
|
||||||
|
{"id": "assembly", "op": "combine", "targets": ["coupon", "top"]},
|
||||||
|
],
|
||||||
|
"result": "assembly",
|
||||||
|
}
|
||||||
|
assembly = worker._recipe(multi, None, "mm")
|
||||||
|
self.assertEqual(len(assembly.Solids()), 2)
|
||||||
|
|
||||||
|
def test_regions_manifest_and_step_roundtrip_are_deterministic(self) -> None:
|
||||||
|
shape = worker._primitive({"shape": "box", "length": 10, "width": 20, "height": 30})
|
||||||
|
regions = [{"name": "fixed", "topology": "face", "selector": {"type": "extreme_face", "axis": "z", "side": "min", "expected_count": 1}}]
|
||||||
|
resolved = worker._regions(shape, regions)
|
||||||
|
prepare = _primitive("box", length=10, width=20, height=30)["operation"]["prepare"]
|
||||||
|
first = worker._manifest(shape, prepare, resolved, None)
|
||||||
|
second = worker._manifest(shape, prepare, resolved, None)
|
||||||
|
self.assertEqual(first, second)
|
||||||
|
self.assertEqual(first["topology"]["solids"], 1)
|
||||||
|
self.assertEqual(first["regions"][0]["count"], 1)
|
||||||
|
self.assertEqual(first["bounding_box"]["size"], [10.0, 20.0, 30.0])
|
||||||
|
self.assertAlmostEqual(first["area"], 2200.0, delta=1e-8)
|
||||||
|
self.assertEqual(first["center_of_mass"], [5.0, 10.0, 15.0])
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
path = Path(directory) / "geometry.step"
|
||||||
|
worker._export_step(shape, path, "mm")
|
||||||
|
worker._require_step_unit(path)
|
||||||
|
import cadquery as cq
|
||||||
|
reopened = cq.importers.importStep(str(path), unit="MM").val()
|
||||||
|
self.assertAlmostEqual(reopened.Volume(), 6000.0, delta=1e-6)
|
||||||
|
|
||||||
|
def test_advanced_recipe_operations_fail_loudly_or_produce_valid_solids(self) -> None:
|
||||||
|
cases = {
|
||||||
|
"revolve": {
|
||||||
|
"profiles": [{"id": "section", "points": [[2, 0, 0], [3, 0, 0], [3, 0, 5], [2, 0, 5]]}],
|
||||||
|
"steps": [{"id": "result", "op": "revolve", "profile": "section", "axis_start": [0, 0, 0], "axis_end": [0, 0, 1], "angle_degrees": 360}],
|
||||||
|
"result": "result",
|
||||||
|
},
|
||||||
|
"sweep": {
|
||||||
|
"profiles": [{"id": "section", "points": [[0, -1, -1], [0, 1, -1], [0, 1, 1], [0, -1, 1]]}],
|
||||||
|
"steps": [{"id": "result", "op": "sweep", "profile": "section", "path": [[0, 0, 0], [10, 0, 0]]}],
|
||||||
|
"result": "result",
|
||||||
|
},
|
||||||
|
"loft": {
|
||||||
|
"profiles": [
|
||||||
|
{"id": "a", "points": [[-2, -2, 0], [2, -2, 0], [2, 2, 0], [-2, 2, 0]]},
|
||||||
|
{"id": "b", "points": [[-1, -1, 10], [1, -1, 10], [1, 1, 10], [-1, 1, 10]]},
|
||||||
|
],
|
||||||
|
"steps": [{"id": "result", "op": "loft", "profiles": ["a", "b"]}],
|
||||||
|
"result": "result",
|
||||||
|
},
|
||||||
|
"fillet": {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 10, "width": 10, "height": 10}},
|
||||||
|
{"id": "result", "op": "fillet", "target": "base", "selector": {"type": "curve_edges", "curve": "line", "expected_count": 12}, "radius": 1},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
},
|
||||||
|
"chamfer": {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 10, "width": 10, "height": 10}},
|
||||||
|
{"id": "result", "op": "chamfer", "target": "base", "selector": {"type": "curve_edges", "curve": "line", "expected_count": 12}, "distance": 0.5},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
},
|
||||||
|
"shell": {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 10, "width": 10, "height": 10}},
|
||||||
|
{"id": "result", "op": "shell", "target": "base", "selector": {"type": "extreme_face", "axis": "z", "side": "max", "expected_count": 1}, "thickness": 1},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
},
|
||||||
|
"mirror": {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 2, "width": 3, "height": 4}},
|
||||||
|
{"id": "result", "op": "mirror", "target": "base", "plane": "YZ"},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
},
|
||||||
|
"linear_pattern": {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "base", "op": "box", "primitive": {"shape": "box", "length": 2, "width": 2, "height": 2}},
|
||||||
|
{"id": "result", "op": "linear_pattern", "target": "base", "vector": [1, 0, 0], "count": 3, "spacing": 4},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for name, recipe in cases.items():
|
||||||
|
with self.subTest(name):
|
||||||
|
shape = worker._recipe(recipe, None, "mm")
|
||||||
|
self.assertTrue(shape.isValid())
|
||||||
|
self.assertGreater(shape.Volume(), 0)
|
||||||
|
|
||||||
|
invalid = cases["fillet"]
|
||||||
|
invalid = json.loads(json.dumps(invalid))
|
||||||
|
invalid["steps"][1]["selector"]["expected_count"] = 11
|
||||||
|
with self.assertRaisesRegex(RuntimeError, "GEOMETRY_SCOPE_COUNT_MISMATCH"):
|
||||||
|
worker._recipe(invalid, None, "mm")
|
||||||
|
|
||||||
|
def test_flange_circular_holes_and_circular_pattern(self) -> None:
|
||||||
|
recipe = {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "flange", "op": "cylinder", "primitive": {"shape": "cylinder", "radius": 20, "height": 5}},
|
||||||
|
{"id": "hole", "op": "cylinder", "primitive": {"shape": "cylinder", "radius": 2, "height": 5}},
|
||||||
|
{"id": "offset_hole", "op": "translate", "target": "hole", "vector": [12, 0, 0]},
|
||||||
|
{"id": "holes", "op": "circular_pattern", "target": "offset_hole", "axis_start": [0, 0, 0], "axis_end": [0, 0, 1], "count": 6, "angle_degrees": 360},
|
||||||
|
{"id": "result", "op": "cut", "target": "flange", "tools": ["holes"]},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
}
|
||||||
|
shape = worker._recipe(recipe, None, "mm")
|
||||||
|
expected = math.pi * 20**2 * 5 - 6 * math.pi * 2**2 * 5
|
||||||
|
self.assertAlmostEqual(shape.Volume(), expected, delta=expected * 1e-8)
|
||||||
|
|
||||||
|
def test_full_job_emits_fixed_package_and_preview(self) -> None:
|
||||||
|
request = _request(
|
||||||
|
"primitive",
|
||||||
|
{"primitive": {"shape": "cylinder", "radius": 4, "height": 12}},
|
||||||
|
regions=[{"name": "ends", "topology": "face", "selector": {"type": "surface_faces", "surface": "plane", "expected_count": 2}}],
|
||||||
|
)
|
||||||
|
get_contract("cad.geometry.prepare@v1").normalize_request(request)
|
||||||
|
|
||||||
|
def execute() -> dict:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
(root / "request").mkdir()
|
||||||
|
(root / "request" / "request.json").write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"job_id": str(uuid4()),
|
||||||
|
"lease_id": str(uuid4()),
|
||||||
|
"request_digest": "a" * 64,
|
||||||
|
"request": request,
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
artifacts = worker.run(root)
|
||||||
|
self.assertEqual(
|
||||||
|
{item["filename"] for item in artifacts}, set(worker.OUTPUT_MEDIA)
|
||||||
|
)
|
||||||
|
self.assertGreater(
|
||||||
|
(root / "output" / "geometry-preview.png").stat().st_size,
|
||||||
|
1000,
|
||||||
|
)
|
||||||
|
return json.loads(
|
||||||
|
(root / "output" / "geometry-manifest.json").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
first = execute()
|
||||||
|
second = execute()
|
||||||
|
self.assertEqual(first, second)
|
||||||
|
self.assertAlmostEqual(first["volume"], math.pi * 4**2 * 12, delta=1e-6)
|
||||||
|
|
||||||
|
def test_external_step_inspect_and_recipe_import(self) -> None:
|
||||||
|
import cadquery as cq
|
||||||
|
|
||||||
|
source = cq.Solid.makeBox(7, 11, 13)
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
input_dir = root / "input" / "geometry"
|
||||||
|
request_dir = root / "request"
|
||||||
|
input_dir.mkdir(parents=True)
|
||||||
|
request_dir.mkdir()
|
||||||
|
source_path = input_dir / "external.step"
|
||||||
|
worker._export_step(source, source_path, "mm")
|
||||||
|
|
||||||
|
for feature, definition in (
|
||||||
|
("inspect", {"geometry": {"input": "geometry"}}),
|
||||||
|
(
|
||||||
|
"recipe",
|
||||||
|
{
|
||||||
|
"recipe": {
|
||||||
|
"profiles": [],
|
||||||
|
"steps": [
|
||||||
|
{"id": "imported", "op": "import_step", "input": "geometry"},
|
||||||
|
{"id": "moved", "op": "translate", "target": "imported", "vector": [20, 0, 0]},
|
||||||
|
{"id": "result", "op": "combine", "targets": ["imported", "moved"]},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
}
|
||||||
|
},
|
||||||
|
),
|
||||||
|
):
|
||||||
|
request = _request(feature, definition)
|
||||||
|
request["inputs"] = [
|
||||||
|
{"key": "geometry", "artifact_id": str(uuid4())}
|
||||||
|
]
|
||||||
|
(request_dir / "request.json").write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"job_id": str(uuid4()),
|
||||||
|
"lease_id": str(uuid4()),
|
||||||
|
"request_digest": "b" * 64,
|
||||||
|
"request": request,
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
artifacts = worker.run(root)
|
||||||
|
self.assertEqual(len(artifacts), 5)
|
||||||
|
manifest = json.loads(
|
||||||
|
(root / "output" / "geometry-manifest.json").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
expected_volume = 7 * 11 * 13 * (2 if feature == "recipe" else 1)
|
||||||
|
self.assertAlmostEqual(
|
||||||
|
manifest["volume"], expected_volume, delta=1e-6
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -58,7 +58,7 @@ class ContextCompactionTests(unittest.TestCase):
|
||||||
|
|
||||||
prepared, stats = prepare_messages_with_stats(
|
prepared, stats = prepare_messages_with_stats(
|
||||||
messages,
|
messages,
|
||||||
reasoning_replay="tool_calls",
|
reasoning_replay="tool_turn",
|
||||||
compact_threshold_chars=10_000,
|
compact_threshold_chars=10_000,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -78,7 +78,7 @@ class ContextCompactionTests(unittest.TestCase):
|
||||||
}, {"role": "tool", "tool_call_id": "tc1", "content": "ok"}]
|
}, {"role": "tool", "tool_call_id": "tc1", "content": "ok"}]
|
||||||
|
|
||||||
prepared = prepare_messages_for_llm(
|
prepared = prepare_messages_for_llm(
|
||||||
messages, reasoning_replay="tool_calls",
|
messages, reasoning_replay="tool_turn",
|
||||||
)
|
)
|
||||||
|
|
||||||
self.assertEqual(prepared[0]["reasoning_content"], "nested reasoning")
|
self.assertEqual(prepared[0]["reasoning_content"], "nested reasoning")
|
||||||
|
|
@ -91,10 +91,54 @@ class ContextCompactionTests(unittest.TestCase):
|
||||||
"provider_specific_fields": {"signature": "opaque"},
|
"provider_specific_fields": {"signature": "opaque"},
|
||||||
}
|
}
|
||||||
|
|
||||||
prepared = prepare_messages_for_llm([message], reasoning_replay="all")
|
prepared = prepare_messages_for_llm(
|
||||||
|
[message], reasoning_replay="conversation"
|
||||||
|
)
|
||||||
|
|
||||||
self.assertEqual(prepared[0], message)
|
self.assertEqual(prepared[0], message)
|
||||||
|
|
||||||
|
def test_conversation_policy_strips_reasoning_from_other_model(self) -> None:
|
||||||
|
messages = [{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "answer",
|
||||||
|
"reasoning_content": "deepseek reasoning",
|
||||||
|
"_model_profile": "deepseek_v4.flash",
|
||||||
|
}]
|
||||||
|
prepared = prepare_messages_for_llm(
|
||||||
|
messages,
|
||||||
|
reasoning_replay="conversation",
|
||||||
|
target_model_profile="glm.flash53",
|
||||||
|
)
|
||||||
|
self.assertNotIn("reasoning_content", prepared[0])
|
||||||
|
self.assertNotIn("_model_profile", prepared[0])
|
||||||
|
|
||||||
|
def test_tool_turn_keeps_only_reasoning_after_latest_user(self) -> None:
|
||||||
|
call = lambda cid, reasoning: {
|
||||||
|
"role": "assistant", "content": None,
|
||||||
|
"reasoning_content": reasoning,
|
||||||
|
"_model_profile": "deepseek_v4.flash",
|
||||||
|
"tool_calls": [{
|
||||||
|
"id": cid, "type": "function",
|
||||||
|
"function": {"name": "shell", "arguments": "{}"},
|
||||||
|
}],
|
||||||
|
}
|
||||||
|
messages = [
|
||||||
|
{"role": "user", "content": "old"},
|
||||||
|
call("old", "old reasoning"),
|
||||||
|
{"role": "tool", "tool_call_id": "old", "content": "old result"},
|
||||||
|
{"role": "assistant", "content": "done"},
|
||||||
|
{"role": "user", "content": "new"},
|
||||||
|
call("new", "new reasoning"),
|
||||||
|
{"role": "tool", "tool_call_id": "new", "content": "new result"},
|
||||||
|
]
|
||||||
|
prepared = prepare_messages_for_llm(
|
||||||
|
messages,
|
||||||
|
reasoning_replay="tool_turn",
|
||||||
|
target_model_profile="deepseek_v4.flash",
|
||||||
|
)
|
||||||
|
self.assertNotIn("reasoning_content", prepared[1])
|
||||||
|
self.assertEqual(prepared[5]["reasoning_content"], "new reasoning")
|
||||||
|
|
||||||
def test_preserves_system_and_recent_messages(self) -> None:
|
def test_preserves_system_and_recent_messages(self) -> None:
|
||||||
messages = [
|
messages = [
|
||||||
{"role": "system", "content": "rules"},
|
{"role": "system", "content": "rules"},
|
||||||
|
|
|
||||||
|
|
@ -146,7 +146,7 @@ class _FakeLLM:
|
||||||
_FAKE_CAPS = SimpleNamespace(
|
_FAKE_CAPS = SimpleNamespace(
|
||||||
reliable_context=100, # 极小上限 → 体量必然超阈值,不用造几十万字符
|
reliable_context=100, # 极小上限 → 体量必然超阈值,不用造几十万字符
|
||||||
family="deepseek_v4", variant="flash",
|
family="deepseek_v4", variant="flash",
|
||||||
reasoning_replay="tool_calls",
|
reasoning_replay="tool_turn",
|
||||||
input_cny_per_mtoken=1.0, output_cny_per_mtoken=2.0,
|
input_cny_per_mtoken=1.0, output_cny_per_mtoken=2.0,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -218,16 +218,24 @@ class MaybeFoldTests(unittest.TestCase):
|
||||||
sess = self._session()
|
sess = self._session()
|
||||||
before = list(sess.messages)
|
before = list(sess.messages)
|
||||||
events = []
|
events = []
|
||||||
with patch.object(cf, "persist_fold") as persist:
|
user_id = uuid4()
|
||||||
|
with patch.object(cf, "persist_fold") as persist, \
|
||||||
|
patch.object(cf, "record_context_fold_failure") as failure:
|
||||||
result = cf.maybe_fold(
|
result = cf.maybe_fold(
|
||||||
sess, _FakeLLM(" "), _FAKE_CAPS, user_id=uuid4(), emit=events.append
|
sess, _FakeLLM(" "), _FAKE_CAPS, user_id=user_id, emit=events.append
|
||||||
)
|
)
|
||||||
self.assertIsNone(result)
|
self.assertIsNone(result)
|
||||||
persist.assert_not_called()
|
persist.assert_not_called()
|
||||||
|
failure.assert_called_once_with(
|
||||||
|
task_id=sess.task_id,
|
||||||
|
user_id=user_id,
|
||||||
|
model_profile="deepseek_v4.flash",
|
||||||
|
reason="empty_summary",
|
||||||
|
)
|
||||||
self.assertEqual(sess.messages, before) # 内存零污染
|
self.assertEqual(sess.messages, before) # 内存零污染
|
||||||
self.assertTrue(any(e.get("type") == "warn" for e in events))
|
self.assertTrue(any(e.get("type") == "warn" for e in events))
|
||||||
|
|
||||||
def test_fold_input_strips_final_reasoning_but_keeps_tool_reasoning(self) -> None:
|
def test_fold_input_strips_reasoning_from_completed_turns(self) -> None:
|
||||||
sess = self._session()
|
sess = self._session()
|
||||||
sess.messages[2]["reasoning_content"] = "final reasoning"
|
sess.messages[2]["reasoning_content"] = "final reasoning"
|
||||||
sess.messages[4] = {
|
sess.messages[4] = {
|
||||||
|
|
@ -254,7 +262,7 @@ class MaybeFoldTests(unittest.TestCase):
|
||||||
final_msg = next(m for m in sent if m.get("content", "").startswith("a000"))
|
final_msg = next(m for m in sent if m.get("content", "").startswith("a000"))
|
||||||
tool_msg = next(m for m in sent if m.get("tool_calls"))
|
tool_msg = next(m for m in sent if m.get("tool_calls"))
|
||||||
self.assertNotIn("reasoning_content", final_msg)
|
self.assertNotIn("reasoning_content", final_msg)
|
||||||
self.assertEqual(tool_msg["reasoning_content"], "tool reasoning")
|
self.assertNotIn("reasoning_content", tool_msg)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ from pathlib import Path
|
||||||
|
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
from core.storage.disk_quota import parse_bytes, scan_user_dir
|
from core.storage.disk_quota import parse_bytes, scan_user_dir, scan_user_trash
|
||||||
|
|
||||||
|
|
||||||
class TestParseBytes(unittest.TestCase):
|
class TestParseBytes(unittest.TestCase):
|
||||||
|
|
@ -75,6 +75,7 @@ class TestScanUserDir(unittest.TestCase):
|
||||||
for hidden in (
|
for hidden in (
|
||||||
".zcbot_cache",
|
".zcbot_cache",
|
||||||
".zcbot_external_results",
|
".zcbot_external_results",
|
||||||
|
".zcbot_artifact_trash",
|
||||||
".zcbot_tmp",
|
".zcbot_tmp",
|
||||||
".memory",
|
".memory",
|
||||||
):
|
):
|
||||||
|
|
@ -84,6 +85,17 @@ class TestScanUserDir(unittest.TestCase):
|
||||||
self.assertEqual(b, 7)
|
self.assertEqual(b, 7)
|
||||||
self.assertEqual(c, 1)
|
self.assertEqual(c, 1)
|
||||||
|
|
||||||
|
def test_trash_is_excluded_from_quota_and_counted_separately(self):
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
root = Path(d)
|
||||||
|
(root / "active.bin").write_bytes(b"active")
|
||||||
|
trash = root / ".zcbot_artifact_trash" / "2026" / "08"
|
||||||
|
trash.mkdir(parents=True)
|
||||||
|
(trash / "deleted.bin").write_bytes(b"deleted")
|
||||||
|
|
||||||
|
self.assertEqual(scan_user_dir(root), (6, 1))
|
||||||
|
self.assertEqual(scan_user_trash(root), (7, 1))
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
|
||||||
|
|
@ -18,6 +18,7 @@ import tempfile
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
import unittest
|
import unittest
|
||||||
|
from contextlib import contextmanager
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from unittest.mock import MagicMock, patch
|
from unittest.mock import MagicMock, patch
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
@ -35,6 +36,22 @@ class FakePool:
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.ensure_calls = []
|
self.ensure_calls = []
|
||||||
self.mark_active_calls = []
|
self.mark_active_calls = []
|
||||||
|
self.active = 0
|
||||||
|
self.capacity = self
|
||||||
|
self.capacity_notices = []
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def foreground(self, user_id, cancel_check=None, wait_notify=None):
|
||||||
|
if wait_notify is not None:
|
||||||
|
for notice in self.capacity_notices:
|
||||||
|
wait_notify(notice)
|
||||||
|
yield True
|
||||||
|
|
||||||
|
def exec_started(self, user_id):
|
||||||
|
self.active += 1
|
||||||
|
|
||||||
|
def exec_finished(self, user_id):
|
||||||
|
self.active -= 1
|
||||||
|
|
||||||
def ensure(self, user_id):
|
def ensure(self, user_id):
|
||||||
name = f"zcbot-sandbox-{user_id}"
|
name = f"zcbot-sandbox-{user_id}"
|
||||||
|
|
@ -161,6 +178,27 @@ class TestShellExec(unittest.TestCase):
|
||||||
self.assertEqual(pool.ensure_calls, [executor.user_id])
|
self.assertEqual(pool.ensure_calls, [executor.user_id])
|
||||||
self.assertEqual(pool.mark_active_calls, [executor.user_id])
|
self.assertEqual(pool.mark_active_calls, [executor.user_id])
|
||||||
|
|
||||||
|
def test_shell_forwards_capacity_queue_events(self):
|
||||||
|
executor, pool, _ = make_executor()
|
||||||
|
pool.capacity_notices = [
|
||||||
|
{"state": "waiting", "reason": "per_user_limit"},
|
||||||
|
{"state": "admitted"},
|
||||||
|
]
|
||||||
|
events = []
|
||||||
|
ctx = make_ctx(executor)
|
||||||
|
ctx.event_emit = events.append
|
||||||
|
proc = MagicMock()
|
||||||
|
proc.communicate.return_value = ("ok\n", "")
|
||||||
|
proc.returncode = 0
|
||||||
|
|
||||||
|
with patch("core.executor_docker.subprocess.Popen", return_value=proc):
|
||||||
|
executor.call_tool("shell", {"command": "echo ok"}, ctx)
|
||||||
|
|
||||||
|
self.assertEqual(events, [
|
||||||
|
{"type": "execution_queue", "state": "waiting", "reason": "per_user_limit"},
|
||||||
|
{"type": "execution_queue", "state": "admitted"},
|
||||||
|
])
|
||||||
|
|
||||||
def test_shell_bad_args(self):
|
def test_shell_bad_args(self):
|
||||||
executor, _, _ = make_executor()
|
executor, _, _ = make_executor()
|
||||||
ctx = make_ctx(executor)
|
ctx = make_ctx(executor)
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,234 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from core.software_contracts import SoftwareContractError, get_contract
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
ADAPTER = ROOT / "windows-node" / "adapters" / "freecad.model.author@v1"
|
||||||
|
sys.path.insert(0, str(ADAPTER))
|
||||||
|
SPEC = importlib.util.spec_from_file_location("freecad_launcher", ADAPTER / "worker.py")
|
||||||
|
assert SPEC and SPEC.loader
|
||||||
|
launcher = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(launcher)
|
||||||
|
from recipe import RecipeError, normalize_model
|
||||||
|
|
||||||
|
|
||||||
|
def model() -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"type": "recipe",
|
||||||
|
"recipe_version": 1,
|
||||||
|
"title": "受控模型",
|
||||||
|
"unit": "mm",
|
||||||
|
"parameters": {"length": 20},
|
||||||
|
"steps": [
|
||||||
|
{
|
||||||
|
"id": "base",
|
||||||
|
"op": "box",
|
||||||
|
"length": {"parameter": "length"},
|
||||||
|
"width": 10,
|
||||||
|
"height": 5,
|
||||||
|
},
|
||||||
|
{"id": "hole", "op": "cylinder", "radius": 2, "height": 5},
|
||||||
|
{"id": "result", "op": "cut", "target": "base", "tools": ["hole"]},
|
||||||
|
],
|
||||||
|
"result": "result",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class FreeCadContractTests(unittest.TestCase):
|
||||||
|
def test_contract_accepts_parameterized_minimal_recipe(self) -> None:
|
||||||
|
request = {
|
||||||
|
"schema_version": 1,
|
||||||
|
"inputs": [],
|
||||||
|
"operation": {"model": model()},
|
||||||
|
"outputs": [],
|
||||||
|
}
|
||||||
|
normalized, digest = get_contract("freecad.model.author@v1").normalize_request(
|
||||||
|
request
|
||||||
|
)
|
||||||
|
self.assertEqual(normalized["operation"]["model"]["result"], "result")
|
||||||
|
self.assertEqual(len(digest), 64)
|
||||||
|
contract = get_contract("freecad.model.author@v1")
|
||||||
|
self.assertEqual(contract.workspace.state_output, "project")
|
||||||
|
self.assertEqual(contract.workspace.preview_outputs, ("preview",))
|
||||||
|
self.assertEqual(
|
||||||
|
set(contract.expected_outputs(normalized)),
|
||||||
|
{
|
||||||
|
"project",
|
||||||
|
"geometry_step",
|
||||||
|
"preview",
|
||||||
|
"model_recipe",
|
||||||
|
"model_manifest",
|
||||||
|
"provenance",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_contract_rejects_raw_expression_strings_and_fcstd_inputs(self) -> None:
|
||||||
|
request = {
|
||||||
|
"schema_version": 1,
|
||||||
|
"inputs": [],
|
||||||
|
"operation": {"model": model()},
|
||||||
|
"outputs": [],
|
||||||
|
}
|
||||||
|
request["operation"]["model"]["steps"][0]["length"] = "length * 2"
|
||||||
|
with self.assertRaises(SoftwareContractError):
|
||||||
|
get_contract("freecad.model.author@v1").normalize_request(request)
|
||||||
|
self.assertEqual(
|
||||||
|
get_contract("freecad.model.author@v1").input_policy["suffixes"],
|
||||||
|
[".step", ".stp"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FreeCadRecipeTests(unittest.TestCase):
|
||||||
|
def test_parameter_references_are_resolved_to_numbers(self) -> None:
|
||||||
|
resolved = normalize_model(model(), input_bound=False)
|
||||||
|
self.assertEqual(resolved["steps"][0]["length"], 20.0)
|
||||||
|
self.assertNotIn("Spreadsheet", json.dumps(resolved))
|
||||||
|
|
||||||
|
def test_forward_reference_and_unbound_step_are_rejected(self) -> None:
|
||||||
|
value = model()
|
||||||
|
value["steps"][0] = {
|
||||||
|
"id": "moved",
|
||||||
|
"op": "translate",
|
||||||
|
"target": "later",
|
||||||
|
"vector": [1, 0, 0],
|
||||||
|
}
|
||||||
|
with self.assertRaisesRegex(RecipeError, "moved"):
|
||||||
|
normalize_model(value, input_bound=False)
|
||||||
|
imported = {
|
||||||
|
"type": "recipe",
|
||||||
|
"recipe_version": 1,
|
||||||
|
"title": "import",
|
||||||
|
"unit": "mm",
|
||||||
|
"steps": [{"id": "part", "op": "import_step", "input": "geometry"}],
|
||||||
|
"result": "part",
|
||||||
|
}
|
||||||
|
with self.assertRaises(RecipeError) as caught:
|
||||||
|
normalize_model(imported, input_bound=False)
|
||||||
|
self.assertEqual(caught.exception.code, "FREECAD_STEP_INPUT_NOT_BOUND")
|
||||||
|
|
||||||
|
|
||||||
|
class FreeCadLauncherTests(unittest.TestCase):
|
||||||
|
def test_version_gate_accepts_only_stable_113_to_before_120(self) -> None:
|
||||||
|
executable = Path(r"C:\Program Files\FreeCAD 1.1\bin\FreeCADCmd.exe")
|
||||||
|
for output in ("FreeCAD 1.1.3", "FreeCAD 1.1.9 Revision: 12345"):
|
||||||
|
with patch.object(
|
||||||
|
subprocess,
|
||||||
|
"run",
|
||||||
|
return_value=subprocess.CompletedProcess([], 0, output, ""),
|
||||||
|
):
|
||||||
|
self.assertTrue(launcher._version(executable).startswith("1.1."))
|
||||||
|
for output in (
|
||||||
|
"FreeCAD 1.1.2",
|
||||||
|
"FreeCAD 1.2.0",
|
||||||
|
"FreeCAD 1.1.3 RC1",
|
||||||
|
"FreeCAD 1.1.4 weekly",
|
||||||
|
"FreeCAD 1.1.4, Libs: 1.1.4devR123",
|
||||||
|
"FreeCAD 1.1.4, Libs: 1.1.4dev.123",
|
||||||
|
):
|
||||||
|
with (
|
||||||
|
patch.object(
|
||||||
|
subprocess,
|
||||||
|
"run",
|
||||||
|
return_value=subprocess.CompletedProcess([], 0, output, ""),
|
||||||
|
),
|
||||||
|
self.assertRaises(launcher.AdapterError),
|
||||||
|
):
|
||||||
|
launcher._version(executable)
|
||||||
|
|
||||||
|
def test_managed_launcher_uses_fake_freecadcmd_with_fixed_inputs(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
job_dir = Path(directory) / "含空格 job"
|
||||||
|
request = {
|
||||||
|
"schema_version": 1,
|
||||||
|
"inputs": [],
|
||||||
|
"operation": {"model": model()},
|
||||||
|
"outputs": [],
|
||||||
|
}
|
||||||
|
record = {
|
||||||
|
"job_id": "job",
|
||||||
|
"lease_id": "lease",
|
||||||
|
"request_digest": "digest",
|
||||||
|
"request": request,
|
||||||
|
}
|
||||||
|
request_path = job_dir / "request" / "request.json"
|
||||||
|
request_path.parent.mkdir(parents=True)
|
||||||
|
request_path.write_text(json.dumps(record), encoding="utf-8")
|
||||||
|
fake = Path(r"C:\Program Files\FreeCAD 1.1\bin\FreeCADCmd.exe")
|
||||||
|
captured: dict[str, object] = {}
|
||||||
|
|
||||||
|
def execute(command, **kwargs):
|
||||||
|
captured["command"] = command
|
||||||
|
captured["environment"] = kwargs["env"]
|
||||||
|
output = job_dir / "output"
|
||||||
|
metadata = output / ".meta"
|
||||||
|
for name in (
|
||||||
|
"model.FCStd",
|
||||||
|
"model.step",
|
||||||
|
"model-recipe.json",
|
||||||
|
"model-manifest.json",
|
||||||
|
):
|
||||||
|
(output / name).write_bytes(b"fake-output")
|
||||||
|
(metadata / "provenance.json").write_bytes(b"{}")
|
||||||
|
(metadata / "preview-mesh.json").write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"vertices": [[0, 0, 0], [1, 0, 0], [0, 1, 0]],
|
||||||
|
"triangles": [[0, 1, 2]],
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
(metadata / "engine-result.json").write_text(
|
||||||
|
'{"status":"succeeded"}', encoding="utf-8"
|
||||||
|
)
|
||||||
|
return subprocess.CompletedProcess(command, 0)
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(launcher, "_find_freecad", return_value=(fake, "1.1.3")),
|
||||||
|
patch.object(launcher.subprocess, "run", side_effect=execute),
|
||||||
|
):
|
||||||
|
artifacts = launcher.run(job_dir)
|
||||||
|
command = captured["command"]
|
||||||
|
environment = captured["environment"]
|
||||||
|
self.assertEqual(command[0], str(fake))
|
||||||
|
self.assertIn("--safe-mode", command)
|
||||||
|
self.assertEqual(Path(command[-1]).name, "freecad_worker.py")
|
||||||
|
self.assertEqual(
|
||||||
|
environment["ZCBOT_FREECAD_JOB_DIR"], str(job_dir.resolve())
|
||||||
|
)
|
||||||
|
self.assertNotIn("PYTHONPATH", {key.upper() for key in environment})
|
||||||
|
self.assertEqual(len(artifacts), 6)
|
||||||
|
|
||||||
|
def test_launcher_uses_fixed_script_safe_mode_and_isolated_configuration(
|
||||||
|
self,
|
||||||
|
) -> None:
|
||||||
|
source = (ADAPTER / "worker.py").read_text(encoding="utf-8")
|
||||||
|
for marker in (
|
||||||
|
"--safe-mode",
|
||||||
|
"--user-cfg",
|
||||||
|
"--system-cfg",
|
||||||
|
"FREECAD_USER_HOME",
|
||||||
|
"FREECAD_USER_DATA",
|
||||||
|
"FREECAD_USER_TEMP",
|
||||||
|
"ZCBOT_FREECAD_JOB_DIR",
|
||||||
|
):
|
||||||
|
self.assertIn(marker, source)
|
||||||
|
for forbidden in ("shell=True", "eval(", "exec(", "requests."):
|
||||||
|
self.assertNotIn(forbidden, source)
|
||||||
|
engine = (ADAPTER / "freecad_worker.py").read_text(encoding="utf-8")
|
||||||
|
self.assertIn("do not rely on __main__", engine)
|
||||||
|
self.assertNotIn("FreeCADGui", engine)
|
||||||
|
self.assertNotIn("Spreadsheet", engine)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -0,0 +1,120 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import unittest
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import uuid4
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
import core.storage.telemetry as telemetry # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
class _CaptureSession:
|
||||||
|
def __init__(self):
|
||||||
|
self.rows = []
|
||||||
|
|
||||||
|
def add(self, row):
|
||||||
|
self.rows.append(row)
|
||||||
|
|
||||||
|
|
||||||
|
class TestHealthTelemetry(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.session = _CaptureSession()
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def fake_scope():
|
||||||
|
yield self.session
|
||||||
|
|
||||||
|
self.scope_patch = patch.object(telemetry, "session_scope", fake_scope)
|
||||||
|
self.scope_patch.start()
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self.scope_patch.stop()
|
||||||
|
|
||||||
|
def _ids(self):
|
||||||
|
return {"task_id": uuid4(), "user_id": uuid4(), "message_id": uuid4()}
|
||||||
|
|
||||||
|
def test_normal_tool_result_does_not_write(self):
|
||||||
|
ids = self._ids()
|
||||||
|
written = telemetry.record_tool_failure(
|
||||||
|
**ids, model_profile="test.fast", tool="read", content="ok",
|
||||||
|
)
|
||||||
|
self.assertFalse(written)
|
||||||
|
self.assertEqual(self.session.rows, [])
|
||||||
|
|
||||||
|
def test_tool_failure_is_structured(self):
|
||||||
|
ids = self._ids()
|
||||||
|
written = telemetry.record_tool_failure(
|
||||||
|
**ids,
|
||||||
|
model_profile="test.fast",
|
||||||
|
tool="shell",
|
||||||
|
content="[exit 1]",
|
||||||
|
arguments={"command": "rg needle docs"},
|
||||||
|
)
|
||||||
|
self.assertTrue(written)
|
||||||
|
row = self.session.rows[0]
|
||||||
|
self.assertEqual(row.kind, telemetry.KIND_TOOL_FAILURE)
|
||||||
|
self.assertEqual(row.units["signature"], "exit N (search/no match)")
|
||||||
|
self.assertEqual(row.units["category"], "failure")
|
||||||
|
|
||||||
|
def test_quality_gate_uses_own_kind(self):
|
||||||
|
ids = self._ids()
|
||||||
|
telemetry.record_tool_failure(
|
||||||
|
**ids,
|
||||||
|
model_profile="test.fast",
|
||||||
|
tool="shell",
|
||||||
|
content="[GATE FAIL] svg_quality_checker: overlap\n[exit 1]",
|
||||||
|
)
|
||||||
|
self.assertEqual(self.session.rows[0].kind, telemetry.KIND_QUALITY_GATE)
|
||||||
|
self.assertEqual(self.session.rows[0].units["category"], "quality_gate")
|
||||||
|
|
||||||
|
def test_quality_gate_is_detected_before_trailing_command_hint(self):
|
||||||
|
ids = self._ids()
|
||||||
|
telemetry.record_tool_failure(
|
||||||
|
**ids,
|
||||||
|
model_profile="deepseek_v4.flash",
|
||||||
|
tool="shell",
|
||||||
|
content=(
|
||||||
|
"[stdout]\n[ERROR] slide.svg - Failed\n"
|
||||||
|
"[GATE FAIL] svg_quality_checker: 1 error\n"
|
||||||
|
"python scripts/svg_to_pptx.py \".\"\n[exit 1]"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(self.session.rows[0].kind, telemetry.KIND_QUALITY_GATE)
|
||||||
|
self.assertEqual(self.session.rows[0].units["category"], "quality_gate")
|
||||||
|
|
||||||
|
def test_control_events_are_structured(self):
|
||||||
|
ids = self._ids()
|
||||||
|
common = {
|
||||||
|
"task_id": ids["task_id"], "user_id": ids["user_id"],
|
||||||
|
"model_profile": "test.fast",
|
||||||
|
}
|
||||||
|
telemetry.record_run_stopped(**common, reason="max_iterations")
|
||||||
|
telemetry.record_agent_guard(
|
||||||
|
**common, tool="edit", guard="same_error", count=4,
|
||||||
|
signature="old_str not found",
|
||||||
|
)
|
||||||
|
telemetry.record_context_fold_failure(
|
||||||
|
**common, reason="empty_summary",
|
||||||
|
)
|
||||||
|
telemetry.record_quality_gate(
|
||||||
|
**common, tool="shell", gate="pptx_full_page_image",
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
[row.kind for row in self.session.rows],
|
||||||
|
[
|
||||||
|
telemetry.KIND_RUN_STOPPED,
|
||||||
|
telemetry.KIND_AGENT_GUARD,
|
||||||
|
telemetry.KIND_CONTEXT_FOLD_FAILURE,
|
||||||
|
telemetry.KIND_QUALITY_GATE,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
self.assertEqual(self.session.rows[0].units["category"], "agent_control")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -75,6 +75,22 @@ class LLMKwargsTests(unittest.TestCase):
|
||||||
kwargs["extra_body"], {"thinking": {"type": "enabled"}}
|
kwargs["extra_body"], {"thinking": {"type": "enabled"}}
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_glm_preserved_thinking_is_nested_in_thinking_body(self) -> None:
|
||||||
|
caps = ModelCapabilities(
|
||||||
|
family="glm", variant="flash53", model_id="zai/glm-5.3-flash",
|
||||||
|
api_key_env="TEST_LLM_API_KEY", thinking_enabled=True,
|
||||||
|
thinking_transport="extra_body", thinking_clear=False,
|
||||||
|
default_reasoning_effort="high",
|
||||||
|
)
|
||||||
|
with patch.dict(os.environ, {"TEST_LLM_API_KEY": "test-key"}):
|
||||||
|
kwargs = LLM(caps)._build_kwargs(
|
||||||
|
[{"role": "user", "content": "hello"}], None, None, "high"
|
||||||
|
)
|
||||||
|
self.assertEqual(kwargs["extra_body"], {
|
||||||
|
"thinking": {"type": "enabled", "clear_thinking": False},
|
||||||
|
"reasoning_effort": "high",
|
||||||
|
})
|
||||||
|
|
||||||
def test_invalid_transport_is_rejected_before_request(self) -> None:
|
def test_invalid_transport_is_rejected_before_request(self) -> None:
|
||||||
llm = self._llm(
|
llm = self._llm(
|
||||||
family="test", thinking_enabled=True, thinking_transport="unknown"
|
family="test", thinking_enabled=True, thinking_transport="unknown"
|
||||||
|
|
@ -94,10 +110,13 @@ class LLMKwargsTests(unittest.TestCase):
|
||||||
self.assertEqual(caps.reasoning_effort_levels, ["low", "high", "max"])
|
self.assertEqual(caps.reasoning_effort_levels, ["low", "high", "max"])
|
||||||
self.assertEqual(caps.default_reasoning_effort, "high")
|
self.assertEqual(caps.default_reasoning_effort, "high")
|
||||||
self.assertEqual(caps.max_output, 8192)
|
self.assertEqual(caps.max_output, 8192)
|
||||||
self.assertEqual(caps.output_cny_per_mtoken, 2.0)
|
self.assertEqual(caps.output_cny_per_mtoken, 4.752)
|
||||||
self.assertEqual(caps.cache_hit_cny_per_mtoken, 0.02)
|
self.assertEqual(caps.cache_hit_cny_per_mtoken, 0.0504)
|
||||||
|
self.assertEqual(
|
||||||
|
caps.pricing["periods"][0]["revision"], "deepseek-v4-20260816"
|
||||||
|
)
|
||||||
self.assertEqual(caps.thinking_transport, "extra_body")
|
self.assertEqual(caps.thinking_transport, "extra_body")
|
||||||
self.assertEqual(caps.reasoning_replay, "tool_calls")
|
self.assertEqual(caps.reasoning_replay, "tool_turn")
|
||||||
|
|
||||||
def test_other_controllable_profiles_declare_transport(self) -> None:
|
def test_other_controllable_profiles_declare_transport(self) -> None:
|
||||||
models_dir = Path(__file__).resolve().parents[1] / "config" / "models"
|
models_dir = Path(__file__).resolve().parents[1] / "config" / "models"
|
||||||
|
|
@ -105,8 +124,12 @@ class LLMKwargsTests(unittest.TestCase):
|
||||||
glm = ModelCapabilities.load("glm.pro52", models_dir)
|
glm = ModelCapabilities.load("glm.pro52", models_dir)
|
||||||
doubao = ModelCapabilities.load("doubao.turbo", models_dir)
|
doubao = ModelCapabilities.load("doubao.turbo", models_dir)
|
||||||
|
|
||||||
self.assertFalse(glm.thinking_enabled)
|
self.assertEqual(glm.profile, "glm.flash53")
|
||||||
|
self.assertTrue(glm.thinking_enabled)
|
||||||
self.assertEqual(glm.thinking_transport, "extra_body")
|
self.assertEqual(glm.thinking_transport, "extra_body")
|
||||||
|
self.assertFalse(glm.thinking_clear)
|
||||||
|
self.assertEqual(glm.reasoning_replay, "conversation")
|
||||||
|
self.assertTrue(glm.native_image_input)
|
||||||
self.assertTrue(doubao.thinking_enabled)
|
self.assertTrue(doubao.thinking_enabled)
|
||||||
self.assertEqual(doubao.thinking_transport, "extra_body")
|
self.assertEqual(doubao.thinking_transport, "extra_body")
|
||||||
self.assertEqual(doubao.default_reasoning_effort, "")
|
self.assertEqual(doubao.default_reasoning_effort, "")
|
||||||
|
|
|
||||||
|
|
@ -1,9 +1,12 @@
|
||||||
"""look_at_image 的问题收敛与兼容兜底测试(不碰网络和数据库)。"""
|
"""look_at_image 的问题收敛与兼容兜底测试(不碰网络和数据库)。"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import tempfile
|
||||||
import unittest
|
import unittest
|
||||||
import uuid
|
import uuid
|
||||||
|
from contextlib import ExitStack
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
from unittest import mock
|
from unittest import mock
|
||||||
|
|
||||||
from core.ark_client import ArkConfig
|
from core.ark_client import ArkConfig
|
||||||
|
|
@ -27,6 +30,69 @@ class LookAtImageQuestionTests(unittest.TestCase):
|
||||||
user_root=Path("."),
|
user_root=Path("."),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_native_vision_model_omits_look_tool_but_keeps_document_ocr(self):
|
||||||
|
from core.tool_registry import ToolContext, build_tools
|
||||||
|
|
||||||
|
ark = ArkConfig(
|
||||||
|
api_key="test",
|
||||||
|
base_url="https://example.invalid",
|
||||||
|
raw={"vision": {"test": {"model_id": "vision-test"}}},
|
||||||
|
)
|
||||||
|
|
||||||
|
def build(native_image_input: bool) -> dict:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp, ExitStack() as stack:
|
||||||
|
root = Path(tmp)
|
||||||
|
stack.enter_context(mock.patch(
|
||||||
|
"core.tool_registry._external_system_status_available",
|
||||||
|
return_value=False,
|
||||||
|
))
|
||||||
|
stack.enter_context(mock.patch(
|
||||||
|
"core.tool_registry._external_systems_available",
|
||||||
|
return_value=False,
|
||||||
|
))
|
||||||
|
stack.enter_context(mock.patch(
|
||||||
|
"core.tool_registry.smtp_configured", return_value=False,
|
||||||
|
))
|
||||||
|
stack.enter_context(mock.patch(
|
||||||
|
"core.tool_registry.wechat_push_available", return_value=False,
|
||||||
|
))
|
||||||
|
stack.enter_context(mock.patch(
|
||||||
|
"core.tool_registry.lfasr_configured", return_value=False,
|
||||||
|
))
|
||||||
|
stack.enter_context(mock.patch(
|
||||||
|
"core.tool_registry.BochaConfig.load", return_value=None,
|
||||||
|
))
|
||||||
|
return build_tools(ToolContext(
|
||||||
|
tool_base=root,
|
||||||
|
ur_path=root,
|
||||||
|
working_dir_path=root,
|
||||||
|
task_id=uuid.uuid4(),
|
||||||
|
uid=uuid.uuid4(),
|
||||||
|
cfg={},
|
||||||
|
caps=SimpleNamespace(
|
||||||
|
enable_run_python=False,
|
||||||
|
native_image_input=native_image_input,
|
||||||
|
),
|
||||||
|
skills=SimpleNamespace(skills={}),
|
||||||
|
cancel_check=None,
|
||||||
|
scheduled_run=True,
|
||||||
|
deferred_actions=SimpleNamespace(),
|
||||||
|
ark_cfg=ark,
|
||||||
|
img_provider="",
|
||||||
|
img_key="",
|
||||||
|
img_cfg=None,
|
||||||
|
img_provider_cfg=None,
|
||||||
|
video_variant="",
|
||||||
|
office_to_pdf_available=False,
|
||||||
|
))
|
||||||
|
|
||||||
|
text_only = build(False)
|
||||||
|
native = build(True)
|
||||||
|
self.assertIn("look_at_image", text_only)
|
||||||
|
self.assertIn("read_document", text_only)
|
||||||
|
self.assertNotIn("look_at_image", native)
|
||||||
|
self.assertIn("read_document", native)
|
||||||
|
|
||||||
def _execute_and_question(self, question=None) -> str:
|
def _execute_and_question(self, question=None) -> str:
|
||||||
captured = {}
|
captured = {}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -35,6 +35,7 @@ def _make_loop(stream_results, nonstream_results):
|
||||||
loop.caps = SimpleNamespace(reliable_context=64_000, family="test", variant="t")
|
loop.caps = SimpleNamespace(reliable_context=64_000, family="test", variant="t")
|
||||||
loop.session = SimpleNamespace(messages=[], task_id="test-task")
|
loop.session = SimpleNamespace(messages=[], task_id="test-task")
|
||||||
loop.user_id = "test-user" # 无 DB 环境:_log_malformed_args 的落库路径应静默跳过
|
loop.user_id = "test-user" # 无 DB 环境:_log_malformed_args 的落库路径应静默跳过
|
||||||
|
loop.user_root = None
|
||||||
# salvage(0.58.24)接在畸形重试前:空 schemas → 未知工具无白名单 → _try_salvage 返 False,
|
# salvage(0.58.24)接在畸形重试前:空 schemas → 未知工具无白名单 → _try_salvage 返 False,
|
||||||
# 这些用例本就测「salvage 救不了 → 落非流式重试」,给个空 executor 即可走到该分支。
|
# 这些用例本就测「salvage 救不了 → 落非流式重试」,给个空 executor 即可走到该分支。
|
||||||
loop.executor = SimpleNamespace(schemas=lambda: [])
|
loop.executor = SimpleNamespace(schemas=lambda: [])
|
||||||
|
|
@ -56,8 +57,8 @@ def _make_loop(stream_results, nonstream_results):
|
||||||
|
|
||||||
|
|
||||||
class TestMalformedRetry(unittest.TestCase):
|
class TestMalformedRetry(unittest.TestCase):
|
||||||
def test_deepseek_write_name_aborts_before_buffered_reasoning_is_emitted(self):
|
def test_deepseek_write_name_resets_streamed_reasoning_before_reroute(self):
|
||||||
"""真实 collect 路径见到 write 名即关流,之前的 reasoning 不重复上屏。"""
|
"""真实 collect 路径实时发 reasoning;见到 write 后重置再改走非流式。"""
|
||||||
reasoning = SimpleNamespace(
|
reasoning = SimpleNamespace(
|
||||||
choices=[SimpleNamespace(delta=SimpleNamespace(
|
choices=[SimpleNamespace(delta=SimpleNamespace(
|
||||||
reasoning_content="thinking", content=None, tool_calls=None,
|
reasoning_content="thinking", content=None, tool_calls=None,
|
||||||
|
|
@ -86,7 +87,44 @@ class TestMalformedRetry(unittest.TestCase):
|
||||||
|
|
||||||
with self.assertRaises(PreferNonstreamToolCall):
|
with self.assertRaises(PreferNonstreamToolCall):
|
||||||
loop._collect_stream_once([])
|
loop._collect_stream_once([])
|
||||||
self.assertEqual(loop.events, [])
|
self.assertEqual(loop.events, [
|
||||||
|
{"type": "reasoning", "delta": "thinking"},
|
||||||
|
{"type": "reasoning_reset"},
|
||||||
|
])
|
||||||
|
|
||||||
|
def test_deepseek_reasoning_is_emitted_before_output_route_is_known(self):
|
||||||
|
"""普通 DeepSeek 回答不能等首个正文 token 才释放已经到达的推理。"""
|
||||||
|
reasoning = SimpleNamespace(
|
||||||
|
choices=[SimpleNamespace(delta=SimpleNamespace(
|
||||||
|
reasoning_content="thinking", content=None, tool_calls=None,
|
||||||
|
))]
|
||||||
|
)
|
||||||
|
text = SimpleNamespace(
|
||||||
|
choices=[SimpleNamespace(delta=SimpleNamespace(
|
||||||
|
reasoning_content=None, content="answer", tool_calls=None,
|
||||||
|
))]
|
||||||
|
)
|
||||||
|
loop = object.__new__(AgentLoop)
|
||||||
|
loop.caps = SimpleNamespace(
|
||||||
|
family="deepseek_v4", default_reasoning_effort=None,
|
||||||
|
)
|
||||||
|
loop.executor = SimpleNamespace(schemas=lambda: [])
|
||||||
|
loop.cancel_check = None
|
||||||
|
loop.events = []
|
||||||
|
loop._emit = loop.events.append
|
||||||
|
loop.llm = SimpleNamespace(
|
||||||
|
chat_stream=lambda **_kwargs: iter([reasoning, text]),
|
||||||
|
)
|
||||||
|
|
||||||
|
with unittest.mock.patch(
|
||||||
|
"core.loop.litellm.stream_chunk_builder", return_value=object()
|
||||||
|
):
|
||||||
|
loop._collect_stream_once([])
|
||||||
|
|
||||||
|
self.assertEqual(loop.events, [
|
||||||
|
{"type": "reasoning", "delta": "thinking"},
|
||||||
|
{"type": "text", "delta": "answer"},
|
||||||
|
])
|
||||||
|
|
||||||
def test_preferred_tool_reroutes_before_arguments_stream(self):
|
def test_preferred_tool_reroutes_before_arguments_stream(self):
|
||||||
"""首包识别 write/edit 后直接非流式,不把它算作一次畸形失败。"""
|
"""首包识别 write/edit 后直接非流式,不把它算作一次畸形失败。"""
|
||||||
|
|
|
||||||
|
|
@ -13,11 +13,12 @@ from core.loop import AgentLoop
|
||||||
|
|
||||||
class _Session:
|
class _Session:
|
||||||
def __init__(self, messages=None):
|
def __init__(self, messages=None):
|
||||||
|
self.task_id = uuid4()
|
||||||
self.messages = list(messages or [])
|
self.messages = list(messages or [])
|
||||||
self.appended = []
|
self.appended = []
|
||||||
self.append_artifacts = []
|
self.append_artifacts = []
|
||||||
|
|
||||||
def append(self, message, *, artifact_refs=None):
|
def append(self, message, *, artifact_refs=None, **_kwargs):
|
||||||
self.messages.append(message)
|
self.messages.append(message)
|
||||||
self.appended.append(message)
|
self.appended.append(message)
|
||||||
self.append_artifacts.append(artifact_refs)
|
self.append_artifacts.append(artifact_refs)
|
||||||
|
|
@ -101,6 +102,48 @@ class PersistedTurnTests(unittest.TestCase):
|
||||||
"version": 1, "scope": "working_dir", "path": "report.pdf",
|
"version": 1, "scope": "working_dir", "path": "report.pdf",
|
||||||
}])
|
}])
|
||||||
|
|
||||||
|
def test_failed_tool_result_records_health_event(self) -> None:
|
||||||
|
session = _Session([{"role": "user", "content": "执行"}])
|
||||||
|
executor = MagicMock()
|
||||||
|
executor.call_tool.return_value = SimpleNamespace(
|
||||||
|
content="[Error] command failed", artifacts=(),
|
||||||
|
)
|
||||||
|
loop = AgentLoop(
|
||||||
|
llm=MagicMock(), executor=executor, session=session,
|
||||||
|
capabilities=SimpleNamespace(
|
||||||
|
max_iterations=2, family="test", variant="model",
|
||||||
|
input_cny_per_mtoken=0, output_cny_per_mtoken=0,
|
||||||
|
cache_hit_cny_per_mtoken=0, pricing={},
|
||||||
|
),
|
||||||
|
user_id=uuid4(), working_dir=Path("."),
|
||||||
|
)
|
||||||
|
loop._maybe_fold_context = MagicMock()
|
||||||
|
tool_call = SimpleNamespace(
|
||||||
|
id="call-1",
|
||||||
|
function=SimpleNamespace(name="shell", arguments='{"command":"false"}'),
|
||||||
|
)
|
||||||
|
responses = [
|
||||||
|
SimpleNamespace(
|
||||||
|
choices=[SimpleNamespace(message=SimpleNamespace(
|
||||||
|
content="", tool_calls=[tool_call],
|
||||||
|
))], usage=None,
|
||||||
|
),
|
||||||
|
SimpleNamespace(
|
||||||
|
choices=[SimpleNamespace(message=SimpleNamespace(
|
||||||
|
content="已说明失败", tool_calls=None,
|
||||||
|
))], usage=None,
|
||||||
|
),
|
||||||
|
]
|
||||||
|
loop._stream_llm = MagicMock(side_effect=[(r, False) for r in responses])
|
||||||
|
with patch("core.loop.record_chat_usage"), \
|
||||||
|
patch("core.loop.record_tool_failure") as failure:
|
||||||
|
result = loop.run_persisted_turn()
|
||||||
|
|
||||||
|
self.assertEqual(result, "已说明失败")
|
||||||
|
failure.assert_called_once()
|
||||||
|
self.assertEqual(failure.call_args.kwargs["tool"], "shell")
|
||||||
|
self.assertEqual(failure.call_args.kwargs["content"], "[Error] command failed")
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,98 @@
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from decimal import Decimal
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
import unittest
|
||||||
|
from unittest.mock import patch
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from core.capabilities import ModelCapabilities
|
||||||
|
from core.pricing import calculate_chat_cost, resolve_chat_price
|
||||||
|
from core.storage.usage import record_chat_usage
|
||||||
|
|
||||||
|
|
||||||
|
MODELS_DIR = Path(__file__).resolve().parents[1] / "config" / "models"
|
||||||
|
|
||||||
|
|
||||||
|
class PricingTests(unittest.TestCase):
|
||||||
|
def test_deepseek_flash_selects_peak_and_off_peak(self) -> None:
|
||||||
|
caps = ModelCapabilities.load("deepseek_v4.flash", MODELS_DIR)
|
||||||
|
# 2026-08-17 是周一;01:00 UTC 为峰时起点,04:00 已回到谷时。
|
||||||
|
peak = resolve_chat_price(
|
||||||
|
caps.pricing,
|
||||||
|
occurred_at=datetime(2026, 8, 17, 1, 0, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
off_peak = resolve_chat_price(
|
||||||
|
caps.pricing,
|
||||||
|
occurred_at=datetime(2026, 8, 17, 4, 0, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
self.assertIsNotNone(peak)
|
||||||
|
self.assertIsNotNone(off_peak)
|
||||||
|
assert peak is not None and off_peak is not None
|
||||||
|
self.assertEqual(peak.tier, "peak")
|
||||||
|
self.assertEqual(peak.input_per_mtoken, Decimal("0.44"))
|
||||||
|
self.assertEqual(off_peak.tier, "default")
|
||||||
|
self.assertEqual(off_peak.output_per_mtoken, Decimal("0.66"))
|
||||||
|
|
||||||
|
def test_deepseek_weekend_is_off_peak(self) -> None:
|
||||||
|
caps = ModelCapabilities.load("deepseek_v4.pro", MODELS_DIR)
|
||||||
|
quote = resolve_chat_price(
|
||||||
|
caps.pricing,
|
||||||
|
occurred_at=datetime(2026, 8, 22, 2, 0, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
assert quote is not None
|
||||||
|
self.assertEqual(quote.tier, "default")
|
||||||
|
|
||||||
|
def test_usd_quote_calculates_cache_split_in_cny(self) -> None:
|
||||||
|
caps = ModelCapabilities.load("deepseek_v4.flash", MODELS_DIR)
|
||||||
|
quote = resolve_chat_price(
|
||||||
|
caps.pricing,
|
||||||
|
occurred_at=datetime(2026, 8, 17, 1, 0, tzinfo=timezone.utc),
|
||||||
|
)
|
||||||
|
assert quote is not None
|
||||||
|
result = calculate_chat_cost(
|
||||||
|
quote,
|
||||||
|
prompt_tokens=1_000_000,
|
||||||
|
completion_tokens=500_000,
|
||||||
|
cache_hit_tokens=800_000,
|
||||||
|
)
|
||||||
|
# miss:0.2*0.44*7.2 + hit:0.8*0.014*7.2 + out:0.5*1.32*7.2
|
||||||
|
self.assertEqual(result.total_cny, Decimal("5.466240"))
|
||||||
|
self.assertEqual(result.cache_miss_tokens, 200_000)
|
||||||
|
|
||||||
|
def test_local_catalog_precedes_litellm_cost_map(self) -> None:
|
||||||
|
caps = ModelCapabilities.load("deepseek_v4.flash", MODELS_DIR)
|
||||||
|
added = []
|
||||||
|
|
||||||
|
class FakeSession:
|
||||||
|
def add(self, row):
|
||||||
|
added.append(row)
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def fake_scope():
|
||||||
|
yield FakeSession()
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch("core.storage.usage.session_scope", fake_scope),
|
||||||
|
patch("core.storage.usage._safe_chat_cost_usd", return_value=Decimal("999")),
|
||||||
|
):
|
||||||
|
cost = record_chat_usage(
|
||||||
|
task_id=None,
|
||||||
|
user_id=uuid4(),
|
||||||
|
message_id=None,
|
||||||
|
model_profile="deepseek_v4.flash",
|
||||||
|
prompt_tokens=1_000_000,
|
||||||
|
completion_tokens=0,
|
||||||
|
pricing=caps.pricing,
|
||||||
|
occurred_at=datetime(2026, 8, 17, 4, 0, tzinfo=timezone.utc),
|
||||||
|
response=SimpleNamespace(created=None),
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(cost, Decimal("1.584000"))
|
||||||
|
self.assertEqual(added[0].units["pricing_revision"], "deepseek-v4-20260816")
|
||||||
|
self.assertEqual(added[0].units["price_tier"], "default")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -0,0 +1,274 @@
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import unittest
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from importlib import util
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import MagicMock, Mock, patch
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from core.capabilities import ModelCapabilities
|
||||||
|
from core.llm import LLM
|
||||||
|
from core.provider_credentials.registry import BY_ID
|
||||||
|
from core.provider_credentials.runtime import resolve_credentials
|
||||||
|
from core.provider_credentials.service import (
|
||||||
|
ProviderCredentialError,
|
||||||
|
RevisionConflict,
|
||||||
|
_hint,
|
||||||
|
_notification_transition,
|
||||||
|
_row_payload,
|
||||||
|
replace_credentials,
|
||||||
|
)
|
||||||
|
from core.provider_credentials.testing import (
|
||||||
|
TestResult,
|
||||||
|
classify_response,
|
||||||
|
test_provider,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def response(status, body):
|
||||||
|
content = json.dumps(body).encode() if isinstance(body, dict) else str(body).encode()
|
||||||
|
return httpx.Response(status, content=content)
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderRegistryTests(unittest.TestCase):
|
||||||
|
def test_credential_groups_are_explicit(self):
|
||||||
|
self.assertEqual(
|
||||||
|
[f.name for f in BY_ID["xfyun_iat"].fields],
|
||||||
|
["appid", "api_key", "api_secret"],
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
[f.name for f in BY_ID["xfyun_lfasr"].fields],
|
||||||
|
["appid", "secret_key"],
|
||||||
|
)
|
||||||
|
self.assertNotIn("ZCBOT_DB_URL", [f.env for p in BY_ID.values() for f in p.fields])
|
||||||
|
|
||||||
|
def test_hint_only_reveals_last_four(self):
|
||||||
|
self.assertEqual(_hint("sk-abcdefgh"), "***efgh")
|
||||||
|
self.assertNotIn("abcdef", _hint("sk-abcdefgh"))
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderClassificationTests(unittest.TestCase):
|
||||||
|
def test_http_402_and_429_are_not_conflated(self):
|
||||||
|
self.assertEqual(classify_response(402, "").status, "exhausted")
|
||||||
|
self.assertEqual(classify_response(429, "rate limit").status, "unreachable")
|
||||||
|
|
||||||
|
def test_deepseek_30_yuan_boundary(self):
|
||||||
|
def request_at(amount):
|
||||||
|
return lambda *a, **k: response(200, {
|
||||||
|
"is_available": True,
|
||||||
|
"balance_infos": [{"currency": "CNY", "total_balance": str(amount)}],
|
||||||
|
})
|
||||||
|
creds = {"api_key": "candidate-secret"}
|
||||||
|
self.assertEqual(
|
||||||
|
test_provider("deepseek", creds, request=request_at("29.99")).status,
|
||||||
|
"low_balance",
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
test_provider("deepseek", creds, request=request_at("30.00")).status,
|
||||||
|
"normal",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_candidate_secret_is_not_returned_in_detail(self):
|
||||||
|
secret = "candidate-super-secret"
|
||||||
|
result = test_provider(
|
||||||
|
"deepseek", {"api_key": secret},
|
||||||
|
request=lambda *a, **k: response(401, {"message": secret}),
|
||||||
|
)
|
||||||
|
self.assertEqual(result.status, "auth_error")
|
||||||
|
self.assertNotIn(secret, result.detail)
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderResolutionTests(unittest.TestCase):
|
||||||
|
def test_database_precedes_environment(self):
|
||||||
|
row = SimpleNamespace(credentials={"api_key": "cipher"})
|
||||||
|
|
||||||
|
class Result:
|
||||||
|
def scalar_one_or_none(self):
|
||||||
|
return row
|
||||||
|
|
||||||
|
class Session:
|
||||||
|
def execute(self, _statement):
|
||||||
|
return Result()
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def scope():
|
||||||
|
yield Session()
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.dict(os.environ, {"DEEPSEEK_API_KEY": "env-key"}),
|
||||||
|
patch("core.storage.session_scope", scope),
|
||||||
|
patch("core.provider_credentials.runtime.decrypt_secret", return_value="db-key") as decrypt,
|
||||||
|
):
|
||||||
|
resolved = resolve_credentials("deepseek")
|
||||||
|
self.assertEqual((resolved.source, resolved.values["api_key"]), ("database", "db-key"))
|
||||||
|
decrypt.assert_called_once_with("cipher", aad="provider:deepseek:api_key")
|
||||||
|
|
||||||
|
def test_environment_fallback_without_database(self):
|
||||||
|
with (
|
||||||
|
patch.dict(os.environ, {"DEEPSEEK_API_KEY": "env-key"}),
|
||||||
|
patch("core.storage.session_scope", side_effect=RuntimeError("no db")),
|
||||||
|
):
|
||||||
|
resolved = resolve_credentials("deepseek")
|
||||||
|
self.assertEqual((resolved.source, resolved.values), ("env", {"api_key": "env-key"}))
|
||||||
|
|
||||||
|
def test_each_xfyun_group_resolves_its_declared_fields(self):
|
||||||
|
env = {
|
||||||
|
"XFYUN_APPID": "app", "XFYUN_API_KEY": "iat-key",
|
||||||
|
"XFYUN_API_SECRET": "iat-secret", "XFYUN_LFASR_SECRET_KEY": "lf-key",
|
||||||
|
}
|
||||||
|
with patch.dict(os.environ, env), patch("core.storage.session_scope", side_effect=RuntimeError("no db")):
|
||||||
|
self.assertEqual(set(resolve_credentials("xfyun_iat").values), {"appid", "api_key", "api_secret"})
|
||||||
|
self.assertEqual(set(resolve_credentials("xfyun_lfasr").values), {"appid", "secret_key"})
|
||||||
|
|
||||||
|
|
||||||
|
class LLMCredentialLifecycleTests(unittest.TestCase):
|
||||||
|
def _caps(self):
|
||||||
|
return ModelCapabilities(
|
||||||
|
family="deepseek_v4", model_id="deepseek/model",
|
||||||
|
api_key_env="DEEPSEEK_API_KEY", thinking_transport="none",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_created_llm_uses_new_key_on_next_request(self):
|
||||||
|
state = {"key": "old-key"}
|
||||||
|
llm = LLM(self._caps(), credential_resolver=lambda _env: state["key"])
|
||||||
|
first = llm._build_kwargs([], None, None, None)
|
||||||
|
state["key"] = "new-key"
|
||||||
|
second = llm._build_kwargs([], None, None, None)
|
||||||
|
self.assertEqual(first["api_key"], "old-key")
|
||||||
|
self.assertEqual(second["api_key"], "new-key")
|
||||||
|
|
||||||
|
def test_built_stream_request_keeps_key_snapshot(self):
|
||||||
|
state = {"key": "old-key"}
|
||||||
|
calls = []
|
||||||
|
llm = LLM(self._caps(), credential_resolver=lambda _env: state["key"])
|
||||||
|
|
||||||
|
def completion(**kwargs):
|
||||||
|
calls.append(kwargs["api_key"])
|
||||||
|
state["key"] = "new-key"
|
||||||
|
return iter([{"chunk": 1}, {"chunk": 2}])
|
||||||
|
|
||||||
|
with patch("core.llm.litellm.completion", side_effect=completion):
|
||||||
|
self.assertEqual(len(list(llm.chat_stream([]))), 2)
|
||||||
|
self.assertEqual(calls, ["old-key"])
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderMutationTests(unittest.TestCase):
|
||||||
|
def test_missing_master_key_refuses_database_save(self):
|
||||||
|
with (
|
||||||
|
patch("core.provider_credentials.service.crypto_configured", return_value=False),
|
||||||
|
self.assertRaisesRegex(ProviderCredentialError, "MASTER_KEY"),
|
||||||
|
):
|
||||||
|
replace_credentials(
|
||||||
|
"deepseek", {"api_key": "candidate"}, expected_revision=0,
|
||||||
|
updated_by=uuid4(),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_failed_candidate_never_opens_database_transaction(self):
|
||||||
|
with (
|
||||||
|
patch("core.provider_credentials.service.crypto_configured", return_value=True),
|
||||||
|
patch("core.provider_credentials.service.test_provider",
|
||||||
|
return_value=TestResult("auth_error", "认证失败")),
|
||||||
|
patch("core.provider_credentials.service.session_scope") as scope,
|
||||||
|
self.assertRaises(ProviderCredentialError),
|
||||||
|
):
|
||||||
|
replace_credentials(
|
||||||
|
"deepseek", {"api_key": "bad-key"}, expected_revision=0,
|
||||||
|
updated_by=uuid4(),
|
||||||
|
)
|
||||||
|
scope.assert_not_called()
|
||||||
|
|
||||||
|
def test_revision_conflict_does_not_overwrite(self):
|
||||||
|
class Session:
|
||||||
|
def execute(self, _statement):
|
||||||
|
return SimpleNamespace(rowcount=0)
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def scope():
|
||||||
|
yield Session()
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch("core.provider_credentials.service.crypto_configured", return_value=True),
|
||||||
|
patch("core.provider_credentials.service.test_provider",
|
||||||
|
return_value=TestResult("normal", "正常")),
|
||||||
|
patch("core.provider_credentials.service.encrypt_secret", return_value="cipher"),
|
||||||
|
patch("core.provider_credentials.service.session_scope", scope),
|
||||||
|
self.assertRaises(RevisionConflict),
|
||||||
|
):
|
||||||
|
replace_credentials(
|
||||||
|
"deepseek", {"api_key": "new-key"}, expected_revision=7,
|
||||||
|
updated_by=uuid4(),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_alert_cooldown_and_recovery(self):
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
low = TestResult("low_balance", "余额低")
|
||||||
|
self.assertEqual(_notification_transition("normal", None, low, now), "low_balance")
|
||||||
|
self.assertIsNone(_notification_transition("low_balance", now, low, now))
|
||||||
|
self.assertEqual(
|
||||||
|
_notification_transition("low_balance", now - timedelta(days=1), low, now),
|
||||||
|
"low_balance",
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
_notification_transition("auth_error", now, TestResult("normal", "正常"), now),
|
||||||
|
"recovered",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_admin_payload_contains_no_ciphertext(self):
|
||||||
|
provider = BY_ID["deepseek"]
|
||||||
|
row = SimpleNamespace(
|
||||||
|
credentials={"api_key": "ciphertext-secret"},
|
||||||
|
credential_hint={"api_key": "***1234"}, revision=2,
|
||||||
|
test_status="normal", test_detail="正常", last_tested_at=None,
|
||||||
|
balance_amount=None, balance_currency=None,
|
||||||
|
)
|
||||||
|
with patch("core.provider_credentials.service.resolve_credentials") as resolve:
|
||||||
|
resolve.return_value = SimpleNamespace(source="database", values={"api_key": "plain-secret"})
|
||||||
|
payload = _row_payload(provider, row)
|
||||||
|
encoded = json.dumps(payload, ensure_ascii=False)
|
||||||
|
self.assertNotIn("ciphertext-secret", encoded)
|
||||||
|
self.assertNotIn("plain-secret", encoded)
|
||||||
|
self.assertIn("***1234", encoded)
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderLeaderTests(unittest.TestCase):
|
||||||
|
def test_unclaimed_round_does_not_test(self):
|
||||||
|
connection = Mock()
|
||||||
|
connection.exec_driver_sql.return_value.scalar.return_value = False
|
||||||
|
engine = MagicMock()
|
||||||
|
engine.connect.return_value.__enter__.return_value = connection
|
||||||
|
with (
|
||||||
|
patch("core.provider_credentials.monitor.get_engine", return_value=engine),
|
||||||
|
patch("core.provider_credentials.monitor.due_provider_ids") as due,
|
||||||
|
):
|
||||||
|
from core.provider_credentials.monitor import run_due_checks
|
||||||
|
self.assertEqual(run_due_checks(), 0)
|
||||||
|
due.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderMigrationTests(unittest.TestCase):
|
||||||
|
def test_0039_creates_single_encrypted_credentials_table(self):
|
||||||
|
path = Path(__file__).resolve().parents[1] / "db" / "migrations" / "versions" / "20260902_1000_0039_provider_credentials.py"
|
||||||
|
spec = util.spec_from_file_location("migration_0039_test", path)
|
||||||
|
module = util.module_from_spec(spec)
|
||||||
|
assert spec.loader is not None
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
def create_table(name, *items):
|
||||||
|
captured["name"] = name
|
||||||
|
captured["columns"] = {item.name for item in items if hasattr(item, "name")}
|
||||||
|
|
||||||
|
with patch.object(module.op, "create_table", side_effect=create_table):
|
||||||
|
module.upgrade()
|
||||||
|
self.assertEqual(captured["name"], "provider_credentials")
|
||||||
|
self.assertTrue({"provider_id", "credentials", "credential_hint", "revision"} <= captured["columns"])
|
||||||
|
self.assertNotIn("api_key", captured["columns"])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -98,6 +98,21 @@ class TestSalvage(unittest.TestCase):
|
||||||
raw = '前缀{"path": "/a", "content": "x"}尾部残渣'
|
raw = '前缀{"path": "/a", "content": "x"}尾部残渣'
|
||||||
self.assertIsNone(salvage_tool_arguments(raw, WRITE_KEYS))
|
self.assertIsNone(salvage_tool_arguments(raw, WRITE_KEYS))
|
||||||
|
|
||||||
|
def test_repeated_identical_json_with_truncated_copy_is_salvaged(self):
|
||||||
|
"""wire 重复同一参数且末尾又截断时,可用两个一致的完整副本恢复。"""
|
||||||
|
good = {"command": "rg -n 'needle' /workspace/report.md"}
|
||||||
|
encoded = json.dumps(good, ensure_ascii=False)
|
||||||
|
raw = encoded + encoded + encoded[:18]
|
||||||
|
self.assertEqual(salvage_tool_arguments(raw, SHELL_KEYS), good)
|
||||||
|
|
||||||
|
def test_distinct_complete_objects_with_trailing_garbage_are_rejected(self):
|
||||||
|
"""存在两个不同的合法参数对象时语义不唯一,不能猜测执行哪一个。"""
|
||||||
|
first = json.dumps({"command": "first"})
|
||||||
|
second = json.dumps({"command": "second"})
|
||||||
|
self.assertIsNone(
|
||||||
|
salvage_tool_arguments(first + second + "{truncated", SHELL_KEYS)
|
||||||
|
)
|
||||||
|
|
||||||
def test_clean_json_at_char0(self):
|
def test_clean_json_at_char0(self):
|
||||||
"""char-0 就是完好 JSON(理论上不会进 salvage,但函数应幂等返回它)。"""
|
"""char-0 就是完好 JSON(理论上不会进 salvage,但函数应幂等返回它)。"""
|
||||||
raw = '{"path": "/a", "content": "x"}'
|
raw = '{"path": "/a", "content": "x"}'
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,213 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import unittest
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from core.sandbox.capacity import ExecCapacity
|
||||||
|
from core.sandbox.pool import SandboxPool
|
||||||
|
from core.sandbox.package_scans import persist_scan
|
||||||
|
from deploy.sandbox.package_scan import scan_packages
|
||||||
|
from core import procs
|
||||||
|
from web.admin import _sandbox_package_stats
|
||||||
|
|
||||||
|
|
||||||
|
class CapacityTests(unittest.TestCase):
|
||||||
|
def test_env_overrides_capacity_config(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch.dict(os.environ, {
|
||||||
|
"ZCBOT_MAX_ACTIVE_EXECS": "5", "ZCBOT_MAX_BACKGROUND_EXECS": "3",
|
||||||
|
"ZCBOT_MAX_ACTIVE_EXECS_PER_USER": "1", "ZCBOT_MIN_MEM_AVAILABLE": "2g",
|
||||||
|
}, clear=False):
|
||||||
|
cap = ExecCapacity(Path(td), {"max_active_execs": 9, "max_background_execs": 8,
|
||||||
|
"max_active_execs_per_user": 7, "min_mem_available": "4g"})
|
||||||
|
self.assertEqual((cap.max_active, cap.max_background, cap.max_per_user, cap.min_mem_available),
|
||||||
|
(5, 3, 1, 2 * 1024**3))
|
||||||
|
|
||||||
|
def test_config_cannot_raise_hard_limits(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td:
|
||||||
|
cap = ExecCapacity(Path(td), {"max_active_execs": 99, "max_background_execs": 99,
|
||||||
|
"max_active_execs_per_user": 99})
|
||||||
|
self.assertEqual((cap.max_active, cap.max_background, cap.max_per_user), (6, 4, 3))
|
||||||
|
|
||||||
|
def test_global_per_user_and_background_limits(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=10**12):
|
||||||
|
cap = ExecCapacity(Path(td), {"max_active_execs": 3, "max_background_execs": 1, "max_active_execs_per_user": 2})
|
||||||
|
a = cap.try_acquire("u1", "background", lease_id="a")
|
||||||
|
b = cap.try_acquire("u1", "foreground", lease_id="b")
|
||||||
|
self.assertIsNotNone(a); self.assertIsNotNone(b)
|
||||||
|
self.assertIsNone(cap.try_acquire("u1", "foreground", lease_id="c"))
|
||||||
|
self.assertIsNone(cap.try_acquire("u2", "background", lease_id="d"))
|
||||||
|
self.assertIsNotNone(cap.try_acquire("u2", "foreground", lease_id="e"))
|
||||||
|
self.assertIsNone(cap.try_acquire("u3", "foreground", lease_id="f"))
|
||||||
|
|
||||||
|
def test_cross_instance_state_and_cancel_queue(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=10**12):
|
||||||
|
one = ExecCapacity(Path(td), {"max_active_execs": 1})
|
||||||
|
two = ExecCapacity(Path(td), {"max_active_execs": 1})
|
||||||
|
lease = one.try_acquire("u1", "foreground", lease_id="held")
|
||||||
|
cancelled = threading.Event()
|
||||||
|
result = []
|
||||||
|
t = threading.Thread(target=lambda: result.append(two.acquire_foreground("u2", cancelled.is_set)))
|
||||||
|
t.start(); time.sleep(.15); cancelled.set(); t.join(2)
|
||||||
|
self.assertEqual(result, [None])
|
||||||
|
self.assertEqual(two.snapshot()["foreground_queued"], 0)
|
||||||
|
one.release(lease)
|
||||||
|
|
||||||
|
def test_foreground_queue_reports_limit_and_admission(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=10**12):
|
||||||
|
cap = ExecCapacity(Path(td), {"max_active_execs": 2, "max_active_execs_per_user": 1})
|
||||||
|
held = cap.try_acquire("u1", "background", lease_id="held")
|
||||||
|
notices = []
|
||||||
|
result = []
|
||||||
|
t = threading.Thread(
|
||||||
|
target=lambda: result.append(cap.acquire_foreground("u1", wait_notify=notices.append))
|
||||||
|
)
|
||||||
|
t.start(); time.sleep(.15)
|
||||||
|
self.assertEqual(notices, [{
|
||||||
|
"state": "waiting", "reason": "per_user_limit",
|
||||||
|
"user_running": 1, "user_limit": 1,
|
||||||
|
"global_running": 1, "global_limit": 2,
|
||||||
|
}])
|
||||||
|
cap.release(held); t.join(2)
|
||||||
|
self.assertEqual(len(result), 1)
|
||||||
|
self.assertIsNotNone(result[0])
|
||||||
|
self.assertEqual(notices[-1], {"state": "admitted"})
|
||||||
|
cap.release(result[0])
|
||||||
|
|
||||||
|
def test_immediate_foreground_admission_does_not_report_queue(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=10**12):
|
||||||
|
cap = ExecCapacity(Path(td), {"max_active_execs": 1})
|
||||||
|
notices = []
|
||||||
|
lease = cap.acquire_foreground("u1", wait_notify=notices.append)
|
||||||
|
self.assertIsNotNone(lease)
|
||||||
|
self.assertEqual(notices, [])
|
||||||
|
cap.release(lease)
|
||||||
|
|
||||||
|
def test_memory_pressure_pauses_new_admission(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.capacity.mem_available_bytes", return_value=100):
|
||||||
|
cap = ExecCapacity(Path(td), {"min_mem_available": "1g"})
|
||||||
|
self.assertIsNone(cap.try_acquire("u", "foreground"))
|
||||||
|
self.assertTrue(cap.snapshot()["memory_paused"])
|
||||||
|
|
||||||
|
|
||||||
|
class PoolTests(unittest.TestCase):
|
||||||
|
def test_defaults_and_docker_tmpfs(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.pool.subprocess.run") as run:
|
||||||
|
run.return_value = MagicMock(returncode=0, stdout="")
|
||||||
|
pool = SandboxPool(Path(td) / "users")
|
||||||
|
self.assertEqual((pool.memory, pool.cpus, pool.idle_ttl, pool.tmp_size), ("4g", "2.0", 600, "1g"))
|
||||||
|
pool._docker_run(uuid4(), "box")
|
||||||
|
argv = run.call_args_list[-1].args[0]
|
||||||
|
self.assertIn("/tmp:exec,size=1g,mode=1777", argv)
|
||||||
|
|
||||||
|
def test_active_exec_not_reaped(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.pool._container_running", return_value=True), patch("core.sandbox.pool.subprocess.run") as run:
|
||||||
|
pool = SandboxPool(Path(td) / "users", idle_ttl=1)
|
||||||
|
uid = uuid4(); pool._last_active[uid] = int(time.time()) - 10
|
||||||
|
pool.exec_started(uid)
|
||||||
|
self.assertEqual(pool.reap_idle(), [])
|
||||||
|
run.assert_not_called()
|
||||||
|
|
||||||
|
def test_scan_failure_does_not_block_idle_removal(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, \
|
||||||
|
patch("core.sandbox.package_scans.scan_and_persist", side_effect=RuntimeError("db down")), \
|
||||||
|
patch("core.sandbox.pool.subprocess.run", return_value=MagicMock(returncode=0)):
|
||||||
|
pool = SandboxPool(Path(td) / "users", idle_ttl=1)
|
||||||
|
uid = uuid4(); pool._last_active[uid] = int(time.time()) - 10
|
||||||
|
self.assertEqual(pool.reap_idle(), [f"zcbot-sandbox-{uid}"])
|
||||||
|
|
||||||
|
def test_finalizer_helper_swallows_scanner_error(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td, patch("core.sandbox.package_scans.scan_and_persist", side_effect=RuntimeError("db down")):
|
||||||
|
pool = SandboxPool(Path(td) / "users")
|
||||||
|
pool._scan_before_remove("box", uuid4(), "foreground")
|
||||||
|
|
||||||
|
|
||||||
|
class PackageScannerTests(unittest.TestCase):
|
||||||
|
def test_empty_scan_does_not_touch_database(self):
|
||||||
|
persist_scan(container_session_id="empty", user_id=uuid4(), execution_kind="foreground",
|
||||||
|
result={"packages": [], "image_digest": "sha256:x", "python_version": "3.12", "total_installed_bytes": 0})
|
||||||
|
|
||||||
|
def test_added_override_bytes_and_symlink_escape(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td:
|
||||||
|
root = Path(td) / "lib"; site = root / "python3.12" / "site-packages"; site.mkdir(parents=True)
|
||||||
|
pkg = site / "demo.py"; pkg.write_text("abc", encoding="utf-8")
|
||||||
|
dist = site / "Demo-2.dist-info"; dist.mkdir()
|
||||||
|
(dist / "METADATA").write_text("Name: Demo\nVersion: 2\n", encoding="utf-8")
|
||||||
|
(dist / "REQUESTED").write_text("", encoding="utf-8")
|
||||||
|
(dist / "RECORD").write_text("demo.py,,3\n", encoding="utf-8")
|
||||||
|
rows = scan_packages(root, {"demo": "1"})
|
||||||
|
self.assertEqual(rows[0]["change"], "override")
|
||||||
|
self.assertEqual(rows[0]["installed_bytes"], 3)
|
||||||
|
self.assertTrue(rows[0]["direct_requested"])
|
||||||
|
self.assertEqual(scan_packages(root, {"demo": "2"})[0]["change"], "reinstall")
|
||||||
|
self.assertEqual(scan_packages(root, {})[0]["change"], "added")
|
||||||
|
|
||||||
|
def test_metadata_symlink_is_ignored(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td:
|
||||||
|
root = Path(td) / "lib"; site = root / "python3.12" / "site-packages"; site.mkdir(parents=True)
|
||||||
|
outside = Path(td) / "outside"; outside.write_text("Name: Bad\nVersion: 1\n")
|
||||||
|
dist = site / "Bad.dist-info"; dist.mkdir()
|
||||||
|
try:
|
||||||
|
(dist / "METADATA").symlink_to(outside)
|
||||||
|
except OSError:
|
||||||
|
self.skipTest("symlink unavailable")
|
||||||
|
self.assertEqual(scan_packages(root, {}), [])
|
||||||
|
|
||||||
|
|
||||||
|
class ProcQueueTests(unittest.TestCase):
|
||||||
|
def test_queued_status_and_cancel(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td:
|
||||||
|
d = Path(td) / "p"; d.mkdir()
|
||||||
|
meta = {"proc_id": "p1", "backend": "docker", "state": "queued"}
|
||||||
|
procs.write_meta(d, meta)
|
||||||
|
self.assertEqual(procs.status_of(meta, d)[0], "queued")
|
||||||
|
self.assertEqual(procs.kill_proc(meta, d), "已取消排队")
|
||||||
|
self.assertEqual(procs.status_of(procs.read_meta(d), d), ("finished", 137))
|
||||||
|
|
||||||
|
def test_start_queued_respects_capacity_then_runs(self):
|
||||||
|
with tempfile.TemporaryDirectory() as td:
|
||||||
|
d = Path(td) / "p"; d.mkdir(); (d / "runner.sh").write_text("true")
|
||||||
|
uid = uuid4()
|
||||||
|
meta = {"proc_id": "p1", "task_id": "t1", "backend": "docker", "state": "queued",
|
||||||
|
"user_id": str(uid), "cwd": "/workspace/demo"}
|
||||||
|
procs.write_meta(d, meta)
|
||||||
|
pool = MagicMock(); pool.capacity.try_acquire.return_value = None
|
||||||
|
self.assertFalse(procs.start_queued_docker(meta, d, pool))
|
||||||
|
pool.capacity.try_acquire.return_value = "bg-p1"
|
||||||
|
pool.run_proc_container.return_value = "zcbot-proc-p1"
|
||||||
|
with patch("core.procs.subprocess.run", return_value=MagicMock(returncode=0, stderr="")):
|
||||||
|
self.assertTrue(procs.start_queued_docker(meta, d, pool))
|
||||||
|
saved = procs.read_meta(d)
|
||||||
|
self.assertEqual(saved["state"], "running")
|
||||||
|
self.assertEqual(saved["capacity_lease_id"], "bg-p1")
|
||||||
|
|
||||||
|
|
||||||
|
class AdminAggregationTests(unittest.TestCase):
|
||||||
|
def test_package_sessions_are_aggregated_by_name_version(self):
|
||||||
|
scans = [
|
||||||
|
SimpleNamespace(container_session_id="s1", user_id=uuid4(), execution_kind="foreground",
|
||||||
|
packages=[{"name":"Demo","version":"1","direct_requested":True,"installed_bytes":100,"change":"added","base_version":None}],
|
||||||
|
finished_at=datetime.now(timezone.utc)),
|
||||||
|
SimpleNamespace(container_session_id="s2", user_id=uuid4(), execution_kind="background",
|
||||||
|
packages=[{"name":"Demo","version":"1","direct_requested":False,"installed_bytes":300,"change":"added","base_version":None}],
|
||||||
|
finished_at=datetime.now(timezone.utc)),
|
||||||
|
]
|
||||||
|
session = MagicMock(); session.execute.return_value.scalars.return_value.all.return_value = scans
|
||||||
|
row = _sandbox_package_stats(session, None)["rows"][0]
|
||||||
|
self.assertEqual(row["session_count"], 2)
|
||||||
|
self.assertEqual(row["user_count"], 2)
|
||||||
|
self.assertEqual(row["foreground_sessions"], 1)
|
||||||
|
self.assertEqual(row["background_sessions"], 1)
|
||||||
|
self.assertEqual(row["direct_sessions"], 1)
|
||||||
|
self.assertEqual(row["average_installed_bytes"], 200)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -59,7 +59,7 @@ def _job(action: str, *, status: str = "succeeded"):
|
||||||
|
|
||||||
|
|
||||||
class SoftwareFollowupTests(unittest.TestCase):
|
class SoftwareFollowupTests(unittest.TestCase):
|
||||||
def test_report_claim_persists_fixed_assistant_message_with_artifacts(self):
|
def test_report_claim_completes_without_persisting_chat_message(self):
|
||||||
job = _job("report")
|
job = _job("report")
|
||||||
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
|
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
|
||||||
session = _Session([job.task_id, task, job])
|
session = _Session([job.task_id, task, job])
|
||||||
|
|
@ -68,20 +68,12 @@ class SoftwareFollowupTests(unittest.TestCase):
|
||||||
def scope():
|
def scope():
|
||||||
yield session
|
yield session
|
||||||
|
|
||||||
with (
|
with patch("web.software_followups.session_scope", scope):
|
||||||
patch("web.software_followups.session_scope", scope),
|
|
||||||
patch("web.software_followups.allocate_message_idx", return_value=7),
|
|
||||||
):
|
|
||||||
claim = claim_followup(job.job_id)
|
claim = claim_followup(job.job_id)
|
||||||
|
|
||||||
self.assertEqual(claim.action, "report")
|
self.assertEqual(claim.action, "report")
|
||||||
self.assertEqual(job.followup_status, "completed")
|
self.assertEqual(job.followup_status, "completed")
|
||||||
self.assertEqual(len(session.added), 1)
|
self.assertEqual(session.added, [])
|
||||||
message = session.added[0]
|
|
||||||
self.assertEqual(message.payload["role"], "assistant")
|
|
||||||
self.assertIn(str(job.job_id), message.payload["content"])
|
|
||||||
self.assertEqual(len(message.artifact_refs), 1)
|
|
||||||
self.assertEqual(message.artifact_refs[0]["label"], "figure.png")
|
|
||||||
|
|
||||||
def test_analyze_claim_persists_internal_turn_and_locks_task(self):
|
def test_analyze_claim_persists_internal_turn_and_locks_task(self):
|
||||||
job = _job("analyze")
|
job = _job("analyze")
|
||||||
|
|
@ -120,7 +112,7 @@ class SoftwareFollowupTests(unittest.TestCase):
|
||||||
self.assertEqual(job.followup_status, "pending")
|
self.assertEqual(job.followup_status, "pending")
|
||||||
self.assertFalse(session.added)
|
self.assertFalse(session.added)
|
||||||
|
|
||||||
def test_failed_job_persists_fixed_assistant_message_without_artifacts(self):
|
def test_failed_job_completes_without_persisting_chat_message(self):
|
||||||
job = _job("analyze", status="failed")
|
job = _job("analyze", status="failed")
|
||||||
job.error = {"code": "COLUMN_MISSING", "detail": " Y2\n does not exist "}
|
job.error = {"code": "COLUMN_MISSING", "detail": " Y2\n does not exist "}
|
||||||
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
|
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
|
||||||
|
|
@ -130,21 +122,14 @@ class SoftwareFollowupTests(unittest.TestCase):
|
||||||
def scope():
|
def scope():
|
||||||
yield session
|
yield session
|
||||||
|
|
||||||
with (
|
with patch("web.software_followups.session_scope", scope):
|
||||||
patch("web.software_followups.session_scope", scope),
|
|
||||||
patch("web.software_followups.allocate_message_idx", return_value=9),
|
|
||||||
):
|
|
||||||
claim = claim_followup(job.job_id)
|
claim = claim_followup(job.job_id)
|
||||||
|
|
||||||
self.assertEqual(claim.action, "report")
|
self.assertEqual(claim.action, "report")
|
||||||
self.assertEqual(job.followup_status, "completed")
|
self.assertEqual(job.followup_status, "completed")
|
||||||
message = session.added[0]
|
self.assertEqual(session.added, [])
|
||||||
self.assertEqual(message.payload["role"], "assistant")
|
|
||||||
self.assertIn("专业软件任务执行失败", message.payload["content"])
|
|
||||||
self.assertIn("Y2 does not exist", message.payload["content"])
|
|
||||||
self.assertEqual(message.artifact_refs, [])
|
|
||||||
|
|
||||||
def test_cancelled_job_persists_fixed_assistant_message(self):
|
def test_cancelled_job_completes_without_persisting_chat_message(self):
|
||||||
job = _job("analyze", status="cancelled")
|
job = _job("analyze", status="cancelled")
|
||||||
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
|
task = SimpleNamespace(task_id=job.task_id, run_status="idle")
|
||||||
session = _Session([job.task_id, task, job])
|
session = _Session([job.task_id, task, job])
|
||||||
|
|
@ -153,19 +138,17 @@ class SoftwareFollowupTests(unittest.TestCase):
|
||||||
def scope():
|
def scope():
|
||||||
yield session
|
yield session
|
||||||
|
|
||||||
with (
|
with patch("web.software_followups.session_scope", scope):
|
||||||
patch("web.software_followups.session_scope", scope),
|
|
||||||
patch("web.software_followups.allocate_message_idx", return_value=10),
|
|
||||||
):
|
|
||||||
claim = claim_followup(job.job_id)
|
claim = claim_followup(job.job_id)
|
||||||
|
|
||||||
self.assertEqual(claim.action, "report")
|
self.assertEqual(claim.action, "report")
|
||||||
self.assertIn("专业软件任务已取消", session.added[0].payload["content"])
|
self.assertEqual(job.followup_status, "completed")
|
||||||
|
self.assertEqual(session.added, [])
|
||||||
|
|
||||||
def test_busy_task_leaves_failed_notification_pending(self):
|
def test_busy_task_does_not_delay_non_chat_result(self):
|
||||||
job = _job("report", status="failed")
|
job = _job("report", status="failed")
|
||||||
task = SimpleNamespace(task_id=job.task_id, run_status="running")
|
task = SimpleNamespace(task_id=job.task_id, run_status="running")
|
||||||
session = _Session([job.task_id, task])
|
session = _Session([job.task_id, task, job])
|
||||||
|
|
||||||
@contextmanager
|
@contextmanager
|
||||||
def scope():
|
def scope():
|
||||||
|
|
@ -174,8 +157,8 @@ class SoftwareFollowupTests(unittest.TestCase):
|
||||||
with patch("web.software_followups.session_scope", scope):
|
with patch("web.software_followups.session_scope", scope):
|
||||||
claim = claim_followup(job.job_id)
|
claim = claim_followup(job.job_id)
|
||||||
|
|
||||||
self.assertIsNone(claim)
|
self.assertEqual(claim.action, "report")
|
||||||
self.assertEqual(job.followup_status, "pending")
|
self.assertEqual(job.followup_status, "completed")
|
||||||
self.assertFalse(session.added)
|
self.assertFalse(session.added)
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -42,16 +42,24 @@ class SoftwareJobToolTests(unittest.TestCase):
|
||||||
)
|
)
|
||||||
self.assertEqual(result["capabilities"][0]["capability"], "origin.plot@v2")
|
self.assertEqual(result["capabilities"][0]["capability"], "origin.plot@v2")
|
||||||
self.assertEqual(result["capabilities"][0]["available_nodes"], 1)
|
self.assertEqual(result["capabilities"][0]["available_nodes"], 1)
|
||||||
self.assertEqual(
|
capabilities = {item["capability"] for item in result["capabilities"]}
|
||||||
result["capabilities"][1]["capability"],
|
self.assertIn("cad.geometry.prepare@v1", capabilities)
|
||||||
"ansys.geometry.inspect@v1",
|
self.assertIn("ansys.mechanical.static_structural@v2", capabilities)
|
||||||
|
self.assertNotIn("ansys.geometry.inspect@v1", capabilities)
|
||||||
|
|
||||||
|
with patch("tools.software_jobs.list_nodes", return_value=nodes):
|
||||||
|
diagnostic = json.loads(
|
||||||
|
SoftwareCapabilityListTool(self.user_id, self.task_id).execute(
|
||||||
|
include_diagnostic=True
|
||||||
|
)
|
||||||
|
)
|
||||||
|
inspect = next(
|
||||||
|
item for item in diagnostic["capabilities"]
|
||||||
|
if item["capability"] == "ansys.geometry.inspect@v1"
|
||||||
)
|
)
|
||||||
self.assertEqual(result["capabilities"][1]["available_nodes"], 0)
|
self.assertEqual(inspect["lifecycle"], "deprecated")
|
||||||
self.assertEqual(
|
self.assertEqual(inspect["visibility"], "diagnostic_only")
|
||||||
result["capabilities"][2]["capability"],
|
self.assertEqual(inspect["replacement"], "cad.geometry.prepare@v1")
|
||||||
"ansys.mechanical.static_structural@v2",
|
|
||||||
)
|
|
||||||
self.assertEqual(result["capabilities"][2]["available_nodes"], 0)
|
|
||||||
|
|
||||||
def test_submit_injects_current_user_and_task(self):
|
def test_submit_injects_current_user_and_task(self):
|
||||||
created = {"job_id": str(uuid4()), "status": "queued"}
|
created = {"job_id": str(uuid4()), "status": "queued"}
|
||||||
|
|
|
||||||
|
|
@ -34,6 +34,7 @@ from core.software_nodes import (
|
||||||
_verify_secret,
|
_verify_secret,
|
||||||
create_enrollment,
|
create_enrollment,
|
||||||
delete_node,
|
delete_node,
|
||||||
|
list_nodes,
|
||||||
update_node_runtime,
|
update_node_runtime,
|
||||||
)
|
)
|
||||||
from web.routers.software_nodes import NodeConnectionManager, _bearer
|
from web.routers.software_nodes import NodeConnectionManager, _bearer
|
||||||
|
|
@ -87,6 +88,8 @@ class SoftwareNodeSecurityTests(unittest.TestCase):
|
||||||
"ansys.geometry.inspect@v1",
|
"ansys.geometry.inspect@v1",
|
||||||
"ansys.mechanical.static_structural@v2",
|
"ansys.mechanical.static_structural@v2",
|
||||||
"blender.scene.author@v3",
|
"blender.scene.author@v3",
|
||||||
|
"cad.geometry.prepare@v1",
|
||||||
|
"freecad.model.author@v1",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.assertEqual(set(result["capabilities"]), set(enrollment.allowed_capabilities))
|
self.assertEqual(set(result["capabilities"]), set(enrollment.allowed_capabilities))
|
||||||
|
|
@ -122,6 +125,30 @@ class SoftwareNodeSecurityTests(unittest.TestCase):
|
||||||
)
|
)
|
||||||
self.assertIsNotNone(node.last_seen_at)
|
self.assertIsNotNone(node.last_seen_at)
|
||||||
|
|
||||||
|
@patch("core.software_nodes.session_scope")
|
||||||
|
def test_admin_node_list_includes_active_job_count(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
node_id = uuid4()
|
||||||
|
node = type("Node", (), {
|
||||||
|
"node_id": node_id,
|
||||||
|
"name": "绘图节点",
|
||||||
|
"status": "online",
|
||||||
|
"capabilities": ["origin.plot@v2"],
|
||||||
|
"node_version": "0.2.0",
|
||||||
|
"os_version": "Windows",
|
||||||
|
"runtime": {},
|
||||||
|
"last_seen_at": datetime.now(timezone.utc),
|
||||||
|
})()
|
||||||
|
node_result = MagicMock()
|
||||||
|
node_result.scalars.return_value.all.return_value = [node]
|
||||||
|
count_result = MagicMock()
|
||||||
|
count_result.all.return_value = [(node_id, 1)]
|
||||||
|
session.execute.side_effect = [node_result, count_result]
|
||||||
|
|
||||||
|
result = list_nodes()
|
||||||
|
|
||||||
|
self.assertEqual(result[0]["active_job_count"], 1)
|
||||||
|
|
||||||
|
|
||||||
class SoftwareNodeConnectionTests(unittest.IsolatedAsyncioTestCase):
|
class SoftwareNodeConnectionTests(unittest.IsolatedAsyncioTestCase):
|
||||||
async def test_new_connection_replaces_old_without_removing_new(self) -> None:
|
async def test_new_connection_replaces_old_without_removing_new(self) -> None:
|
||||||
|
|
@ -799,6 +826,11 @@ class SoftwareJobProtocolTests(unittest.TestCase):
|
||||||
self.assertEqual(offer["node_id"], node.node_id)
|
self.assertEqual(offer["node_id"], node.node_id)
|
||||||
self.assertEqual(offer["payload"]["job_id"], str(jobs[1].job_id))
|
self.assertEqual(offer["payload"]["job_id"], str(jobs[1].job_id))
|
||||||
self.assertEqual(offer["payload"]["request"]["operation"]["plot"]["type"], "line")
|
self.assertEqual(offer["payload"]["request"]["operation"]["plot"]["type"], "line")
|
||||||
|
self.assertEqual(
|
||||||
|
offer["payload"]["request_summary"]["display_name"],
|
||||||
|
"Origin 科研绘图",
|
||||||
|
)
|
||||||
|
self.assertIn("title", offer["payload"]["request_summary"])
|
||||||
|
|
||||||
def test_input_download_rechecks_file_digest(self) -> None:
|
def test_input_download_rechecks_file_digest(self) -> None:
|
||||||
source = (
|
source = (
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,8 @@ class StaticVendorTests(unittest.TestCase):
|
||||||
self.assertIn('id="right-tab-jobs"', html)
|
self.assertIn('id="right-tab-jobs"', html)
|
||||||
self.assertIn('id="right-jobs-panel"', html)
|
self.assertIn('id="right-jobs-panel"', html)
|
||||||
self.assertIn('id="software-job-badge"', html)
|
self.assertIn('id="software-job-badge"', html)
|
||||||
|
self.assertIn(".workspace-badge[hidden]", html)
|
||||||
|
self.assertIn(".sj-entry-dot.succeeded, .sj-entry-dot.failed", html)
|
||||||
self.assertIn(".right-workspace-panel {", html)
|
self.assertIn(".right-workspace-panel {", html)
|
||||||
self.assertLess(
|
self.assertLess(
|
||||||
html.index('id="right-tab-jobs"'),
|
html.index('id="right-tab-jobs"'),
|
||||||
|
|
@ -50,6 +52,7 @@ class StaticVendorTests(unittest.TestCase):
|
||||||
self.assertIn("续接自 Job ${sourceJobId.slice(0, 8)}", source)
|
self.assertIn("续接自 Job ${sourceJobId.slice(0, 8)}", source)
|
||||||
self.assertIn("<dt>来源 Job</dt>", source)
|
self.assertIn("<dt>来源 Job</dt>", source)
|
||||||
self.assertIn("refreshCurrentTaskAfterSoftwareJob(job.task_id)", source)
|
self.assertIn("refreshCurrentTaskAfterSoftwareJob(job.task_id)", source)
|
||||||
|
self.assertNotIn('else if (active.length) entryDot.classList.add("active")', source)
|
||||||
self.assertIn("job.execution_runtime", source)
|
self.assertIn("job.execution_runtime", source)
|
||||||
self.assertIn("执行版本未记录", source)
|
self.assertIn("执行版本未记录", source)
|
||||||
self.assertIn("执行版本</dt><dd>${escapeHtml(jobRuntimeText(job))}", source)
|
self.assertIn("执行版本</dt><dd>${escapeHtml(jobRuntimeText(job))}", source)
|
||||||
|
|
@ -69,7 +72,7 @@ class StaticVendorTests(unittest.TestCase):
|
||||||
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")
|
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")
|
||||||
|
|
||||||
self.assertIn('id="node-enrollment-modal" class="modal"', html)
|
self.assertIn('id="node-enrollment-modal" class="modal"', html)
|
||||||
self.assertIn("生成 Windows Node 注册码", html)
|
self.assertIn("生成专业软件节点注册码", html)
|
||||||
self.assertIn('"/v1/admin/software-node-enrollments"', admin_js)
|
self.assertIn('"/v1/admin/software-node-enrollments"', admin_js)
|
||||||
self.assertNotIn('capabilities: ["origin.plot@v2"]', admin_js)
|
self.assertNotIn('capabilities: ["origin.plot@v2"]', admin_js)
|
||||||
self.assertIn("节点能力由客户端自动发现并上报", html)
|
self.assertIn("节点能力由客户端自动发现并上报", html)
|
||||||
|
|
@ -87,10 +90,80 @@ class StaticVendorTests(unittest.TestCase):
|
||||||
self.assertIn('apiSend("PATCH", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
|
self.assertIn('apiSend("PATCH", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
|
||||||
self.assertIn('apiSend("DELETE", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
|
self.assertIn('apiSend("DELETE", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
|
||||||
self.assertIn("最近心跳", admin_js)
|
self.assertIn("最近心跳", admin_js)
|
||||||
|
self.assertIn("活动任务", admin_js)
|
||||||
|
self.assertIn("node.active_job_count", admin_js)
|
||||||
self.assertIn("重新启用", admin_js)
|
self.assertIn("重新启用", admin_js)
|
||||||
self.assertIn("永久删除", admin_js)
|
self.assertIn("永久删除", admin_js)
|
||||||
self.assertNotIn("localStorage.setItem", admin_js)
|
self.assertNotIn("localStorage.setItem", admin_js)
|
||||||
|
|
||||||
|
def test_admin_uses_compact_overview_and_grouped_workspace_tabs(self) -> None:
|
||||||
|
html = ADMIN_HTML.read_text(encoding="utf-8")
|
||||||
|
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
self.assertIn('id="ops-summary" class="ops-summary"', admin_js)
|
||||||
|
self.assertIn(".ops-summary {", html)
|
||||||
|
self.assertIn('class="summary-primary"', admin_js)
|
||||||
|
self.assertIn('class="summary-secondary"', admin_js)
|
||||||
|
labels = (
|
||||||
|
"任务运行",
|
||||||
|
"执行容量",
|
||||||
|
"专业软件节点",
|
||||||
|
"工具健康",
|
||||||
|
"活跃用户",
|
||||||
|
"Token 与缓存",
|
||||||
|
"成本",
|
||||||
|
"存储",
|
||||||
|
)
|
||||||
|
positions = []
|
||||||
|
for label in labels:
|
||||||
|
self.assertIn(f'label: "{label}"', admin_js)
|
||||||
|
positions.append(admin_js.index(f'label: "{label}"'))
|
||||||
|
self.assertEqual(positions, sorted(positions))
|
||||||
|
self.assertIn("function renderRuntimeDetails()", admin_js)
|
||||||
|
self.assertIn("function renderCapacitySection()", admin_js)
|
||||||
|
self.assertIn("function renderUsageSection(d)", admin_js)
|
||||||
|
for tab in ("运行", "用量", "用户", "资源", "集成", "质量"):
|
||||||
|
self.assertIn(f'"{tab}"', admin_js)
|
||||||
|
self.assertIn('class="workspace-tabs"', admin_js)
|
||||||
|
self.assertIn('role="tabpanel"', admin_js)
|
||||||
|
self.assertIn('tab: "run", target: "s-runtime"', admin_js)
|
||||||
|
self.assertIn('tab: "run", target: "s-sandbox-capacity"', admin_js)
|
||||||
|
self.assertIn('tab: "resources", target: "s-windows-node"', admin_js)
|
||||||
|
self.assertIn('tab: "quality", target: "s-toolfail"', admin_js)
|
||||||
|
self.assertIn('tab: "users", target: "s-users"', admin_js)
|
||||||
|
self.assertIn('tab: "usage", target: "s-usage"', admin_js)
|
||||||
|
self.assertIn('tab: "resources", target: "s-storage"', admin_js)
|
||||||
|
self.assertIn('unit: softwareNodesLoaded ? "在线 / 总数"', admin_js)
|
||||||
|
self.assertIn('value: fmtTokens(tokens7d), unit: "近 7 天"', admin_js)
|
||||||
|
self.assertIn("storageTotals.physical_bytes", admin_js)
|
||||||
|
self.assertIn("repeat(auto-fit, minmax(190px, 1fr))", html)
|
||||||
|
self.assertIn('apiGet("/v1/admin/storage/users?page=0&page_size=1")', admin_js)
|
||||||
|
self.assertIn('id="s-sandbox-capacity"', admin_js)
|
||||||
|
self.assertIn('id="s-sandbox-packages"', admin_js)
|
||||||
|
self.assertIn('sandboxCapacityData = await apiGet("/v1/admin/sandbox/capacity")', admin_js)
|
||||||
|
self.assertNotIn("containerStatusHTML(capacity)", admin_js)
|
||||||
|
self.assertIn('data-summary-tab="${escapeHtml(tab)}"', admin_js)
|
||||||
|
self.assertIn("function renderCapacityDrawer()", admin_js)
|
||||||
|
self.assertIn("function openCapacityDrawer()", admin_js)
|
||||||
|
self.assertIn("const TOOL_HEALTH_REFRESH_MS = 90000", admin_js)
|
||||||
|
self.assertIn("if (toolFailuresRequest) return toolFailuresRequest", admin_js)
|
||||||
|
self.assertIn("loadToolFailures();", admin_js)
|
||||||
|
self.assertNotIn('activeTab === "quality"', admin_js)
|
||||||
|
self.assertIn('item.category === "failure"', admin_js)
|
||||||
|
self.assertIn('item.category === "agent_control"', admin_js)
|
||||||
|
self.assertIn("代理控制与上下文异常(近 24h)", admin_js)
|
||||||
|
self.assertIn('id="capacity-drawer" class="capacity-drawer"', html)
|
||||||
|
self.assertIn(".capacity-drawer-panel {", html)
|
||||||
|
|
||||||
|
def test_changelog_loads_incremental_pages(self) -> None:
|
||||||
|
source = (JS_DIR / "changelog.js").read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
self.assertIn("const PAGE_SIZE = 5", source)
|
||||||
|
self.assertIn("offset=${_nextOffset}", source)
|
||||||
|
self.assertIn("d.has_more", source)
|
||||||
|
self.assertIn("加载更多", source)
|
||||||
|
self.assertNotIn("/v1/changelog?limit=50", source)
|
||||||
|
|
||||||
def test_dev_html_uses_local_markdown_vendor_assets(self) -> None:
|
def test_dev_html_uses_local_markdown_vendor_assets(self) -> None:
|
||||||
html = DEV_HTML.read_text(encoding="utf-8")
|
html = DEV_HTML.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
@ -140,6 +213,17 @@ class StaticVendorTests(unittest.TestCase):
|
||||||
self.assertIn(":focus-visible", css)
|
self.assertIn(":focus-visible", css)
|
||||||
self.assertIn("@media (max-width: 640px)", css)
|
self.assertIn("@media (max-width: 640px)", css)
|
||||||
|
|
||||||
|
def test_admin_entry_lives_in_main_conversation_panel(self) -> None:
|
||||||
|
html = DEV_HTML.read_text(encoding="utf-8")
|
||||||
|
header = html.split("<header>", 1)[1].split("</header>", 1)[0]
|
||||||
|
conversation_head = html.split('id="pane-mid"', 1)[1].split(
|
||||||
|
'id="chat-meta"', 1
|
||||||
|
)[0]
|
||||||
|
|
||||||
|
self.assertNotIn('id="hd-admin"', header)
|
||||||
|
self.assertIn('id="hd-admin"', conversation_head)
|
||||||
|
self.assertIn('href="/static/admin.html"', conversation_head)
|
||||||
|
|
||||||
def test_file_toolbar_keeps_hidden_toggle_and_groups_add_actions(self) -> None:
|
def test_file_toolbar_keeps_hidden_toggle_and_groups_add_actions(self) -> None:
|
||||||
html = DEV_HTML.read_text(encoding="utf-8")
|
html = DEV_HTML.read_text(encoding="utf-8")
|
||||||
files_js = (JS_DIR / "files.js").read_text(encoding="utf-8")
|
files_js = (JS_DIR / "files.js").read_text(encoding="utf-8")
|
||||||
|
|
@ -191,6 +275,14 @@ class StaticVendorTests(unittest.TestCase):
|
||||||
self.assertNotIn('<div class="role">科研助手</div>', frontend)
|
self.assertNotIn('<div class="role">科研助手</div>', frontend)
|
||||||
self.assertIn(".tool-call.activity > summary", css)
|
self.assertIn(".tool-call.activity > summary", css)
|
||||||
self.assertIn("#task-progress-dock .task-progress", css)
|
self.assertIn("#task-progress-dock .task-progress", css)
|
||||||
|
self.assertIn(".conversation-rail-caption", html)
|
||||||
|
self.assertIn('id="conversation-side-rail"', html)
|
||||||
|
self.assertLess(html.index('id="msg-outline-rail"'), html.index('id="software-result-rail"'))
|
||||||
|
self.assertIn(".rail-count.attention", html)
|
||||||
|
self.assertIn("toggleConversationRailPinned", frontend)
|
||||||
|
self.assertIn("syncConversationSideRail", frontend)
|
||||||
|
self.assertIn("点击固定展开或收起导航", frontend)
|
||||||
|
self.assertIn("个新结果", frontend)
|
||||||
|
|
||||||
def test_frontend_surfaces_context_and_cache_stats(self) -> None:
|
def test_frontend_surfaces_context_and_cache_stats(self) -> None:
|
||||||
src = _frontend_source()
|
src = _frontend_source()
|
||||||
|
|
@ -211,6 +303,13 @@ class StaticVendorTests(unittest.TestCase):
|
||||||
self.assertIn("await yieldToStreamPaint()", chat_js)
|
self.assertIn("await yieldToStreamPaint()", chat_js)
|
||||||
self.assertIn('document.visibilityState === "hidden"', chat_js)
|
self.assertIn('document.visibilityState === "hidden"', chat_js)
|
||||||
|
|
||||||
|
def test_sse_consumer_resets_abandoned_reasoning_stream(self) -> None:
|
||||||
|
chat_js = (JS_DIR / "chat.js").read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
self.assertIn("function resetReasonSeg(run)", chat_js)
|
||||||
|
self.assertIn('t === "reasoning_reset"', chat_js)
|
||||||
|
self.assertIn("resetReasonSeg(ctx)", chat_js)
|
||||||
|
|
||||||
def test_custom_task_name_is_optional_with_explicit_directory(self) -> None:
|
def test_custom_task_name_is_optional_with_explicit_directory(self) -> None:
|
||||||
html = DEV_HTML.read_text(encoding="utf-8")
|
html = DEV_HTML.read_text(encoding="utf-8")
|
||||||
newtask_js = (JS_DIR / "newtask.js").read_text(encoding="utf-8")
|
newtask_js = (JS_DIR / "newtask.js").read_text(encoding="utf-8")
|
||||||
|
|
|
||||||
|
|
@ -32,6 +32,27 @@ class StorageMigrationTests(unittest.TestCase):
|
||||||
self.assertIn("ix_scheduled_jobs_due_active", rendered)
|
self.assertIn("ix_scheduled_jobs_due_active", rendered)
|
||||||
self.assertIn("USING brin", rendered)
|
self.assertIn("USING brin", rendered)
|
||||||
|
|
||||||
|
def test_0040_upgrade_compiles_health_event_indexes(self) -> None:
|
||||||
|
statements: list[str] = []
|
||||||
|
|
||||||
|
def capture(sql, *multiparams, **params):
|
||||||
|
statements.append(str(sql.compile(dialect=postgresql.dialect())))
|
||||||
|
|
||||||
|
engine = create_mock_engine("postgresql+psycopg://", capture)
|
||||||
|
operations = Operations(MigrationContext.configure(engine.connect()))
|
||||||
|
migration = importlib.import_module(
|
||||||
|
"db.migrations.versions.20260903_1400_0040_tool_health_indexes"
|
||||||
|
)
|
||||||
|
with patch.object(migration, "op", operations):
|
||||||
|
migration.upgrade()
|
||||||
|
|
||||||
|
rendered = "\n".join(statements)
|
||||||
|
self.assertIn("ix_usage_health_kind_created", rendered)
|
||||||
|
self.assertIn("ux_usage_tool_failure_message", rendered)
|
||||||
|
self.assertIn("tool_failure", rendered)
|
||||||
|
self.assertIn("agent_guard", rendered)
|
||||||
|
self.assertIn("context_fold_failure", rendered)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
|
||||||
|
|
@ -75,6 +75,15 @@ class TestSystemPromptPaths(unittest.TestCase):
|
||||||
self.assertIn(str(wd), prompt)
|
self.assertIn(str(wd), prompt)
|
||||||
self.assertNotIn("/workspace/数据资源展示", prompt)
|
self.assertNotIn("/workspace/数据资源展示", prompt)
|
||||||
|
|
||||||
|
def test_mermaid_visual_guidance_is_backend_independent(self):
|
||||||
|
for backend in ("docker", "host"):
|
||||||
|
with self.subTest(backend=backend):
|
||||||
|
prompt, _, _ = _build(backend)
|
||||||
|
self.assertIn("## Mermaid 视觉规范", prompt)
|
||||||
|
self.assertIn("`classDef` 按角色分组", prompt)
|
||||||
|
self.assertIn("不要输出全灰单色图", prompt)
|
||||||
|
self.assertIn("用户指定品牌色", prompt)
|
||||||
|
|
||||||
def test_rename_guard_only_when_tool_is_available(self):
|
def test_rename_guard_only_when_tool_is_available(self):
|
||||||
interactive, _, _ = _build("docker", allow_rename=True)
|
interactive, _, _ = _build("docker", allow_rename=True)
|
||||||
scheduled, _, _ = _build("docker", allow_rename=False)
|
scheduled, _, _ = _build("docker", allow_rename=False)
|
||||||
|
|
|
||||||
|
|
@ -11,36 +11,67 @@ from unittest.mock import patch
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
import core.toolfail as tf # noqa: E402
|
import core.toolfail as tf # noqa: E402
|
||||||
|
from core.storage.telemetry import ( # noqa: E402
|
||||||
|
KIND_AGENT_GUARD,
|
||||||
|
KIND_EMPTY_RESPONSE,
|
||||||
|
KIND_RUN_ERROR,
|
||||||
|
KIND_TOOL_FAILURE,
|
||||||
|
KIND_TOOL_MALFORMED,
|
||||||
|
)
|
||||||
|
from core.tool_failure import structured_failure # noqa: E402
|
||||||
|
|
||||||
NOW = datetime.now(timezone.utc)
|
NOW = datetime.now(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
class _FakeSession:
|
class _FakeSession:
|
||||||
"""按 SQL 关键字分流四段查询:messages / tool_malformed / run_error / empty_response。"""
|
"""工具健康只允许一次 usage_events 查询。"""
|
||||||
|
|
||||||
def __init__(self, msg_rows, malformed_rows, run_rows, empty_rows):
|
def __init__(self, rows):
|
||||||
self._msg_rows = msg_rows
|
self._rows = rows
|
||||||
self._malformed_rows = malformed_rows
|
self.calls = 0
|
||||||
self._run_rows = run_rows
|
|
||||||
self._empty_rows = empty_rows
|
|
||||||
|
|
||||||
def execute(self, clause, params=None):
|
def execute(self, clause, params=None):
|
||||||
sql = str(clause)
|
sql = str(clause)
|
||||||
if "tool_malformed" in sql:
|
self.calls += 1
|
||||||
rows = self._malformed_rows
|
if "from usage_events" not in sql or "from messages" in sql:
|
||||||
elif "run_error" in sql:
|
raise AssertionError(sql)
|
||||||
rows = self._run_rows
|
return SimpleNamespace(fetchall=lambda: self._rows)
|
||||||
elif "empty_response" in sql:
|
|
||||||
rows = self._empty_rows
|
|
||||||
else:
|
def _structured_row(row):
|
||||||
rows = self._msg_rows
|
task_id, user_id, created_at, tool, content = row[:5]
|
||||||
return SimpleNamespace(fetchall=lambda: rows)
|
arguments = None
|
||||||
|
if len(row) > 6 and isinstance(row[6], dict):
|
||||||
|
calls = row[6].get("tool_calls") or []
|
||||||
|
if calls:
|
||||||
|
arguments = (calls[0].get("function") or {}).get("arguments")
|
||||||
|
units = structured_failure(tool, content, arguments=arguments)
|
||||||
|
assert units is not None
|
||||||
|
return (
|
||||||
|
KIND_TOOL_FAILURE, task_id, user_id, created_at, None, tool,
|
||||||
|
None, None, None, units["failure_kind"], units["signature"],
|
||||||
|
units["category"], units["sample"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _scan(msg_rows, malformed_rows, run_rows=None, empty_rows=None, **kw):
|
def _scan(msg_rows, malformed_rows, run_rows=None, empty_rows=None, **kw):
|
||||||
|
event_rows = [_structured_row(row) for row in msg_rows]
|
||||||
|
event_rows.extend((
|
||||||
|
KIND_TOOL_MALFORMED, task_id, user_id, created_at, None, tool,
|
||||||
|
err, head, tail, None, None, None, None,
|
||||||
|
) for task_id, user_id, created_at, tool, err, head, tail in malformed_rows)
|
||||||
|
event_rows.extend((
|
||||||
|
KIND_RUN_ERROR, task_id, user_id, created_at, None, None,
|
||||||
|
err, None, None, None, None, None, None,
|
||||||
|
) for task_id, user_id, created_at, err in (run_rows or []))
|
||||||
|
event_rows.extend((
|
||||||
|
KIND_EMPTY_RESPONSE, task_id, user_id, created_at, model_profile, None,
|
||||||
|
None, None, None, None, None, None, None,
|
||||||
|
) for task_id, user_id, created_at, model_profile in (empty_rows or []))
|
||||||
|
|
||||||
@contextmanager
|
@contextmanager
|
||||||
def fake_scope():
|
def fake_scope():
|
||||||
yield _FakeSession(msg_rows, malformed_rows, run_rows or [], empty_rows or [])
|
yield _FakeSession(event_rows)
|
||||||
|
|
||||||
with patch.object(tf, "session_scope", fake_scope):
|
with patch.object(tf, "session_scope", fake_scope):
|
||||||
return tf.scan_tool_failures(**kw)
|
return tf.scan_tool_failures(**kw)
|
||||||
|
|
@ -73,8 +104,8 @@ class TestToolfailMalformed(unittest.TestCase):
|
||||||
out = _scan([], rows, days=7, min_count=2, min_tasks=2)
|
out = _scan([], rows, days=7, min_count=2, min_tasks=2)
|
||||||
self.assertEqual(len(out), 1)
|
self.assertEqual(len(out), 1)
|
||||||
|
|
||||||
def test_merges_with_message_clusters(self):
|
def test_merges_structured_and_malformed_clusters(self):
|
||||||
"""两段数据源并存互不干扰:messages 报错与 malformed 各出各的 cluster。"""
|
"""同表不同 kind 并存互不干扰:普通失败与 malformed 各出一组。"""
|
||||||
msg_rows = [
|
msg_rows = [
|
||||||
("t1", "u1", NOW, "glob", "[Error] base path not found"),
|
("t1", "u1", NOW, "glob", "[Error] base path not found"),
|
||||||
("t2", "u2", NOW, "glob", "[Error] base path not found"),
|
("t2", "u2", NOW, "glob", "[Error] base path not found"),
|
||||||
|
|
@ -184,6 +215,25 @@ class TestToolfailEmptyResponse(unittest.TestCase):
|
||||||
self.assertEqual(out, [])
|
self.assertEqual(out, [])
|
||||||
|
|
||||||
|
|
||||||
|
class TestStructuredHealthEvents(unittest.TestCase):
|
||||||
|
def test_agent_control_is_separate_from_operational_failure(self):
|
||||||
|
rows = [(
|
||||||
|
KIND_AGENT_GUARD, f"t{i}", "u1", NOW, "deepseek_v4.pro", "edit",
|
||||||
|
None, None, None, "guard", "same_error: old_str not found",
|
||||||
|
"agent_control", "same_error, count=4",
|
||||||
|
) for i in range(2)]
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def fake_scope():
|
||||||
|
yield _FakeSession(rows)
|
||||||
|
|
||||||
|
with patch.object(tf, "session_scope", fake_scope):
|
||||||
|
out = tf.scan_tool_failures(days=1, min_count=2, min_tasks=2)
|
||||||
|
self.assertEqual(len(out), 1)
|
||||||
|
self.assertEqual(out[0]["category"], "agent_control")
|
||||||
|
self.assertEqual(out[0]["kind"], "guard")
|
||||||
|
|
||||||
|
|
||||||
class TestToolWireHealth(unittest.TestCase):
|
class TestToolWireHealth(unittest.TestCase):
|
||||||
def test_aggregates_recovery_rates_and_totals(self):
|
def test_aggregates_recovery_rates_and_totals(self):
|
||||||
rows = [
|
rows = [
|
||||||
|
|
@ -226,6 +276,34 @@ class TestToolWireHealth(unittest.TestCase):
|
||||||
self.assertIsNone(out["total"]["recovery_rate_24h"])
|
self.assertIsNone(out["total"]["recovery_rate_24h"])
|
||||||
|
|
||||||
|
|
||||||
|
class TestToolHealthCache(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
tf._clear_tool_health_cache()
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
tf._clear_tool_health_cache()
|
||||||
|
|
||||||
|
def test_failure_scan_is_reused_within_ttl(self):
|
||||||
|
expected = [{"tool": "shell"}]
|
||||||
|
with patch.object(tf, "scan_tool_failures", return_value=expected) as scan:
|
||||||
|
first = tf.scan_tool_failures_cached(days=7, min_count=3, min_tasks=1)
|
||||||
|
second = tf.scan_tool_failures_cached(days=7, min_count=3, min_tasks=1)
|
||||||
|
|
||||||
|
self.assertIs(first, expected)
|
||||||
|
self.assertIs(second, expected)
|
||||||
|
scan.assert_called_once_with(days=7, min_count=3, min_tasks=1)
|
||||||
|
|
||||||
|
def test_wire_scan_normalizes_window_before_caching(self):
|
||||||
|
expected = {"days": 90, "rows": [], "total": {}}
|
||||||
|
with patch.object(tf, "scan_tool_wire_health", return_value=expected) as scan:
|
||||||
|
first = tf.scan_tool_wire_health_cached(days=365)
|
||||||
|
second = tf.scan_tool_wire_health_cached(days=90)
|
||||||
|
|
||||||
|
self.assertIs(first, expected)
|
||||||
|
self.assertIs(second, expected)
|
||||||
|
scan.assert_called_once_with(days=90)
|
||||||
|
|
||||||
|
|
||||||
class TestProviderCriticalAlert(unittest.TestCase):
|
class TestProviderCriticalAlert(unittest.TestCase):
|
||||||
def setUp(self):
|
def setUp(self):
|
||||||
tf._alerted_at.clear()
|
tf._alerted_at.clear()
|
||||||
|
|
|
||||||
|
|
@ -194,8 +194,16 @@ class TasksCrudTests(unittest.TestCase):
|
||||||
tid = _client.post("/v1/tasks", json={"name": "状态闸任务"}, headers=_AUTH).json()["task_id"]
|
tid = _client.post("/v1/tasks", json={"name": "状态闸任务"}, headers=_AUTH).json()["task_id"]
|
||||||
# idle 时 cancel → 409
|
# idle 时 cancel → 409
|
||||||
self.assertEqual(_client.post(f"/v1/tasks/{tid}/cancel", headers=_AUTH).status_code, 409)
|
self.assertEqual(_client.post(f"/v1/tasks/{tid}/cancel", headers=_AUTH).status_code, 409)
|
||||||
# running 时 clear → 409;回 idle 后 clear → 200 且归零
|
# running 时 cancel 可重试:第一次切 cancelling,第二次幂等补发信号。
|
||||||
_set_run_status(tid, "running")
|
_set_run_status(tid, "running")
|
||||||
|
with patch("web.routers.messages.broker.request_cancel") as request_cancel:
|
||||||
|
first = _client.post(f"/v1/tasks/{tid}/cancel", headers=_AUTH)
|
||||||
|
second = _client.post(f"/v1/tasks/{tid}/cancel", headers=_AUTH)
|
||||||
|
self.assertEqual(first.status_code, 202, first.text)
|
||||||
|
self.assertEqual(second.status_code, 202, second.text)
|
||||||
|
self.assertEqual(request_cancel.call_count, 2)
|
||||||
|
|
||||||
|
# cancelling 时 clear → 409;回 idle 后 clear → 200 且归零
|
||||||
self.assertEqual(_client.post(f"/v1/tasks/{tid}/clear", headers=_AUTH).status_code, 409)
|
self.assertEqual(_client.post(f"/v1/tasks/{tid}/clear", headers=_AUTH).status_code, 409)
|
||||||
_set_run_status(tid, "idle")
|
_set_run_status(tid, "idle")
|
||||||
d = _client.post(f"/v1/tasks/{tid}/clear", headers=_AUTH).json()
|
d = _client.post(f"/v1/tasks/{tid}/clear", headers=_AUTH).json()
|
||||||
|
|
|
||||||
|
|
@ -73,11 +73,17 @@ class PublicEndpointTests(unittest.TestCase):
|
||||||
self.assertIn("brand", body)
|
self.assertIn("brand", body)
|
||||||
|
|
||||||
def test_changelog_public(self):
|
def test_changelog_public(self):
|
||||||
r = _client.get("/v1/changelog?limit=2")
|
r = _client.get("/v1/changelog?limit=2&offset=0")
|
||||||
self.assertEqual(r.status_code, 200)
|
self.assertEqual(r.status_code, 200)
|
||||||
self.assertIn("entries", r.json())
|
first = r.json()
|
||||||
|
self.assertEqual(len(first["entries"]), 2)
|
||||||
|
self.assertGreaterEqual(first["total"], 2)
|
||||||
|
self.assertEqual(first["next_offset"], 2)
|
||||||
|
second = _client.get("/v1/changelog?limit=2&offset=2").json()
|
||||||
|
self.assertNotEqual(first["entries"][0]["version"], second["entries"][0]["version"])
|
||||||
# limit 越界 clamp,不 500
|
# limit 越界 clamp,不 500
|
||||||
self.assertEqual(_client.get("/v1/changelog?limit=9999").status_code, 200)
|
self.assertEqual(_client.get("/v1/changelog?limit=9999").status_code, 200)
|
||||||
|
self.assertEqual(_client.get("/v1/changelog?offset=-1").status_code, 200)
|
||||||
|
|
||||||
def test_root_redirects_to_dev_spa(self):
|
def test_root_redirects_to_dev_spa(self):
|
||||||
r = _client.get("/", follow_redirects=False)
|
r = _client.get("/", follow_redirects=False)
|
||||||
|
|
@ -111,6 +117,12 @@ class AuthGateTests(unittest.TestCase):
|
||||||
("POST", "/v1/tasks"),
|
("POST", "/v1/tasks"),
|
||||||
("POST", "/v1/asr/transcribe"),
|
("POST", "/v1/asr/transcribe"),
|
||||||
("GET", "/v1/admin/overview"),
|
("GET", "/v1/admin/overview"),
|
||||||
|
("GET", "/v1/admin/provider-credentials"),
|
||||||
|
("PUT", "/v1/admin/provider-credentials/deepseek"),
|
||||||
|
("POST", "/v1/admin/provider-credentials/deepseek/test"),
|
||||||
|
("DELETE", "/v1/admin/provider-credentials/deepseek"),
|
||||||
|
("GET", "/v1/admin/sandbox/capacity"),
|
||||||
|
("GET", "/v1/admin/sandbox/packages"),
|
||||||
("GET", "/v1/admin/software-nodes"),
|
("GET", "/v1/admin/software-nodes"),
|
||||||
("GET", "/v1/software-jobs"),
|
("GET", "/v1/software-jobs"),
|
||||||
("POST", "/v1/software-jobs/00000000-0000-0000-0000-000000000000/cancel"),
|
("POST", "/v1/software-jobs/00000000-0000-0000-0000-000000000000/cancel"),
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,30 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from web.sinks import WebEventSink
|
||||||
|
|
||||||
|
|
||||||
|
class _Broker:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.events: list[tuple] = []
|
||||||
|
|
||||||
|
def emit(self, task_id, event) -> None:
|
||||||
|
self.events.append((task_id, event))
|
||||||
|
|
||||||
|
|
||||||
|
class WebEventSinkTests(unittest.TestCase):
|
||||||
|
def test_done_is_deferred_to_worker_close(self) -> None:
|
||||||
|
broker = _Broker()
|
||||||
|
task_id = uuid4()
|
||||||
|
sink = WebEventSink(broker, task_id)
|
||||||
|
|
||||||
|
sink.emit({"type": "text", "delta": "ok"})
|
||||||
|
sink.emit({"type": "done"})
|
||||||
|
|
||||||
|
self.assertEqual(broker.events, [(task_id, {"type": "text", "delta": "ok"})])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -9,17 +9,28 @@ PROJECT = ROOT / "Zcbot.WindowsNode"
|
||||||
|
|
||||||
|
|
||||||
class WindowsNodeSourceTests(unittest.TestCase):
|
class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
def test_project_targets_net10_windows_forms_with_json_schema_validator(self) -> None:
|
def test_project_targets_net10_wpf_with_windows_forms_interop(self) -> None:
|
||||||
tree = ET.parse(PROJECT / "Zcbot.WindowsNode.csproj")
|
tree = ET.parse(PROJECT / "Zcbot.WindowsNode.csproj")
|
||||||
root = tree.getroot()
|
root = tree.getroot()
|
||||||
self.assertEqual(root.findtext("./PropertyGroup/TargetFramework"), "net10.0-windows")
|
self.assertEqual(root.findtext("./PropertyGroup/TargetFramework"), "net10.0-windows")
|
||||||
self.assertEqual(root.findtext("./PropertyGroup/UseWindowsForms"), "true")
|
self.assertEqual(root.findtext("./PropertyGroup/UseWindowsForms"), "true")
|
||||||
|
self.assertEqual(root.findtext("./PropertyGroup/UseWPF"), "true")
|
||||||
self.assertEqual(root.findtext("./PropertyGroup/OutputType"), "WinExe")
|
self.assertEqual(root.findtext("./PropertyGroup/OutputType"), "WinExe")
|
||||||
|
self.assertEqual(
|
||||||
|
root.findtext("./PropertyGroup/ApplicationIcon"), "Assets\\zcbot.ico"
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
[item.attrib["Include"] for item in root.findall("./ItemGroup/Resource")],
|
||||||
|
["Assets\\zcbot.ico"],
|
||||||
|
)
|
||||||
|
icon = PROJECT / "Assets" / "zcbot.ico"
|
||||||
|
self.assertTrue(icon.is_file())
|
||||||
|
self.assertEqual(icon.read_bytes()[:6], b"\x00\x00\x01\x00\x07\x00")
|
||||||
packages = root.findall("./ItemGroup/PackageReference")
|
packages = root.findall("./ItemGroup/PackageReference")
|
||||||
self.assertEqual([item.attrib["Include"] for item in packages], ["JsonSchema.Net"])
|
self.assertEqual([item.attrib["Include"] for item in packages], ["JsonSchema.Net"])
|
||||||
|
|
||||||
def test_node_protocol_and_secret_storage_markers_are_present(self) -> None:
|
def test_node_protocol_and_secret_storage_markers_are_present(self) -> None:
|
||||||
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs"))
|
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.rglob("*.cs"))
|
||||||
for marker in (
|
for marker in (
|
||||||
"v1/software-nodes/enroll",
|
"v1/software-nodes/enroll",
|
||||||
"v1/software-nodes/connect",
|
"v1/software-nodes/connect",
|
||||||
|
|
@ -29,16 +40,129 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
"SetAccessRuleProtection(isProtected: true",
|
"SetAccessRuleProtection(isProtected: true",
|
||||||
'"adapter.json"',
|
'"adapter.json"',
|
||||||
"NotifyIcon",
|
"NotifyIcon",
|
||||||
"ConfigurationForm",
|
"MainWindow",
|
||||||
"TrayIconFactory.Create",
|
"TrayIconFactory.Create",
|
||||||
'"--headless"',
|
'"--headless"',
|
||||||
):
|
):
|
||||||
self.assertIn(marker, source)
|
self.assertIn(marker, source)
|
||||||
|
|
||||||
|
def test_gui_and_headless_share_the_composition_root(self) -> None:
|
||||||
|
program = (PROJECT / "Program.cs").read_text(encoding="utf-8")
|
||||||
|
composition = (
|
||||||
|
PROJECT / "Bootstrap" / "NodeCompositionRoot.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
controller = (
|
||||||
|
PROJECT / "Application" / "NodeApplicationController.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
self.assertEqual(program.count("new NodeCompositionRoot"), 2)
|
||||||
|
self.assertIn("CreateApplicationController()", program)
|
||||||
|
self.assertIn("RunHeadlessAsync(shutdown.Token)", program)
|
||||||
|
self.assertIn("new App(compositionRoot.CreateApplicationController())", program)
|
||||||
|
self.assertIn("return app.Run()", program)
|
||||||
|
self.assertNotIn("new TrayApplicationContext", program)
|
||||||
|
self.assertIn("new NodeConnectionLoop(config, Paths", composition)
|
||||||
|
self.assertIn("stopTask ??= StopOnceAsync(mode)", controller)
|
||||||
|
self.assertIn("await StopConnectionAsync()", controller)
|
||||||
|
|
||||||
|
def test_wpf_overview_registration_and_settings_use_view_model_commands(self) -> None:
|
||||||
|
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
view_model = (
|
||||||
|
PROJECT / "Presentation" / "ViewModels" / "MainWindowViewModel.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
startup = (PROJECT / "StartupRegistration.cs").read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
for binding in (
|
||||||
|
"ShowOverviewCommand",
|
||||||
|
"ShowSettingsCommand",
|
||||||
|
"RegisterCommand",
|
||||||
|
"ReconnectCommand",
|
||||||
|
"ResetIdentityCommand",
|
||||||
|
"IsStartupEnabled",
|
||||||
|
"DataRootPage.CurrentRoot",
|
||||||
|
):
|
||||||
|
self.assertIn(f"{{Binding {binding}", xaml)
|
||||||
|
self.assertIn('PasswordChanged="EnrollmentCodeBox_OnPasswordChanged"', xaml)
|
||||||
|
self.assertIn("CopyDiagnosticsCommand", xaml)
|
||||||
|
self.assertNotIn("OpenLegacyCommand", xaml + view_model)
|
||||||
|
self.assertNotIn("NodeApplicationController", xaml + view_model)
|
||||||
|
self.assertIn("interface IStartupRegistration", startup)
|
||||||
|
self.assertIn("RegisterRequested += RegisterFromWpfAsync", tray)
|
||||||
|
self.assertIn("ReconnectRequested += ReconnectFromWpfAsync", tray)
|
||||||
|
self.assertIn("ResetIdentityRequested += ResetIdentityFromWpfAsync", tray)
|
||||||
|
|
||||||
|
def test_wpf_software_page_uses_runtime_services_and_tray_dialogs(self) -> None:
|
||||||
|
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
page = (
|
||||||
|
PROJECT / "Presentation" / "ViewModels" / "SoftwarePageViewModel.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
service = (
|
||||||
|
PROJECT / "Presentation" / "Services" / "SoftwareManagementService.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
|
for binding in (
|
||||||
|
"ShowSoftwareCommand",
|
||||||
|
"SoftwarePage.RefreshCommand",
|
||||||
|
"SoftwarePage.CancelInstallCommand",
|
||||||
|
"SoftwarePage.Cards",
|
||||||
|
"ChooseLocationCommand",
|
||||||
|
"ClearLocationCommand",
|
||||||
|
"InstallRuntimeCommand",
|
||||||
|
):
|
||||||
|
self.assertIn(f"{{Binding {binding}", xaml)
|
||||||
|
self.assertIn("ISoftwareManagementService", page + service)
|
||||||
|
self.assertIn("SoftwareLocationService.SaveConfiguredPath", service)
|
||||||
|
self.assertIn("SoftwareLocationService.ClearConfiguredPath", service)
|
||||||
|
self.assertIn("ManagedRuntimeInstaller.InstallAsync", service)
|
||||||
|
self.assertIn("AdapterCatalog.HasActiveExecution", service)
|
||||||
|
self.assertIn("ChooseLocationRequested += ChooseSoftwareLocation", tray)
|
||||||
|
self.assertIn("ConfirmInstallRequested += ConfirmRuntimeInstall", tray)
|
||||||
|
self.assertNotIn("OpenFileDialog", page + service)
|
||||||
|
|
||||||
|
def test_wpf_theme_has_selected_navigation_status_and_responsive_actions(self) -> None:
|
||||||
|
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
theme = (
|
||||||
|
PROJECT / "Presentation" / "Themes" / "LightTheme.xaml"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
self.assertGreaterEqual(xaml.count("<ToggleButton"), 3)
|
||||||
|
self.assertIn('IsChecked="{Binding IsOverviewPage, Mode=OneWay}"', xaml)
|
||||||
|
self.assertIn('IsChecked="{Binding IsSoftwarePage, Mode=OneWay}"', xaml)
|
||||||
|
self.assertIn('IsChecked="{Binding IsSettingsPage, Mode=OneWay}"', xaml)
|
||||||
|
self.assertIn('Binding="{Binding StatusTone}"', xaml)
|
||||||
|
self.assertIn("SoftwarePage.IsBusy", xaml)
|
||||||
|
self.assertGreaterEqual(xaml.count("<WrapPanel"), 3)
|
||||||
|
self.assertIn("HasOperationMessage", xaml)
|
||||||
|
for token in (
|
||||||
|
"SidebarSelectedBrush",
|
||||||
|
"SuccessSurfaceBrush",
|
||||||
|
"WarningSurfaceBrush",
|
||||||
|
"ErrorSurfaceBrush",
|
||||||
|
"CardBorderStyle",
|
||||||
|
"SectionTitleStyle",
|
||||||
|
):
|
||||||
|
self.assertIn(token, theme)
|
||||||
|
self.assertIn('<ControlTemplate TargetType="Button">', theme)
|
||||||
|
self.assertIn('<ControlTemplate TargetType="ToggleButton">', theme)
|
||||||
|
|
||||||
def test_workspace_storage_is_global_by_workspace_not_user_directory(self) -> None:
|
def test_workspace_storage_is_global_by_workspace_not_user_directory(self) -> None:
|
||||||
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
|
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
|
||||||
store = (PROJECT / "WorkspaceStore.cs").read_text(encoding="utf-8")
|
store = (PROJECT / "WorkspaceStore.cs").read_text(encoding="utf-8")
|
||||||
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
|
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
self.assertIn('Path.Combine(root, "workspaces")', models)
|
self.assertIn('Path.Combine(root, "workspaces")', models)
|
||||||
self.assertIn('Binding(job).WorkspaceId.ToString("D")', store)
|
self.assertIn('Binding(job).WorkspaceId.ToString("D")', store)
|
||||||
self.assertNotIn("UserId", store)
|
self.assertNotIn("UserId", store)
|
||||||
|
|
@ -48,7 +172,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertIn("workspaceStore.Restore", runner)
|
self.assertIn("workspaceStore.Restore", runner)
|
||||||
|
|
||||||
def test_runtime_auto_reports_all_discovered_adapter_capabilities(self) -> None:
|
def test_runtime_auto_reports_all_discovered_adapter_capabilities(self) -> None:
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
|
||||||
config_store = (PROJECT / "NodeConfigStore.cs").read_text(encoding="utf-8")
|
config_store = (PROJECT / "NodeConfigStore.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
"var installedCapabilities = adapters.All.Select(item => item.Capability).ToArray()",
|
"var installedCapabilities = adapters.All.Select(item => item.Capability).ToArray()",
|
||||||
|
|
@ -62,27 +186,42 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
def test_node_does_not_expose_arbitrary_execution_primitives(self) -> None:
|
def test_node_does_not_expose_arbitrary_execution_primitives(self) -> None:
|
||||||
source = "\n".join(
|
source = "\n".join(
|
||||||
path.read_text(encoding="utf-8")
|
path.read_text(encoding="utf-8")
|
||||||
for path in PROJECT.glob("*.cs")
|
for path in PROJECT.rglob("*.cs")
|
||||||
if path.name not in {
|
if path.name not in {
|
||||||
"AdapterProcessRunner.cs",
|
"AdapterExecutionService.cs",
|
||||||
|
"ProcessSupervisor.cs",
|
||||||
"ConfigurationForm.cs",
|
"ConfigurationForm.cs",
|
||||||
"SoftwareRuntimeManager.cs",
|
"TrayHost.cs",
|
||||||
|
"SoftwareLocationService.cs",
|
||||||
|
"ManagedRuntimeInstaller.cs",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
for forbidden in ("Process.Start", "cmd.exe", "powershell.exe", "LabTalk"):
|
for forbidden in ("Process.Start", "cmd.exe", "powershell.exe", "LabTalk"):
|
||||||
self.assertNotIn(forbidden, source)
|
self.assertNotIn(forbidden, source)
|
||||||
|
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
|
||||||
self.assertIn('"explorer.exe"', form)
|
encoding="utf-8"
|
||||||
self.assertIn("startInfo.ArgumentList.Add(dataRoot.Text)", form)
|
)
|
||||||
self.assertNotIn("UseShellExecute = true", form)
|
self.assertIn('"explorer.exe"', tray)
|
||||||
|
self.assertIn("startInfo.ArgumentList.Add(path)", tray)
|
||||||
|
self.assertNotIn("UseShellExecute = true", tray)
|
||||||
|
|
||||||
def test_data_root_is_configurable_and_migrated_before_switching(self) -> None:
|
def test_data_root_is_configurable_and_migrated_before_switching(self) -> None:
|
||||||
settings = (PROJECT / "NodeDataRoot.cs").read_text(encoding="utf-8")
|
settings = (PROJECT / "NodeDataRoot.cs").read_text(encoding="utf-8")
|
||||||
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
|
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
page = (
|
||||||
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
|
PROJECT / "Presentation" / "ViewModels" / "DataRootPageViewModel.cs"
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
).read_text(encoding="utf-8")
|
||||||
|
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
|
||||||
|
controller = (
|
||||||
|
PROJECT / "Application" / "NodeApplicationController.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
composition = (
|
||||||
|
PROJECT / "Bootstrap" / "NodeCompositionRoot.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
|
||||||
self.assertIn('"ZCBOT_WINDOWS_NODE_DATA_DIR"', settings)
|
self.assertIn('"ZCBOT_WINDOWS_NODE_DATA_DIR"', settings)
|
||||||
self.assertIn("Environment.SpecialFolder.LocalApplicationData", settings)
|
self.assertIn("Environment.SpecialFolder.LocalApplicationData", settings)
|
||||||
|
|
@ -99,15 +238,13 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
settings.index("VerifyTree(source, staging"),
|
settings.index("VerifyTree(source, staging"),
|
||||||
settings.index("Directory.Move(staging, target)"),
|
settings.index("Directory.Move(staging, target)"),
|
||||||
)
|
)
|
||||||
self.assertIn("DataRootMigrationRequested", form)
|
self.assertIn("MigrationRequested", page)
|
||||||
self.assertIn("jobInbox.HasPendingJobs", form)
|
self.assertIn("service.HasPendingJobs", page)
|
||||||
self.assertIn(
|
self.assertIn("saveDataRoot(result.TargetDirectory)", controller)
|
||||||
"NodeDataRootSettings.SaveUserRoot(result.TargetDirectory)", tray
|
self.assertIn("NodeDataRootSettings.SaveUserRoot", composition)
|
||||||
)
|
self.assertIn("controller.MigrateDataRootAsync(target)", tray)
|
||||||
self.assertIn("jobPipelines.Values.Concat(exportPipelines.Values)", connection)
|
self.assertIn("jobPipelines.Values.Concat(exportPipelines.Values)", connection)
|
||||||
self.assertIn(
|
self.assertIn("Path.GetPathRoot(paths.RootDirectory)", connection)
|
||||||
"Path.GetPathRoot(NodePaths.ForCurrentMachine().RootDirectory)", connection
|
|
||||||
)
|
|
||||||
|
|
||||||
def test_config_field_names_do_not_serialize_plain_node_token(self) -> None:
|
def test_config_field_names_do_not_serialize_plain_node_token(self) -> None:
|
||||||
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
|
models = (PROJECT / "NodeModels.cs").read_text(encoding="utf-8")
|
||||||
|
|
@ -116,28 +253,46 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertNotIn("string NodeToken", stored_record)
|
self.assertNotIn("string NodeToken", stored_record)
|
||||||
|
|
||||||
def test_configuration_ui_never_displays_or_copies_token(self) -> None:
|
def test_configuration_ui_never_displays_or_copies_token(self) -> None:
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
|
||||||
self.assertIn("CreateTextBox(usePassword: true)", form)
|
encoding="utf-8"
|
||||||
self.assertIn("UseSystemPasswordChar = usePassword", form)
|
)
|
||||||
self.assertNotIn("NodeToken", form)
|
view_model = (
|
||||||
self.assertIn("Clipboard.SetText(BuildDiagnosticText(currentConfig))", form)
|
PROJECT / "Presentation" / "ViewModels" / "MainWindowViewModel.cs"
|
||||||
diagnostics = form.split("private string BuildDiagnosticText", 1)[1].split(
|
).read_text(encoding="utf-8")
|
||||||
"private async Task RunAnsysAcceptanceAsync", 1
|
diagnostic_service = (
|
||||||
)[0]
|
PROJECT / "Application" / "DiagnosticService.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
self.assertIn("<PasswordBox", xaml)
|
||||||
|
self.assertNotIn("NodeToken", xaml + view_model + diagnostic_service)
|
||||||
|
self.assertIn("Clipboard.SetText(text)", tray)
|
||||||
|
diagnostics = diagnostic_service.split("public string Build", 1)[1]
|
||||||
self.assertNotIn("NodeToken", diagnostics)
|
self.assertNotIn("NodeToken", diagnostics)
|
||||||
self.assertIn("内容不包含 Node Token", form)
|
self.assertIn("内容不包含 Node Token", view_model)
|
||||||
self.assertIn("清除本机身份", form)
|
self.assertIn("清除本机身份", xaml)
|
||||||
self.assertIn("管理后台删除或禁用云端旧节点", form)
|
self.assertIn("管理后台删除或禁用云端旧节点", tray)
|
||||||
|
|
||||||
def test_ansys_acceptance_ui_runs_only_fixed_bundled_suite_before_gate(self) -> None:
|
def test_ansys_acceptance_ui_runs_only_fixed_bundled_suite_before_gate(self) -> None:
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
view_model = (
|
||||||
|
PROJECT / "Presentation" / "ViewModels" / "AnsysAcceptanceViewModel.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
service = (
|
||||||
|
PROJECT / "Application" / "AnsysAcceptanceService.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
adapters = (PROJECT / "NodeAdapters.cs").read_text(encoding="utf-8")
|
adapters = (PROJECT / "NodeAdapters.cs").read_text(encoding="utf-8")
|
||||||
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
|
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
|
||||||
self.assertIn("ANSYS Mechanical 真机验收", form)
|
encoding="utf-8"
|
||||||
self.assertIn("运行内置基准验收", form)
|
)
|
||||||
self.assertIn("AcceptanceReportPassed", form)
|
self.assertIn("ANSYS Mechanical 真机验收", xaml)
|
||||||
self.assertIn('"ZCBOT_ANSYS_242_VALIDATED", "1"', form)
|
self.assertIn("运行内置基准验收", xaml)
|
||||||
self.assertIn("EnvironmentVariableTarget.Machine", form)
|
self.assertIn("ReportPassed", service)
|
||||||
|
self.assertIn('GateEnvironmentVariable,\n "1"', service)
|
||||||
|
self.assertIn("EnvironmentVariableTarget.Machine", service)
|
||||||
self.assertIn("SupportsLocalAcceptance", adapters)
|
self.assertIn("SupportsLocalAcceptance", adapters)
|
||||||
self.assertIn('Path.Combine(descriptor.DirectoryPath, "acceptance.py")', runner)
|
self.assertIn('Path.Combine(descriptor.DirectoryPath, "acceptance.py")', runner)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
|
|
@ -145,114 +300,106 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
)
|
)
|
||||||
self.assertIn('["--work-root", root, "--repeat", "20"', runner)
|
self.assertIn('["--work-root", root, "--repeat", "20"', runner)
|
||||||
self.assertIn("Adapter acceptance cannot run while a dispatched job is active", runner)
|
self.assertIn("Adapter acceptance cannot run while a dispatched job is active", runner)
|
||||||
acceptance = form.split("private async Task RunAnsysAcceptanceAsync", 1)[1].split(
|
self.assertIn("service.RunAsync(workRoot", view_model)
|
||||||
"private void EnableAnsysGate", 1
|
self.assertNotIn("new OpenFileDialog", view_model + service)
|
||||||
)[0]
|
self.assertNotIn("Process.Start", view_model + service)
|
||||||
self.assertNotIn("new OpenFileDialog", acceptance)
|
|
||||||
self.assertNotIn("Process.Start", acceptance)
|
|
||||||
|
|
||||||
def test_professional_software_ui_uses_fixed_per_runtime_installers(self) -> None:
|
def test_professional_software_ui_uses_fixed_per_runtime_installers(self) -> None:
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
|
||||||
manager = (PROJECT / "SoftwareRuntimeManager.cs").read_text(encoding="utf-8")
|
encoding="utf-8"
|
||||||
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
|
)
|
||||||
|
service = (
|
||||||
|
PROJECT / "Presentation" / "Services" / "SoftwareManagementService.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
catalog = (PROJECT / "Runtime" / "SoftwareCatalog.cs").read_text(encoding="utf-8")
|
||||||
|
manager = (PROJECT / "Runtime" / "SoftwareLocationService.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
installer = (PROJECT / "Runtime" / "ManagedRuntimeInstaller.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
self.assertIn('CreateSectionTitle("专业软件")', form)
|
self.assertIn('Text="本机专业软件"', xaml)
|
||||||
self.assertIn('CreateButton("自动检测", 92)', form)
|
self.assertIn('Content="自动检测"', xaml)
|
||||||
self.assertIn('CreateButton("选择位置", 92)', form)
|
self.assertIn('Content="选择位置"', xaml)
|
||||||
self.assertIn('CreateButton("安装 / 更新环境", 150', form)
|
self.assertIn('Content="安装 / 更新环境"', xaml)
|
||||||
self.assertIn("panel.SetColumnSpan(path, 2)", form)
|
self.assertIn('IsReadOnly="True"', xaml)
|
||||||
self.assertIn("path.ReadOnly = true", form)
|
self.assertIn("ManagedRuntimeInstaller.InstallAsync", service)
|
||||||
software_panel = form.split("private Control CreateSoftwarePanel", 1)[1].split(
|
self.assertIn("item.HasActiveJobs", service)
|
||||||
"private void RefreshSoftwareCards", 1
|
for runtime_id in ('"origin"', '"ansys"', '"blender"', '"freecad"', '"cad"'):
|
||||||
)[0]
|
self.assertIn(runtime_id, catalog)
|
||||||
self.assertIn("panel.SetColumnSpan(status, 2)", software_panel)
|
self.assertIn("SoftwarePathKind.None", catalog)
|
||||||
self.assertIn("panel.SetColumnSpan(actions, 2)", software_panel)
|
self.assertIn('"https://pypi.org/simple/"', catalog)
|
||||||
self.assertIn("SoftwareRuntimeManager.InstallRuntimeAsync", form)
|
self.assertIn("definition.DefaultIndexUrl ?? DefaultIndexUrl", installer)
|
||||||
self.assertIn("item.HasActiveJobs", form)
|
self.assertIn("runtime-only 软件不接受应用路径", manager)
|
||||||
for runtime_id in ('"origin"', '"ansys"', '"blender"'):
|
|
||||||
self.assertIn(runtime_id, manager)
|
|
||||||
self.assertIn('@"Software\\Zcbot\\WindowsNode\\Software"', manager)
|
self.assertIn('@"Software\\Zcbot\\WindowsNode\\Software"', manager)
|
||||||
self.assertIn("RegistryHive.LocalMachine", manager)
|
self.assertIn("RegistryHive.LocalMachine", manager)
|
||||||
self.assertIn("RegistryHive.CurrentUser", manager)
|
self.assertIn("RegistryHive.CurrentUser", manager)
|
||||||
self.assertIn('"ZCBOT_BLENDER_EXE"', manager)
|
self.assertIn('"ZCBOT_BLENDER_EXE"', catalog)
|
||||||
self.assertIn('"AWP_ROOT242"', manager)
|
self.assertIn('"ZCBOT_FREECAD_CMD"', catalog)
|
||||||
self.assertIn('"ZCBOT_ORIGIN_EXE"', manager)
|
self.assertIn('"FreeCADCmd.exe"', catalog)
|
||||||
self.assertIn('"Origin*.exe"', manager)
|
self.assertIn('"AWP_ROOT242"', catalog)
|
||||||
|
self.assertIn('"ZCBOT_ORIGIN_EXE"', catalog)
|
||||||
|
self.assertIn('"Origin*.exe"', catalog)
|
||||||
self.assertIn("OriginExecutables", manager)
|
self.assertIn("OriginExecutables", manager)
|
||||||
self.assertIn('Path.Combine(runtimes, definition.RuntimeId)', manager)
|
self.assertIn('Path.Combine(runtimes, definition.RuntimeId)', installer)
|
||||||
self.assertIn("ArgumentList.Add(argument)", manager)
|
self.assertIn("ArgumentList.Add(argument)", installer)
|
||||||
self.assertNotIn('"cmd.exe"', manager.lower())
|
self.assertNotIn('"cmd.exe"', installer.lower())
|
||||||
self.assertNotIn('"powershell.exe"', manager.lower())
|
self.assertNotIn('"powershell.exe"', installer.lower())
|
||||||
self.assertIn("SoftwareRuntimeManager.ApplyProcessEnvironment", runner)
|
self.assertIn("SoftwareLocationService.ApplyProcessEnvironment", runner)
|
||||||
|
self.assertIn("AdapterCatalog.HasActiveExecution", installer)
|
||||||
|
|
||||||
def test_configuration_window_is_resizable_and_dpi_safe(self) -> None:
|
def test_configuration_window_is_resizable_and_dpi_safe(self) -> None:
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
|
||||||
self.assertIn("ClientSize = new Size(1080, 1050)", form)
|
encoding="utf-8"
|
||||||
self.assertIn("MinimumSize = new Size(760, 560)", form)
|
|
||||||
self.assertIn("FormBorderStyle.Sizable", form)
|
|
||||||
self.assertIn("AutoScaleMode.Dpi", form)
|
|
||||||
self.assertIn("AutoScroll = true", form)
|
|
||||||
self.assertNotIn("MaximumSize = new Size(410", form)
|
|
||||||
self.assertIn("注册码默认 10 分钟有效", form)
|
|
||||||
self.assertIn("成功注册一次后立即失效", form)
|
|
||||||
self.assertIn("CreateSection", form)
|
|
||||||
self.assertIn("注册并连接", form)
|
|
||||||
self.assertNotIn("ContentWidth", form)
|
|
||||||
self.assertIn("new Padding(24, 20, 24, 20)", form)
|
|
||||||
self.assertIn("new ColumnStyle(SizeType.Percent, 100)", form)
|
|
||||||
self.assertIn('AddTab(tabs, "节点概览")', form)
|
|
||||||
self.assertIn('AddTab(tabs, "专业软件")', form)
|
|
||||||
self.assertIn('AddTab(tabs, "本机任务")', form)
|
|
||||||
self.assertIn('AddTab(tabs, "运行设置")', form)
|
|
||||||
self.assertIn("SizeMode = TabSizeMode.FillToRight", form)
|
|
||||||
self.assertIn("new NavigationTabControl", form)
|
|
||||||
self.assertIn("DrawMode = TabDrawMode.OwnerDrawFixed", form)
|
|
||||||
self.assertIn("Appearance = TabAppearance.FlatButtons", form)
|
|
||||||
self.assertNotIn("protected override void OnResize", form)
|
|
||||||
self.assertIn("tab.Controls.Add(content)", form)
|
|
||||||
self.assertIn("CreateSectionTitle(\"节点信息\")", form)
|
|
||||||
self.assertIn("CreateSubsectionTitle(\"可用软件\")", form)
|
|
||||||
self.assertIn("heading.Controls.Add(state, 1, 0)", form)
|
|
||||||
self.assertNotIn("headingHint", form)
|
|
||||||
self.assertIn("panel.SetColumnSpan(path, 2)", form)
|
|
||||||
self.assertIn("panel.SetColumnSpan(status, 2)", form)
|
|
||||||
self.assertIn("panel.SetColumnSpan(actions, 2)", form)
|
|
||||||
self.assertIn("BorderedTableLayoutPanel", form)
|
|
||||||
self.assertIn(
|
|
||||||
'runtime.Software == "OriginPro" ? "Origin" : runtime.Software', form
|
|
||||||
)
|
)
|
||||||
self.assertIn("FormatCapabilitySummary(adapters.All)", form)
|
self.assertIn('Width="1040"', xaml)
|
||||||
self.assertIn("GroupBy(entry => entry.SoftwareName", form)
|
self.assertIn('MinWidth="760"', xaml)
|
||||||
self.assertIn("GroupBy(entry => entry.CapabilityName", form)
|
self.assertIn('MinHeight="560"', xaml)
|
||||||
self.assertIn("ShortCapabilityName(adapter.DisplayName, softwareName)", form)
|
self.assertIn("<ScrollViewer", xaml)
|
||||||
self.assertIn('CreateButton("立即重连", 112, primary: true)', form)
|
self.assertIn('Content="注册并连接"', xaml)
|
||||||
self.assertIn("ReconnectRequested?.Invoke()", form)
|
self.assertIn('Content="节点概览"', xaml)
|
||||||
self.assertIn("registrationCard.Visible = !registered", form)
|
self.assertIn('Content="专业软件"', xaml)
|
||||||
self.assertIn("reconnect.Visible = registered", form)
|
self.assertIn('Content="本机任务"', xaml)
|
||||||
self.assertIn("resetIdentity.Visible = registered", form)
|
self.assertIn('Content="运行设置"', xaml)
|
||||||
self.assertIn('CreateSectionTitle("本机任务")', form)
|
self.assertIn("<DataGrid", xaml)
|
||||||
self.assertIn("DataGridView", form)
|
self.assertFalse((PROJECT / "ConfigurationForm.cs").exists())
|
||||||
self.assertIn("jobInbox.ReadJobSnapshots()", form)
|
self.assertFalse((PROJECT / "TrayApplicationContext.cs").exists())
|
||||||
self.assertIn('"software_running" => "软件执行中"', form)
|
|
||||||
self.assertNotIn("HttpClient", form)
|
|
||||||
|
|
||||||
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
|
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
|
||||||
self.assertIn("form.ReconnectRequested += RestartConnection", tray)
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
self.assertNotIn("legacyForm", tray)
|
||||||
self.assertIn("statusItem.Text = CompactMenuText(status.Message, 24)", tray)
|
self.assertIn("statusItem.Text = CompactMenuText(status.Message, 24)", tray)
|
||||||
self.assertIn('singleLine[..(length - 1)] + "…"', tray)
|
self.assertIn('singleLine[..(length - 1)] + "…"', tray)
|
||||||
|
|
||||||
def test_tray_exit_waits_for_or_cancels_active_jobs(self) -> None:
|
def test_tray_exit_waits_for_or_cancels_active_jobs(self) -> None:
|
||||||
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
|
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
|
||||||
|
host_loop = (PROJECT / "Host" / "NodeConnectionLoop.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
controller = (
|
||||||
|
PROJECT / "Application" / "NodeApplicationController.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
|
||||||
self.assertIn("private async Task ExitNodeAsync()", tray)
|
self.assertIn("private async Task ExitNodeAsync()", tray)
|
||||||
self.assertIn("if (exiting || config is null", tray)
|
self.assertIn("if (exiting)", tray)
|
||||||
self.assertIn("ReadJobSnapshots()", tray)
|
self.assertIn("controller.ActiveJobCount", tray)
|
||||||
self.assertIn("MessageBoxButtons.YesNoCancel", tray)
|
self.assertIn("MessageBoxButtons.YesNoCancel", tray)
|
||||||
self.assertIn("connectionLoop?.CancelActiveJobsForExit()", tray)
|
self.assertIn("NodeShutdownMode.CancelJobs", tray)
|
||||||
self.assertIn("tray.ContextMenuStrip.Enabled = false", tray)
|
self.assertIn("tray.ContextMenuStrip.Enabled = false", tray)
|
||||||
self.assertLess(tray.index("await connectionTask"), tray.index("ExitThread();"))
|
self.assertLess(tray.index("await controller.StopAsync"), tray.index("shutdown();"))
|
||||||
self.assertIn("internal void CancelActiveJobsForExit()", connection)
|
self.assertIn("stopTask ??= StopOnceAsync(mode)", controller)
|
||||||
|
self.assertIn("connection?.CancelActiveJobsForExit()", controller)
|
||||||
|
self.assertIn("await StopConnectionAsync()", controller)
|
||||||
|
self.assertIn("public void CancelActiveJobsForExit()", host_loop)
|
||||||
|
self.assertIn("coordinator.CancelActiveJobsForExit()", host_loop)
|
||||||
self.assertIn("adapters.Find(job.Capability)?.Cancel(job.JobId)", connection)
|
self.assertIn("adapters.Find(job.Capability)?.Cancel(job.JobId)", connection)
|
||||||
self.assertIn("inputDownloader.DownloadAsync(job, forcedStop.Token)", connection)
|
self.assertIn("inputDownloader.DownloadAsync(job, forcedStop.Token)", connection)
|
||||||
self.assertIn('"NODE_EXIT_CANCELLED"', connection)
|
self.assertIn('"NODE_EXIT_CANCELLED"', connection)
|
||||||
|
|
@ -260,11 +407,20 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
|
|
||||||
def test_local_job_monitor_is_persisted_and_software_neutral(self) -> None:
|
def test_local_job_monitor_is_persisted_and_software_neutral(self) -> None:
|
||||||
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
|
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
monitor = (
|
||||||
|
PROJECT / "Application" / "JobMonitorService.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
|
||||||
models = (PROJECT / "JobMonitorModels.cs").read_text(encoding="utf-8")
|
models = (PROJECT / "JobMonitorModels.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn('"state.json"', inbox)
|
repository = (PROJECT / "Jobs" / "JobRepository.cs").read_text(
|
||||||
self.assertIn("ReadJobSnapshots", inbox)
|
encoding="utf-8"
|
||||||
self.assertIn("AtomicWrite(path, content, overwrite: true)", inbox)
|
)
|
||||||
|
state_machine = (PROJECT / "Jobs" / "JobStateMachine.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
self.assertIn('"state.json"', monitor)
|
||||||
|
self.assertIn("ReadSnapshots", monitor)
|
||||||
|
self.assertIn("AtomicFile.Write(path, content, overwrite: true)", repository)
|
||||||
for stage in (
|
for stage in (
|
||||||
"accepted",
|
"accepted",
|
||||||
"downloading_inputs",
|
"downloading_inputs",
|
||||||
|
|
@ -275,26 +431,36 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
"failed",
|
"failed",
|
||||||
"cancelled",
|
"cancelled",
|
||||||
):
|
):
|
||||||
self.assertIn(f'"{stage}"', inbox + connection)
|
self.assertIn(
|
||||||
|
f'"{stage}"',
|
||||||
|
inbox + monitor + repository + state_machine + connection,
|
||||||
|
)
|
||||||
self.assertIn("JobDisplaySnapshot", models)
|
self.assertIn("JobDisplaySnapshot", models)
|
||||||
self.assertNotIn("origin_running", inbox + connection + models)
|
self.assertNotIn("origin_running", inbox + monitor + connection + models)
|
||||||
|
|
||||||
def test_startup_and_runtime_installation_are_owned_by_the_ui(self) -> None:
|
def test_startup_and_runtime_installation_are_owned_by_the_ui(self) -> None:
|
||||||
startup = (PROJECT / "StartupRegistration.cs").read_text(encoding="utf-8")
|
startup = (PROJECT / "StartupRegistration.cs").read_text(encoding="utf-8")
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
xaml = (PROJECT / "Presentation" / "MainWindow.xaml").read_text(
|
||||||
manager = (PROJECT / "SoftwareRuntimeManager.cs").read_text(encoding="utf-8")
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
view_model = (
|
||||||
|
PROJECT / "Presentation" / "ViewModels" / "MainWindowViewModel.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
installer = (PROJECT / "Runtime" / "ManagedRuntimeInstaller.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
self.assertFalse((ROOT / "install-windows-node.bat").exists())
|
self.assertFalse((ROOT / "install-windows-node.bat").exists())
|
||||||
self.assertIn('Text = "登录 Windows 后自动启动节点"', form)
|
self.assertIn('Content="登录 Windows 后自动启动节点"', xaml)
|
||||||
self.assertIn("StartupRegistration.SetEnabled", form)
|
self.assertIn("startupRegistration.SetEnabled", view_model)
|
||||||
self.assertIn('@"Software\\Microsoft\\Windows\\CurrentVersion\\Run"', startup)
|
self.assertIn('@"Software\\Microsoft\\Windows\\CurrentVersion\\Run"', startup)
|
||||||
self.assertIn("Environment.ProcessPath", startup)
|
self.assertIn("Environment.ProcessPath", startup)
|
||||||
self.assertIn("FindPython312Async", manager)
|
self.assertIn("FindPython312Async", installer)
|
||||||
self.assertIn('new BootstrapPython("py.exe", ["-3.12"])', manager)
|
self.assertIn('new BootstrapPython("py.exe", ["-3.12"])', installer)
|
||||||
self.assertIn('new BootstrapPython("python.exe", [])', manager)
|
self.assertIn('new BootstrapPython("python.exe", [])', installer)
|
||||||
self.assertIn('"ZCBOT_PIP_INDEX_URL"', manager)
|
self.assertIn('"ZCBOT_PIP_INDEX_URL"', installer)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
'"https://pypi.tuna.tsinghua.edu.cn/simple/"', manager
|
'"https://pypi.tuna.tsinghua.edu.cn/simple/"', installer
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_publish_output_contains_the_complete_node_payload(self) -> None:
|
def test_publish_output_contains_the_complete_node_payload(self) -> None:
|
||||||
|
|
@ -331,6 +497,22 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
"adapters\\blender.scene.author@v3\\blender.scene.author.v3.json",
|
"adapters\\blender.scene.author@v3\\blender.scene.author.v3.json",
|
||||||
project,
|
project,
|
||||||
)
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"..\\..\\software-contracts\\cad.geometry.prepare.v1.json",
|
||||||
|
project,
|
||||||
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"adapters\\cad.geometry.prepare@v1\\cad.geometry.prepare.v1.json",
|
||||||
|
project,
|
||||||
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"..\\..\\software-contracts\\freecad.model.author.v1.json",
|
||||||
|
project,
|
||||||
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"adapters\\freecad.model.author@v1\\freecad.model.author.v1.json",
|
||||||
|
project,
|
||||||
|
)
|
||||||
self.assertNotIn("software-contracts\\%(Filename)%(Extension)", project)
|
self.assertNotIn("software-contracts\\%(Filename)%(Extension)", project)
|
||||||
self.assertFalse((ROOT / "install-windows-node.ps1").exists())
|
self.assertFalse((ROOT / "install-windows-node.ps1").exists())
|
||||||
self.assertFalse((ROOT / "install-origin-runtime.ps1").exists())
|
self.assertFalse((ROOT / "install-origin-runtime.ps1").exists())
|
||||||
|
|
@ -354,30 +536,36 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertFalse((ROOT / "package-windows-node.ps1").exists())
|
self.assertFalse((ROOT / "package-windows-node.ps1").exists())
|
||||||
|
|
||||||
def test_auth_rejection_is_distinct_from_http_websocket_handshake_failure(self) -> None:
|
def test_auth_rejection_is_distinct_from_http_websocket_handshake_failure(self) -> None:
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
client = (PROJECT / "Protocol" / "NodeProtocolClient.cs").read_text(
|
||||||
self.assertIn('socket.HttpStatusCode is HttpStatusCode.Unauthorized', connection)
|
encoding="utf-8"
|
||||||
self.assertIn('or HttpStatusCode.Forbidden', connection)
|
)
|
||||||
self.assertIn("WebSocket 握手被拒绝,请检查服务端或反向代理", connection)
|
session = (PROJECT / "Host" / "NodeSession.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn("catch (NodeEndpointException exception)", connection)
|
self.assertIn('socket.HttpStatusCode is HttpStatusCode.Unauthorized', client)
|
||||||
self.assertIn("throw new NodeEndpointException", connection)
|
self.assertIn('or HttpStatusCode.Forbidden', client)
|
||||||
self.assertIn("(int?)result.CloseStatus == 4003", connection)
|
self.assertIn("WebSocket 握手被拒绝,请检查服务端或反向代理", session)
|
||||||
self.assertIn("节点身份已被服务端拒绝", connection)
|
self.assertIn("catch (NodeEndpointException exception)", session)
|
||||||
self.assertNotIn("Node credentials were rejected", connection)
|
self.assertIn("throw new NodeEndpointException", client)
|
||||||
|
self.assertIn("(int?)result.CloseStatus == 4003", client)
|
||||||
|
self.assertIn("节点身份已被服务端拒绝", client)
|
||||||
|
self.assertNotIn("Node credentials were rejected", client + session)
|
||||||
|
|
||||||
def test_adapter_runtime_probe_is_worker_owned_and_reported(self) -> None:
|
def test_adapter_runtime_probe_is_worker_owned_and_reported(self) -> None:
|
||||||
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
|
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
worker = (
|
worker = (
|
||||||
ROOT / "adapters" / "origin.plot@v2" / "worker.py"
|
ROOT / "adapters" / "origin.plot@v2" / "worker.py"
|
||||||
).read_text(encoding="utf-8")
|
).read_text(encoding="utf-8")
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn('r"Origin.ApplicationSI\\CLSID"', worker)
|
self.assertIn('r"Origin.ApplicationSI\\CLSID"', worker)
|
||||||
self.assertIn('["--probe"]', runner)
|
self.assertIn('["--probe"]', runner)
|
||||||
self.assertIn('root.GetProperty("adapter_version")', runner)
|
self.assertIn('root.GetProperty("adapter_version")', runner)
|
||||||
self.assertIn('startInfo.Environment["PYTHONUTF8"] = "1"', runner)
|
self.assertIn('startInfo.Environment["PYTHONUTF8"] = "1"', runner)
|
||||||
self.assertIn('startInfo.Environment["PYTHONIOENCODING"] = "utf-8"', runner)
|
self.assertIn('startInfo.Environment["PYTHONIOENCODING"] = "utf-8"', runner)
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
diagnostics = (
|
||||||
self.assertIn('AppendLine($"Adapter: {adapter.AdapterVersion}")', form)
|
PROJECT / "Application" / "DiagnosticService.cs"
|
||||||
self.assertIn('$"{group.Key} · 版本未知"', form)
|
).read_text(encoding="utf-8")
|
||||||
|
self.assertIn('AppendLine($"Adapter: {adapter.AdapterVersion}")', diagnostics)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
"&& !jobInbox.HasPendingJobs",
|
"&& !jobInbox.HasPendingJobs",
|
||||||
connection,
|
connection,
|
||||||
|
|
@ -385,7 +573,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertIn("originAdapter?.HasActiveJobs", connection)
|
self.assertIn("originAdapter?.HasActiveJobs", connection)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
"ReadRecoverableJobs().Any(item => item.Terminal is null)",
|
"ReadRecoverableJobs().Any(item => item.Terminal is null)",
|
||||||
(PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8"),
|
(PROJECT / "Jobs" / "JobRepository.cs").read_text(encoding="utf-8"),
|
||||||
)
|
)
|
||||||
self.assertNotIn("CreateInstance", worker)
|
self.assertNotIn("CreateInstance", worker)
|
||||||
for marker in (
|
for marker in (
|
||||||
|
|
@ -398,7 +586,22 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
|
|
||||||
def test_job_offer_is_persisted_before_acceptance(self) -> None:
|
def test_job_offer_is_persisted_before_acceptance(self) -> None:
|
||||||
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
|
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
monitor = (
|
||||||
|
PROJECT / "Application" / "JobMonitorService.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
repository = (PROJECT / "Jobs" / "JobRepository.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
recovery = (PROJECT / "Jobs" / "JobRecoveryService.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
atomic_file = (PROJECT / "Persistence" / "AtomicFile.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
|
||||||
|
protocol_client = (PROJECT / "Protocol" / "NodeProtocolClient.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
adapters = (PROJECT / "NodeAdapters.cs").read_text(encoding="utf-8")
|
adapters = (PROJECT / "NodeAdapters.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn("adapters.Find(capability)", inbox)
|
self.assertIn("adapters.Find(capability)", inbox)
|
||||||
self.assertIn("adapter.ValidateRequest(request)", inbox)
|
self.assertIn("adapter.ValidateRequest(request)", inbox)
|
||||||
|
|
@ -411,44 +614,51 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertNotIn('AppContext.BaseDirectory, "software-contracts"', adapters)
|
self.assertNotIn('AppContext.BaseDirectory, "software-contracts"', adapters)
|
||||||
self.assertNotIn("IsValidOriginRequest", inbox)
|
self.assertNotIn("IsValidOriginRequest", inbox)
|
||||||
self.assertNotIn("IsValidPlot", inbox)
|
self.assertNotIn("IsValidPlot", inbox)
|
||||||
self.assertIn('root.TryGetProperty("input_transfers"', inbox)
|
self.assertIn('root.TryGetProperty("input_transfers"', inbox + monitor)
|
||||||
|
self.assertIn('root.TryGetProperty("request_summary"', inbox + monitor)
|
||||||
|
self.assertIn("ReadDisplayTitle(root)", monitor)
|
||||||
|
self.assertIn("foreach (var action in operation.EnumerateObject())", monitor)
|
||||||
self.assertIn("transfers.GetArrayLength() > 16", inbox)
|
self.assertIn("transfers.GetArrayLength() > 16", inbox)
|
||||||
self.assertNotIn("transfers.GetArrayLength() is < 1 or > 16", inbox)
|
self.assertNotIn("transfers.GetArrayLength() is < 1 or > 16", inbox)
|
||||||
self.assertIn('"input", key, filename', inbox)
|
self.assertIn('"input", key, filename', repository)
|
||||||
self.assertIn("FileOptions.WriteThrough", inbox)
|
self.assertIn("FileOptions.WriteThrough", atomic_file)
|
||||||
self.assertIn("stream.Flush(flushToDisk: true)", inbox)
|
self.assertIn("stream.Flush(flushToDisk: true)", atomic_file)
|
||||||
new_record = inbox.split("var record =", 1)[1].split("private static JsonElement?", 1)[0]
|
new_record = inbox.split("var record =", 1)[1].split("private static JsonElement?", 1)[0]
|
||||||
self.assertLess(
|
self.assertLess(
|
||||||
new_record.index("AtomicWrite(requestPath, record"),
|
new_record.index("AtomicFile.Write(requestPath, record"),
|
||||||
new_record.index("JobOfferResult.Accept"),
|
new_record.index("JobOfferResult.Accept"),
|
||||||
)
|
)
|
||||||
self.assertIn('offerResult.Accepted ? "job_accept" : "job_reject"', connection)
|
self.assertIn('offerResult.Accepted ? "job_accept" : "job_reject"', connection)
|
||||||
self.assertIn("sendLock.WaitAsync", connection)
|
self.assertIn("sendLock.WaitAsync", protocol_client)
|
||||||
self.assertIn("!jobInbox.HasPendingJobs", connection)
|
self.assertIn("!jobInbox.HasPendingJobs", connection)
|
||||||
self.assertIn("capability_runtime = capabilityRuntime", connection)
|
self.assertIn("capability_runtime = capabilityRuntime", connection)
|
||||||
self.assertIn("adapter.RunAsync(job)", connection)
|
self.assertIn("adapter.RunAsync(job)", connection)
|
||||||
self.assertIn("ReportRecoverableJobsAsync", connection)
|
self.assertIn("ReportRecoverableJobsAsync", connection)
|
||||||
self.assertIn("ConcurrentDictionary<Guid, Task> jobPipelines", connection)
|
self.assertIn("ConcurrentDictionary<Guid, Task> jobPipelines", connection)
|
||||||
self.assertIn("StartJobPipeline(socket, acceptedJob)", connection)
|
self.assertIn("StartJobPipeline(client, acceptedJob)", connection)
|
||||||
self.assertIn("inputDownloader.DownloadAsync(job, forcedStop.Token)", connection)
|
self.assertIn("inputDownloader.DownloadAsync(job, forcedStop.Token)", connection)
|
||||||
self.assertNotIn("CreateLinkedTokenSource(cancellationToken, forcedStop.Token)", connection)
|
self.assertNotIn("CreateLinkedTokenSource(cancellationToken, forcedStop.Token)", connection)
|
||||||
self.assertIn('stage = "uploading_outputs"', connection)
|
self.assertIn('stage = "uploading_outputs"', connection)
|
||||||
self.assertIn('stage = "software_running"', connection)
|
self.assertIn('stage = "software_running"', connection)
|
||||||
self.assertNotIn('stage = "origin_running"', connection)
|
self.assertNotIn('stage = "origin_running"', connection)
|
||||||
self.assertIn("&& !job.UploadComplete", connection)
|
self.assertIn("JobRecoveryAction.ResumeOutputUpload", connection)
|
||||||
self.assertIn("&& !job.CloudTerminal", connection)
|
self.assertIn("job.UploadComplete || job.CloudTerminal", recovery)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
"await Task.WhenAll(jobPipelines.Values.Concat(exportPipelines.Values).ToArray())",
|
"await Task.WhenAll(jobPipelines.Values.Concat(exportPipelines.Values).ToArray())",
|
||||||
connection,
|
connection,
|
||||||
)
|
)
|
||||||
self.assertIn("StartJobPipeline(socket, job)", connection)
|
self.assertIn("StartJobPipeline(client, job, recovery: true)", connection)
|
||||||
self.assertIn('stage = "downloading_inputs"', connection)
|
self.assertIn('stage = "downloading_inputs"', connection)
|
||||||
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', inbox)
|
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', repository)
|
||||||
self.assertIn("AtomicWrite(requestPath, updated, overwrite: true)", inbox)
|
self.assertIn("AtomicFile.Write(requestPath, updated, overwrite: true)", inbox)
|
||||||
|
|
||||||
downloader = (PROJECT / "JobInputDownloader.cs").read_text(encoding="utf-8")
|
downloader = (PROJECT / "JobInputDownloader.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', downloader)
|
http_client = (PROJECT / "Transfers" / "NodeHttpClient.cs").read_text(
|
||||||
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', downloader)
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', http_client)
|
||||||
|
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', http_client)
|
||||||
|
self.assertIn("SendWithRetryAsync", downloader)
|
||||||
self.assertIn("HttpCompletionOption.ResponseHeadersRead", downloader)
|
self.assertIn("HttpCompletionOption.ResponseHeadersRead", downloader)
|
||||||
self.assertIn("IncrementalHash.CreateHash", downloader)
|
self.assertIn("IncrementalHash.CreateHash", downloader)
|
||||||
self.assertIn("total > expectedSize", downloader)
|
self.assertIn("total > expectedSize", downloader)
|
||||||
|
|
@ -456,8 +666,13 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertNotIn("Process.Start", downloader)
|
self.assertNotIn("Process.Start", downloader)
|
||||||
|
|
||||||
def test_adapter_worker_launch_is_manifest_driven_and_terminal_driven(self) -> None:
|
def test_adapter_worker_launch_is_manifest_driven_and_terminal_driven(self) -> None:
|
||||||
runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8")
|
runner = (PROJECT / "Adapters" / "AdapterExecutionService.cs").read_text(
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
supervisor = (PROJECT / "Adapters" / "ProcessSupervisor.cs").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
|
||||||
project = (PROJECT / "Zcbot.WindowsNode.csproj").read_text(encoding="utf-8")
|
project = (PROJECT / "Zcbot.WindowsNode.csproj").read_text(encoding="utf-8")
|
||||||
worker = (
|
worker = (
|
||||||
ROOT / "adapters" / "origin.plot@v2" / "worker.py"
|
ROOT / "adapters" / "origin.plot@v2" / "worker.py"
|
||||||
|
|
@ -473,8 +688,8 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', runner)
|
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', runner)
|
||||||
self.assertIn('"NODE_RESTARTED_DURING_JOB"', runner)
|
self.assertIn('"NODE_RESTARTED_DURING_JOB"', runner)
|
||||||
self.assertIn("CancellationTokenSource.CreateLinkedTokenSource", runner)
|
self.assertIn("CancellationTokenSource.CreateLinkedTokenSource", runner)
|
||||||
self.assertIn("process.Kill(entireProcessTree: true)", runner)
|
self.assertIn("process.Kill(entireProcessTree: true)", supervisor)
|
||||||
self.assertIn('type.GetString() == "job_cancel"', connection)
|
self.assertIn('message.Type == "job_cancel"', connection)
|
||||||
self.assertIn('"cancelled", "USER_CANCELLED"', connection)
|
self.assertIn('"cancelled", "USER_CANCELLED"', connection)
|
||||||
self.assertIn("..\\adapters\\**\\*", project)
|
self.assertIn("..\\adapters\\**\\*", project)
|
||||||
self.assertIn("descriptor.Manifest.WorkerTimeoutMinutes", runner)
|
self.assertIn("descriptor.Manifest.WorkerTimeoutMinutes", runner)
|
||||||
|
|
@ -495,24 +710,29 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
|
|
||||||
def test_origin_adapter_can_be_packaged_without_building_node(self) -> None:
|
def test_origin_adapter_can_be_packaged_without_building_node(self) -> None:
|
||||||
script = (ROOT / "package-origin-adapter.bat").read_text(encoding="utf-8")
|
script = (ROOT / "package-origin-adapter.bat").read_text(encoding="utf-8")
|
||||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
connection = (PROJECT / "Jobs" / "JobCoordinator.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn('"adapters\\origin.plot@v2\\adapter.json"', script)
|
self.assertIn('"adapters\\origin.plot@v2\\adapter.json"', script)
|
||||||
self.assertIn('"adapters\\origin.plot@v2\\worker.py"', script)
|
self.assertIn('"adapters\\origin.plot@v2\\worker.py"', script)
|
||||||
self.assertIn('"adapters\\origin.plot@v2\\acceptance.py"', script)
|
self.assertIn('"adapters\\origin.plot@v2\\acceptance.py"', script)
|
||||||
self.assertIn('"..\\software-contracts\\origin.plot.v2.json"', script)
|
self.assertIn('"..\\software-contracts\\origin.plot.v2.json"', script)
|
||||||
self.assertNotIn("dotnet", script.lower())
|
self.assertNotIn("dotnet", script.lower())
|
||||||
|
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
diagnostics = (
|
||||||
self.assertIn("Contract SHA-256: {adapter.ContractSha256}", form)
|
PROJECT / "Application" / "DiagnosticService.cs"
|
||||||
self.assertIn("Workspace protocol:", form)
|
).read_text(encoding="utf-8")
|
||||||
self.assertIn("Workspace state:", form)
|
self.assertIn("Contract SHA-256: {adapter.ContractSha256}", diagnostics)
|
||||||
|
self.assertIn("Workspace protocol:", diagnostics)
|
||||||
|
self.assertIn("Workspace state:", diagnostics)
|
||||||
self.assertIn("workspace_protocol_version", connection)
|
self.assertIn("workspace_protocol_version", connection)
|
||||||
self.assertIn("contract_sha256", connection)
|
self.assertIn("contract_sha256", connection)
|
||||||
|
|
||||||
uploader = (PROJECT / "JobOutputUploader.cs").read_text(encoding="utf-8")
|
uploader = (PROJECT / "JobOutputUploader.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', uploader)
|
http_client = (PROJECT / "Transfers" / "NodeHttpClient.cs").read_text(
|
||||||
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', uploader)
|
encoding="utf-8"
|
||||||
self.assertIn('DefaultRequestHeaders.Add("X-Lease-Id"', uploader)
|
)
|
||||||
|
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', http_client)
|
||||||
|
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', http_client)
|
||||||
|
self.assertIn('request.Headers.Add("X-Lease-Id"', http_client)
|
||||||
self.assertIn("SHA256.HashDataAsync", uploader)
|
self.assertIn("SHA256.HashDataAsync", uploader)
|
||||||
self.assertIn("upload-complete.json", connection + uploader)
|
self.assertIn("upload-complete.json", connection + uploader)
|
||||||
self.assertIn("adapter.PreviewOutputIds", connection)
|
self.assertIn("adapter.PreviewOutputIds", connection)
|
||||||
|
|
@ -536,11 +756,17 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertNotIn('".tmp-" + Guid.NewGuid()', uploader)
|
self.assertNotIn('".tmp-" + Guid.NewGuid()', uploader)
|
||||||
self.assertNotIn("Process.Start", uploader)
|
self.assertNotIn("Process.Start", uploader)
|
||||||
|
|
||||||
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
|
tray = (PROJECT / "Presentation" / "Tray" / "TrayHost.cs").read_text(
|
||||||
self.assertIn("_ = form.Handle", tray)
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
controller = (
|
||||||
|
PROJECT / "Application" / "NodeApplicationController.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
self.assertNotIn("legacyForm", tray)
|
||||||
|
self.assertIn("window.ShowAndActivate()", tray)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
'UpdateStatus(NodeStatus.Create(NodeState.Connecting, "正在连接 zcbot…"))',
|
'Publish(NodeStatus.Create(NodeState.Connecting, "正在连接 zcbot…"))',
|
||||||
tray,
|
controller,
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_origin_analysis_adapter_can_be_packaged_without_building_node(self) -> None:
|
def test_origin_analysis_adapter_can_be_packaged_without_building_node(self) -> None:
|
||||||
|
|
@ -572,9 +798,36 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertIn("tar.exe -a -c -f", script)
|
self.assertIn("tar.exe -a -c -f", script)
|
||||||
self.assertNotIn("dotnet", script.lower())
|
self.assertNotIn("dotnet", script.lower())
|
||||||
|
|
||||||
|
def test_cad_adapter_can_be_packaged_without_building_node(self) -> None:
|
||||||
|
script = (ROOT / "package-cad-adapter.bat").read_text(encoding="utf-8")
|
||||||
|
worker = (
|
||||||
|
ROOT / "adapters" / "cad.geometry.prepare@v1" / "worker.py"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
self.assertIn("adapters\\cad.geometry.prepare@v1\\adapter.json", script)
|
||||||
|
self.assertIn("adapters\\cad.geometry.prepare@v1\\worker.py", script)
|
||||||
|
self.assertIn("adapters\\cad.geometry.prepare@v1\\requirements.txt", script)
|
||||||
|
self.assertIn("..\\software-contracts\\cad.geometry.prepare.v1.json", script)
|
||||||
|
self.assertIn("tar.exe -a -c -f", script)
|
||||||
|
self.assertNotIn("dotnet", script.lower())
|
||||||
|
self.assertIn("INPUT_PATH_ESCAPES_JOB", worker)
|
||||||
|
self.assertIn("OUTPUT_PATH_ESCAPES_JOB", worker)
|
||||||
|
self.assertIn("STEP_ROUNDTRIP_VOLUME_MISMATCH", worker)
|
||||||
|
for forbidden in ("subprocess", "eval(", "exec(", "os.system", "requests."):
|
||||||
|
self.assertNotIn(forbidden, worker)
|
||||||
|
|
||||||
|
def test_freecad_adapter_can_be_packaged_without_building_node(self) -> None:
|
||||||
|
script = (ROOT / "package-freecad-adapter.bat").read_text(encoding="utf-8")
|
||||||
|
for filename in ("adapter.json", "worker.py", "freecad_worker.py", "recipe.py", "preview_renderer.py", "requirements.txt", "acceptance.py"):
|
||||||
|
self.assertIn(filename, script)
|
||||||
|
self.assertIn("..\\software-contracts\\freecad.model.author.v1.json", script)
|
||||||
|
self.assertIn("tar.exe -a -c -f", script)
|
||||||
|
self.assertNotIn("dotnet", script.lower())
|
||||||
|
|
||||||
def test_local_job_monitor_never_shows_update_before_acceptance(self) -> None:
|
def test_local_job_monitor_never_shows_update_before_acceptance(self) -> None:
|
||||||
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
|
monitor = (
|
||||||
self.assertIn("state.UpdatedAt > acceptedAt", inbox)
|
PROJECT / "Application" / "JobMonitorService.cs"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
self.assertIn("state.UpdatedAt > acceptedAt", monitor)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue