Compare commits
5 Commits
054aaf8e8c
...
8aafe3272f
| Author | SHA1 | Date |
|---|---|---|
|
|
8aafe3272f | |
|
|
ded8cba54c | |
|
|
3e0a719f80 | |
|
|
faa2279f00 | |
|
|
b016f8ee8c |
|
|
@ -36,8 +36,11 @@
|
||||||
|
|
||||||
## 文档与版本
|
## 文档与版本
|
||||||
|
|
||||||
- 版本号、`CHANGELOG.md`、`PROGRESS.md` 在 push 前统一更新一次,不按每个 commit 更新。
|
- 开发 push 与正式发布分离:功能开发期间允许持续 commit / push,不因此提升版本号,也不新增已发布的 `CHANGELOG.md` 数字版本条目。
|
||||||
- push 前更新 `PROGRESS.md`:补“已完成关键能力”条目,状态表变化随之更新,新增或删除模块时同步文件清单。
|
- 尚未发布但需要预先整理的用户文案写在 `CHANGELOG.md` 顶部 `## Unreleased`;该区不会被前端更新日志接口解析。正式发布时再把它改成 `## <版本> — <日期>`。
|
||||||
|
- 版本号与用户版 `CHANGELOG.md` 只在功能稳定、准备上线时通过单独的 release commit 统一更新;同一次发布中校准 `PROGRESS.md`,不按每个开发 commit 更新。
|
||||||
|
- 阶段性成果可随开发更新 `PROGRESS.md`;正式发布前补“已完成关键能力”条目,状态表变化随之更新,新增或删除模块时同步文件清单。
|
||||||
|
- 未完成且不能让线上用户接触的功能,应在独立功能分支开发、待稳定后合并生产分支;若必须提前合并或部署,则使用默认关闭的 feature flag,并限制为管理员或测试账号启用。
|
||||||
- 版本号唯一事实源是 `core/__init__.py::__version__`:
|
- 版本号唯一事实源是 `core/__init__.py::__version__`:
|
||||||
- patch:bug 修复、重构、调参、新 skill、样式;
|
- patch:bug 修复、重构、调参、新 skill、样式;
|
||||||
- minor:成批新功能或明显对外行为变化;
|
- minor:成批新功能或明显对外行为变化;
|
||||||
|
|
@ -59,4 +62,3 @@
|
||||||
## 领域语境
|
## 领域语境
|
||||||
|
|
||||||
主要使用方是中国建筑材料科学研究总院,核心语境是无机非金属材料研发与生产,包括水泥、混凝土、玻璃、陶瓷、耐火材料和新型建材。典型任务是配方研发、性能测试、XRD/SEM/热分析、实验数据建模,以及申报书、调研报告、专利和论文写作。默认按材料研发而非建筑施工、结构计算或 BIM 理解需求。
|
主要使用方是中国建筑材料科学研究总院,核心语境是无机非金属材料研发与生产,包括水泥、混凝土、玻璃、陶瓷、耐火材料和新型建材。典型任务是配方研发、性能测试、XRD/SEM/热分析、实验数据建模,以及申报书、调研报告、专利和论文写作。默认按材料研发而非建筑施工、结构计算或 BIM 理解需求。
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,13 @@
|
||||||
> 面向用户的更新说明,由 `GET /v1/changelog` 解析后在前端「点版本号」弹层展示。
|
> 面向用户的更新说明,由 `GET /v1/changelog` 解析后在前端「点版本号」弹层展示。
|
||||||
> 维护口径:只记**用户可感知**的变化("能干什么了"而不是"改了哪个模块"),纯内部重构 / 修复不记,
|
> 维护口径:只记**用户可感知**的变化("能干什么了"而不是"改了哪个模块"),纯内部重构 / 修复不记,
|
||||||
> 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。
|
> 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。
|
||||||
|
> 开发中的用户文案可先写入 `## Unreleased`;该区不会被前端解析,正式发布时再替换为数字版本和日期。
|
||||||
> 工程口径的完整记录见 `PROGRESS.md` / git log。
|
> 工程口径的完整记录见 `PROGRESS.md` / git log。
|
||||||
|
|
||||||
|
## Unreleased
|
||||||
|
|
||||||
|
- 新增专业软件 Job 中心:Agent 可提交、查询和停止 Windows Node 上的受控软件任务;用户可在右下角跨对话查看进度、收到完成通知,并回到原对话分析结果。
|
||||||
|
|
||||||
## 0.65.2 — 2026-08-13
|
## 0.65.2 — 2026-08-13
|
||||||
|
|
||||||
- 粘贴、拖拽或选择附件发送后,对话中的用户消息会保留附件卡片;图片同时显示可点击放大的缩略图,刷新历史对话后也不会消失。
|
- 粘贴、拖拽或选择附件发送后,对话中的用户消息会保留附件卡片;图片同时显示可点击放大的缩略图,刷新历史对话后也不会消失。
|
||||||
|
|
|
||||||
16
DESIGN.md
16
DESIGN.md
|
|
@ -460,11 +460,21 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
||||||
|
|
||||||
第一阶段以 `docs/windows-node-mvp-intranet.md` 为实现契约:Windows Node 只作为受控执行节点,通过出站 HTTP/WS 主动连接 zcbot;首批能力固定为 `origin.plot@v1`。长期方案中的 mTLS、Service/DesktopRunner 双进程、完整租约与多节点调度暂不进入 MVP,但 URL path、Node ID、Bearer Header 和任务协议保留原位升级空间。
|
第一阶段以 `docs/windows-node-mvp-intranet.md` 为实现契约:Windows Node 只作为受控执行节点,通过出站 HTTP/WS 主动连接 zcbot;首批能力固定为 `origin.plot@v1`。长期方案中的 mTLS、Service/DesktopRunner 双进程、完整租约与多节点调度暂不进入 MVP,但 URL path、Node ID、Bearer Header 和任务协议保留原位升级空间。
|
||||||
|
|
||||||
云端控制面使用独立的 `compute_node_enrollments` 与 `compute_nodes`,不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵,数据库只保存 SHA-256 摘要;节点注册在行锁事务中校验有效期、预期名称和允许能力,成功后原子消费。每个节点获得独立高熵 Token,数据库只保存 bcrypt 强哈希,明文仅在注册响应出现一次。
|
云端控制面使用独立的 `software_node_enrollments` 与 `software_nodes`,不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵,数据库只保存 SHA-256 摘要;节点注册在行锁事务中校验有效期、预期名称和允许能力,成功后原子消费。每个节点获得独立高熵 Token,数据库只保存 bcrypt 强哈希,明文仅在注册响应出现一次。
|
||||||
|
|
||||||
Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/compute/nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活,新连接关闭旧连接;`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态,再关闭现有连接;断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。
|
Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/software-nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活,新连接关闭旧连接;`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态,再关闭现有连接;断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。
|
||||||
|
|
||||||
首批只落注册、认证、心跳、状态与禁用基础链路。`compute_jobs`、任务 offer/accept、Origin Worker、输入输出传输、重连对账和 Token 轮换属于后续垂直闭环,不以任意命令或脚本接口临时代替。
|
第二阶段已增加 `software_jobs`(专业软件任务)账本与 `origin.plot@v1` 的 offer/accept 骨架。用户只能在本人 task 下以幂等键提交固定 schema;云端规范化请求并记录 SHA-256,按当前进程真实在线、能力匹配、健康且有空闲 slot 的 Node 创建短期 offer。Node 再次校验 schema、图形类型和输出格式,使用 write-through、flush 与原子 rename 先落本机任务目录,再回 `job_accept`;重复 job 只有 digest 一致才接受。过期或发送失败的 offer 回到队列,lease、Node 和 digest 不匹配的响应被拒绝。Node 接收后云端进入 `dispatched` 而非 `running`,并将 slot 降为 0;只有固定 Worker 真正启动后才进入 `origin_running`。
|
||||||
|
|
||||||
|
第三阶段补齐输入下载与恢复状态协议:`input_id` 固定为用户已有 artifact UUID,提交时快照文件名、大小和 SHA-256,只允许 CSV/XLSX/JSON 且不超过 100 MiB。Node 以自身 Bearer 身份访问任务绑定的只读下载端点,流式写入本 job 的 `input/`,同时限制声明大小并校验 SHA-256,完成后原子 rename;不暴露工作区路径。Node 会原子读取/补报 `terminal.json`,断线后云端把活动任务标记 `disconnected` 并保留 Node/lease,重连按 job、lease、digest 恢复下载或幂等补报终态,不自动重派。
|
||||||
|
|
||||||
|
第四阶段落地固定 Origin Worker:Node 仅从管理员安装的固定 Python 运行时启动随程序发布的 `worker.py`,参数只有本机 job 目录;请求不能指定脚本、解释器或文件路径。Worker 使用 `originpro` 生成 OPJU、PNG、SVG、PDF、plot spec 和 provenance,校验产物签名并原子写入终态;当前受控图形仅含 line、scatter、line_scatter 和双栏出版布局。进程内 pipeline 按 job 去重,并脱离单次 WebSocket 的取消令牌运行;连接中断只延迟状态/终态上报。Node 进程若在 Worker 启动后重启,则保守失败而不重复驱动 Origin,避免无法证明的双执行。
|
||||||
|
|
||||||
|
第五阶段完成输出上传与发布:Node 只按固定 manifest ID 逐项流式 PUT,并携带 Node、lease、request digest 与内容摘要;云端重新绑定任务身份,不信 Node 提供的路径或媒体类型。文件先进入用户根下隐藏暂存区,固定文件名、单文件/总大小和 SHA-256 全部验证后,目录级原子移动到 `<working_dir>/origin/<job_id>/`,再登记平台 artifact UUID 并写成功终态。重复 PUT、complete 和重连均按摘要幂等;部分上传不可见,只有完整集合才能发布。
|
||||||
|
|
||||||
|
第六阶段增加用户级 Job 中心与 Agent typed tools。`software_capability_list` 只暴露固定能力及当前在线空闲节点数,`software_job_submit/status/cancel` 在构造时绑定当前 user/task,模型不能跨用户或跨对话指定归属。右下角 Job 中心按用户聚合各对话任务,活动期短轮询、空闲期降频;终态变化通知用户,成功任务可回到原对话发起分析。取消采用协作协议:未派发任务直接终止,已派发任务先进入 `cancelling`,云端通过 WebSocket 发送并在心跳时重放 `job_cancel`,Node 杀死固定 Worker 进程树后回报 `cancelled`;终态写入仍由云端账本裁决。
|
||||||
|
|
||||||
|
后续仍需实现 Token 轮换;不得以任意命令或脚本接口临时代替。当前 Job 中心采用轮询而非用户事件推送,单活与 offer 选择仍只覆盖单 Web 进程;生产启用多实例前必须增加 Redis/PG fencing 或固定路由到单一控制面实例。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`。
|
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`。
|
||||||
|
|
||||||
最后更新:2026-08-13(用户消息结构化附件与图片预览,bump 0.65.2)
|
最后更新:2026-08-13(专业软件 Job 中心开发中,未发版)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -22,6 +22,8 @@
|
||||||
|
|
||||||
### 2026-08-13
|
### 2026-08-13
|
||||||
|
|
||||||
|
- **08-13 / Unreleased / 专业软件 Job 中心**:Agent 新增固定能力发现、提交、状态和取消工具,后端提供用户级跨对话任务列表与协作取消,Windows Node 可终止固定 Worker 进程树并幂等回报取消终态;Web 右下角展示活动/最近任务、进度、完成通知、停止与回到原对话分析入口。相关 Python 78 项、JavaScript 语法、Python 编译、.NET build 与 diff 检查通过;数据库保持在 0031,未连库、未执行 migration。
|
||||||
|
|
||||||
- **08-13 / 0.65.2 / 用户消息结构化附件 + 对话内图片预览**:新增 0031 `messages.attachment_refs`,新客户端将附件作为结构化字段发送,数据库正文仅保留用户自然语言;后端按 task working_dir 校验路径并在内存模型上下文中补附件提示,旧客户端与历史正文标记继续兼容。用户消息即时态和历史态统一展示附件 chip,图片额外显示可点击缩略图;相关 Python 34 项、全部前端 Node 26 项、Python/JavaScript 语法、Alembic 单 head 与 diff 检查通过,未连接或迁移生产 DB。
|
- **08-13 / 0.65.2 / 用户消息结构化附件 + 对话内图片预览**:新增 0031 `messages.attachment_refs`,新客户端将附件作为结构化字段发送,数据库正文仅保留用户自然语言;后端按 task working_dir 校验路径并在内存模型上下文中补附件提示,旧客户端与历史正文标记继续兼容。用户消息即时态和历史态统一展示附件 chip,图片额外显示可点击缩略图;相关 Python 34 项、全部前端 Node 26 项、Python/JavaScript 语法、Alembic 单 head 与 diff 检查通过,未连接或迁移生产 DB。
|
||||||
|
|
||||||
### 2026-08-12
|
### 2026-08-12
|
||||||
|
|
@ -341,7 +343,7 @@ core/llm_transport.py 438 ← wire 层健壮性:畸形/吐空检测+留
|
||||||
core/tool_registry.py 264 ← 声明式工具注册表((组名,gate,factory);secret/host 工具按实际能力 gate)
|
core/tool_registry.py 264 ← 声明式工具注册表((组名,gate,factory);secret/host 工具按实际能力 gate)
|
||||||
core/context.py 95 ← LLM 调用前压缩旧 tool / load_skill 消息(带压力门槛),保 tool_call 协议字段
|
core/context.py 95 ← LLM 调用前压缩旧 tool / load_skill 消息(带压力门槛),保 tool_call 协议字段
|
||||||
core/external_systems/*.py ← 外部系统目录/用户授权/凭据加密 + 通用 OpenAPI/MCP connector
|
core/external_systems/*.py ← 外部系统目录/用户授权/凭据加密 + 通用 OpenAPI/MCP connector
|
||||||
core/compute_nodes.py ← Windows Node 注册码、身份认证与运行状态
|
core/software_nodes.py ← Windows Node 注册码、身份认证与运行状态
|
||||||
core/sinks.py 101
|
core/sinks.py 101
|
||||||
core/paths.py 50 ← task_dir db form 归一
|
core/paths.py 50 ← task_dir db form 归一
|
||||||
core/probe.py 243
|
core/probe.py 243
|
||||||
|
|
@ -363,7 +365,7 @@ tools/{base,output,fs,shell,run_python,skill_tool,skill_authoring,media_common,s
|
||||||
main.py ~210 ← 入口:web / db / probe / user / sandbox check
|
main.py ~210 ← 入口:web / db / probe / user / sandbox check
|
||||||
db/migrations/versions/ 0001-0030
|
db/migrations/versions/ 0001-0030
|
||||||
web/app.py ~210 ← 工厂 + lifespan 编排(07-23 拆分;路由在 routers/,协程在 background 等)
|
web/app.py ~210 ← 工厂 + lifespan 编排(07-23 拆分;路由在 routers/,协程在 background 等)
|
||||||
web/routers/*.py ← 含 external_systems 用户连接与 compute_nodes 节点路由
|
web/routers/*.py ← 含 external_systems 用户连接与 software_nodes 节点路由
|
||||||
web/{background,scheduler_runner,wechat_runner}.py ← lifespan 后台协程按域析出
|
web/{background,scheduler_runner,wechat_runner}.py ← lifespan 后台协程按域析出
|
||||||
web/{runs,common,schemas,model_gate,userfiles}.py ← BG worker/共享 helper/请求体/档位门控/路径安全
|
web/{runs,common,schemas,model_gate,userfiles}.py ← BG worker/共享 helper/请求体/档位门控/路径安全
|
||||||
web/auth.py ~190 ← 邮箱密码 + platform_key → JWT
|
web/auth.py ~190 ← 邮箱密码 + platform_key → JWT
|
||||||
|
|
|
||||||
36
RUN.md
36
RUN.md
|
|
@ -538,6 +538,18 @@ sudo bash deploy/update_bluegreen.sh
|
||||||
> 蓝绿部署机(生产,已 bootstrap)一律用 `sudo bash deploy/update_bluegreen.sh`(见上节)。
|
> 蓝绿部署机(生产,已 bootstrap)一律用 `sudo bash deploy/update_bluegreen.sh`(见上节)。
|
||||||
> 下面的 `update.sh` 是单实例版:host backend 机 / 内网机 / 未切蓝绿的部署继续用。
|
> 下面的 `update.sh` 是单实例版:host backend 机 / 内网机 / 未切蓝绿的部署继续用。
|
||||||
|
|
||||||
|
开发 push 与正式发布是两个动作:开发期间可以持续 push,但不提升
|
||||||
|
`core/__init__.py::__version__`,也不新增 `CHANGELOG.md` 的数字版本条目。需要提前整理的
|
||||||
|
用户文案写在 `## Unreleased`,前端不会展示。功能稳定后创建单独的 release commit,统一完成:
|
||||||
|
|
||||||
|
1. 将 `## Unreleased` 改为 `## <版本> — <日期>`,只保留用户可感知的变化;
|
||||||
|
2. 更新 `core/__init__.py::__version__`;
|
||||||
|
3. 校准 `PROGRESS.md`,并按变更类型同步本文件或 `DESIGN.md`;
|
||||||
|
4. 部署并验证 `/healthz` 与 `/v1/changelog`。
|
||||||
|
|
||||||
|
未完成且不能向线上用户开放的功能应留在功能分支,稳定后再合并生产分支;必须提前部署时,
|
||||||
|
使用默认关闭且仅向管理员或测试账号开放的 feature flag。
|
||||||
|
|
||||||
**一把梭(单实例):`deploy/update.sh`**
|
**一把梭(单实例):`deploy/update.sh`**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -1042,14 +1054,16 @@ sudo xfs_quota -x -c "limit -p bhard=10g zcbot_<user_uuid>" /opt
|
||||||
|
|
||||||
### Windows Node 内网 MVP(开发中)
|
### Windows Node 内网 MVP(开发中)
|
||||||
|
|
||||||
先执行 `alembic upgrade head` 创建 `compute_node_enrollments` 和 `compute_nodes`。不要在未确认目标数据库时运行迁移;本机 `.env` 的 `ZCBOT_DB_URL` 可能是生产隧道。
|
先执行 `alembic upgrade head` 创建 `software_node_enrollments`、`software_nodes` 和 `software_jobs`。不要在未确认目标数据库时运行迁移;本机 `.env` 的 `ZCBOT_DB_URL` 可能是生产隧道。
|
||||||
|
|
||||||
云端当前提供:
|
云端当前提供:
|
||||||
|
|
||||||
- 管理员 `POST /v1/admin/compute-node-enrollments` 创建一次性注册码;
|
- 管理员 `POST /v1/admin/software-node-enrollments` 创建一次性注册码;
|
||||||
- Node `POST /v1/compute/nodes/enroll` 注册并一次性取得 `node_id`、`node_token`;
|
- Node `POST /v1/software-nodes/enroll` 注册并一次性取得 `node_id`、`node_token`;
|
||||||
- Node 携带 `Authorization: Bearer <node_token>` 和 `X-Node-Id` 连接 `WS /v1/compute/nodes/connect`;
|
- Node 携带 `Authorization: Bearer <node_token>` 和 `X-Node-Id` 连接 `WS /v1/software-nodes/connect`;
|
||||||
- 管理员 `GET /v1/admin/compute-nodes` 查看节点,`PATCH /v1/admin/compute-nodes/{node_id}` 启停节点,`DELETE /v1/admin/compute-nodes/{node_id}` 永久删除节点身份。
|
- 用户或 Agent 通过 `POST /v1/tasks/{task_id}/software-jobs` 提交专业软件任务;
|
||||||
|
- 用户通过 `GET /v1/software-jobs` 查看本人跨对话任务,可用 `task_id`、`active_only` 和 `limit` 筛选,`POST /v1/software-jobs/{job_id}/cancel` 请求停止;
|
||||||
|
- 管理员 `GET /v1/admin/software-nodes` 查看节点,`PATCH /v1/admin/software-nodes/{node_id}` 启停节点,`DELETE /v1/admin/software-nodes/{node_id}` 永久删除节点身份。
|
||||||
|
|
||||||
Node API 只能绑定受控内网地址并由安全组限制来源 IP。当前 HTTP/WS 链路不加密;跨安全域、公网或不可信终端接入前,必须先升级 HTTPS/WSS。多 Web 实例部署时,Node API 暂时固定路由到单一实例,直至 Connection Manager 增加跨实例 fencing。
|
Node API 只能绑定受控内网地址并由安全组限制来源 IP。当前 HTTP/WS 链路不加密;跨安全域、公网或不可信终端接入前,必须先升级 HTTPS/WSS。多 Web 实例部署时,Node API 暂时固定路由到单一实例,直至 Connection Manager 增加跨实例 fencing。
|
||||||
|
|
||||||
|
|
@ -1064,11 +1078,21 @@ windows-node/Zcbot.WindowsNode/bin/Debug/net10.0-windows/Zcbot.WindowsNode.exe e
|
||||||
windows-node/Zcbot.WindowsNode/bin/Debug/net10.0-windows/Zcbot.WindowsNode.exe
|
windows-node/Zcbot.WindowsNode/bin/Debug/net10.0-windows/Zcbot.WindowsNode.exe
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Origin Worker 使用独立固定 Python 运行时。先确认该交互式 Windows 账号已安装并可启动 Origin/OriginPro,再由管理员安装运行时(不要复用服务端 `.venv`):
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
windows-node\install-origin-runtime.ps1 -BootstrapPython D:\programs\Python312\python.exe
|
||||||
|
```
|
||||||
|
|
||||||
|
默认解释器为 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\python.exe`。如需放在其他受管目录,设置机器级 `ZCBOT_ORIGIN_PYTHON` 为绝对 `python.exe` 路径后重启 Node。运行时固定依赖见 `windows-node/origin-worker/requirements.txt`;任务请求无权选择解释器、脚本或路径。当前 Worker 支持 CSV/XLSX/JSON 输入,`line`、`scatter`、`line_scatter` 与 OPJU/PNG/SVG/PDF 输出。成功产物由 Node 流式上传,全部校验通过后发布到任务工作目录 `origin/<job_id>/`;上传中断会在重连时幂等续传。
|
||||||
|
|
||||||
|
Web 用户登录后,右下角 Job 中心会聚合本人最近任务。活动任务约 4 秒刷新一次,空闲时降为约 30 秒;停止已派发任务是协作取消,状态先显示“正在停止”,Node 在线时立即接收,断线后在下次连接或心跳时重放。Agent 可调用 `software_capability_list`、`software_job_submit`、`software_job_status` 和 `software_job_cancel`;提交工具只创建固定 schema 的持久任务,不会阻塞当前对话等待完成。
|
||||||
|
|
||||||
注册配置写入 `%ProgramData%\Zcbot\WindowsNode\node.json`;Token 使用 DPAPI `LocalMachine` 加密,ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号注册并运行 Node。当前 MVP 以该账号的登录后计划任务启动,不安装 Windows Service。
|
注册配置写入 `%ProgramData%\Zcbot\WindowsNode\node.json`;Token 使用 DPAPI `LocalMachine` 加密,ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号注册并运行 Node。当前 MVP 以该账号的登录后计划任务启动,不安装 Windows Service。
|
||||||
|
|
||||||
直接双击 EXE 启动托盘 UI:红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开配置窗。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless`。
|
直接双击 EXE 启动托盘 UI:红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开配置窗。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless`。
|
||||||
|
|
||||||
经 nginx 反代时,`/v1/compute/nodes/connect` 必须单独透传 WebSocket Upgrade/Connection 头并设置长连接超时,配置见 `deploy/nginx/zcbot.conf.example`。若注册成功后节点持续显示“连接中断,等待重连”,先用 WebSocket 握手检查该路径;返回普通 HTTP 404 通常表示请求落入了清空 `Connection` 头的默认 location。
|
经 nginx 反代时,`/v1/software-nodes/connect` 必须单独透传 WebSocket Upgrade/Connection 头并设置长连接超时,配置见 `deploy/nginx/zcbot.conf.example`。若注册成功后节点持续显示“连接中断,等待重连”,先用 WebSocket 握手检查该路径;返回普通 HTTP 404 通常表示请求落入了清空 `Connection` 头的默认 location。
|
||||||
|
|
||||||
|
|
||||||
- **入口**:`main.py`(`web / db / probe / user`)→ `core/agent_builder.py::build_agent`
|
- **入口**:`main.py`(`web / db / probe / user`)→ `core/agent_builder.py::build_agent`
|
||||||
|
|
|
||||||
|
|
@ -59,7 +59,7 @@ def register_published_artifacts(
|
||||||
continue
|
continue
|
||||||
current_path = _rel(root, path)
|
current_path = _rel(root, path)
|
||||||
label = str(ref.get("label") or "")
|
label = str(ref.get("label") or "")
|
||||||
media_type = mimetypes.guess_type(path.name)[0]
|
media_type = str(ref.get("media_type") or "") or mimetypes.guess_type(path.name)[0]
|
||||||
size_bytes = path.stat().st_size
|
size_bytes = path.stat().st_size
|
||||||
content_sha256 = _hash_file(path)
|
content_sha256 = _hash_file(path)
|
||||||
statement = pg_insert(Artifact).values(
|
statement = pg_insert(Artifact).values(
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,757 @@
|
||||||
|
"""专业软件任务的校验、幂等持久化和 offer 状态机。"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from hashlib import sha256
|
||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
from sqlalchemy import desc, select
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from core.software_nodes import SUPPORTED_CAPABILITIES
|
||||||
|
from core.storage.engine import session_scope
|
||||||
|
from core.storage.models import Artifact, SoftwareJob, SoftwareNode, Task
|
||||||
|
|
||||||
|
OFFER_SECONDS = 60
|
||||||
|
ALLOWED_PLOT_TYPES = frozenset(
|
||||||
|
{"line", "scatter", "line_scatter"}
|
||||||
|
)
|
||||||
|
ALLOWED_OUTPUT_FORMATS = frozenset({"opju", "png", "svg", "pdf"})
|
||||||
|
ALLOWED_INPUT_SUFFIXES = frozenset({".csv", ".xlsx", ".json"})
|
||||||
|
MAX_INPUT_BYTES = 100 * 1024 * 1024
|
||||||
|
MAX_OUTPUT_ARTIFACT_BYTES = 256 * 1024 * 1024
|
||||||
|
class SoftwareJobError(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
MAX_OUTPUT_TOTAL_BYTES = 512 * 1024 * 1024
|
||||||
|
OUTPUT_ARTIFACTS = {
|
||||||
|
"project": ("project.opju", "application/x-origin-project", "opju"),
|
||||||
|
"figure_png": ("figure.png", "image/png", "png"),
|
||||||
|
"figure_svg": ("figure.svg", "image/svg+xml", "svg"),
|
||||||
|
"figure_pdf": ("figure.pdf", "application/pdf", "pdf"),
|
||||||
|
"plot_spec": ("plot-spec.json", "application/json", None),
|
||||||
|
"provenance": ("provenance.json", "application/json", None),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _has_only(value: dict, fields: set[str]) -> bool:
|
||||||
|
return set(value).issubset(fields)
|
||||||
|
|
||||||
|
|
||||||
|
def _canonical_request(request: dict) -> tuple[dict, str]:
|
||||||
|
if not isinstance(request, dict) or set(request) != {"schema_version", "input", "plot", "output"}:
|
||||||
|
raise SoftwareJobError("invalid origin plot request fields")
|
||||||
|
if request.get("schema_version") != 1:
|
||||||
|
raise SoftwareJobError("unsupported origin plot schema version")
|
||||||
|
input_spec = request.get("input")
|
||||||
|
plot = request.get("plot")
|
||||||
|
output = request.get("output")
|
||||||
|
if not all(isinstance(item, dict) for item in (input_spec, plot, output)):
|
||||||
|
raise SoftwareJobError("origin plot request sections must be objects")
|
||||||
|
if not _has_only(input_spec, {"input_id", "sheet"}):
|
||||||
|
raise SoftwareJobError("unsupported origin input fields")
|
||||||
|
try:
|
||||||
|
UUID(str(input_spec.get("input_id") or ""))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise SoftwareJobError("input.input_id must be an artifact UUID") from exc
|
||||||
|
if "sheet" in input_spec and (
|
||||||
|
not isinstance(input_spec["sheet"], str) or not 1 <= len(input_spec["sheet"]) <= 128
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("input.sheet must be a string")
|
||||||
|
if not _has_only(
|
||||||
|
plot,
|
||||||
|
{"type", "x", "y", "template", "title", "x_axis", "y_axis", "legend", "error_bars"},
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("unsupported origin plot fields")
|
||||||
|
if plot.get("type") not in ALLOWED_PLOT_TYPES:
|
||||||
|
raise SoftwareJobError("unsupported origin plot type")
|
||||||
|
if "title" in plot and (
|
||||||
|
not isinstance(plot["title"], str) or len(plot["title"]) > 500
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("plot.title must be a string")
|
||||||
|
if plot.get("template", "publication_double_column") != "publication_double_column":
|
||||||
|
raise SoftwareJobError("unsupported origin plot template")
|
||||||
|
x_column = plot.get("x")
|
||||||
|
y_columns = plot.get("y")
|
||||||
|
if not isinstance(x_column, str) or not 1 <= len(x_column) <= 128:
|
||||||
|
raise SoftwareJobError("plot.x must be a column name")
|
||||||
|
if isinstance(y_columns, str):
|
||||||
|
y_columns = [y_columns]
|
||||||
|
if (
|
||||||
|
not isinstance(y_columns, list)
|
||||||
|
or not 1 <= len(y_columns) <= 16
|
||||||
|
or len(y_columns) != len(set(y_columns))
|
||||||
|
or any(not isinstance(item, str) or not 1 <= len(item) <= 128 for item in y_columns)
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("plot.y must contain 1 to 16 unique column names")
|
||||||
|
for axis_name in ("x_axis", "y_axis"):
|
||||||
|
axis = plot.get(axis_name)
|
||||||
|
if axis is not None and (
|
||||||
|
not isinstance(axis, dict)
|
||||||
|
or not _has_only(axis, {"title", "unit", "scale"})
|
||||||
|
or axis.get("scale", "linear") != "linear"
|
||||||
|
or any(
|
||||||
|
name in axis and not isinstance(axis[name], str)
|
||||||
|
for name in ("title", "unit")
|
||||||
|
)
|
||||||
|
):
|
||||||
|
raise SoftwareJobError(f"invalid {axis_name}")
|
||||||
|
legend = plot.get("legend")
|
||||||
|
if legend is not None and (
|
||||||
|
not isinstance(legend, dict)
|
||||||
|
or not _has_only(legend, {"enabled", "position"})
|
||||||
|
or ("enabled" in legend and not isinstance(legend["enabled"], bool))
|
||||||
|
or legend.get("enabled", True) is not True
|
||||||
|
or legend.get("position", "top_right") != "top_right"
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("invalid plot.legend")
|
||||||
|
if plot.get("error_bars") is not None:
|
||||||
|
raise SoftwareJobError("error bars are not supported in origin.plot@v1")
|
||||||
|
if not _has_only(output, {"formats", "dpi", "capture_screenshots", "record_video"}):
|
||||||
|
raise SoftwareJobError("unsupported origin output fields")
|
||||||
|
if any(
|
||||||
|
name in output and not isinstance(output[name], bool)
|
||||||
|
for name in ("capture_screenshots", "record_video")
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("origin output capture flags must be boolean")
|
||||||
|
if output.get("record_video", False):
|
||||||
|
raise SoftwareJobError("origin video recording is not supported")
|
||||||
|
dpi = output.get("dpi", 300)
|
||||||
|
if not isinstance(dpi, int) or isinstance(dpi, bool) or not 72 <= dpi <= 1200:
|
||||||
|
raise SoftwareJobError("output.dpi must be between 72 and 1200")
|
||||||
|
formats = output.get("formats")
|
||||||
|
if (
|
||||||
|
not isinstance(formats, list)
|
||||||
|
or not formats
|
||||||
|
or len(formats) != len(set(formats))
|
||||||
|
or any(item not in ALLOWED_OUTPUT_FORMATS for item in formats)
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("output.formats contains unsupported values")
|
||||||
|
encoded = json.dumps(request, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
||||||
|
if len(encoded.encode("utf-8")) > 256 * 1024:
|
||||||
|
raise SoftwareJobError("origin plot request is too large")
|
||||||
|
normalized = json.loads(encoded)
|
||||||
|
return normalized, sha256(encoded.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _job_dict(row: SoftwareJob) -> dict:
|
||||||
|
return {
|
||||||
|
"job_id": str(row.job_id),
|
||||||
|
"task_id": str(row.task_id),
|
||||||
|
"capability": row.capability,
|
||||||
|
"request_digest": row.request_digest,
|
||||||
|
"node_id": str(row.node_id) if row.node_id else None,
|
||||||
|
"status": row.status,
|
||||||
|
"stage": row.stage,
|
||||||
|
"progress": row.progress,
|
||||||
|
"metrics": row.metrics,
|
||||||
|
"error": row.error,
|
||||||
|
"artifact_manifest": row.artifact_manifest,
|
||||||
|
"created_at": row.created_at.isoformat() if row.created_at else None,
|
||||||
|
"started_at": row.started_at.isoformat() if row.started_at else None,
|
||||||
|
"terminal_at": row.terminal_at.isoformat() if row.terminal_at else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def list_jobs(
|
||||||
|
user_id: UUID,
|
||||||
|
*,
|
||||||
|
task_id: UUID | None = None,
|
||||||
|
active_only: bool = False,
|
||||||
|
limit: int = 50,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""列出用户的软件任务;用于全局 Job 中心和 Agent 查询。"""
|
||||||
|
limit = max(1, min(int(limit), 100))
|
||||||
|
with session_scope() as session:
|
||||||
|
statement = (
|
||||||
|
select(SoftwareJob, Task.name, SoftwareNode.name)
|
||||||
|
.join(Task, Task.task_id == SoftwareJob.task_id)
|
||||||
|
.outerjoin(SoftwareNode, SoftwareNode.node_id == SoftwareJob.node_id)
|
||||||
|
.where(SoftwareJob.user_id == user_id)
|
||||||
|
)
|
||||||
|
if task_id is not None:
|
||||||
|
statement = statement.where(SoftwareJob.task_id == task_id)
|
||||||
|
if active_only:
|
||||||
|
statement = statement.where(
|
||||||
|
SoftwareJob.status.in_(
|
||||||
|
{
|
||||||
|
"queued",
|
||||||
|
"offered",
|
||||||
|
"dispatched",
|
||||||
|
"running",
|
||||||
|
"disconnected",
|
||||||
|
"cancelling",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
rows = session.execute(
|
||||||
|
statement.order_by(
|
||||||
|
desc(SoftwareJob.created_at), desc(SoftwareJob.job_id)
|
||||||
|
).limit(limit)
|
||||||
|
).all()
|
||||||
|
results: list[dict] = []
|
||||||
|
for job, task_name, node_name in rows:
|
||||||
|
item = _job_dict(job)
|
||||||
|
item.update(
|
||||||
|
{
|
||||||
|
"task_name": task_name,
|
||||||
|
"node_name": node_name,
|
||||||
|
"input": job.input_manifest,
|
||||||
|
"request_summary": _request_summary(job),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
results.append(item)
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def _request_summary(job: SoftwareJob) -> dict:
|
||||||
|
plot = job.request.get("plot") or {}
|
||||||
|
output = job.request.get("output") or {}
|
||||||
|
return {
|
||||||
|
"display_name": (
|
||||||
|
"Origin 科研绘图"
|
||||||
|
if job.capability == "origin.plot@v1"
|
||||||
|
else job.capability
|
||||||
|
),
|
||||||
|
"title": str(plot.get("title") or ""),
|
||||||
|
"formats": list(output.get("formats") or []),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def request_job_cancel(user_id: UUID, job_id: UUID) -> tuple[dict, dict | None]:
|
||||||
|
"""持久化取消意图;queued 直接终止,已分派任务返回 Node 消息。"""
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
with session_scope() as session:
|
||||||
|
job = session.execute(
|
||||||
|
select(SoftwareJob).where(
|
||||||
|
SoftwareJob.job_id == job_id,
|
||||||
|
SoftwareJob.user_id == user_id,
|
||||||
|
).with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if job is None:
|
||||||
|
raise SoftwareJobError("job not found")
|
||||||
|
if job.status in {"succeeded", "failed", "cancelled"}:
|
||||||
|
return _job_dict(job), None
|
||||||
|
if job.status == "queued" or job.node_id is None or job.lease_id is None:
|
||||||
|
job.status = "cancelled"
|
||||||
|
job.stage = "terminal"
|
||||||
|
job.error = {"code": "USER_CANCELLED", "detail": "Cancelled before dispatch."}
|
||||||
|
job.terminal_at = now
|
||||||
|
return _job_dict(job), None
|
||||||
|
job.status = "cancelling"
|
||||||
|
job.stage = "cancel_requested"
|
||||||
|
payload = {
|
||||||
|
"job_id": str(job.job_id),
|
||||||
|
"lease_id": str(job.lease_id),
|
||||||
|
"request_digest": job.request_digest,
|
||||||
|
}
|
||||||
|
return _job_dict(job), {"node_id": job.node_id, "payload": payload}
|
||||||
|
|
||||||
|
|
||||||
|
def pending_node_cancellations(node_id: UUID) -> list[dict]:
|
||||||
|
"""节点重连或心跳时重放未确认的取消意图。"""
|
||||||
|
with session_scope() as session:
|
||||||
|
rows = session.execute(
|
||||||
|
select(SoftwareJob).where(
|
||||||
|
SoftwareJob.node_id == node_id,
|
||||||
|
SoftwareJob.status == "cancelling",
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"job_id": str(job.job_id),
|
||||||
|
"lease_id": str(job.lease_id),
|
||||||
|
"request_digest": job.request_digest,
|
||||||
|
}
|
||||||
|
for job in rows
|
||||||
|
if job.lease_id is not None
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def create_job(
|
||||||
|
user_id: UUID,
|
||||||
|
task_id: UUID,
|
||||||
|
*,
|
||||||
|
idempotency_key: str,
|
||||||
|
capability: str,
|
||||||
|
request: dict,
|
||||||
|
) -> tuple[dict, bool]:
|
||||||
|
key = idempotency_key.strip()
|
||||||
|
if not key or len(key) > 200:
|
||||||
|
raise SoftwareJobError("idempotency_key must contain 1 to 200 characters")
|
||||||
|
if capability not in SUPPORTED_CAPABILITIES:
|
||||||
|
raise SoftwareJobError("unsupported capability")
|
||||||
|
normalized, digest = _canonical_request(request)
|
||||||
|
with session_scope() as session:
|
||||||
|
task = session.execute(
|
||||||
|
select(Task.task_id).where(Task.task_id == task_id, Task.user_id == user_id)
|
||||||
|
).first()
|
||||||
|
if task is None:
|
||||||
|
raise SoftwareJobError("task not found")
|
||||||
|
artifact_id = UUID(normalized["input"]["input_id"])
|
||||||
|
artifact = session.execute(
|
||||||
|
select(Artifact).where(
|
||||||
|
Artifact.artifact_id == artifact_id,
|
||||||
|
Artifact.user_id == user_id,
|
||||||
|
Artifact.status == "active",
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if artifact is None:
|
||||||
|
raise SoftwareJobError("input artifact not found")
|
||||||
|
suffix = "." + artifact.current_path.rsplit(".", 1)[-1].lower() if "." in artifact.current_path else ""
|
||||||
|
if suffix not in ALLOWED_INPUT_SUFFIXES:
|
||||||
|
raise SoftwareJobError("input artifact type is not supported")
|
||||||
|
if (
|
||||||
|
artifact.size_bytes is None
|
||||||
|
or artifact.size_bytes < 0
|
||||||
|
or artifact.size_bytes > MAX_INPUT_BYTES
|
||||||
|
or not artifact.content_sha256
|
||||||
|
or len(artifact.content_sha256) != 64
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("input artifact metadata is incomplete or too large")
|
||||||
|
input_manifest = {
|
||||||
|
"artifact_id": str(artifact.artifact_id),
|
||||||
|
"filename": artifact.current_path.replace("\\", "/").rsplit("/", 1)[-1],
|
||||||
|
"size_bytes": artifact.size_bytes,
|
||||||
|
"sha256": artifact.content_sha256,
|
||||||
|
}
|
||||||
|
existing = session.execute(
|
||||||
|
select(SoftwareJob).where(
|
||||||
|
SoftwareJob.user_id == user_id,
|
||||||
|
SoftwareJob.idempotency_key == key,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if existing is not None:
|
||||||
|
if (
|
||||||
|
existing.task_id != task_id
|
||||||
|
or existing.capability != capability
|
||||||
|
or existing.request_digest != digest
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("idempotency key was already used for a different request")
|
||||||
|
return _job_dict(existing), False
|
||||||
|
row = SoftwareJob(
|
||||||
|
job_id=uuid4(),
|
||||||
|
user_id=user_id,
|
||||||
|
task_id=task_id,
|
||||||
|
idempotency_key=key,
|
||||||
|
capability=capability,
|
||||||
|
request=normalized,
|
||||||
|
request_digest=digest,
|
||||||
|
input_manifest=input_manifest,
|
||||||
|
status="queued",
|
||||||
|
stage="",
|
||||||
|
metrics={},
|
||||||
|
error={},
|
||||||
|
artifact_manifest=[],
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with session.begin_nested():
|
||||||
|
session.add(row)
|
||||||
|
session.flush()
|
||||||
|
return _job_dict(row), True
|
||||||
|
except IntegrityError:
|
||||||
|
existing = session.execute(
|
||||||
|
select(SoftwareJob).where(
|
||||||
|
SoftwareJob.user_id == user_id,
|
||||||
|
SoftwareJob.idempotency_key == key,
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
if (
|
||||||
|
existing.task_id != task_id
|
||||||
|
or existing.capability != capability
|
||||||
|
or existing.request_digest != digest
|
||||||
|
):
|
||||||
|
raise SoftwareJobError(
|
||||||
|
"idempotency key was already used for a different request"
|
||||||
|
)
|
||||||
|
return _job_dict(existing), False
|
||||||
|
|
||||||
|
|
||||||
|
def get_job(user_id: UUID, job_id: UUID) -> dict | None:
|
||||||
|
with session_scope() as session:
|
||||||
|
row = session.execute(
|
||||||
|
select(SoftwareJob).where(SoftwareJob.job_id == job_id, SoftwareJob.user_id == user_id)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
return _job_dict(row) if row else None
|
||||||
|
|
||||||
|
|
||||||
|
def offer_next_job(node_ids: set[UUID]) -> dict | None:
|
||||||
|
"""从当前进程实际在线的节点中选择一个,为最早 queued job 创建短租约。"""
|
||||||
|
if not node_ids:
|
||||||
|
return None
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
with session_scope() as session:
|
||||||
|
expired = session.execute(
|
||||||
|
select(SoftwareJob)
|
||||||
|
.where(
|
||||||
|
SoftwareJob.status == "offered",
|
||||||
|
SoftwareJob.lease_expires_at <= now,
|
||||||
|
)
|
||||||
|
.with_for_update(skip_locked=True)
|
||||||
|
).scalars()
|
||||||
|
for item in expired:
|
||||||
|
item.status = "queued"
|
||||||
|
item.node_id = None
|
||||||
|
item.lease_id = None
|
||||||
|
item.lease_expires_at = None
|
||||||
|
job = session.execute(
|
||||||
|
select(SoftwareJob)
|
||||||
|
.where(SoftwareJob.status == "queued")
|
||||||
|
.order_by(SoftwareJob.created_at, SoftwareJob.job_id)
|
||||||
|
.with_for_update(skip_locked=True)
|
||||||
|
.limit(1)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if job is None:
|
||||||
|
return None
|
||||||
|
busy_node_ids = set(
|
||||||
|
session.execute(
|
||||||
|
select(SoftwareJob.node_id).where(
|
||||||
|
SoftwareJob.node_id.is_not(None),
|
||||||
|
SoftwareJob.status.in_({"offered", "dispatched", "running"}),
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
nodes = session.execute(
|
||||||
|
select(SoftwareNode)
|
||||||
|
.where(SoftwareNode.node_id.in_(node_ids), SoftwareNode.status == "online")
|
||||||
|
.order_by(SoftwareNode.last_seen_at.desc())
|
||||||
|
).scalars()
|
||||||
|
node = next(
|
||||||
|
(
|
||||||
|
item
|
||||||
|
for item in nodes
|
||||||
|
if item.node_id not in busy_node_ids
|
||||||
|
and job.capability in item.capabilities
|
||||||
|
and int((item.runtime or {}).get("available_slots") or 0) > 0
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
if node is None:
|
||||||
|
return None
|
||||||
|
lease_id = uuid4()
|
||||||
|
expires_at = now + timedelta(seconds=OFFER_SECONDS)
|
||||||
|
job.node_id = node.node_id
|
||||||
|
job.lease_id = lease_id
|
||||||
|
job.lease_expires_at = expires_at
|
||||||
|
job.status = "offered"
|
||||||
|
return {
|
||||||
|
"node_id": node.node_id,
|
||||||
|
"payload": {
|
||||||
|
"job_id": str(job.job_id),
|
||||||
|
"lease_id": str(lease_id),
|
||||||
|
"lease_expires_at": expires_at.isoformat(),
|
||||||
|
"capability": job.capability,
|
||||||
|
"request_digest": job.request_digest,
|
||||||
|
"request": job.request,
|
||||||
|
"input_transfer": {
|
||||||
|
**job.input_manifest,
|
||||||
|
"download_path": f"/v1/software-jobs/{job.job_id}/input",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_job_input(node_id: UUID, job_id: UUID) -> dict | None:
|
||||||
|
"""返回任务绑定的 artifact 定位信息;调用方仍需在 user_root 内安全解析。"""
|
||||||
|
with session_scope() as session:
|
||||||
|
job = session.execute(
|
||||||
|
select(SoftwareJob).where(
|
||||||
|
SoftwareJob.job_id == job_id,
|
||||||
|
SoftwareJob.node_id == node_id,
|
||||||
|
SoftwareJob.status.in_({"offered", "dispatched", "running", "disconnected"}),
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if job is None:
|
||||||
|
return None
|
||||||
|
artifact_id = UUID(job.input_manifest["artifact_id"])
|
||||||
|
artifact = session.execute(
|
||||||
|
select(Artifact).where(
|
||||||
|
Artifact.artifact_id == artifact_id,
|
||||||
|
Artifact.user_id == job.user_id,
|
||||||
|
Artifact.status == "active",
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if artifact is None:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"user_id": job.user_id,
|
||||||
|
"current_path": artifact.current_path,
|
||||||
|
**job.input_manifest,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_job_output_context(node_id: UUID, job_id: UUID, lease_id: UUID, digest: str) -> dict | None:
|
||||||
|
"""返回 Node 输出上传上下文,不向 Node 暴露任何云端文件路径。"""
|
||||||
|
with session_scope() as session:
|
||||||
|
row = session.execute(
|
||||||
|
select(SoftwareJob, Task.working_dir)
|
||||||
|
.join(Task, Task.task_id == SoftwareJob.task_id)
|
||||||
|
.where(SoftwareJob.job_id == job_id)
|
||||||
|
).one_or_none()
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
job, working_dir = row
|
||||||
|
if (
|
||||||
|
job.node_id != node_id
|
||||||
|
or job.lease_id != lease_id
|
||||||
|
or job.request_digest != digest
|
||||||
|
or job.status not in {"dispatched", "running", "disconnected", "succeeded"}
|
||||||
|
):
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"user_id": job.user_id,
|
||||||
|
"task_id": job.task_id,
|
||||||
|
"working_dir": working_dir,
|
||||||
|
"request": job.request,
|
||||||
|
"status": job.status,
|
||||||
|
"artifact_manifest": job.artifact_manifest,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def validate_output_manifest(request: dict, manifest: object) -> list[dict]:
|
||||||
|
if not isinstance(manifest, list):
|
||||||
|
raise SoftwareJobError("job artifact manifest must be a list")
|
||||||
|
requested_formats = set(request.get("output", {}).get("formats") or [])
|
||||||
|
expected_ids = {"plot_spec", "provenance"}
|
||||||
|
expected_ids.update(
|
||||||
|
artifact_id
|
||||||
|
for artifact_id, (_, _, output_format) in OUTPUT_ARTIFACTS.items()
|
||||||
|
if output_format in requested_formats
|
||||||
|
)
|
||||||
|
if len(manifest) != len(expected_ids):
|
||||||
|
raise SoftwareJobError("job artifact manifest is incomplete")
|
||||||
|
normalized: list[dict] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
total = 0
|
||||||
|
for raw in manifest:
|
||||||
|
if not isinstance(raw, dict) or set(raw) != {
|
||||||
|
"artifact_id", "filename", "media_type", "size_bytes", "sha256"
|
||||||
|
}:
|
||||||
|
raise SoftwareJobError("job artifact manifest entry is invalid")
|
||||||
|
local_id = raw.get("artifact_id")
|
||||||
|
if local_id not in expected_ids or local_id in seen:
|
||||||
|
raise SoftwareJobError("job artifact manifest identity is invalid")
|
||||||
|
filename, media_type, _ = OUTPUT_ARTIFACTS[local_id]
|
||||||
|
size = raw.get("size_bytes")
|
||||||
|
digest = raw.get("sha256")
|
||||||
|
if raw.get("filename") != filename or raw.get("media_type") != media_type:
|
||||||
|
raise SoftwareJobError("job artifact manifest metadata does not match its identity")
|
||||||
|
if not isinstance(size, int) or isinstance(size, bool) or not 1 <= size <= MAX_OUTPUT_ARTIFACT_BYTES:
|
||||||
|
raise SoftwareJobError("job output artifact size is invalid")
|
||||||
|
if not isinstance(digest, str) or not re.fullmatch(r"[0-9a-f]{64}", digest):
|
||||||
|
raise SoftwareJobError("job output artifact digest is invalid")
|
||||||
|
total += size
|
||||||
|
seen.add(local_id)
|
||||||
|
normalized.append(dict(raw))
|
||||||
|
if seen != expected_ids or total > MAX_OUTPUT_TOTAL_BYTES:
|
||||||
|
raise SoftwareJobError("job artifact manifest is incomplete or too large")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def abandon_offer(node_id: UUID, payload: dict) -> None:
|
||||||
|
"""WebSocket 发送失败时只回滚仍属于该连接租约的 offer。"""
|
||||||
|
try:
|
||||||
|
job_id = UUID(str(payload.get("job_id", "")))
|
||||||
|
lease_id = UUID(str(payload.get("lease_id", "")))
|
||||||
|
except ValueError:
|
||||||
|
return
|
||||||
|
with session_scope() as session:
|
||||||
|
job = session.execute(
|
||||||
|
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if (
|
||||||
|
job is not None
|
||||||
|
and job.status == "offered"
|
||||||
|
and job.node_id == node_id
|
||||||
|
and job.lease_id == lease_id
|
||||||
|
):
|
||||||
|
job.status = "queued"
|
||||||
|
job.node_id = None
|
||||||
|
job.lease_id = None
|
||||||
|
job.lease_expires_at = None
|
||||||
|
|
||||||
|
|
||||||
|
def respond_to_offer(node_id: UUID, *, accepted: bool, payload: dict) -> None:
|
||||||
|
try:
|
||||||
|
job_id = UUID(str(payload.get("job_id", "")))
|
||||||
|
lease_id = UUID(str(payload.get("lease_id", "")))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise SoftwareJobError("invalid job offer response identity") from exc
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
with session_scope() as session:
|
||||||
|
job = session.execute(
|
||||||
|
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if job is None or job.node_id != node_id or job.lease_id != lease_id:
|
||||||
|
raise SoftwareJobError("job offer is stale or does not belong to this node")
|
||||||
|
if (
|
||||||
|
accepted
|
||||||
|
and job.status in {"dispatched", "running", "cancelling", "succeeded", "failed", "cancelled"}
|
||||||
|
and payload.get("request_digest") == job.request_digest
|
||||||
|
):
|
||||||
|
return
|
||||||
|
if job.status != "offered":
|
||||||
|
raise SoftwareJobError("job offer is stale or does not belong to this node")
|
||||||
|
if job.lease_expires_at is None or job.lease_expires_at <= now:
|
||||||
|
job.status = "queued"
|
||||||
|
job.node_id = None
|
||||||
|
job.lease_id = None
|
||||||
|
job.lease_expires_at = None
|
||||||
|
raise SoftwareJobError("job offer has expired")
|
||||||
|
if accepted:
|
||||||
|
if payload.get("request_digest") != job.request_digest:
|
||||||
|
raise SoftwareJobError("job request digest mismatch")
|
||||||
|
job.status = "dispatched"
|
||||||
|
job.stage = "accepted"
|
||||||
|
job.error = {}
|
||||||
|
else:
|
||||||
|
job.status = "queued"
|
||||||
|
job.node_id = None
|
||||||
|
job.lease_id = None
|
||||||
|
job.lease_expires_at = None
|
||||||
|
job.error = {"code": "node_rejected", "detail": str(payload.get("reason") or "")[:500]}
|
||||||
|
|
||||||
|
|
||||||
|
def update_job_state(node_id: UUID, payload: dict) -> None:
|
||||||
|
job_id, lease_id, digest = _message_identity(payload)
|
||||||
|
stage = str(payload.get("stage") or "")
|
||||||
|
progress = payload.get("progress")
|
||||||
|
metrics = payload.get("metrics") or {}
|
||||||
|
if not stage or len(stage) > 100:
|
||||||
|
raise SoftwareJobError("job stage is required")
|
||||||
|
if not isinstance(progress, int) or isinstance(progress, bool) or not 0 <= progress <= 100:
|
||||||
|
raise SoftwareJobError("job progress must be between 0 and 100")
|
||||||
|
if not isinstance(metrics, dict) or len(json.dumps(metrics, ensure_ascii=False)) > 64 * 1024:
|
||||||
|
raise SoftwareJobError("job metrics are invalid")
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
with session_scope() as session:
|
||||||
|
job = session.execute(
|
||||||
|
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
_assert_job_message(job, node_id, lease_id, digest)
|
||||||
|
if job.status in {"succeeded", "failed", "cancelled", "cancelling"}:
|
||||||
|
return
|
||||||
|
if not _can_accept_state(job.status):
|
||||||
|
raise SoftwareJobError("job state cannot advance from its current status")
|
||||||
|
job.status = (
|
||||||
|
"dispatched"
|
||||||
|
if stage in {"accepted", "waiting_input", "ready_to_run"}
|
||||||
|
else "running"
|
||||||
|
)
|
||||||
|
job.stage = stage
|
||||||
|
job.progress = progress
|
||||||
|
job.metrics = metrics
|
||||||
|
if job.status == "running" and job.started_at is None:
|
||||||
|
job.started_at = now
|
||||||
|
|
||||||
|
|
||||||
|
def record_job_terminal(node_id: UUID, payload: dict) -> None:
|
||||||
|
job_id, lease_id, digest = _message_identity(payload)
|
||||||
|
terminal_status = payload.get("status")
|
||||||
|
if terminal_status not in {"succeeded", "failed", "cancelled"}:
|
||||||
|
raise SoftwareJobError("invalid job terminal status")
|
||||||
|
error = payload.get("error") or {}
|
||||||
|
manifest = payload.get("artifact_manifest") or []
|
||||||
|
if not isinstance(error, dict) or len(json.dumps(error, ensure_ascii=False)) > 64 * 1024:
|
||||||
|
raise SoftwareJobError("job terminal error is invalid")
|
||||||
|
if not isinstance(manifest, list) or len(json.dumps(manifest, ensure_ascii=False)) > 256 * 1024:
|
||||||
|
raise SoftwareJobError("job artifact manifest is invalid")
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
with session_scope() as session:
|
||||||
|
job = session.execute(
|
||||||
|
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
_assert_job_message(job, node_id, lease_id, digest)
|
||||||
|
if terminal_status == "succeeded":
|
||||||
|
expected = validate_output_manifest(
|
||||||
|
job.request,
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"artifact_id": item.get("source_artifact_id"),
|
||||||
|
"filename": item.get("filename"),
|
||||||
|
"media_type": item.get("media_type"),
|
||||||
|
"size_bytes": item.get("size_bytes"),
|
||||||
|
"sha256": item.get("sha256"),
|
||||||
|
}
|
||||||
|
for item in manifest
|
||||||
|
if isinstance(item, dict)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
if len(expected) != len(manifest) or any(
|
||||||
|
not isinstance(item.get("artifact_id"), str)
|
||||||
|
or not _is_uuid(item["artifact_id"])
|
||||||
|
or not isinstance(item.get("path"), str)
|
||||||
|
or not item["path"].startswith(f"origin/{job.job_id}/")
|
||||||
|
for item in manifest
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("successful job artifacts have not been published")
|
||||||
|
if job.status in {"succeeded", "failed", "cancelled"}:
|
||||||
|
if job.status != terminal_status:
|
||||||
|
raise SoftwareJobError("job terminal status conflicts with existing terminal")
|
||||||
|
return
|
||||||
|
if job.status not in {"offered", "dispatched", "running", "disconnected", "cancelling"}:
|
||||||
|
raise SoftwareJobError("job terminal cannot advance from its current status")
|
||||||
|
job.status = terminal_status
|
||||||
|
job.stage = "terminal"
|
||||||
|
job.progress = 100 if terminal_status == "succeeded" else job.progress
|
||||||
|
job.error = error
|
||||||
|
job.artifact_manifest = manifest
|
||||||
|
job.terminal_at = now
|
||||||
|
|
||||||
|
|
||||||
|
def _is_uuid(value: str) -> bool:
|
||||||
|
try:
|
||||||
|
UUID(value)
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def mark_node_jobs_disconnected(node_id: UUID) -> None:
|
||||||
|
"""连接丢失后保留 Node 归属和 lease,禁止任务被自动重派。"""
|
||||||
|
with session_scope() as session:
|
||||||
|
jobs = session.execute(
|
||||||
|
select(SoftwareJob)
|
||||||
|
.where(
|
||||||
|
SoftwareJob.node_id == node_id,
|
||||||
|
SoftwareJob.status.in_({"dispatched", "running"}),
|
||||||
|
)
|
||||||
|
.with_for_update()
|
||||||
|
).scalars()
|
||||||
|
for job in jobs:
|
||||||
|
job.status = "disconnected"
|
||||||
|
|
||||||
|
|
||||||
|
def _can_accept_state(status: str) -> bool:
|
||||||
|
return status in {"offered", "dispatched", "running", "disconnected"}
|
||||||
|
|
||||||
|
|
||||||
|
def _message_identity(payload: dict) -> tuple[UUID, UUID, str]:
|
||||||
|
try:
|
||||||
|
job_id = UUID(str(payload.get("job_id", "")))
|
||||||
|
lease_id = UUID(str(payload.get("lease_id", "")))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise SoftwareJobError("invalid job message identity") from exc
|
||||||
|
digest = str(payload.get("request_digest") or "")
|
||||||
|
if len(digest) != 64:
|
||||||
|
raise SoftwareJobError("invalid job request digest")
|
||||||
|
return job_id, lease_id, digest
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_job_message(
|
||||||
|
job: SoftwareJob | None,
|
||||||
|
node_id: UUID,
|
||||||
|
lease_id: UUID,
|
||||||
|
digest: str,
|
||||||
|
) -> None:
|
||||||
|
if (
|
||||||
|
job is None
|
||||||
|
or job.node_id != node_id
|
||||||
|
or job.lease_id != lease_id
|
||||||
|
or job.request_digest != digest
|
||||||
|
):
|
||||||
|
raise SoftwareJobError("job message does not belong to this node or lease")
|
||||||
|
|
@ -11,13 +11,13 @@ import bcrypt
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
|
|
||||||
from core.storage.engine import session_scope
|
from core.storage.engine import session_scope
|
||||||
from core.storage.models import ComputeNode, ComputeNodeEnrollment
|
from core.storage.models import SoftwareNode, SoftwareNodeEnrollment
|
||||||
|
|
||||||
SUPPORTED_CAPABILITIES = frozenset({"origin.plot@v1"})
|
SUPPORTED_CAPABILITIES = frozenset({"origin.plot@v1"})
|
||||||
MAX_ENROLLMENT_FAILURES = 5
|
MAX_ENROLLMENT_FAILURES = 5
|
||||||
|
|
||||||
|
|
||||||
class ComputeNodeError(Exception):
|
class SoftwareNodeError(Exception):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -46,12 +46,12 @@ def create_enrollment(
|
||||||
) -> dict:
|
) -> dict:
|
||||||
allowed = list(dict.fromkeys(capabilities or ["origin.plot@v1"]))
|
allowed = list(dict.fromkeys(capabilities or ["origin.plot@v1"]))
|
||||||
if not allowed or any(item not in SUPPORTED_CAPABILITIES for item in allowed):
|
if not allowed or any(item not in SUPPORTED_CAPABILITIES for item in allowed):
|
||||||
raise ComputeNodeError("unsupported capability")
|
raise SoftwareNodeError("unsupported capability")
|
||||||
if not 60 <= ttl_seconds <= 3600:
|
if not 60 <= ttl_seconds <= 3600:
|
||||||
raise ComputeNodeError("ttl_seconds must be between 60 and 3600")
|
raise SoftwareNodeError("ttl_seconds must be between 60 and 3600")
|
||||||
code = "ZCN-" + secrets.token_hex(16).upper()
|
code = "ZCN-" + secrets.token_hex(16).upper()
|
||||||
expires_at = datetime.now(timezone.utc) + timedelta(seconds=ttl_seconds)
|
expires_at = datetime.now(timezone.utc) + timedelta(seconds=ttl_seconds)
|
||||||
row = ComputeNodeEnrollment(
|
row = SoftwareNodeEnrollment(
|
||||||
enrollment_id=uuid4(),
|
enrollment_id=uuid4(),
|
||||||
code_hash=_enrollment_digest(code),
|
code_hash=_enrollment_digest(code),
|
||||||
expected_name=(expected_name or "").strip() or None,
|
expected_name=(expected_name or "").strip() or None,
|
||||||
|
|
@ -80,22 +80,22 @@ def enroll_node(
|
||||||
name = node_name.strip()
|
name = node_name.strip()
|
||||||
requested = list(dict.fromkeys(capabilities))
|
requested = list(dict.fromkeys(capabilities))
|
||||||
if not name or not requested:
|
if not name or not requested:
|
||||||
raise ComputeNodeError("node_name and capabilities are required")
|
raise SoftwareNodeError("node_name and capabilities are required")
|
||||||
now = datetime.now(timezone.utc)
|
now = datetime.now(timezone.utc)
|
||||||
error: str | None = None
|
error: str | None = None
|
||||||
with session_scope() as session:
|
with session_scope() as session:
|
||||||
enrollment = session.execute(
|
enrollment = session.execute(
|
||||||
select(ComputeNodeEnrollment)
|
select(SoftwareNodeEnrollment)
|
||||||
.where(
|
.where(
|
||||||
ComputeNodeEnrollment.code_hash == _enrollment_digest(enrollment_code),
|
SoftwareNodeEnrollment.code_hash == _enrollment_digest(enrollment_code),
|
||||||
ComputeNodeEnrollment.consumed_at.is_(None),
|
SoftwareNodeEnrollment.consumed_at.is_(None),
|
||||||
ComputeNodeEnrollment.expires_at > now,
|
SoftwareNodeEnrollment.expires_at > now,
|
||||||
ComputeNodeEnrollment.failed_attempts < MAX_ENROLLMENT_FAILURES,
|
SoftwareNodeEnrollment.failed_attempts < MAX_ENROLLMENT_FAILURES,
|
||||||
)
|
)
|
||||||
.with_for_update()
|
.with_for_update()
|
||||||
).scalar_one_or_none()
|
).scalar_one_or_none()
|
||||||
if enrollment is None:
|
if enrollment is None:
|
||||||
raise ComputeNodeError("invalid or expired enrollment code")
|
raise SoftwareNodeError("invalid or expired enrollment code")
|
||||||
enrollment.failed_attempts += 1
|
enrollment.failed_attempts += 1
|
||||||
if enrollment.expected_name and enrollment.expected_name != name:
|
if enrollment.expected_name and enrollment.expected_name != name:
|
||||||
error = "node name does not match enrollment"
|
error = "node name does not match enrollment"
|
||||||
|
|
@ -103,7 +103,7 @@ def enroll_node(
|
||||||
error = "capability is not allowed by enrollment"
|
error = "capability is not allowed by enrollment"
|
||||||
else:
|
else:
|
||||||
existing = session.execute(
|
existing = session.execute(
|
||||||
select(ComputeNode.node_id).where(ComputeNode.install_id == install_id)
|
select(SoftwareNode.node_id).where(SoftwareNode.install_id == install_id)
|
||||||
).first()
|
).first()
|
||||||
if existing is not None:
|
if existing is not None:
|
||||||
error = "install is already enrolled"
|
error = "install is already enrolled"
|
||||||
|
|
@ -111,7 +111,7 @@ def enroll_node(
|
||||||
token = secrets.token_urlsafe(48)
|
token = secrets.token_urlsafe(48)
|
||||||
node_id = uuid4()
|
node_id = uuid4()
|
||||||
session.add(
|
session.add(
|
||||||
ComputeNode(
|
SoftwareNode(
|
||||||
node_id=node_id,
|
node_id=node_id,
|
||||||
name=name,
|
name=name,
|
||||||
install_id=install_id,
|
install_id=install_id,
|
||||||
|
|
@ -125,7 +125,7 @@ def enroll_node(
|
||||||
)
|
)
|
||||||
enrollment.consumed_at = now
|
enrollment.consumed_at = now
|
||||||
if error is not None:
|
if error is not None:
|
||||||
raise ComputeNodeError(error)
|
raise SoftwareNodeError(error)
|
||||||
return {
|
return {
|
||||||
"node_id": str(node_id),
|
"node_id": str(node_id),
|
||||||
"node_token": token,
|
"node_token": token,
|
||||||
|
|
@ -136,13 +136,13 @@ def enroll_node(
|
||||||
|
|
||||||
def authenticate_node(node_id: UUID, token: str) -> dict:
|
def authenticate_node(node_id: UUID, token: str) -> dict:
|
||||||
with session_scope() as session:
|
with session_scope() as session:
|
||||||
node = session.get(ComputeNode, node_id)
|
node = session.get(SoftwareNode, node_id)
|
||||||
if (
|
if (
|
||||||
node is None
|
node is None
|
||||||
or node.status == "disabled"
|
or node.status == "disabled"
|
||||||
or not _verify_secret(token, node.token_hash)
|
or not _verify_secret(token, node.token_hash)
|
||||||
):
|
):
|
||||||
raise ComputeNodeError("invalid node credentials")
|
raise SoftwareNodeError("invalid node credentials")
|
||||||
return {
|
return {
|
||||||
"node_id": node.node_id,
|
"node_id": node.node_id,
|
||||||
"install_id": node.install_id,
|
"install_id": node.install_id,
|
||||||
|
|
@ -152,9 +152,9 @@ def authenticate_node(node_id: UUID, token: str) -> dict:
|
||||||
|
|
||||||
def update_node_runtime(node_id: UUID, *, status: str, runtime: dict) -> None:
|
def update_node_runtime(node_id: UUID, *, status: str, runtime: dict) -> None:
|
||||||
with session_scope() as session:
|
with session_scope() as session:
|
||||||
node = session.get(ComputeNode, node_id)
|
node = session.get(SoftwareNode, node_id)
|
||||||
if node is None or node.status == "disabled":
|
if node is None or node.status == "disabled":
|
||||||
raise ComputeNodeError("node is disabled or missing")
|
raise SoftwareNodeError("node is disabled or missing")
|
||||||
node.status = status
|
node.status = status
|
||||||
node.runtime = runtime
|
node.runtime = runtime
|
||||||
node.last_seen_at = datetime.now(timezone.utc)
|
node.last_seen_at = datetime.now(timezone.utc)
|
||||||
|
|
@ -163,14 +163,14 @@ def update_node_runtime(node_id: UUID, *, status: str, runtime: dict) -> None:
|
||||||
def mark_node_offline(node_id: UUID) -> None:
|
def mark_node_offline(node_id: UUID) -> None:
|
||||||
"""仅把活动节点转离线;管理员禁用态不可被断线收尾覆盖。"""
|
"""仅把活动节点转离线;管理员禁用态不可被断线收尾覆盖。"""
|
||||||
with session_scope() as session:
|
with session_scope() as session:
|
||||||
node = session.get(ComputeNode, node_id)
|
node = session.get(SoftwareNode, node_id)
|
||||||
if node is not None and node.status != "disabled":
|
if node is not None and node.status != "disabled":
|
||||||
node.status = "offline"
|
node.status = "offline"
|
||||||
|
|
||||||
|
|
||||||
def set_node_disabled(node_id: UUID, disabled: bool) -> bool:
|
def set_node_disabled(node_id: UUID, disabled: bool) -> bool:
|
||||||
with session_scope() as session:
|
with session_scope() as session:
|
||||||
node = session.get(ComputeNode, node_id)
|
node = session.get(SoftwareNode, node_id)
|
||||||
if node is None:
|
if node is None:
|
||||||
return False
|
return False
|
||||||
node.status = "disabled" if disabled else "offline"
|
node.status = "disabled" if disabled else "offline"
|
||||||
|
|
@ -180,7 +180,7 @@ def set_node_disabled(node_id: UUID, disabled: bool) -> bool:
|
||||||
def delete_node(node_id: UUID) -> bool:
|
def delete_node(node_id: UUID) -> bool:
|
||||||
"""撤销并物理删除节点身份;当前节点表没有任务历史外键。"""
|
"""撤销并物理删除节点身份;当前节点表没有任务历史外键。"""
|
||||||
with session_scope() as session:
|
with session_scope() as session:
|
||||||
node = session.get(ComputeNode, node_id)
|
node = session.get(SoftwareNode, node_id)
|
||||||
if node is None:
|
if node is None:
|
||||||
return False
|
return False
|
||||||
session.delete(node)
|
session.delete(node)
|
||||||
|
|
@ -190,7 +190,7 @@ def delete_node(node_id: UUID) -> bool:
|
||||||
def list_nodes() -> list[dict]:
|
def list_nodes() -> list[dict]:
|
||||||
with session_scope() as session:
|
with session_scope() as session:
|
||||||
rows = (
|
rows = (
|
||||||
session.execute(select(ComputeNode).order_by(ComputeNode.created_at))
|
session.execute(select(SoftwareNode).order_by(SoftwareNode.created_at))
|
||||||
.scalars()
|
.scalars()
|
||||||
.all()
|
.all()
|
||||||
)
|
)
|
||||||
|
|
@ -424,10 +424,10 @@ class ChannelBinding(Base):
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class ComputeNodeEnrollment(Base):
|
class SoftwareNodeEnrollment(Base):
|
||||||
"""Windows Node 一次性注册码;数据库只保存不可逆摘要。"""
|
"""Windows Node 一次性注册码;数据库只保存不可逆摘要。"""
|
||||||
|
|
||||||
__tablename__ = "compute_node_enrollments"
|
__tablename__ = "software_node_enrollments"
|
||||||
enrollment_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
enrollment_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
||||||
code_hash: Mapped[str] = mapped_column(Text, nullable=False, unique=True)
|
code_hash: Mapped[str] = mapped_column(Text, nullable=False, unique=True)
|
||||||
expected_name: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
expected_name: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
|
@ -443,10 +443,10 @@ class ComputeNodeEnrollment(Base):
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class ComputeNode(Base):
|
class SoftwareNode(Base):
|
||||||
"""平台托管的 Windows 执行节点身份与最后一次运行态。"""
|
"""平台托管的 Windows 执行节点身份与最后一次运行态。"""
|
||||||
|
|
||||||
__tablename__ = "compute_nodes"
|
__tablename__ = "software_nodes"
|
||||||
node_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
node_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
||||||
name: Mapped[str] = mapped_column(Text, nullable=False)
|
name: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
install_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), nullable=False, unique=True)
|
install_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), nullable=False, unique=True)
|
||||||
|
|
@ -465,6 +465,49 @@ class ComputeNode(Base):
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareJob(Base):
|
||||||
|
"""专业软件任务账本;请求只保存规范化参数和输入引用。"""
|
||||||
|
|
||||||
|
__tablename__ = "software_jobs"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("user_id", "idempotency_key", name="uq_software_jobs_user_idempotency"),
|
||||||
|
Index("ix_software_jobs_status_created", "status", "created_at"),
|
||||||
|
Index("ix_software_jobs_node_status", "node_id", "status"),
|
||||||
|
)
|
||||||
|
|
||||||
|
job_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
|
||||||
|
user_id: Mapped[UUID] = mapped_column(
|
||||||
|
PG_UUID(as_uuid=True), ForeignKey("users.user_id", ondelete="CASCADE"), nullable=False
|
||||||
|
)
|
||||||
|
task_id: Mapped[UUID] = mapped_column(
|
||||||
|
PG_UUID(as_uuid=True), ForeignKey("tasks.task_id", ondelete="CASCADE"), nullable=False
|
||||||
|
)
|
||||||
|
idempotency_key: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
capability: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
request: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
|
||||||
|
request_digest: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
input_manifest: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
|
||||||
|
node_id: Mapped[Optional[UUID]] = mapped_column(
|
||||||
|
PG_UUID(as_uuid=True), ForeignKey("software_nodes.node_id", ondelete="SET NULL"), nullable=True
|
||||||
|
)
|
||||||
|
lease_id: Mapped[Optional[UUID]] = mapped_column(PG_UUID(as_uuid=True), nullable=True)
|
||||||
|
lease_expires_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
status: Mapped[str] = mapped_column(Text, nullable=False, default="queued", server_default="queued")
|
||||||
|
stage: Mapped[str] = mapped_column(Text, nullable=False, default="", server_default="")
|
||||||
|
progress: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
|
||||||
|
metrics: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
|
||||||
|
error: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
|
||||||
|
artifact_manifest: Mapped[list[Any]] = mapped_column(JSONB, nullable=False, default=list)
|
||||||
|
started_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
terminal_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||||
|
)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), server_default=func.now(), onupdate=func.now(), nullable=False
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class ExternalSystemDefinition(Base):
|
class ExternalSystemDefinition(Base):
|
||||||
"""管理员维护的可信外部系统目录;不含任何用户凭据。"""
|
"""管理员维护的可信外部系统目录;不含任何用户凭据。"""
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -47,6 +47,12 @@ from tools.run_python import RunPythonTool
|
||||||
from tools.schedule import (
|
from tools.schedule import (
|
||||||
ScheduleCancelTool, ScheduleCreateTool, ScheduleListTool, ScheduleUpdateTool,
|
ScheduleCancelTool, ScheduleCreateTool, ScheduleListTool, ScheduleUpdateTool,
|
||||||
)
|
)
|
||||||
|
from tools.software_jobs import (
|
||||||
|
SoftwareCapabilityListTool,
|
||||||
|
SoftwareJobCancelTool,
|
||||||
|
SoftwareJobStatusTool,
|
||||||
|
SoftwareJobSubmitTool,
|
||||||
|
)
|
||||||
from tools.seedance import SeedanceTool
|
from tools.seedance import SeedanceTool
|
||||||
from tools.seedream import SeedreamTool
|
from tools.seedream import SeedreamTool
|
||||||
from tools.send_email import SendEmailTool, smtp_configured
|
from tools.send_email import SendEmailTool, smtp_configured
|
||||||
|
|
@ -207,6 +213,14 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
|
||||||
ScheduleCancelTool(ctx.uid, **base),
|
ScheduleCancelTool(ctx.uid, **base),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
def _software_jobs() -> list:
|
||||||
|
return [
|
||||||
|
SoftwareCapabilityListTool(ctx.uid, ctx.task_id, **base),
|
||||||
|
SoftwareJobSubmitTool(ctx.uid, ctx.task_id, **base),
|
||||||
|
SoftwareJobStatusTool(ctx.uid, ctx.task_id, **base),
|
||||||
|
SoftwareJobCancelTool(ctx.uid, ctx.task_id, **base),
|
||||||
|
]
|
||||||
|
|
||||||
def _send_email() -> list:
|
def _send_email() -> list:
|
||||||
# base_dir 用 working_dir_path(宿主 task 目录):send_email 在宿主进程读附件,
|
# base_dir 用 working_dir_path(宿主 task 目录):send_email 在宿主进程读附件,
|
||||||
# docker 下 agent 给的相对路径相对容器 workdir=task_dir,翻回宿主即 working_dir_path;
|
# docker 下 agent 给的相对路径相对容器 workdir=task_dir,翻回宿主即 working_dir_path;
|
||||||
|
|
@ -294,6 +308,7 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
|
||||||
("skill_authoring", lambda: True, _skill_authoring),
|
("skill_authoring", lambda: True, _skill_authoring),
|
||||||
# 定时 run 内不挂 schedule_*(防任务造任务自我繁殖);仅交互对话可建/管 job。
|
# 定时 run 内不挂 schedule_*(防任务造任务自我繁殖);仅交互对话可建/管 job。
|
||||||
("schedules", lambda: not ctx.scheduled_run, _schedules),
|
("schedules", lambda: not ctx.scheduled_run, _schedules),
|
||||||
|
("software_jobs", lambda: not ctx.scheduled_run, _software_jobs),
|
||||||
("send_email", smtp_configured, _send_email),
|
("send_email", smtp_configured, _send_email),
|
||||||
("wechat_push", wechat_push_available, _wechat_push),
|
("wechat_push", wechat_push_available, _wechat_push),
|
||||||
("run_python", lambda: ctx.caps.enable_run_python, _run_python),
|
("run_python", lambda: ctx.caps.enable_run_python, _run_python),
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,102 @@
|
||||||
|
"""Rename software nodes and add the professional software job ledger.
|
||||||
|
|
||||||
|
Revision ID: 0032
|
||||||
|
Revises: 0031
|
||||||
|
Create Date: 2026-08-13
|
||||||
|
"""
|
||||||
|
from collections.abc import Sequence
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
from sqlalchemy.dialects import postgresql
|
||||||
|
|
||||||
|
revision: str = "0032"
|
||||||
|
down_revision: str | None = "0031"
|
||||||
|
branch_labels: str | Sequence[str] | None = None
|
||||||
|
depends_on: str | Sequence[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.rename_table("compute_node_enrollments", "software_node_enrollments")
|
||||||
|
op.rename_table("compute_nodes", "software_nodes")
|
||||||
|
op.execute(
|
||||||
|
"ALTER INDEX ix_compute_nodes_status RENAME TO ix_software_nodes_status"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
|
||||||
|
"compute_node_enrollments_pkey TO software_node_enrollments_pkey"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
|
||||||
|
"compute_node_enrollments_code_hash_key TO software_node_enrollments_code_hash_key"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
|
||||||
|
"compute_node_enrollments_created_by_fkey TO software_node_enrollments_created_by_fkey"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_nodes RENAME CONSTRAINT "
|
||||||
|
"compute_nodes_pkey TO software_nodes_pkey"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_nodes RENAME CONSTRAINT "
|
||||||
|
"compute_nodes_install_id_key TO software_nodes_install_id_key"
|
||||||
|
)
|
||||||
|
op.create_table(
|
||||||
|
"software_jobs",
|
||||||
|
sa.Column("job_id", postgresql.UUID(as_uuid=True), primary_key=True),
|
||||||
|
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.user_id", ondelete="CASCADE"), nullable=False),
|
||||||
|
sa.Column("task_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("tasks.task_id", ondelete="CASCADE"), nullable=False),
|
||||||
|
sa.Column("idempotency_key", sa.Text(), nullable=False),
|
||||||
|
sa.Column("capability", sa.Text(), nullable=False),
|
||||||
|
sa.Column("request", postgresql.JSONB(), nullable=False),
|
||||||
|
sa.Column("request_digest", sa.Text(), nullable=False),
|
||||||
|
sa.Column("input_manifest", postgresql.JSONB(), nullable=False),
|
||||||
|
sa.Column("node_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("software_nodes.node_id", ondelete="SET NULL"), nullable=True),
|
||||||
|
sa.Column("lease_id", postgresql.UUID(as_uuid=True), nullable=True),
|
||||||
|
sa.Column("lease_expires_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("status", sa.Text(), server_default="queued", nullable=False),
|
||||||
|
sa.Column("stage", sa.Text(), server_default="", nullable=False),
|
||||||
|
sa.Column("progress", sa.Integer(), server_default="0", nullable=False),
|
||||||
|
sa.Column("metrics", postgresql.JSONB(), nullable=False),
|
||||||
|
sa.Column("error", postgresql.JSONB(), nullable=False),
|
||||||
|
sa.Column("artifact_manifest", postgresql.JSONB(), nullable=False),
|
||||||
|
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("terminal_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.UniqueConstraint("user_id", "idempotency_key", name="uq_software_jobs_user_idempotency"),
|
||||||
|
)
|
||||||
|
op.create_index("ix_software_jobs_status_created", "software_jobs", ["status", "created_at"])
|
||||||
|
op.create_index("ix_software_jobs_node_status", "software_jobs", ["node_id", "status"])
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_index("ix_software_jobs_node_status", table_name="software_jobs")
|
||||||
|
op.drop_index("ix_software_jobs_status_created", table_name="software_jobs")
|
||||||
|
op.drop_table("software_jobs")
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_nodes RENAME CONSTRAINT "
|
||||||
|
"software_nodes_install_id_key TO compute_nodes_install_id_key"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_nodes RENAME CONSTRAINT "
|
||||||
|
"software_nodes_pkey TO compute_nodes_pkey"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
|
||||||
|
"software_node_enrollments_created_by_fkey TO compute_node_enrollments_created_by_fkey"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
|
||||||
|
"software_node_enrollments_code_hash_key TO compute_node_enrollments_code_hash_key"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
|
||||||
|
"software_node_enrollments_pkey TO compute_node_enrollments_pkey"
|
||||||
|
)
|
||||||
|
op.execute(
|
||||||
|
"ALTER INDEX ix_software_nodes_status RENAME TO ix_compute_nodes_status"
|
||||||
|
)
|
||||||
|
op.rename_table("software_nodes", "compute_nodes")
|
||||||
|
op.rename_table("software_node_enrollments", "compute_node_enrollments")
|
||||||
|
|
@ -60,7 +60,7 @@ server {
|
||||||
|
|
||||||
# ★ Windows Node 长连接:注册走普通 POST,注册后的节点控制通道走 WebSocket。
|
# ★ Windows Node 长连接:注册走普通 POST,注册后的节点控制通道走 WebSocket。
|
||||||
# 必须独立于下面会清空 Connection 头的默认 location;15s 应用心跳保持链路活跃。
|
# 必须独立于下面会清空 Connection 头的默认 location;15s 应用心跳保持链路活跃。
|
||||||
location = /v1/compute/nodes/connect {
|
location = /v1/software-nodes/connect {
|
||||||
proxy_pass http://zcbot_backend;
|
proxy_pass http://zcbot_backend;
|
||||||
proxy_http_version 1.1;
|
proxy_http_version 1.1;
|
||||||
proxy_set_header Upgrade $http_upgrade;
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
|
|
||||||
|
|
@ -34,7 +34,7 @@ Windows Node 不是完整的本地 zcbot:
|
||||||
### 2.1 目标
|
### 2.1 目标
|
||||||
|
|
||||||
- 云端 zcbot 可以发现节点能力、容量、软件版本和在线状态。
|
- 云端 zcbot 可以发现节点能力、容量、软件版本和在线状态。
|
||||||
- 用户可以提交、查询、取消长时间 Windows 计算任务。
|
- 用户可以提交、查询、取消长时间运行的专业软件任务。
|
||||||
- 网络中断、云端重启或节点重启后,任务可以确定性对账。
|
- 网络中断、云端重启或节点重启后,任务可以确定性对账。
|
||||||
- 节点可以上报阶段、进度、结构化指标、日志摘要和事件截图。
|
- 节点可以上报阶段、进度、结构化指标、日志摘要和事件截图。
|
||||||
- 中间产物和最终产物支持校验、断点上传与按需导入工作目录。
|
- 中间产物和最终产物支持校验、断点上传与按需导入工作目录。
|
||||||
|
|
@ -79,7 +79,7 @@ flowchart LR
|
||||||
|---|---|
|
|---|---|
|
||||||
| `NodeRegistry` | 节点注册、证书指纹、启停、能力和管理员标签 |
|
| `NodeRegistry` | 节点注册、证书指纹、启停、能力和管理员标签 |
|
||||||
| `NodeConnectionManager` | WSS 连接、心跳、消息 ACK、同节点单活连接 |
|
| `NodeConnectionManager` | WSS 连接、心跳、消息 ACK、同节点单活连接 |
|
||||||
| `ComputeJobService` | 用户授权、幂等提交、节点选择、租约、取消和终态 |
|
| `SoftwareJobService` | 用户授权、幂等提交、节点选择、租约、取消和终态 |
|
||||||
| `ComputeTransferService` | 输入下载凭证、分块上传、SHA-256、容量与保留期 |
|
| `ComputeTransferService` | 输入下载凭证、分块上传、SHA-256、容量与保留期 |
|
||||||
| `ComputeBroker` | 把任务事件推送到 Web UI;不承担持久化事实源 |
|
| `ComputeBroker` | 把任务事件推送到 Web UI;不承担持久化事实源 |
|
||||||
| `ComputeTools` | agent 可调用的能力发现、提交、查询、取消、产物导入工具 |
|
| `ComputeTools` | agent 可调用的能力发现、提交、查询、取消、产物导入工具 |
|
||||||
|
|
@ -157,7 +157,7 @@ sequenceDiagram
|
||||||
participant Node as Windows Node
|
participant Node as Windows Node
|
||||||
Admin->>Cloud: 创建一次性 enrollment token
|
Admin->>Cloud: 创建一次性 enrollment token
|
||||||
Node->>Node: 生成设备密钥对
|
Node->>Node: 生成设备密钥对
|
||||||
Node->>Cloud: POST /v1/compute/nodes/enroll
|
Node->>Cloud: POST /v1/software-nodes/enroll
|
||||||
Cloud->>Cloud: 消耗 token,创建 node_id
|
Cloud->>Cloud: 消耗 token,创建 node_id
|
||||||
Cloud-->>Node: 客户端证书、CA、云端地址
|
Cloud-->>Node: 客户端证书、CA、云端地址
|
||||||
Node->>Node: 私钥写入 Windows Certificate Store
|
Node->>Node: 私钥写入 Windows Certificate Store
|
||||||
|
|
@ -178,7 +178,7 @@ sequenceDiagram
|
||||||
节点连接:
|
节点连接:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
WSS /v1/compute/nodes/connect
|
WSS /v1/software-nodes/connect
|
||||||
```
|
```
|
||||||
|
|
||||||
统一消息 envelope:
|
统一消息 envelope:
|
||||||
|
|
@ -252,13 +252,13 @@ Node 在 `hello` 和心跳中声明由本机可信配置生成的能力:
|
||||||
建议新增三张表,不复用外部系统连接表:
|
建议新增三张表,不复用外部系统连接表:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
compute_nodes(
|
software_nodes(
|
||||||
node_id pk, name, cert_fingerprint, status,
|
node_id pk, name, cert_fingerprint, status,
|
||||||
labels jsonb, capabilities jsonb, config_revision,
|
labels jsonb, capabilities jsonb, config_revision,
|
||||||
last_seen_at, disabled_at, created_at, updated_at
|
last_seen_at, disabled_at, created_at, updated_at
|
||||||
)
|
)
|
||||||
|
|
||||||
compute_jobs(
|
software_jobs(
|
||||||
job_id pk, user_id fk, task_id fk, tool_call_id,
|
job_id pk, user_id fk, task_id fk, tool_call_id,
|
||||||
capability, schema_version, request jsonb,
|
capability, schema_version, request jsonb,
|
||||||
idempotency_key, request_digest,
|
idempotency_key, request_digest,
|
||||||
|
|
@ -268,7 +268,7 @@ compute_jobs(
|
||||||
created_at, started_at, terminal_at, updated_at
|
created_at, started_at, terminal_at, updated_at
|
||||||
)
|
)
|
||||||
|
|
||||||
compute_job_events(
|
software_job_events(
|
||||||
event_id pk, job_id fk, sequence,
|
event_id pk, job_id fk, sequence,
|
||||||
kind, level, payload jsonb, created_at
|
kind, level, payload jsonb, created_at
|
||||||
)
|
)
|
||||||
|
|
@ -352,7 +352,7 @@ Node 不访问整个用户 workspace。云端只为显式引用的文件创建
|
||||||
大产物不通过 WSS 消息传输。Node 使用 HTTPS 分块上传:
|
大产物不通过 WSS 消息传输。Node 使用 HTTPS 分块上传:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
POST /v1/compute/jobs/{job_id}/artifacts/upload-session
|
POST /v1/software-jobs/{job_id}/artifacts/upload-session
|
||||||
PUT /v1/compute/transfers/{transfer_id}/parts/{part_number}
|
PUT /v1/compute/transfers/{transfer_id}/parts/{part_number}
|
||||||
POST /v1/compute/transfers/{transfer_id}/complete
|
POST /v1/compute/transfers/{transfer_id}/complete
|
||||||
```
|
```
|
||||||
|
|
@ -364,7 +364,7 @@ POST /v1/compute/transfers/{transfer_id}/complete
|
||||||
Node 上传完成后先进入:
|
Node 上传完成后先进入:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
<user_root>/.zcbot_cache/<task_id>/compute_jobs/<job_id>/
|
<user_root>/.zcbot_cache/<task_id>/software_jobs/<job_id>/
|
||||||
```
|
```
|
||||||
|
|
||||||
该目录默认隐藏且有 TTL。用户或 agent 明确导入后复制到:
|
该目录默认隐藏且有 TTL。用户或 agent 明确导入后复制到:
|
||||||
|
|
@ -379,10 +379,10 @@ Node 上传完成后先进入:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
compute_capability_list
|
compute_capability_list
|
||||||
compute_job_submit
|
software_job_submit
|
||||||
compute_job_status
|
software_job_status
|
||||||
compute_job_cancel
|
software_job_cancel
|
||||||
compute_job_artifact_import
|
software_job_artifact_import
|
||||||
```
|
```
|
||||||
|
|
||||||
- capability list 只返回用户有权使用且有健康节点承载的能力;
|
- capability list 只返回用户有权使用且有健康节点承载的能力;
|
||||||
|
|
@ -418,7 +418,7 @@ process.cancel_adapter
|
||||||
- 优先捕获目标窗口;
|
- 优先捕获目标窗口;
|
||||||
- 敏感信息上传前遮罩;
|
- 敏感信息上传前遮罩;
|
||||||
- 视频默认关闭,显式启用时建议 1280×720、5–10 FPS、H.264 分段;
|
- 视频默认关闭,显式启用时建议 1280×720、5–10 FPS、H.264 分段;
|
||||||
- 采集失败不得使计算任务失败;
|
- 采集失败不得使专业软件任务失败;
|
||||||
- 模型只按需读取关键帧,不持续消费完整视频。
|
- 模型只按需读取关键帧,不持续消费完整视频。
|
||||||
|
|
||||||
## 12. Origin 首批适配器
|
## 12. Origin 首批适配器
|
||||||
|
|
|
||||||
|
|
@ -60,7 +60,7 @@ sequenceDiagram
|
||||||
participant N as Windows Node
|
participant N as Windows Node
|
||||||
A->>Z: 创建一次性注册码
|
A->>Z: 创建一次性注册码
|
||||||
A->>N: 输入内网地址和注册码
|
A->>N: 输入内网地址和注册码
|
||||||
N->>Z: HTTP POST /v1/compute/nodes/enroll
|
N->>Z: HTTP POST /v1/software-nodes/enroll
|
||||||
Z->>Z: 校验并原子消费注册码
|
Z->>Z: 校验并原子消费注册码
|
||||||
Z-->>N: node_id + node_token + 配置
|
Z-->>N: node_id + node_token + 配置
|
||||||
N->>N: DPAPI 加密保存 node_token
|
N->>N: DPAPI 加密保存 node_token
|
||||||
|
|
@ -71,7 +71,7 @@ sequenceDiagram
|
||||||
注册请求:
|
注册请求:
|
||||||
|
|
||||||
```http
|
```http
|
||||||
POST http://zcbot.internal:8765/v1/compute/nodes/enroll
|
POST http://zcbot.internal:8765/v1/software-nodes/enroll
|
||||||
Content-Type: application/json
|
Content-Type: application/json
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -116,7 +116,7 @@ Content-Type: application/json
|
||||||
### 4.1 WS 连接
|
### 4.1 WS 连接
|
||||||
|
|
||||||
```http
|
```http
|
||||||
GET ws://zcbot.internal:8765/v1/compute/nodes/connect
|
GET ws://zcbot.internal:8765/v1/software-nodes/connect
|
||||||
Authorization: Bearer <node_token>
|
Authorization: Bearer <node_token>
|
||||||
X-Node-Id: <node_id>
|
X-Node-Id: <node_id>
|
||||||
Upgrade: websocket
|
Upgrade: websocket
|
||||||
|
|
@ -170,13 +170,13 @@ RDP:不向公网开放,使用 VPN、堡垒机或云安全登录
|
||||||
云端首期只增加:
|
云端首期只增加:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
compute_nodes(
|
software_nodes(
|
||||||
node_id pk, name, install_id, token_hash, status,
|
node_id pk, name, install_id, token_hash, status,
|
||||||
capabilities jsonb, last_seen_at,
|
capabilities jsonb, last_seen_at,
|
||||||
created_at, updated_at
|
created_at, updated_at
|
||||||
)
|
)
|
||||||
|
|
||||||
compute_jobs(
|
software_jobs(
|
||||||
job_id pk, user_id fk, task_id fk,
|
job_id pk, user_id fk, task_id fk,
|
||||||
idempotency_key, capability, request jsonb,
|
idempotency_key, capability, request jsonb,
|
||||||
node_id fk, status, progress,
|
node_id fk, status, progress,
|
||||||
|
|
@ -200,6 +200,8 @@ Node 断线且本地任务可能仍在执行时标记 `disconnected`,不得自
|
||||||
|
|
||||||
## 7. Origin 任务闭环
|
## 7. Origin 任务闭环
|
||||||
|
|
||||||
|
当前实现进度:云端任务账本、幂等提交、短期 offer、Node 本地原子保存与 accept/reject 已落地。输入以任务绑定的 artifact UUID 下载,Node 流式校验大小和 SHA-256 后原子保存。固定 Worker 使用管理员安装的隔离 Python 运行时与随程序发布的 `worker.py` 驱动 Origin,生成 OPJU、PNG、SVG、PDF、plot spec、provenance 和原子 `terminal.json`;运行不绑定单次 WebSocket,断线后继续执行。同一进程按 job 去重,Node 重启后不重复启动已留启动标记但无可信终态的任务。成功产物逐项流式上传到云端隐藏暂存区,云端复核任务身份、固定文件名、大小和 SHA-256 后,一次性发布到 `<working_dir>/origin/<job_id>/` 并登记平台 artifact UUID;Node 以 `upload-complete.json` 恢复中断上传。
|
||||||
|
|
||||||
```text
|
```text
|
||||||
用户上传 CSV/XLSX
|
用户上传 CSV/XLSX
|
||||||
→ zcbot 生成受控 plot spec
|
→ zcbot 生成受控 plot spec
|
||||||
|
|
|
||||||
|
|
@ -8,6 +8,10 @@ SAMPLE = """# 更新日志
|
||||||
|
|
||||||
> 导语在首个条目前,应被丢弃。
|
> 导语在首个条目前,应被丢弃。
|
||||||
|
|
||||||
|
## Unreleased
|
||||||
|
|
||||||
|
- 尚未发布,不能出现在用户版接口中
|
||||||
|
|
||||||
## 0.55.0 — 2026-07-09
|
## 0.55.0 — 2026-07-09
|
||||||
|
|
||||||
- 条目 A
|
- 条目 A
|
||||||
|
|
@ -36,6 +40,12 @@ class ChangelogParseTests(unittest.TestCase):
|
||||||
def test_preamble_before_first_entry_is_dropped(self) -> None:
|
def test_preamble_before_first_entry_is_dropped(self) -> None:
|
||||||
es = _parse_changelog(SAMPLE)
|
es = _parse_changelog(SAMPLE)
|
||||||
self.assertNotIn("导语", es[0]["body_md"])
|
self.assertNotIn("导语", es[0]["body_md"])
|
||||||
|
self.assertNotIn("尚未发布", es[0]["body_md"])
|
||||||
|
|
||||||
|
def test_unreleased_section_is_not_returned(self) -> None:
|
||||||
|
es = _parse_changelog(SAMPLE)
|
||||||
|
self.assertNotIn("Unreleased", [e["version"] for e in es])
|
||||||
|
self.assertEqual([e["version"] for e in es], ["0.55.0", "0.54.1", "0.53.0"])
|
||||||
|
|
||||||
def test_real_changelog_file_parses(self) -> None:
|
def test_real_changelog_file_parses(self) -> None:
|
||||||
# 仓库里的真文件:格式坏了(条目头打错 / 空正文)在 CI 就翻车,不等用户点开才发现
|
# 仓库里的真文件:格式坏了(条目头打错 / 空正文)在 CI 就翻车,不等用户点开才发现
|
||||||
|
|
|
||||||
|
|
@ -1,109 +0,0 @@
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import importlib
|
|
||||||
import unittest
|
|
||||||
from unittest.mock import AsyncMock, patch
|
|
||||||
from uuid import uuid4
|
|
||||||
|
|
||||||
from alembic.migration import MigrationContext
|
|
||||||
from alembic.operations import Operations
|
|
||||||
from sqlalchemy import create_mock_engine
|
|
||||||
from sqlalchemy.dialects import postgresql
|
|
||||||
|
|
||||||
from core.compute_nodes import (
|
|
||||||
_enrollment_digest,
|
|
||||||
_hash_secret,
|
|
||||||
_verify_secret,
|
|
||||||
delete_node,
|
|
||||||
)
|
|
||||||
from web.routers.compute_nodes import NodeConnectionManager, _bearer
|
|
||||||
|
|
||||||
|
|
||||||
class ComputeNodeSecurityTests(unittest.TestCase):
|
|
||||||
def test_secret_hash_is_salted_and_verifiable(self) -> None:
|
|
||||||
first = _hash_secret("node-secret")
|
|
||||||
second = _hash_secret("node-secret")
|
|
||||||
self.assertNotEqual(first, second)
|
|
||||||
self.assertNotIn("node-secret", first)
|
|
||||||
self.assertTrue(_verify_secret("node-secret", first))
|
|
||||||
self.assertFalse(_verify_secret("wrong", first))
|
|
||||||
|
|
||||||
def test_bearer_parser_rejects_query_style_or_missing_token(self) -> None:
|
|
||||||
self.assertEqual(_bearer("Bearer abc"), "abc")
|
|
||||||
with self.assertRaisesRegex(Exception, "missing node bearer token"):
|
|
||||||
_bearer(None)
|
|
||||||
|
|
||||||
def test_enrollment_digest_does_not_store_plaintext(self) -> None:
|
|
||||||
digest = _enrollment_digest("ZCN-ABC")
|
|
||||||
self.assertEqual(len(digest), 64)
|
|
||||||
self.assertNotIn("ZCN-ABC", digest)
|
|
||||||
|
|
||||||
|
|
||||||
class ComputeNodeConnectionTests(unittest.IsolatedAsyncioTestCase):
|
|
||||||
async def test_new_connection_replaces_old_without_removing_new(self) -> None:
|
|
||||||
manager = NodeConnectionManager()
|
|
||||||
node_id = uuid4()
|
|
||||||
old = AsyncMock()
|
|
||||||
new = AsyncMock()
|
|
||||||
|
|
||||||
await manager.activate(node_id, old)
|
|
||||||
await manager.activate(node_id, new)
|
|
||||||
|
|
||||||
old.close.assert_awaited_once_with(
|
|
||||||
code=4001, reason="replaced by a newer connection"
|
|
||||||
)
|
|
||||||
self.assertFalse(await manager.remove(node_id, old))
|
|
||||||
self.assertTrue(await manager.remove(node_id, new))
|
|
||||||
|
|
||||||
async def test_admin_close_removes_and_closes_connection(self) -> None:
|
|
||||||
manager = NodeConnectionManager()
|
|
||||||
node_id = uuid4()
|
|
||||||
websocket = AsyncMock()
|
|
||||||
await manager.activate(node_id, websocket)
|
|
||||||
await manager.close(node_id)
|
|
||||||
websocket.close.assert_awaited_once_with(code=4003, reason="node disabled")
|
|
||||||
self.assertFalse(await manager.remove(node_id, websocket))
|
|
||||||
|
|
||||||
|
|
||||||
class ComputeNodeMigrationTests(unittest.TestCase):
|
|
||||||
def test_0030_upgrade_compiles_as_postgresql_ddl(self) -> None:
|
|
||||||
statements: list[str] = []
|
|
||||||
|
|
||||||
def capture(sql, *multiparams, **params):
|
|
||||||
statements.append(str(sql.compile(dialect=postgresql.dialect())))
|
|
||||||
|
|
||||||
engine = create_mock_engine("postgresql+psycopg://", capture)
|
|
||||||
operations = Operations(MigrationContext.configure(engine.connect()))
|
|
||||||
migration = importlib.import_module(
|
|
||||||
"db.migrations.versions.20260812_2000_0030_compute_nodes"
|
|
||||||
)
|
|
||||||
with patch.object(migration, "op", operations):
|
|
||||||
migration.upgrade()
|
|
||||||
|
|
||||||
rendered = "\n".join(statements)
|
|
||||||
self.assertIn("compute_node_enrollments", rendered)
|
|
||||||
self.assertIn("compute_nodes", rendered)
|
|
||||||
self.assertIn("ix_compute_nodes_status", rendered)
|
|
||||||
|
|
||||||
|
|
||||||
class ComputeNodeDeleteTests(unittest.TestCase):
|
|
||||||
@patch("core.compute_nodes.session_scope")
|
|
||||||
def test_delete_node_removes_existing_identity(self, session_scope) -> None:
|
|
||||||
session = session_scope.return_value.__enter__.return_value
|
|
||||||
node = object()
|
|
||||||
session.get.return_value = node
|
|
||||||
|
|
||||||
self.assertTrue(delete_node(uuid4()))
|
|
||||||
session.delete.assert_called_once_with(node)
|
|
||||||
|
|
||||||
@patch("core.compute_nodes.session_scope")
|
|
||||||
def test_delete_node_reports_missing_identity(self, session_scope) -> None:
|
|
||||||
session = session_scope.return_value.__enter__.return_value
|
|
||||||
session.get.return_value = None
|
|
||||||
|
|
||||||
self.assertFalse(delete_node(uuid4()))
|
|
||||||
session.delete.assert_not_called()
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
unittest.main()
|
|
||||||
|
|
@ -0,0 +1,46 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
WORKER_PATH = (
|
||||||
|
Path(__file__).resolve().parents[1] / "windows-node" / "origin-worker" / "worker.py"
|
||||||
|
)
|
||||||
|
SPEC = importlib.util.spec_from_file_location("zcbot_origin_worker", WORKER_PATH)
|
||||||
|
assert SPEC and SPEC.loader
|
||||||
|
worker = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(worker)
|
||||||
|
|
||||||
|
|
||||||
|
class OriginWorkerUnitTests(unittest.TestCase):
|
||||||
|
def test_csv_and_json_inputs_are_read_without_origin(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
csv_path = root / "input.csv"
|
||||||
|
csv_path.write_text("x,y\n1,2\n3,4\n", encoding="utf-8")
|
||||||
|
self.assertEqual(worker._read_rows(csv_path, None), (["x", "y"], [["1", "2"], ["3", "4"]]))
|
||||||
|
|
||||||
|
json_path = root / "input.json"
|
||||||
|
json_path.write_text(json.dumps([{"x": 1, "y": 2}, {"x": 3, "y": 4}]), encoding="utf-8")
|
||||||
|
self.assertEqual(worker._read_rows(json_path, None), (["x", "y"], [[1, 2], [3, 4]]))
|
||||||
|
|
||||||
|
def test_manifest_uses_stable_id_and_streaming_digest(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
path = Path(directory) / "plot-spec.json"
|
||||||
|
path.write_text("{}", encoding="utf-8")
|
||||||
|
manifest = worker._manifest(path, "application/json")
|
||||||
|
self.assertEqual(manifest["artifact_id"], "plot_spec")
|
||||||
|
self.assertEqual(manifest["sha256"], worker._file_sha256(path))
|
||||||
|
self.assertEqual(manifest["size_bytes"], 2)
|
||||||
|
|
||||||
|
def test_axis_title_includes_units(self) -> None:
|
||||||
|
self.assertEqual(worker._axis_title({"title": "Stress", "unit": "MPa"}, "Y"), "Stress (MPa)")
|
||||||
|
self.assertEqual(worker._axis_title(None, "Time"), "Time")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -0,0 +1,80 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import unittest
|
||||||
|
from unittest.mock import patch
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from tools.software_jobs import (
|
||||||
|
SoftwareCapabilityListTool,
|
||||||
|
SoftwareJobCancelTool,
|
||||||
|
SoftwareJobStatusTool,
|
||||||
|
SoftwareJobSubmitTool,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareJobToolTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.user_id = uuid4()
|
||||||
|
self.task_id = uuid4()
|
||||||
|
|
||||||
|
def test_capability_list_reports_current_capacity(self):
|
||||||
|
nodes = [
|
||||||
|
{
|
||||||
|
"status": "online",
|
||||||
|
"capabilities": ["origin.plot@v1"],
|
||||||
|
"runtime": {"available_slots": 1},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"status": "offline",
|
||||||
|
"capabilities": ["origin.plot@v1"],
|
||||||
|
"runtime": {"available_slots": 1},
|
||||||
|
},
|
||||||
|
]
|
||||||
|
with patch("tools.software_jobs.list_nodes", return_value=nodes):
|
||||||
|
result = json.loads(
|
||||||
|
SoftwareCapabilityListTool(self.user_id, self.task_id).execute()
|
||||||
|
)
|
||||||
|
self.assertEqual(result["capabilities"][0]["available_nodes"], 1)
|
||||||
|
|
||||||
|
def test_submit_injects_current_user_and_task(self):
|
||||||
|
created = {"job_id": str(uuid4()), "status": "queued"}
|
||||||
|
tool = SoftwareJobSubmitTool(self.user_id, self.task_id)
|
||||||
|
with patch("tools.software_jobs.create_job", return_value=(created, True)) as create:
|
||||||
|
result = json.loads(tool.execute("origin.plot@v1", {"input_id": str(uuid4())}))
|
||||||
|
self.assertTrue(result["created"])
|
||||||
|
self.assertEqual(create.call_args.args[:2], (self.user_id, self.task_id))
|
||||||
|
self.assertEqual(create.call_args.kwargs["capability"], "origin.plot@v1")
|
||||||
|
|
||||||
|
def test_status_and_cancel_reject_cross_task_job(self):
|
||||||
|
foreign = {"job_id": str(uuid4()), "task_id": str(uuid4())}
|
||||||
|
with patch("tools.software_jobs.get_job", return_value=foreign):
|
||||||
|
status = SoftwareJobStatusTool(self.user_id, self.task_id).execute(
|
||||||
|
foreign["job_id"]
|
||||||
|
)
|
||||||
|
cancel = SoftwareJobCancelTool(self.user_id, self.task_id).execute(
|
||||||
|
foreign["job_id"]
|
||||||
|
)
|
||||||
|
self.assertIn("not found", status)
|
||||||
|
self.assertIn("not found", cancel)
|
||||||
|
|
||||||
|
def test_cancel_uses_user_scoped_service(self):
|
||||||
|
job_id = uuid4()
|
||||||
|
current = {"job_id": str(job_id), "task_id": str(self.task_id)}
|
||||||
|
cancelled = {**current, "status": "cancelled"}
|
||||||
|
with (
|
||||||
|
patch("tools.software_jobs.get_job", return_value=current),
|
||||||
|
patch(
|
||||||
|
"tools.software_jobs.request_job_cancel",
|
||||||
|
return_value=(cancelled, None),
|
||||||
|
) as request_cancel,
|
||||||
|
):
|
||||||
|
result = json.loads(
|
||||||
|
SoftwareJobCancelTool(self.user_id, self.task_id).execute(str(job_id))
|
||||||
|
)
|
||||||
|
self.assertEqual(result["status"], "cancelled")
|
||||||
|
request_cancel.assert_called_once_with(self.user_id, job_id)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -0,0 +1,379 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import AsyncMock, patch
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from alembic.migration import MigrationContext
|
||||||
|
from alembic.operations import Operations
|
||||||
|
from sqlalchemy import create_mock_engine
|
||||||
|
from sqlalchemy.dialects import postgresql
|
||||||
|
|
||||||
|
from core.software_nodes import (
|
||||||
|
_enrollment_digest,
|
||||||
|
_hash_secret,
|
||||||
|
_verify_secret,
|
||||||
|
delete_node,
|
||||||
|
)
|
||||||
|
from core.software_jobs import (
|
||||||
|
_canonical_request,
|
||||||
|
abandon_offer,
|
||||||
|
mark_node_jobs_disconnected,
|
||||||
|
record_job_terminal,
|
||||||
|
respond_to_offer,
|
||||||
|
update_job_state,
|
||||||
|
validate_output_manifest,
|
||||||
|
list_jobs,
|
||||||
|
request_job_cancel,
|
||||||
|
)
|
||||||
|
from web.routers.software_nodes import NodeConnectionManager, _bearer
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareNodeSecurityTests(unittest.TestCase):
|
||||||
|
def test_secret_hash_is_salted_and_verifiable(self) -> None:
|
||||||
|
first = _hash_secret("node-secret")
|
||||||
|
second = _hash_secret("node-secret")
|
||||||
|
self.assertNotEqual(first, second)
|
||||||
|
self.assertNotIn("node-secret", first)
|
||||||
|
self.assertTrue(_verify_secret("node-secret", first))
|
||||||
|
self.assertFalse(_verify_secret("wrong", first))
|
||||||
|
|
||||||
|
def test_bearer_parser_rejects_query_style_or_missing_token(self) -> None:
|
||||||
|
self.assertEqual(_bearer("Bearer abc"), "abc")
|
||||||
|
with self.assertRaisesRegex(Exception, "missing node bearer token"):
|
||||||
|
_bearer(None)
|
||||||
|
|
||||||
|
def test_enrollment_digest_does_not_store_plaintext(self) -> None:
|
||||||
|
digest = _enrollment_digest("ZCN-ABC")
|
||||||
|
self.assertEqual(len(digest), 64)
|
||||||
|
self.assertNotIn("ZCN-ABC", digest)
|
||||||
|
|
||||||
|
def test_websocket_auth_rejection_uses_explicit_application_close_code(self) -> None:
|
||||||
|
source = (
|
||||||
|
Path(__file__).resolve().parents[1] / "web" / "routers" / "software_nodes.py"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
rejection = source.split("except (ValueError, SoftwareNodeError):", 1)[1].split(
|
||||||
|
"await node_connections.activate", 1
|
||||||
|
)[0]
|
||||||
|
self.assertLess(
|
||||||
|
rejection.index("await websocket.accept()"), rejection.index("await websocket.close")
|
||||||
|
)
|
||||||
|
self.assertIn('code=4003, reason="invalid node credentials"', rejection)
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareNodeConnectionTests(unittest.IsolatedAsyncioTestCase):
|
||||||
|
async def test_new_connection_replaces_old_without_removing_new(self) -> None:
|
||||||
|
manager = NodeConnectionManager()
|
||||||
|
node_id = uuid4()
|
||||||
|
old = AsyncMock()
|
||||||
|
new = AsyncMock()
|
||||||
|
|
||||||
|
await manager.activate(node_id, old)
|
||||||
|
await manager.activate(node_id, new)
|
||||||
|
|
||||||
|
old.close.assert_awaited_once_with(
|
||||||
|
code=4001, reason="replaced by a newer connection"
|
||||||
|
)
|
||||||
|
self.assertFalse(await manager.remove(node_id, old))
|
||||||
|
self.assertTrue(await manager.remove(node_id, new))
|
||||||
|
|
||||||
|
async def test_admin_close_removes_and_closes_connection(self) -> None:
|
||||||
|
manager = NodeConnectionManager()
|
||||||
|
node_id = uuid4()
|
||||||
|
websocket = AsyncMock()
|
||||||
|
await manager.activate(node_id, websocket)
|
||||||
|
await manager.close(node_id)
|
||||||
|
websocket.close.assert_awaited_once_with(code=4003, reason="node disabled")
|
||||||
|
self.assertFalse(await manager.remove(node_id, websocket))
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareNodeMigrationTests(unittest.TestCase):
|
||||||
|
def test_0030_upgrade_compiles_as_postgresql_ddl(self) -> None:
|
||||||
|
statements: list[str] = []
|
||||||
|
|
||||||
|
def capture(sql, *multiparams, **params):
|
||||||
|
statements.append(str(sql.compile(dialect=postgresql.dialect())))
|
||||||
|
|
||||||
|
engine = create_mock_engine("postgresql+psycopg://", capture)
|
||||||
|
operations = Operations(MigrationContext.configure(engine.connect()))
|
||||||
|
migration = importlib.import_module(
|
||||||
|
"db.migrations.versions.20260812_2000_0030_compute_nodes"
|
||||||
|
)
|
||||||
|
with patch.object(migration, "op", operations):
|
||||||
|
migration.upgrade()
|
||||||
|
|
||||||
|
rendered = "\n".join(statements)
|
||||||
|
self.assertIn("compute_node_enrollments", rendered)
|
||||||
|
self.assertIn("compute_nodes", rendered)
|
||||||
|
self.assertIn("ix_compute_nodes_status", rendered)
|
||||||
|
|
||||||
|
def test_0032_upgrade_compiles_as_postgresql_ddl(self) -> None:
|
||||||
|
statements: list[str] = []
|
||||||
|
|
||||||
|
def capture(sql, *multiparams, **params):
|
||||||
|
statements.append(str(sql.compile(dialect=postgresql.dialect())))
|
||||||
|
|
||||||
|
engine = create_mock_engine("postgresql+psycopg://", capture)
|
||||||
|
operations = Operations(MigrationContext.configure(engine.connect()))
|
||||||
|
migration = importlib.import_module(
|
||||||
|
"db.migrations.versions.20260813_1600_0032_software_jobs"
|
||||||
|
)
|
||||||
|
with patch.object(migration, "op", operations):
|
||||||
|
migration.upgrade()
|
||||||
|
|
||||||
|
rendered = "\n".join(statements)
|
||||||
|
self.assertIn("ALTER TABLE compute_node_enrollments RENAME TO software_node_enrollments", rendered)
|
||||||
|
self.assertIn("ALTER TABLE compute_nodes RENAME TO software_nodes", rendered)
|
||||||
|
self.assertIn("software_jobs", rendered)
|
||||||
|
self.assertIn("uq_software_jobs_user_idempotency", rendered)
|
||||||
|
self.assertIn("ix_software_jobs_status_created", rendered)
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareJobProtocolTests(unittest.TestCase):
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_queued_job_cancels_without_node_message(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
job = type("Job", (), {})()
|
||||||
|
job.job_id = uuid4(); job.user_id = uuid4(); job.task_id = uuid4()
|
||||||
|
job.capability = "origin.plot@v1"; job.request_digest = "a" * 64
|
||||||
|
job.node_id = None; job.lease_id = None; job.status = "queued"; job.stage = ""
|
||||||
|
job.progress = 0; job.metrics = {}; job.error = {}; job.artifact_manifest = []
|
||||||
|
job.created_at = None; job.started_at = None; job.terminal_at = None
|
||||||
|
session.execute.return_value.scalar_one_or_none.return_value = job
|
||||||
|
result, node_message = request_job_cancel(job.user_id, job.job_id)
|
||||||
|
self.assertEqual(result["status"], "cancelled")
|
||||||
|
self.assertIsNone(node_message)
|
||||||
|
self.assertEqual(job.error["code"], "USER_CANCELLED")
|
||||||
|
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_running_job_persists_cancel_before_sending(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
job = type("Job", (), {})()
|
||||||
|
job.job_id = uuid4(); job.user_id = uuid4(); job.task_id = uuid4()
|
||||||
|
job.capability = "origin.plot@v1"; job.request_digest = "b" * 64
|
||||||
|
job.node_id = uuid4(); job.lease_id = uuid4(); job.status = "running"
|
||||||
|
job.stage = "origin_running"; job.progress = 10; job.metrics = {}; job.error = {}
|
||||||
|
job.artifact_manifest = []; job.created_at = None; job.started_at = None; job.terminal_at = None
|
||||||
|
session.execute.return_value.scalar_one_or_none.return_value = job
|
||||||
|
result, node_message = request_job_cancel(job.user_id, job.job_id)
|
||||||
|
self.assertEqual(result["status"], "cancelling")
|
||||||
|
self.assertEqual(node_message["node_id"], job.node_id)
|
||||||
|
self.assertEqual(node_message["payload"]["lease_id"], str(job.lease_id))
|
||||||
|
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_job_list_is_enriched_for_job_center(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
job = type("Job", (), {})()
|
||||||
|
job.job_id = uuid4(); job.task_id = uuid4(); job.capability = "origin.plot@v1"
|
||||||
|
job.request_digest = "c" * 64; job.node_id = uuid4(); job.status = "running"
|
||||||
|
job.stage = "origin_running"; job.progress = 20; job.metrics = {}; job.error = {}
|
||||||
|
job.artifact_manifest = []; job.input_manifest = {"filename": "input.xlsx"}
|
||||||
|
job.request = {"plot": {"title": "Test"}, "output": {"formats": ["png"]}}
|
||||||
|
job.created_at = None; job.started_at = None; job.terminal_at = None
|
||||||
|
session.execute.return_value.all.return_value = [(job, "材料仿真", "LAB-01")]
|
||||||
|
results = list_jobs(uuid4(), limit=10)
|
||||||
|
self.assertEqual(results[0]["task_name"], "材料仿真")
|
||||||
|
self.assertEqual(results[0]["node_name"], "LAB-01")
|
||||||
|
self.assertEqual(results[0]["request_summary"]["display_name"], "Origin 科研绘图")
|
||||||
|
|
||||||
|
def test_origin_request_is_canonical_and_rejects_extra_fields(self) -> None:
|
||||||
|
request = {
|
||||||
|
"schema_version": 1,
|
||||||
|
"input": {"input_id": str(uuid4()), "sheet": "Sheet1"},
|
||||||
|
"plot": {"type": "line", "x": "x", "y": ["y"]},
|
||||||
|
"output": {"formats": ["png", "opju"]},
|
||||||
|
}
|
||||||
|
normalized, digest = _canonical_request(request)
|
||||||
|
self.assertEqual(normalized, request)
|
||||||
|
self.assertEqual(len(digest), 64)
|
||||||
|
with self.assertRaisesRegex(Exception, "invalid origin plot request fields"):
|
||||||
|
_canonical_request({**request, "script": "anything"})
|
||||||
|
with self.assertRaisesRegex(Exception, "unsupported origin plot fields"):
|
||||||
|
_canonical_request({**request, "plot": {**request["plot"], "script": "anything"}})
|
||||||
|
with self.assertRaisesRegex(Exception, "input.input_id must be an artifact UUID"):
|
||||||
|
_canonical_request({**request, "input": {"input_id": "C:\\data.csv"}})
|
||||||
|
|
||||||
|
def test_origin_request_rejects_unimplemented_plot_semantics(self) -> None:
|
||||||
|
request = {
|
||||||
|
"schema_version": 1,
|
||||||
|
"input": {"input_id": str(uuid4())},
|
||||||
|
"plot": {"type": "scatter", "x": "time", "y": ["a", "b"]},
|
||||||
|
"output": {"formats": ["png"], "dpi": 300},
|
||||||
|
}
|
||||||
|
for plot, message in (
|
||||||
|
({**request["plot"], "template": "custom"}, "unsupported origin plot template"),
|
||||||
|
({**request["plot"], "x_axis": {"scale": "log10"}}, "invalid x_axis"),
|
||||||
|
({**request["plot"], "legend": {"enabled": False}}, "invalid plot.legend"),
|
||||||
|
({**request["plot"], "y": ["a", "a"]}, "plot.y must contain"),
|
||||||
|
):
|
||||||
|
with self.subTest(message=message), self.assertRaisesRegex(Exception, message):
|
||||||
|
_canonical_request({**request, "plot": plot})
|
||||||
|
with self.assertRaisesRegex(Exception, "video recording is not supported"):
|
||||||
|
_canonical_request(
|
||||||
|
{**request, "output": {"formats": ["png"], "record_video": True}}
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_output_manifest_matches_exact_requested_formats(self) -> None:
|
||||||
|
request = {"output": {"formats": ["opju", "png"]}}
|
||||||
|
manifest = [
|
||||||
|
{"artifact_id": "project", "filename": "project.opju", "media_type": "application/x-origin-project", "size_bytes": 10, "sha256": "a" * 64},
|
||||||
|
{"artifact_id": "figure_png", "filename": "figure.png", "media_type": "image/png", "size_bytes": 20, "sha256": "b" * 64},
|
||||||
|
{"artifact_id": "plot_spec", "filename": "plot-spec.json", "media_type": "application/json", "size_bytes": 30, "sha256": "c" * 64},
|
||||||
|
{"artifact_id": "provenance", "filename": "provenance.json", "media_type": "application/json", "size_bytes": 40, "sha256": "d" * 64},
|
||||||
|
]
|
||||||
|
self.assertEqual(validate_output_manifest(request, manifest), manifest)
|
||||||
|
with self.assertRaisesRegex(Exception, "incomplete"):
|
||||||
|
validate_output_manifest(request, manifest[:-1])
|
||||||
|
with self.assertRaisesRegex(Exception, "metadata"):
|
||||||
|
validate_output_manifest(
|
||||||
|
request,
|
||||||
|
[{**manifest[0], "filename": "anything.opju"}, *manifest[1:]],
|
||||||
|
)
|
||||||
|
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_stale_offer_cannot_be_accepted_by_another_node(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
job = type("Job", (), {})()
|
||||||
|
job.node_id = uuid4()
|
||||||
|
job.lease_id = uuid4()
|
||||||
|
job.status = "offered"
|
||||||
|
session.execute.return_value.scalar_one_or_none.return_value = job
|
||||||
|
with self.assertRaisesRegex(Exception, "stale or does not belong"):
|
||||||
|
respond_to_offer(
|
||||||
|
uuid4(),
|
||||||
|
accepted=True,
|
||||||
|
payload={"job_id": str(uuid4()), "lease_id": str(job.lease_id)},
|
||||||
|
)
|
||||||
|
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_failed_delivery_only_abandons_matching_offer(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
node_id = uuid4()
|
||||||
|
lease_id = uuid4()
|
||||||
|
job = type("Job", (), {})()
|
||||||
|
job.node_id = node_id
|
||||||
|
job.lease_id = lease_id
|
||||||
|
job.status = "offered"
|
||||||
|
session.execute.return_value.scalar_one_or_none.return_value = job
|
||||||
|
abandon_offer(
|
||||||
|
node_id,
|
||||||
|
{"job_id": str(uuid4()), "lease_id": str(lease_id)},
|
||||||
|
)
|
||||||
|
self.assertEqual(job.status, "queued")
|
||||||
|
self.assertIsNone(job.node_id)
|
||||||
|
|
||||||
|
def test_dispatcher_excludes_nodes_with_active_jobs(self) -> None:
|
||||||
|
source = (
|
||||||
|
Path(__file__).resolve().parents[1] / "core" / "software_jobs.py"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
self.assertIn('{"offered", "dispatched", "running"}', source)
|
||||||
|
self.assertIn("item.node_id not in busy_node_ids", source)
|
||||||
|
|
||||||
|
def test_input_download_rechecks_file_digest(self) -> None:
|
||||||
|
source = (
|
||||||
|
Path(__file__).resolve().parents[1]
|
||||||
|
/ "web" / "routers" / "software_nodes.py"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
self.assertIn("digest = sha256()", source)
|
||||||
|
self.assertIn('digest.hexdigest() != item["sha256"]', source)
|
||||||
|
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_job_state_restores_disconnected_job(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
node_id = uuid4()
|
||||||
|
lease_id = uuid4()
|
||||||
|
digest = "a" * 64
|
||||||
|
job = type("Job", (), {})()
|
||||||
|
job.node_id = node_id
|
||||||
|
job.lease_id = lease_id
|
||||||
|
job.request_digest = digest
|
||||||
|
job.status = "disconnected"
|
||||||
|
job.started_at = None
|
||||||
|
session.execute.return_value.scalar_one_or_none.return_value = job
|
||||||
|
update_job_state(node_id, {
|
||||||
|
"job_id": str(uuid4()),
|
||||||
|
"lease_id": str(lease_id),
|
||||||
|
"request_digest": digest,
|
||||||
|
"stage": "waiting_input",
|
||||||
|
"progress": 0,
|
||||||
|
"metrics": {},
|
||||||
|
})
|
||||||
|
self.assertEqual(job.status, "dispatched")
|
||||||
|
self.assertEqual(job.stage, "waiting_input")
|
||||||
|
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_ready_to_run_is_not_reported_as_running(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
node_id = uuid4()
|
||||||
|
lease_id = uuid4()
|
||||||
|
digest = "c" * 64
|
||||||
|
job = type("Job", (), {})()
|
||||||
|
job.node_id = node_id
|
||||||
|
job.lease_id = lease_id
|
||||||
|
job.request_digest = digest
|
||||||
|
job.status = "dispatched"
|
||||||
|
job.started_at = None
|
||||||
|
session.execute.return_value.scalar_one_or_none.return_value = job
|
||||||
|
update_job_state(node_id, {
|
||||||
|
"job_id": str(uuid4()), "lease_id": str(lease_id),
|
||||||
|
"request_digest": digest, "stage": "ready_to_run",
|
||||||
|
"progress": 5, "metrics": {"input_bytes": 10},
|
||||||
|
})
|
||||||
|
self.assertEqual(job.status, "dispatched")
|
||||||
|
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_terminal_replay_is_idempotent(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
node_id = uuid4()
|
||||||
|
lease_id = uuid4()
|
||||||
|
digest = "b" * 64
|
||||||
|
job = type("Job", (), {})()
|
||||||
|
job.node_id = node_id
|
||||||
|
job.lease_id = lease_id
|
||||||
|
job.request_digest = digest
|
||||||
|
job.status = "failed"
|
||||||
|
session.execute.return_value.scalar_one_or_none.return_value = job
|
||||||
|
record_job_terminal(node_id, {
|
||||||
|
"job_id": str(uuid4()),
|
||||||
|
"lease_id": str(lease_id),
|
||||||
|
"request_digest": digest,
|
||||||
|
"status": "failed",
|
||||||
|
"error": {"code": "TEST"},
|
||||||
|
"artifact_manifest": [],
|
||||||
|
})
|
||||||
|
self.assertEqual(job.status, "failed")
|
||||||
|
|
||||||
|
@patch("core.software_jobs.session_scope")
|
||||||
|
def test_disconnect_does_not_requeue_active_jobs(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
first = type("Job", (), {"status": "running"})()
|
||||||
|
second = type("Job", (), {"status": "dispatched"})()
|
||||||
|
session.execute.return_value.scalars.return_value = [first, second]
|
||||||
|
mark_node_jobs_disconnected(uuid4())
|
||||||
|
self.assertEqual(first.status, "disconnected")
|
||||||
|
self.assertEqual(second.status, "disconnected")
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareNodeDeleteTests(unittest.TestCase):
|
||||||
|
@patch("core.software_nodes.session_scope")
|
||||||
|
def test_delete_node_removes_existing_identity(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
node = object()
|
||||||
|
session.get.return_value = node
|
||||||
|
|
||||||
|
self.assertTrue(delete_node(uuid4()))
|
||||||
|
session.delete.assert_called_once_with(node)
|
||||||
|
|
||||||
|
@patch("core.software_nodes.session_scope")
|
||||||
|
def test_delete_node_reports_missing_identity(self, session_scope) -> None:
|
||||||
|
session = session_scope.return_value.__enter__.return_value
|
||||||
|
session.get.return_value = None
|
||||||
|
|
||||||
|
self.assertFalse(delete_node(uuid4()))
|
||||||
|
session.delete.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -0,0 +1,65 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from web.routers.software_nodes import _publish_software_job_outputs
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareOutputPublishTests(unittest.TestCase):
|
||||||
|
def test_complete_set_moves_atomically_and_can_be_replayed(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
job_id = uuid4()
|
||||||
|
working_dir = root / "research"
|
||||||
|
staging = root / ".zcbot_software_job_staging" / str(job_id)
|
||||||
|
staging.mkdir(parents=True)
|
||||||
|
working_dir.mkdir()
|
||||||
|
content = b"origin-result"
|
||||||
|
(staging / "figure.png").write_bytes(content)
|
||||||
|
manifest = [{
|
||||||
|
"artifact_id": "figure_png",
|
||||||
|
"filename": "figure.png",
|
||||||
|
"media_type": "image/png",
|
||||||
|
"size_bytes": len(content),
|
||||||
|
"sha256": hashlib.sha256(content).hexdigest(),
|
||||||
|
}]
|
||||||
|
context = {
|
||||||
|
"user_id": uuid4(),
|
||||||
|
"task_id": uuid4(),
|
||||||
|
"working_dir": "research",
|
||||||
|
}
|
||||||
|
|
||||||
|
def register(**kwargs):
|
||||||
|
return tuple({
|
||||||
|
"version": 2,
|
||||||
|
"scope": "working_dir",
|
||||||
|
"path": ref["path"],
|
||||||
|
"label": ref["label"],
|
||||||
|
"artifact_id": str(uuid4()),
|
||||||
|
} for ref in kwargs["refs"])
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch("web.routers.software_nodes.load_user_root", return_value=root),
|
||||||
|
patch(
|
||||||
|
"web.routers.software_nodes.register_published_artifacts",
|
||||||
|
side_effect=register,
|
||||||
|
),
|
||||||
|
):
|
||||||
|
first = _publish_software_job_outputs(job_id, context, manifest)
|
||||||
|
second = _publish_software_job_outputs(job_id, context, manifest)
|
||||||
|
|
||||||
|
published = working_dir / "origin" / str(job_id) / "figure.png"
|
||||||
|
self.assertEqual(published.read_bytes(), content)
|
||||||
|
self.assertFalse(staging.exists())
|
||||||
|
self.assertEqual(first[0]["source_artifact_id"], "figure_png")
|
||||||
|
self.assertEqual(first[0]["path"], f"origin/{job_id}/figure.png")
|
||||||
|
self.assertEqual(second[0]["source_artifact_id"], "figure_png")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
|
|
@ -26,19 +26,29 @@ def _frontend_source() -> str:
|
||||||
|
|
||||||
|
|
||||||
class StaticVendorTests(unittest.TestCase):
|
class StaticVendorTests(unittest.TestCase):
|
||||||
|
def test_dev_console_has_software_job_center(self) -> None:
|
||||||
|
html = DEV_HTML.read_text(encoding="utf-8")
|
||||||
|
source = (JS_DIR / "software_jobs.js").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('id="software-job-center"', html)
|
||||||
|
self.assertIn('id="software-job-toggle"', html)
|
||||||
|
self.assertIn('"/v1/software-jobs?limit=50"', source)
|
||||||
|
self.assertIn("/cancel`", source)
|
||||||
|
self.assertIn("分析结果", source)
|
||||||
|
|
||||||
def test_admin_can_create_windows_node_enrollment_code(self) -> None:
|
def test_admin_can_create_windows_node_enrollment_code(self) -> None:
|
||||||
html = ADMIN_HTML.read_text(encoding="utf-8")
|
html = ADMIN_HTML.read_text(encoding="utf-8")
|
||||||
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")
|
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")
|
||||||
|
|
||||||
self.assertIn('id="node-enrollment-modal" class="modal"', html)
|
self.assertIn('id="node-enrollment-modal" class="modal"', html)
|
||||||
self.assertIn("生成 Windows Node 注册码", html)
|
self.assertIn("生成 Windows Node 注册码", html)
|
||||||
self.assertIn('"/v1/admin/compute-node-enrollments"', admin_js)
|
self.assertIn('"/v1/admin/software-node-enrollments"', admin_js)
|
||||||
self.assertIn('capabilities: ["origin.plot@v1"]', admin_js)
|
self.assertIn('capabilities: ["origin.plot@v1"]', admin_js)
|
||||||
|
self.assertIn('origin.health === "ready"', admin_js)
|
||||||
self.assertIn("ttl_seconds: 600", admin_js)
|
self.assertIn("ttl_seconds: 600", admin_js)
|
||||||
self.assertIn("navigator.clipboard.writeText(value)", admin_js)
|
self.assertIn("navigator.clipboard.writeText(value)", admin_js)
|
||||||
self.assertIn('apiGet("/v1/admin/compute-nodes")', admin_js)
|
self.assertIn('apiGet("/v1/admin/software-nodes")', admin_js)
|
||||||
self.assertIn('apiSend("PATCH", `/v1/admin/compute-nodes/${node.node_id}`', admin_js)
|
self.assertIn('apiSend("PATCH", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
|
||||||
self.assertIn('apiSend("DELETE", `/v1/admin/compute-nodes/${node.node_id}`', admin_js)
|
self.assertIn('apiSend("DELETE", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
|
||||||
self.assertIn("最近心跳", admin_js)
|
self.assertIn("最近心跳", admin_js)
|
||||||
self.assertIn("重新启用", admin_js)
|
self.assertIn("重新启用", admin_js)
|
||||||
self.assertIn("永久删除", admin_js)
|
self.assertIn("永久删除", admin_js)
|
||||||
|
|
|
||||||
|
|
@ -110,8 +110,10 @@ class AuthGateTests(unittest.TestCase):
|
||||||
("POST", "/v1/tasks"),
|
("POST", "/v1/tasks"),
|
||||||
("POST", "/v1/asr/transcribe"),
|
("POST", "/v1/asr/transcribe"),
|
||||||
("GET", "/v1/admin/overview"),
|
("GET", "/v1/admin/overview"),
|
||||||
("GET", "/v1/admin/compute-nodes"),
|
("GET", "/v1/admin/software-nodes"),
|
||||||
("DELETE", "/v1/admin/compute-nodes/00000000-0000-0000-0000-000000000000"),
|
("GET", "/v1/software-jobs"),
|
||||||
|
("POST", "/v1/software-jobs/00000000-0000-0000-0000-000000000000/cancel"),
|
||||||
|
("DELETE", "/v1/admin/software-nodes/00000000-0000-0000-0000-000000000000"),
|
||||||
("GET", "/v1/admin/tool-wire-health"),
|
("GET", "/v1/admin/tool-wire-health"),
|
||||||
("GET", "/v1/admin/external-system-definitions"),
|
("GET", "/v1/admin/external-system-definitions"),
|
||||||
("GET", "/v1/admin/external-system-users"),
|
("GET", "/v1/admin/external-system-users"),
|
||||||
|
|
|
||||||
|
|
@ -20,8 +20,8 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
def test_node_protocol_and_secret_storage_markers_are_present(self) -> None:
|
def test_node_protocol_and_secret_storage_markers_are_present(self) -> None:
|
||||||
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs"))
|
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs"))
|
||||||
for marker in (
|
for marker in (
|
||||||
"v1/compute/nodes/enroll",
|
"v1/software-nodes/enroll",
|
||||||
"v1/compute/nodes/connect",
|
"v1/software-nodes/connect",
|
||||||
'SetRequestHeader("Authorization"',
|
'SetRequestHeader("Authorization"',
|
||||||
'SetRequestHeader("X-Node-Id"',
|
'SetRequestHeader("X-Node-Id"',
|
||||||
"DataProtectionScope.LocalMachine",
|
"DataProtectionScope.LocalMachine",
|
||||||
|
|
@ -35,7 +35,11 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertIn(marker, source)
|
self.assertIn(marker, source)
|
||||||
|
|
||||||
def test_node_does_not_expose_arbitrary_execution_primitives(self) -> None:
|
def test_node_does_not_expose_arbitrary_execution_primitives(self) -> None:
|
||||||
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs"))
|
source = "\n".join(
|
||||||
|
path.read_text(encoding="utf-8")
|
||||||
|
for path in PROJECT.glob("*.cs")
|
||||||
|
if path.name != "OriginWorkerRunner.cs"
|
||||||
|
)
|
||||||
for forbidden in ("Process.Start", "cmd.exe", "powershell.exe", "LabTalk"):
|
for forbidden in ("Process.Start", "cmd.exe", "powershell.exe", "LabTalk"):
|
||||||
self.assertNotIn(forbidden, source)
|
self.assertNotIn(forbidden, source)
|
||||||
|
|
||||||
|
|
@ -56,7 +60,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
|
|
||||||
def test_configuration_window_is_resizable_and_dpi_safe(self) -> None:
|
def test_configuration_window_is_resizable_and_dpi_safe(self) -> None:
|
||||||
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
|
||||||
self.assertIn("ClientSize = new Size(880, 720)", form)
|
self.assertIn("ClientSize = new Size(840, 680)", form)
|
||||||
self.assertIn("FormBorderStyle.Sizable", form)
|
self.assertIn("FormBorderStyle.Sizable", form)
|
||||||
self.assertIn("AutoScaleMode.Dpi", form)
|
self.assertIn("AutoScaleMode.Dpi", form)
|
||||||
self.assertIn("AutoScroll = true", form)
|
self.assertIn("AutoScroll = true", form)
|
||||||
|
|
@ -65,12 +69,17 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertIn("成功注册一次后立即失效", form)
|
self.assertIn("成功注册一次后立即失效", form)
|
||||||
self.assertIn("CreateCard", form)
|
self.assertIn("CreateCard", form)
|
||||||
self.assertIn("注册并连接", form)
|
self.assertIn("注册并连接", form)
|
||||||
self.assertIn("ContentWidth = 800", form)
|
self.assertIn("ContentWidth = 760", form)
|
||||||
self.assertIn("节点能力", form)
|
|
||||||
self.assertIn("Origin 绘图", form)
|
self.assertIn("Origin 绘图", form)
|
||||||
self.assertIn("当前 MVP 内置声明该协议,不需要手工配置", form)
|
self.assertIn('CreateCapabilityRow("Origin 绘图", "origin.plot@v1")', form)
|
||||||
|
self.assertIn('CreateButton("立即重连", 112, primary: true)', form)
|
||||||
|
self.assertIn("ReconnectRequested?.Invoke()", form)
|
||||||
self.assertIn("registrationCard.Visible = !registered", form)
|
self.assertIn("registrationCard.Visible = !registered", form)
|
||||||
self.assertIn("registeredActionsCard.Visible = registered", form)
|
self.assertIn("reconnect.Visible = registered", form)
|
||||||
|
self.assertIn("resetIdentity.Visible = registered", form)
|
||||||
|
|
||||||
|
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
|
||||||
|
self.assertIn("form.ReconnectRequested += RestartConnection", tray)
|
||||||
|
|
||||||
def test_startup_task_is_login_scoped_and_runs_the_fixed_node_executable(self) -> None:
|
def test_startup_task_is_login_scoped_and_runs_the_fixed_node_executable(self) -> None:
|
||||||
script = (ROOT / "install-startup.ps1").read_text(encoding="utf-8")
|
script = (ROOT / "install-startup.ps1").read_text(encoding="utf-8")
|
||||||
|
|
@ -80,6 +89,109 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
||||||
self.assertIn("-RunLevel Limited", script)
|
self.assertIn("-RunLevel Limited", script)
|
||||||
self.assertNotIn("-RunLevel Highest", script)
|
self.assertNotIn("-RunLevel Highest", script)
|
||||||
|
|
||||||
|
def test_auth_rejection_is_distinct_from_http_websocket_handshake_failure(self) -> None:
|
||||||
|
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('socket.HttpStatusCode is HttpStatusCode.Unauthorized', connection)
|
||||||
|
self.assertIn('or HttpStatusCode.Forbidden', connection)
|
||||||
|
self.assertIn("WebSocket 握手被拒绝,请检查服务端或反向代理", connection)
|
||||||
|
self.assertIn("catch (NodeEndpointException exception)", connection)
|
||||||
|
self.assertIn("throw new NodeEndpointException", connection)
|
||||||
|
self.assertIn("(int?)result.CloseStatus == 4003", connection)
|
||||||
|
self.assertIn("节点身份已被服务端拒绝", connection)
|
||||||
|
self.assertNotIn("Node credentials were rejected", connection)
|
||||||
|
|
||||||
|
def test_origin_runtime_probe_is_read_only_and_reported(self) -> None:
|
||||||
|
probe = (PROJECT / "OriginRuntimeProbe.cs").read_text(encoding="utf-8")
|
||||||
|
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('AutomationProgId = @"Origin.ApplicationSI\\CLSID"', probe)
|
||||||
|
self.assertIn("RegistryHive.LocalMachine", probe)
|
||||||
|
self.assertIn("RegistryHive.CurrentUser", probe)
|
||||||
|
self.assertIn('new("OriginPro", version, "0.2.0", health, detail)', probe)
|
||||||
|
self.assertIn(
|
||||||
|
'available_slots = origin.Health == "ready" && !jobInbox.HasPendingJobs ? 1 : 0',
|
||||||
|
connection,
|
||||||
|
)
|
||||||
|
self.assertNotIn("CreateInstance", probe)
|
||||||
|
self.assertNotIn("Process.Start", probe)
|
||||||
|
for marker in (
|
||||||
|
"software_version = origin.SoftwareVersion",
|
||||||
|
"adapter_version = origin.AdapterVersion",
|
||||||
|
"health = origin.Health",
|
||||||
|
"detail = origin.Detail",
|
||||||
|
):
|
||||||
|
self.assertIn(marker, connection)
|
||||||
|
|
||||||
|
def test_job_offer_is_persisted_before_acceptance(self) -> None:
|
||||||
|
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
|
||||||
|
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('capabilityValue.GetString() != "origin.plot@v1"', inbox)
|
||||||
|
self.assertIn("PlotTypes.Contains", inbox)
|
||||||
|
self.assertIn("OutputFormats.Contains", inbox)
|
||||||
|
self.assertIn("FileOptions.WriteThrough", inbox)
|
||||||
|
self.assertIn("stream.Flush(flushToDisk: true)", inbox)
|
||||||
|
new_record = inbox.split("var record =", 1)[1].split("private static JsonElement?", 1)[0]
|
||||||
|
self.assertLess(
|
||||||
|
new_record.index("AtomicWrite(requestPath, record"),
|
||||||
|
new_record.index("JobOfferResult.Accept"),
|
||||||
|
)
|
||||||
|
self.assertIn('offerResult.Accepted ? "job_accept" : "job_reject"', connection)
|
||||||
|
self.assertIn("sendLock.WaitAsync", connection)
|
||||||
|
self.assertIn("!jobInbox.HasPendingJobs ? 1 : 0", connection)
|
||||||
|
self.assertIn("ReportRecoverableJobsAsync", connection)
|
||||||
|
self.assertIn("ConcurrentDictionary<Guid, Task> jobPipelines", connection)
|
||||||
|
self.assertIn("StartJobPipeline(socket, acceptedJob)", connection)
|
||||||
|
self.assertIn("inputDownloader.DownloadAsync(job, CancellationToken.None)", connection)
|
||||||
|
self.assertIn('stage = "uploading_outputs"', connection)
|
||||||
|
self.assertIn("&& !job.UploadComplete", connection)
|
||||||
|
self.assertIn("StartJobPipeline(socket, job)", connection)
|
||||||
|
self.assertIn('stage = "waiting_input"', connection)
|
||||||
|
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', inbox)
|
||||||
|
self.assertIn("AtomicWrite(requestPath, updated, overwrite: true)", inbox)
|
||||||
|
|
||||||
|
downloader = (PROJECT / "JobInputDownloader.cs").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', downloader)
|
||||||
|
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', downloader)
|
||||||
|
self.assertIn("HttpCompletionOption.ResponseHeadersRead", downloader)
|
||||||
|
self.assertIn("IncrementalHash.CreateHash", downloader)
|
||||||
|
self.assertIn("total > expectedSize", downloader)
|
||||||
|
self.assertIn("File.Move(temporaryPath, destination, overwrite: false)", downloader)
|
||||||
|
self.assertNotIn("Process.Start", downloader)
|
||||||
|
|
||||||
|
def test_origin_worker_launch_is_fixed_and_terminal_driven(self) -> None:
|
||||||
|
runner = (PROJECT / "OriginWorkerRunner.cs").read_text(encoding="utf-8")
|
||||||
|
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
||||||
|
project = (PROJECT / "Zcbot.WindowsNode.csproj").read_text(encoding="utf-8")
|
||||||
|
worker = (ROOT / "origin-worker" / "worker.py").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('Environment.GetEnvironmentVariable("ZCBOT_ORIGIN_PYTHON")', runner)
|
||||||
|
self.assertIn('Path.Combine(paths.RootDirectory, "runtimes", "origin", "python.exe")', runner)
|
||||||
|
self.assertIn("UseShellExecute = false", runner)
|
||||||
|
self.assertIn("startInfo.ArgumentList.Add(workerScript)", runner)
|
||||||
|
self.assertIn("startInfo.ArgumentList.Add(jobDirectory)", runner)
|
||||||
|
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', runner)
|
||||||
|
self.assertIn('"NODE_RESTARTED_DURING_JOB"', runner)
|
||||||
|
self.assertIn("CancellationTokenSource.CreateLinkedTokenSource", runner)
|
||||||
|
self.assertIn("process.Kill(entireProcessTree: true)", runner)
|
||||||
|
self.assertIn('type.GetString() == "job_cancel"', connection)
|
||||||
|
self.assertIn('"cancelled", "USER_CANCELLED"', connection)
|
||||||
|
self.assertIn("origin-worker\\worker.py", project)
|
||||||
|
self.assertIn("if op.oext:", worker)
|
||||||
|
self.assertIn("op.exit()", worker)
|
||||||
|
self.assertIn("op.new_graph", worker)
|
||||||
|
self.assertIn("layer.add_plot", worker)
|
||||||
|
self.assertIn("op.save", worker)
|
||||||
|
self.assertIn("graph.save_fig", worker)
|
||||||
|
self.assertIn('_atomic_json(job_dir / "terminal.json"', worker)
|
||||||
|
for forbidden in ("subprocess", "eval(", "exec(", "os.system"):
|
||||||
|
self.assertNotIn(forbidden, worker)
|
||||||
|
|
||||||
|
uploader = (PROJECT / "JobOutputUploader.cs").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', uploader)
|
||||||
|
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', uploader)
|
||||||
|
self.assertIn('DefaultRequestHeaders.Add("X-Lease-Id"', uploader)
|
||||||
|
self.assertIn("SHA256.HashDataAsync", uploader)
|
||||||
|
self.assertIn("upload-complete.json", connection + uploader)
|
||||||
|
self.assertNotIn("Process.Start", uploader)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,122 @@
|
||||||
|
"""Agent 可调用的专业软件任务工具。"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from uuid import UUID, uuid4
|
||||||
|
|
||||||
|
from core.software_jobs import (
|
||||||
|
SoftwareJobError,
|
||||||
|
create_job,
|
||||||
|
get_job,
|
||||||
|
list_jobs,
|
||||||
|
request_job_cancel,
|
||||||
|
)
|
||||||
|
from core.software_nodes import SUPPORTED_CAPABILITIES, list_nodes
|
||||||
|
from .base import Tool
|
||||||
|
|
||||||
|
|
||||||
|
class _SoftwareJobTool(Tool):
|
||||||
|
def __init__(self, user_id: UUID, task_id: UUID, **kwargs) -> None:
|
||||||
|
super().__init__(**kwargs)
|
||||||
|
self.user_id = user_id
|
||||||
|
self.task_id = task_id
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareCapabilityListTool(_SoftwareJobTool):
|
||||||
|
name = "software_capability_list"
|
||||||
|
description = "List professional software capabilities available through managed Windows nodes."
|
||||||
|
parameters = {"type": "object", "properties": {}, "additionalProperties": False}
|
||||||
|
|
||||||
|
def execute(self) -> str:
|
||||||
|
nodes = list_nodes()
|
||||||
|
items = [{
|
||||||
|
"capability": item,
|
||||||
|
"display_name": "Origin 科研绘图" if item == "origin.plot@v1" else item,
|
||||||
|
"available_nodes": sum(
|
||||||
|
1
|
||||||
|
for node in nodes
|
||||||
|
if node["status"] == "online"
|
||||||
|
and item in (node.get("capabilities") or [])
|
||||||
|
and (node.get("runtime") or {}).get("available_slots", 0) > 0
|
||||||
|
),
|
||||||
|
} for item in sorted(SUPPORTED_CAPABILITIES)]
|
||||||
|
return json.dumps({"capabilities": items}, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareJobSubmitTool(_SoftwareJobTool):
|
||||||
|
name = "software_job_submit"
|
||||||
|
description = (
|
||||||
|
"Submit a persistent professional software job for the current task. "
|
||||||
|
"Return immediately with job_id; do not poll continuously or wait for completion."
|
||||||
|
)
|
||||||
|
parameters = {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"capability": {"type": "string", "enum": sorted(SUPPORTED_CAPABILITIES)},
|
||||||
|
"request": {"type": "object"},
|
||||||
|
"idempotency_key": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Stable unique key for this exact submission; omit to generate one.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["capability", "request"],
|
||||||
|
"additionalProperties": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
def execute(self, capability: str, request: dict, idempotency_key: str = "") -> str:
|
||||||
|
try:
|
||||||
|
job, created = create_job(
|
||||||
|
self.user_id,
|
||||||
|
self.task_id,
|
||||||
|
idempotency_key=idempotency_key.strip() or str(uuid4()),
|
||||||
|
capability=capability,
|
||||||
|
request=request,
|
||||||
|
)
|
||||||
|
return json.dumps({**job, "created": created}, ensure_ascii=False)
|
||||||
|
except SoftwareJobError as exc:
|
||||||
|
return f"[Error] {exc}"
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareJobStatusTool(_SoftwareJobTool):
|
||||||
|
name = "software_job_status"
|
||||||
|
description = "Check one software job, or list recent jobs in the current task when job_id is omitted."
|
||||||
|
parameters = {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"job_id": {"type": "string"}},
|
||||||
|
"additionalProperties": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
def execute(self, job_id: str = "") -> str:
|
||||||
|
try:
|
||||||
|
if job_id.strip():
|
||||||
|
item = get_job(self.user_id, UUID(job_id.strip()))
|
||||||
|
if item is None or item["task_id"] != str(self.task_id):
|
||||||
|
return "[Error] software job not found"
|
||||||
|
return json.dumps(item, ensure_ascii=False)
|
||||||
|
return json.dumps(
|
||||||
|
{"results": list_jobs(self.user_id, task_id=self.task_id, limit=20)},
|
||||||
|
ensure_ascii=False,
|
||||||
|
)
|
||||||
|
except ValueError:
|
||||||
|
return "[Error] invalid job_id"
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareJobCancelTool(_SoftwareJobTool):
|
||||||
|
name = "software_job_cancel"
|
||||||
|
description = "Request cancellation of a software job in the current task after the user asks to stop it."
|
||||||
|
parameters = {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"job_id": {"type": "string"}},
|
||||||
|
"required": ["job_id"],
|
||||||
|
"additionalProperties": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
def execute(self, job_id: str) -> str:
|
||||||
|
try:
|
||||||
|
item = get_job(self.user_id, UUID(job_id.strip()))
|
||||||
|
if item is None or item["task_id"] != str(self.task_id):
|
||||||
|
return "[Error] software job not found"
|
||||||
|
job, _ = request_job_cancel(self.user_id, UUID(job_id.strip()))
|
||||||
|
return json.dumps(job, ensure_ascii=False)
|
||||||
|
except (ValueError, SoftwareJobError) as exc:
|
||||||
|
return f"[Error] {exc}"
|
||||||
|
|
@ -49,7 +49,7 @@ from .background import (
|
||||||
from .broker import broker
|
from .broker import broker
|
||||||
from .routers.asr import register_asr_routes
|
from .routers.asr import register_asr_routes
|
||||||
from .routers.authroutes import register_auth_routes
|
from .routers.authroutes import register_auth_routes
|
||||||
from .routers.compute_nodes import register_compute_node_routes
|
from .routers.software_nodes import register_software_node_routes
|
||||||
from .routers.external_systems import register_external_system_routes
|
from .routers.external_systems import register_external_system_routes
|
||||||
from .routers.files import register_file_routes
|
from .routers.files import register_file_routes
|
||||||
from .routers.kb import register_kb_routes
|
from .routers.kb import register_kb_routes
|
||||||
|
|
@ -206,7 +206,9 @@ def create_app() -> FastAPI:
|
||||||
register_asr_routes(app, require_user=require_user, auth_cfg=auth_cfg)
|
register_asr_routes(app, require_user=require_user, auth_cfg=auth_cfg)
|
||||||
register_task_routes(app, require_user=require_user)
|
register_task_routes(app, require_user=require_user)
|
||||||
register_message_routes(app, require_user=require_user)
|
register_message_routes(app, require_user=require_user)
|
||||||
register_compute_node_routes(app, require_admin=require_admin)
|
register_software_node_routes(
|
||||||
|
app, require_user=require_user, require_admin=require_admin
|
||||||
|
)
|
||||||
|
|
||||||
# ───────────── 管理后台(admin-only)─────────────
|
# ───────────── 管理后台(admin-only)─────────────
|
||||||
register_admin_routes(app, require_admin)
|
register_admin_routes(app, require_admin)
|
||||||
|
|
|
||||||
|
|
@ -1,158 +0,0 @@
|
||||||
"""Windows Node MVP 的注册、管理与长连接端点。"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from uuid import UUID
|
|
||||||
|
|
||||||
from fastapi import Depends, HTTPException, WebSocket, WebSocketDisconnect, status
|
|
||||||
|
|
||||||
from core.compute_nodes import (
|
|
||||||
ComputeNodeError,
|
|
||||||
authenticate_node,
|
|
||||||
create_enrollment,
|
|
||||||
delete_node,
|
|
||||||
enroll_node,
|
|
||||||
list_nodes,
|
|
||||||
mark_node_offline,
|
|
||||||
set_node_disabled,
|
|
||||||
update_node_runtime,
|
|
||||||
)
|
|
||||||
from web.schemas import (
|
|
||||||
ComputeEnrollmentCreateRequest,
|
|
||||||
ComputeNodeDisableRequest,
|
|
||||||
ComputeNodeEnrollRequest,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class NodeConnectionManager:
|
|
||||||
def __init__(self) -> None:
|
|
||||||
self._connections: dict[UUID, WebSocket] = {}
|
|
||||||
self._lock = asyncio.Lock()
|
|
||||||
|
|
||||||
async def activate(self, node_id: UUID, websocket: WebSocket) -> None:
|
|
||||||
async with self._lock:
|
|
||||||
old = self._connections.get(node_id)
|
|
||||||
self._connections[node_id] = websocket
|
|
||||||
if old is not None and old is not websocket:
|
|
||||||
await old.close(code=4001, reason="replaced by a newer connection")
|
|
||||||
|
|
||||||
async def remove(self, node_id: UUID, websocket: WebSocket) -> bool:
|
|
||||||
async with self._lock:
|
|
||||||
if self._connections.get(node_id) is websocket:
|
|
||||||
self._connections.pop(node_id, None)
|
|
||||||
return True
|
|
||||||
return False
|
|
||||||
|
|
||||||
async def close(self, node_id: UUID) -> None:
|
|
||||||
async with self._lock:
|
|
||||||
websocket = self._connections.pop(node_id, None)
|
|
||||||
if websocket is not None:
|
|
||||||
await websocket.close(code=4003, reason="node disabled")
|
|
||||||
|
|
||||||
|
|
||||||
node_connections = NodeConnectionManager()
|
|
||||||
|
|
||||||
|
|
||||||
def _bearer(authorization: str | None) -> str:
|
|
||||||
scheme, _, token = (authorization or "").partition(" ")
|
|
||||||
if scheme.lower() != "bearer" or not token:
|
|
||||||
raise ComputeNodeError("missing node bearer token")
|
|
||||||
return token
|
|
||||||
|
|
||||||
|
|
||||||
def register_compute_node_routes(app, *, require_admin) -> None:
|
|
||||||
@app.post(
|
|
||||||
"/v1/compute/nodes/enroll",
|
|
||||||
tags=["compute-nodes"],
|
|
||||||
status_code=status.HTTP_201_CREATED,
|
|
||||||
)
|
|
||||||
def node_enroll(body: ComputeNodeEnrollRequest):
|
|
||||||
try:
|
|
||||||
return enroll_node(**body.model_dump())
|
|
||||||
except ComputeNodeError as exc:
|
|
||||||
raise HTTPException(400, str(exc)) from exc
|
|
||||||
|
|
||||||
@app.websocket("/v1/compute/nodes/connect")
|
|
||||||
async def node_connect(websocket: WebSocket):
|
|
||||||
try:
|
|
||||||
node_id = UUID(websocket.headers.get("x-node-id", ""))
|
|
||||||
token = _bearer(websocket.headers.get("authorization"))
|
|
||||||
identity = await asyncio.to_thread(authenticate_node, node_id, token)
|
|
||||||
except (ValueError, ComputeNodeError):
|
|
||||||
await websocket.close(code=1008, reason="invalid node credentials")
|
|
||||||
return
|
|
||||||
await websocket.accept()
|
|
||||||
await node_connections.activate(node_id, websocket)
|
|
||||||
try:
|
|
||||||
await websocket.send_json({"type": "connected", "heartbeat_seconds": 15})
|
|
||||||
while True:
|
|
||||||
message = await websocket.receive_json()
|
|
||||||
message_type = message.get("type")
|
|
||||||
payload = message.get("payload") or {}
|
|
||||||
if message_type not in {"hello", "heartbeat"} or not isinstance(
|
|
||||||
payload, dict
|
|
||||||
):
|
|
||||||
await websocket.send_json(
|
|
||||||
{"type": "error", "code": "unsupported_message"}
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
if payload.get("install_id") and payload["install_id"] != str(
|
|
||||||
identity["install_id"]
|
|
||||||
):
|
|
||||||
await websocket.close(code=1008, reason="install identity mismatch")
|
|
||||||
return
|
|
||||||
await asyncio.to_thread(
|
|
||||||
update_node_runtime,
|
|
||||||
node_id,
|
|
||||||
status="online",
|
|
||||||
runtime=payload,
|
|
||||||
)
|
|
||||||
await websocket.send_json(
|
|
||||||
{"type": "ack", "message_id": message.get("message_id")}
|
|
||||||
)
|
|
||||||
except (ComputeNodeError, WebSocketDisconnect, RuntimeError, ValueError):
|
|
||||||
pass
|
|
||||||
finally:
|
|
||||||
if await node_connections.remove(node_id, websocket):
|
|
||||||
await asyncio.to_thread(mark_node_offline, node_id)
|
|
||||||
|
|
||||||
@app.post("/v1/admin/compute-node-enrollments", tags=["admin"])
|
|
||||||
def admin_create_compute_enrollment(
|
|
||||||
body: ComputeEnrollmentCreateRequest,
|
|
||||||
user_id: UUID = Depends(require_admin), # noqa: B008
|
|
||||||
):
|
|
||||||
try:
|
|
||||||
return create_enrollment(user_id, **body.model_dump())
|
|
||||||
except ComputeNodeError as exc:
|
|
||||||
raise HTTPException(400, str(exc)) from exc
|
|
||||||
|
|
||||||
@app.get("/v1/admin/compute-nodes", tags=["admin"])
|
|
||||||
def admin_compute_nodes(user_id: UUID = Depends(require_admin)): # noqa: B008
|
|
||||||
return {"results": list_nodes()}
|
|
||||||
|
|
||||||
@app.patch("/v1/admin/compute-nodes/{node_id}", tags=["admin"])
|
|
||||||
async def admin_disable_compute_node(
|
|
||||||
node_id: UUID,
|
|
||||||
body: ComputeNodeDisableRequest,
|
|
||||||
user_id: UUID = Depends(require_admin), # noqa: B008
|
|
||||||
):
|
|
||||||
if not await asyncio.to_thread(set_node_disabled, node_id, body.disabled):
|
|
||||||
raise HTTPException(404, "compute node not found")
|
|
||||||
if body.disabled:
|
|
||||||
await node_connections.close(node_id)
|
|
||||||
return {
|
|
||||||
"node_id": str(node_id),
|
|
||||||
"status": "disabled" if body.disabled else "offline",
|
|
||||||
}
|
|
||||||
|
|
||||||
@app.delete("/v1/admin/compute-nodes/{node_id}", tags=["admin"])
|
|
||||||
async def admin_delete_compute_node(
|
|
||||||
node_id: UUID,
|
|
||||||
user_id: UUID = Depends(require_admin), # noqa: B008
|
|
||||||
):
|
|
||||||
# 先撤掉在线连接,避免删除后的旧 socket 继续上报运行态。
|
|
||||||
await node_connections.close(node_id)
|
|
||||||
if not await asyncio.to_thread(delete_node, node_id):
|
|
||||||
raise HTTPException(404, "compute node not found")
|
|
||||||
return {"node_id": str(node_id), "status": "deleted"}
|
|
||||||
|
|
@ -0,0 +1,576 @@
|
||||||
|
"""Windows Node MVP 的注册、管理与长连接端点。"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
from hashlib import sha256
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from fastapi import Depends, Header, HTTPException, Request, WebSocket, WebSocketDisconnect, status
|
||||||
|
from fastapi.responses import FileResponse
|
||||||
|
|
||||||
|
from core.software_nodes import (
|
||||||
|
SoftwareNodeError,
|
||||||
|
authenticate_node,
|
||||||
|
create_enrollment,
|
||||||
|
delete_node,
|
||||||
|
enroll_node,
|
||||||
|
list_nodes,
|
||||||
|
mark_node_offline,
|
||||||
|
set_node_disabled,
|
||||||
|
update_node_runtime,
|
||||||
|
)
|
||||||
|
from core.software_jobs import (
|
||||||
|
MAX_OUTPUT_ARTIFACT_BYTES,
|
||||||
|
MAX_OUTPUT_TOTAL_BYTES,
|
||||||
|
OUTPUT_ARTIFACTS,
|
||||||
|
abandon_offer,
|
||||||
|
create_job,
|
||||||
|
get_job,
|
||||||
|
get_job_input,
|
||||||
|
get_job_output_context,
|
||||||
|
mark_node_jobs_disconnected,
|
||||||
|
offer_next_job,
|
||||||
|
pending_node_cancellations,
|
||||||
|
record_job_terminal,
|
||||||
|
list_jobs,
|
||||||
|
request_job_cancel,
|
||||||
|
respond_to_offer,
|
||||||
|
update_job_state,
|
||||||
|
validate_output_manifest,
|
||||||
|
SoftwareJobError,
|
||||||
|
)
|
||||||
|
from core.artifact_lifecycle import register_published_artifacts
|
||||||
|
from web.schemas import (
|
||||||
|
SoftwareEnrollmentCreateRequest,
|
||||||
|
SoftwareJobCreateRequest,
|
||||||
|
SoftwareNodeDisableRequest,
|
||||||
|
SoftwareNodeEnrollRequest,
|
||||||
|
)
|
||||||
|
from web.userfiles import load_user_root, safe_join
|
||||||
|
|
||||||
|
|
||||||
|
class NodeConnectionManager:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._connections: dict[UUID, WebSocket] = {}
|
||||||
|
self._send_locks: dict[UUID, asyncio.Lock] = {}
|
||||||
|
self._lock = asyncio.Lock()
|
||||||
|
|
||||||
|
async def activate(self, node_id: UUID, websocket: WebSocket) -> None:
|
||||||
|
async with self._lock:
|
||||||
|
old = self._connections.get(node_id)
|
||||||
|
self._connections[node_id] = websocket
|
||||||
|
self._send_locks.setdefault(node_id, asyncio.Lock())
|
||||||
|
if old is not None and old is not websocket:
|
||||||
|
await old.close(code=4001, reason="replaced by a newer connection")
|
||||||
|
|
||||||
|
async def remove(self, node_id: UUID, websocket: WebSocket) -> bool:
|
||||||
|
async with self._lock:
|
||||||
|
if self._connections.get(node_id) is websocket:
|
||||||
|
self._connections.pop(node_id, None)
|
||||||
|
self._send_locks.pop(node_id, None)
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def close(self, node_id: UUID) -> None:
|
||||||
|
async with self._lock:
|
||||||
|
websocket = self._connections.pop(node_id, None)
|
||||||
|
self._send_locks.pop(node_id, None)
|
||||||
|
if websocket is not None:
|
||||||
|
await websocket.close(code=4003, reason="node disabled")
|
||||||
|
|
||||||
|
async def node_ids(self) -> set[UUID]:
|
||||||
|
async with self._lock:
|
||||||
|
return set(self._connections)
|
||||||
|
|
||||||
|
async def send(self, node_id: UUID, message: dict) -> bool:
|
||||||
|
async with self._lock:
|
||||||
|
websocket = self._connections.get(node_id)
|
||||||
|
send_lock = self._send_locks.get(node_id)
|
||||||
|
if websocket is None or send_lock is None:
|
||||||
|
return False
|
||||||
|
async with send_lock:
|
||||||
|
await websocket.send_json(message)
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def send_on(self, node_id: UUID, websocket: WebSocket, message: dict) -> bool:
|
||||||
|
async with self._lock:
|
||||||
|
current = self._connections.get(node_id)
|
||||||
|
send_lock = self._send_locks.get(node_id)
|
||||||
|
if current is not websocket or send_lock is None:
|
||||||
|
return False
|
||||||
|
async with send_lock:
|
||||||
|
await websocket.send_json(message)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
node_connections = NodeConnectionManager()
|
||||||
|
|
||||||
|
|
||||||
|
def _bearer(authorization: str | None) -> str:
|
||||||
|
scheme, _, token = (authorization or "").partition(" ")
|
||||||
|
if scheme.lower() != "bearer" or not token:
|
||||||
|
raise SoftwareNodeError("missing node bearer token")
|
||||||
|
return token
|
||||||
|
|
||||||
|
|
||||||
|
def _authenticate_output_request(
|
||||||
|
job_id: UUID,
|
||||||
|
authorization: str | None,
|
||||||
|
x_node_id: str,
|
||||||
|
x_lease_id: str,
|
||||||
|
x_request_digest: str,
|
||||||
|
) -> tuple[UUID, UUID, dict]:
|
||||||
|
try:
|
||||||
|
node_id = UUID(x_node_id)
|
||||||
|
lease_id = UUID(x_lease_id)
|
||||||
|
authenticate_node(node_id, _bearer(authorization))
|
||||||
|
except (ValueError, SoftwareNodeError) as exc:
|
||||||
|
raise HTTPException(401, "invalid node credentials or job identity") from exc
|
||||||
|
context = get_job_output_context(node_id, job_id, lease_id, x_request_digest)
|
||||||
|
if context is None:
|
||||||
|
raise HTTPException(404, "software job output target not found")
|
||||||
|
return node_id, lease_id, context
|
||||||
|
|
||||||
|
|
||||||
|
def _hash_file(path: Path) -> str:
|
||||||
|
digest = sha256()
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||||
|
digest.update(chunk)
|
||||||
|
return digest.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _reject_symlink_path(root: Path, target: Path) -> None:
|
||||||
|
root = root.resolve()
|
||||||
|
current = root
|
||||||
|
for part in target.relative_to(root).parts:
|
||||||
|
current = current / part
|
||||||
|
if current.is_symlink():
|
||||||
|
raise HTTPException(409, "software job output path contains a symbolic link")
|
||||||
|
|
||||||
|
|
||||||
|
def _publish_software_job_outputs(job_id: UUID, context: dict, manifest: list[dict]) -> list[dict]:
|
||||||
|
root = load_user_root(context["user_id"])
|
||||||
|
working_dir = safe_join(root, context["working_dir"])
|
||||||
|
staging = safe_join(root, f".zcbot_software_job_staging/{job_id}")
|
||||||
|
relative_output = Path("origin") / str(job_id)
|
||||||
|
destination = safe_join(working_dir, relative_output.as_posix())
|
||||||
|
source = staging if staging.is_dir() else destination
|
||||||
|
_reject_symlink_path(root, source)
|
||||||
|
_reject_symlink_path(root, destination)
|
||||||
|
for item in manifest:
|
||||||
|
path = source / item["filename"]
|
||||||
|
if (
|
||||||
|
not path.is_file()
|
||||||
|
or path.stat().st_size != item["size_bytes"]
|
||||||
|
or _hash_file(path) != item["sha256"]
|
||||||
|
):
|
||||||
|
raise SoftwareJobError(f"uploaded artifact is missing or invalid: {item['artifact_id']}")
|
||||||
|
if source == staging:
|
||||||
|
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
if destination.exists():
|
||||||
|
raise SoftwareJobError("software job output destination already exists unexpectedly")
|
||||||
|
os.replace(staging, destination)
|
||||||
|
try:
|
||||||
|
staging.parent.rmdir()
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
refs = tuple({
|
||||||
|
"path": (relative_output / item["filename"]).as_posix(),
|
||||||
|
"label": item["filename"],
|
||||||
|
"media_type": item["media_type"],
|
||||||
|
} for item in manifest)
|
||||||
|
published_refs = register_published_artifacts(
|
||||||
|
user_id=context["user_id"],
|
||||||
|
task_id=context["task_id"],
|
||||||
|
user_root=root,
|
||||||
|
working_dir=working_dir,
|
||||||
|
refs=refs,
|
||||||
|
)
|
||||||
|
refs_by_path = {item["path"]: item for item in published_refs}
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
**item,
|
||||||
|
"source_artifact_id": item["artifact_id"],
|
||||||
|
"artifact_id": refs_by_path[(relative_output / item["filename"]).as_posix()]["artifact_id"],
|
||||||
|
"path": (relative_output / item["filename"]).as_posix(),
|
||||||
|
}
|
||||||
|
for item in manifest
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def register_software_node_routes(app, *, require_user, require_admin) -> None:
|
||||||
|
@app.post(
|
||||||
|
"/v1/software-nodes/enroll",
|
||||||
|
tags=["software-nodes"],
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
)
|
||||||
|
def node_enroll(body: SoftwareNodeEnrollRequest):
|
||||||
|
try:
|
||||||
|
return enroll_node(**body.model_dump())
|
||||||
|
except SoftwareNodeError as exc:
|
||||||
|
raise HTTPException(400, str(exc)) from exc
|
||||||
|
|
||||||
|
@app.get("/v1/software-jobs/{job_id}/input", tags=["software-nodes"])
|
||||||
|
def download_software_job_input(
|
||||||
|
job_id: UUID,
|
||||||
|
authorization: str | None = Header(default=None),
|
||||||
|
x_node_id: str = Header(default=""),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
node_id = UUID(x_node_id)
|
||||||
|
authenticate_node(node_id, _bearer(authorization))
|
||||||
|
except (ValueError, SoftwareNodeError) as exc:
|
||||||
|
raise HTTPException(401, "invalid node credentials") from exc
|
||||||
|
item = get_job_input(node_id, job_id)
|
||||||
|
if item is None:
|
||||||
|
raise HTTPException(404, "software job input not found")
|
||||||
|
target = safe_join(load_user_root(item["user_id"]), item["current_path"])
|
||||||
|
if not target.is_file():
|
||||||
|
raise HTTPException(404, "software job input file not found")
|
||||||
|
stat = target.stat()
|
||||||
|
if stat.st_size != item["size_bytes"]:
|
||||||
|
raise HTTPException(409, "software job input changed after submission")
|
||||||
|
digest = sha256()
|
||||||
|
with target.open("rb") as handle:
|
||||||
|
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||||
|
digest.update(chunk)
|
||||||
|
if digest.hexdigest() != item["sha256"]:
|
||||||
|
raise HTTPException(409, "software job input changed after submission")
|
||||||
|
return FileResponse(
|
||||||
|
path=str(target),
|
||||||
|
filename=item["filename"],
|
||||||
|
media_type="application/octet-stream",
|
||||||
|
headers={
|
||||||
|
"Cache-Control": "no-store",
|
||||||
|
"X-Content-SHA256": item["sha256"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
@app.put(
|
||||||
|
"/v1/software-jobs/{job_id}/outputs/{artifact_id}",
|
||||||
|
tags=["software-nodes"],
|
||||||
|
status_code=status.HTTP_204_NO_CONTENT,
|
||||||
|
)
|
||||||
|
async def upload_software_job_output(
|
||||||
|
job_id: UUID,
|
||||||
|
artifact_id: str,
|
||||||
|
request: Request,
|
||||||
|
authorization: str | None = Header(default=None),
|
||||||
|
x_node_id: str = Header(default=""),
|
||||||
|
x_lease_id: str = Header(default=""),
|
||||||
|
x_request_digest: str = Header(default=""),
|
||||||
|
x_content_sha256: str = Header(default=""),
|
||||||
|
x_content_length: int = Header(default=-1),
|
||||||
|
):
|
||||||
|
_, _, context = await asyncio.to_thread(
|
||||||
|
_authenticate_output_request,
|
||||||
|
job_id, authorization, x_node_id, x_lease_id, x_request_digest,
|
||||||
|
)
|
||||||
|
metadata = OUTPUT_ARTIFACTS.get(artifact_id)
|
||||||
|
if metadata is None:
|
||||||
|
raise HTTPException(400, "unsupported output artifact identity")
|
||||||
|
filename, _, output_format = metadata
|
||||||
|
requested_formats = set(context["request"].get("output", {}).get("formats") or [])
|
||||||
|
if output_format is not None and output_format not in requested_formats:
|
||||||
|
raise HTTPException(400, "output artifact was not requested")
|
||||||
|
if not 1 <= x_content_length <= MAX_OUTPUT_ARTIFACT_BYTES:
|
||||||
|
raise HTTPException(400, "output artifact size is invalid")
|
||||||
|
if len(x_content_sha256) != 64 or any(c not in "0123456789abcdef" for c in x_content_sha256):
|
||||||
|
raise HTTPException(400, "output artifact digest is invalid")
|
||||||
|
root = load_user_root(context["user_id"])
|
||||||
|
published = safe_join(
|
||||||
|
safe_join(root, context["working_dir"]),
|
||||||
|
f"origin/{job_id}/{filename}",
|
||||||
|
)
|
||||||
|
if published.is_file():
|
||||||
|
if published.stat().st_size == x_content_length and _hash_file(published) == x_content_sha256:
|
||||||
|
return None
|
||||||
|
raise HTTPException(409, "published output conflicts with uploaded artifact")
|
||||||
|
staging = safe_join(root, f".zcbot_software_job_staging/{job_id}")
|
||||||
|
_reject_symlink_path(root, staging)
|
||||||
|
staging.mkdir(parents=True, exist_ok=True)
|
||||||
|
destination = staging / filename
|
||||||
|
if destination.is_file():
|
||||||
|
if destination.stat().st_size == x_content_length and _hash_file(destination) == x_content_sha256:
|
||||||
|
return None
|
||||||
|
raise HTTPException(409, "uploaded output conflicts with existing staging file")
|
||||||
|
staged_total = sum(
|
||||||
|
item.stat().st_size for item in staging.iterdir() if item.is_file()
|
||||||
|
)
|
||||||
|
if staged_total + x_content_length > MAX_OUTPUT_TOTAL_BYTES:
|
||||||
|
raise HTTPException(413, "software job outputs exceed the total size limit")
|
||||||
|
temporary = destination.with_name(destination.name + ".tmp-" + os.urandom(8).hex())
|
||||||
|
digest = sha256()
|
||||||
|
total = 0
|
||||||
|
try:
|
||||||
|
with temporary.open("xb") as handle:
|
||||||
|
async for chunk in request.stream():
|
||||||
|
total += len(chunk)
|
||||||
|
if total > x_content_length or total > MAX_OUTPUT_ARTIFACT_BYTES:
|
||||||
|
raise HTTPException(413, "output artifact exceeded declared size")
|
||||||
|
digest.update(chunk)
|
||||||
|
handle.write(chunk)
|
||||||
|
handle.flush()
|
||||||
|
os.fsync(handle.fileno())
|
||||||
|
if total != x_content_length or digest.hexdigest() != x_content_sha256:
|
||||||
|
raise HTTPException(400, "output artifact did not match declared metadata")
|
||||||
|
os.replace(temporary, destination)
|
||||||
|
finally:
|
||||||
|
temporary.unlink(missing_ok=True)
|
||||||
|
return None
|
||||||
|
|
||||||
|
@app.post("/v1/software-jobs/{job_id}/outputs/complete", tags=["software-nodes"])
|
||||||
|
async def complete_software_job_outputs(
|
||||||
|
job_id: UUID,
|
||||||
|
request: Request,
|
||||||
|
authorization: str | None = Header(default=None),
|
||||||
|
x_node_id: str = Header(default=""),
|
||||||
|
x_lease_id: str = Header(default=""),
|
||||||
|
x_request_digest: str = Header(default=""),
|
||||||
|
):
|
||||||
|
node_id, lease_id, context = await asyncio.to_thread(
|
||||||
|
_authenticate_output_request,
|
||||||
|
job_id, authorization, x_node_id, x_lease_id, x_request_digest,
|
||||||
|
)
|
||||||
|
body = await request.json()
|
||||||
|
if not isinstance(body, dict):
|
||||||
|
raise HTTPException(400, "output completion body must be an object")
|
||||||
|
try:
|
||||||
|
manifest = validate_output_manifest(context["request"], body.get("artifact_manifest"))
|
||||||
|
published = await asyncio.to_thread(
|
||||||
|
_publish_software_job_outputs, job_id, context, manifest
|
||||||
|
)
|
||||||
|
terminal = {
|
||||||
|
"job_id": str(job_id),
|
||||||
|
"lease_id": str(lease_id),
|
||||||
|
"request_digest": x_request_digest,
|
||||||
|
"status": "succeeded",
|
||||||
|
"error": {},
|
||||||
|
"artifact_manifest": published,
|
||||||
|
}
|
||||||
|
await asyncio.to_thread(record_job_terminal, node_id, terminal)
|
||||||
|
except (SoftwareJobError, KeyError, TypeError) as exc:
|
||||||
|
raise HTTPException(409, str(exc)) from exc
|
||||||
|
return {"status": "succeeded", "artifact_manifest": published}
|
||||||
|
|
||||||
|
@app.websocket("/v1/software-nodes/connect")
|
||||||
|
async def node_connect(websocket: WebSocket):
|
||||||
|
try:
|
||||||
|
node_id = UUID(websocket.headers.get("x-node-id", ""))
|
||||||
|
token = _bearer(websocket.headers.get("authorization"))
|
||||||
|
identity = await asyncio.to_thread(authenticate_node, node_id, token)
|
||||||
|
except (ValueError, SoftwareNodeError):
|
||||||
|
# 握手前 close 会被 ASGI 统一表现为 HTTP 403,客户端无法区分
|
||||||
|
# “凭据无效”和“代理/路由没有正确转发 WebSocket”。先升级再用
|
||||||
|
# 应用关闭码给已持有 Node ID/Token 的节点返回明确诊断。
|
||||||
|
await websocket.accept()
|
||||||
|
await websocket.close(code=4003, reason="invalid node credentials")
|
||||||
|
return
|
||||||
|
await websocket.accept()
|
||||||
|
await node_connections.activate(node_id, websocket)
|
||||||
|
try:
|
||||||
|
await node_connections.send_on(
|
||||||
|
node_id,
|
||||||
|
websocket,
|
||||||
|
{"type": "connected", "heartbeat_seconds": 15},
|
||||||
|
)
|
||||||
|
for cancel in await asyncio.to_thread(pending_node_cancellations, node_id):
|
||||||
|
await node_connections.send_on(
|
||||||
|
node_id, websocket, {"type": "job_cancel", "payload": cancel}
|
||||||
|
)
|
||||||
|
while True:
|
||||||
|
message = await websocket.receive_json()
|
||||||
|
message_type = message.get("type")
|
||||||
|
payload = message.get("payload") or {}
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
await node_connections.send_on(node_id, websocket,
|
||||||
|
{"type": "error", "code": "unsupported_message"}
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if message_type in {"job_accept", "job_reject"}:
|
||||||
|
await asyncio.to_thread(
|
||||||
|
respond_to_offer,
|
||||||
|
node_id,
|
||||||
|
accepted=message_type == "job_accept",
|
||||||
|
payload=payload,
|
||||||
|
)
|
||||||
|
await node_connections.send_on(node_id, websocket,
|
||||||
|
{"type": "ack", "message_id": message.get("message_id")}
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if message_type in {"job_state", "job_terminal"}:
|
||||||
|
if message_type == "job_terminal" and payload.get("status") == "succeeded":
|
||||||
|
await node_connections.send_on(node_id, websocket,
|
||||||
|
{
|
||||||
|
"type": "error",
|
||||||
|
"code": "outputs_not_published",
|
||||||
|
"message_id": message.get("message_id"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
handler = (
|
||||||
|
update_job_state
|
||||||
|
if message_type == "job_state"
|
||||||
|
else record_job_terminal
|
||||||
|
)
|
||||||
|
await asyncio.to_thread(handler, node_id, payload)
|
||||||
|
await node_connections.send_on(node_id, websocket,
|
||||||
|
{"type": "ack", "message_id": message.get("message_id")}
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if message_type not in {"hello", "heartbeat"}:
|
||||||
|
await node_connections.send_on(node_id, websocket,
|
||||||
|
{"type": "error", "code": "unsupported_message"}
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if payload.get("install_id") and payload["install_id"] != str(
|
||||||
|
identity["install_id"]
|
||||||
|
):
|
||||||
|
await websocket.close(code=1008, reason="install identity mismatch")
|
||||||
|
return
|
||||||
|
await asyncio.to_thread(
|
||||||
|
update_node_runtime,
|
||||||
|
node_id,
|
||||||
|
status="online",
|
||||||
|
runtime=payload,
|
||||||
|
)
|
||||||
|
await node_connections.send_on(node_id, websocket,
|
||||||
|
{"type": "ack", "message_id": message.get("message_id")}
|
||||||
|
)
|
||||||
|
for cancel in await asyncio.to_thread(pending_node_cancellations, node_id):
|
||||||
|
await node_connections.send_on(
|
||||||
|
node_id, websocket, {"type": "job_cancel", "payload": cancel}
|
||||||
|
)
|
||||||
|
offer = await asyncio.to_thread(
|
||||||
|
offer_next_job, await node_connections.node_ids()
|
||||||
|
)
|
||||||
|
if offer is not None:
|
||||||
|
delivered = await node_connections.send(
|
||||||
|
offer["node_id"],
|
||||||
|
{"type": "job_offer", "payload": offer["payload"]},
|
||||||
|
)
|
||||||
|
if not delivered:
|
||||||
|
await asyncio.to_thread(
|
||||||
|
abandon_offer, offer["node_id"], offer["payload"]
|
||||||
|
)
|
||||||
|
except (
|
||||||
|
SoftwareJobError,
|
||||||
|
SoftwareNodeError,
|
||||||
|
WebSocketDisconnect,
|
||||||
|
RuntimeError,
|
||||||
|
ValueError,
|
||||||
|
):
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
if await node_connections.remove(node_id, websocket):
|
||||||
|
await asyncio.to_thread(mark_node_offline, node_id)
|
||||||
|
await asyncio.to_thread(mark_node_jobs_disconnected, node_id)
|
||||||
|
|
||||||
|
@app.post("/v1/tasks/{task_id}/software-jobs", tags=["software-jobs"])
|
||||||
|
async def submit_software_job(
|
||||||
|
task_id: UUID,
|
||||||
|
body: SoftwareJobCreateRequest,
|
||||||
|
user_id: UUID = Depends(require_user), # noqa: B008
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
job, created = await asyncio.to_thread(
|
||||||
|
create_job, user_id, task_id, **body.model_dump()
|
||||||
|
)
|
||||||
|
offer = await asyncio.to_thread(
|
||||||
|
offer_next_job, await node_connections.node_ids()
|
||||||
|
)
|
||||||
|
if offer is not None:
|
||||||
|
delivered = await node_connections.send(
|
||||||
|
offer["node_id"], {"type": "job_offer", "payload": offer["payload"]}
|
||||||
|
)
|
||||||
|
if not delivered:
|
||||||
|
await asyncio.to_thread(
|
||||||
|
abandon_offer, offer["node_id"], offer["payload"]
|
||||||
|
)
|
||||||
|
return {**job, "created": created}
|
||||||
|
except SoftwareJobError as exc:
|
||||||
|
detail = str(exc)
|
||||||
|
raise HTTPException(404 if detail == "task not found" else 400, detail) from exc
|
||||||
|
|
||||||
|
@app.get("/v1/software-jobs", tags=["software-jobs"])
|
||||||
|
def read_software_jobs(
|
||||||
|
task_id: UUID | None = None,
|
||||||
|
active_only: bool = False,
|
||||||
|
limit: int = 50,
|
||||||
|
user_id: UUID = Depends(require_user), # noqa: B008
|
||||||
|
):
|
||||||
|
return {
|
||||||
|
"results": list_jobs(
|
||||||
|
user_id, task_id=task_id, active_only=active_only, limit=limit
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
@app.post("/v1/software-jobs/{job_id}/cancel", tags=["software-jobs"])
|
||||||
|
async def cancel_software_job(
|
||||||
|
job_id: UUID,
|
||||||
|
user_id: UUID = Depends(require_user), # noqa: B008
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
job, message = await asyncio.to_thread(request_job_cancel, user_id, job_id)
|
||||||
|
except SoftwareJobError as exc:
|
||||||
|
raise HTTPException(404, str(exc)) from exc
|
||||||
|
if message is not None:
|
||||||
|
await node_connections.send(
|
||||||
|
message["node_id"],
|
||||||
|
{"type": "job_cancel", "payload": message["payload"]},
|
||||||
|
)
|
||||||
|
return job
|
||||||
|
|
||||||
|
@app.get("/v1/software-jobs/{job_id}", tags=["software-jobs"])
|
||||||
|
def read_software_job(
|
||||||
|
job_id: UUID,
|
||||||
|
user_id: UUID = Depends(require_user), # noqa: B008
|
||||||
|
):
|
||||||
|
job = get_job(user_id, job_id)
|
||||||
|
if job is None:
|
||||||
|
raise HTTPException(404, "software job not found")
|
||||||
|
return job
|
||||||
|
|
||||||
|
@app.post("/v1/admin/software-node-enrollments", tags=["admin"])
|
||||||
|
def admin_create_software_enrollment(
|
||||||
|
body: SoftwareEnrollmentCreateRequest,
|
||||||
|
user_id: UUID = Depends(require_admin), # noqa: B008
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return create_enrollment(user_id, **body.model_dump())
|
||||||
|
except SoftwareNodeError as exc:
|
||||||
|
raise HTTPException(400, str(exc)) from exc
|
||||||
|
|
||||||
|
@app.get("/v1/admin/software-nodes", tags=["admin"])
|
||||||
|
def admin_software_nodes(user_id: UUID = Depends(require_admin)): # noqa: B008
|
||||||
|
return {"results": list_nodes()}
|
||||||
|
|
||||||
|
@app.patch("/v1/admin/software-nodes/{node_id}", tags=["admin"])
|
||||||
|
async def admin_disable_software_node(
|
||||||
|
node_id: UUID,
|
||||||
|
body: SoftwareNodeDisableRequest,
|
||||||
|
user_id: UUID = Depends(require_admin), # noqa: B008
|
||||||
|
):
|
||||||
|
if not await asyncio.to_thread(set_node_disabled, node_id, body.disabled):
|
||||||
|
raise HTTPException(404, "software node not found")
|
||||||
|
if body.disabled:
|
||||||
|
await node_connections.close(node_id)
|
||||||
|
return {
|
||||||
|
"node_id": str(node_id),
|
||||||
|
"status": "disabled" if body.disabled else "offline",
|
||||||
|
}
|
||||||
|
|
||||||
|
@app.delete("/v1/admin/software-nodes/{node_id}", tags=["admin"])
|
||||||
|
async def admin_delete_software_node(
|
||||||
|
node_id: UUID,
|
||||||
|
user_id: UUID = Depends(require_admin), # noqa: B008
|
||||||
|
):
|
||||||
|
# 先撤掉在线连接,避免删除后的旧 socket 继续上报运行态。
|
||||||
|
await node_connections.close(node_id)
|
||||||
|
if not await asyncio.to_thread(delete_node, node_id):
|
||||||
|
raise HTTPException(404, "software node not found")
|
||||||
|
return {"node_id": str(node_id), "status": "deleted"}
|
||||||
|
|
@ -116,13 +116,13 @@ class ExternalSystemCredentialsRequest(BaseModel):
|
||||||
credentials: dict[str, str] = Field(default_factory=dict)
|
credentials: dict[str, str] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
class ComputeEnrollmentCreateRequest(BaseModel):
|
class SoftwareEnrollmentCreateRequest(BaseModel):
|
||||||
expected_name: str = ""
|
expected_name: str = ""
|
||||||
capabilities: list[str] = Field(default_factory=lambda: ["origin.plot@v1"])
|
capabilities: list[str] = Field(default_factory=lambda: ["origin.plot@v1"])
|
||||||
ttl_seconds: int = 600
|
ttl_seconds: int = 600
|
||||||
|
|
||||||
|
|
||||||
class ComputeNodeEnrollRequest(BaseModel):
|
class SoftwareNodeEnrollRequest(BaseModel):
|
||||||
enrollment_code: str
|
enrollment_code: str
|
||||||
node_name: str
|
node_name: str
|
||||||
install_id: UUID
|
install_id: UUID
|
||||||
|
|
@ -131,5 +131,11 @@ class ComputeNodeEnrollRequest(BaseModel):
|
||||||
capabilities: list[str]
|
capabilities: list[str]
|
||||||
|
|
||||||
|
|
||||||
class ComputeNodeDisableRequest(BaseModel):
|
class SoftwareNodeDisableRequest(BaseModel):
|
||||||
disabled: bool = True
|
disabled: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
class SoftwareJobCreateRequest(BaseModel):
|
||||||
|
idempotency_key: str
|
||||||
|
capability: str = "origin.plot@v1"
|
||||||
|
request: dict = Field(default_factory=dict)
|
||||||
|
|
|
||||||
|
|
@ -1069,6 +1069,34 @@
|
||||||
overflow: hidden; text-overflow: ellipsis; white-space: nowrap; margin-top: 2px;
|
overflow: hidden; text-overflow: ellipsis; white-space: nowrap; margin-top: 2px;
|
||||||
}
|
}
|
||||||
.proc-toast.fail { border-color: rgba(192,57,43,0.5); }
|
.proc-toast.fail { border-color: rgba(192,57,43,0.5); }
|
||||||
|
#software-job-center {
|
||||||
|
position: fixed; right: 16px; bottom: 16px; z-index: 115; display: none;
|
||||||
|
width: min(380px, calc(100vw - 24px)); font-size: 13px;
|
||||||
|
}
|
||||||
|
#software-job-center.show { display: block; }
|
||||||
|
#software-job-toggle {
|
||||||
|
margin-left: auto; display: flex; align-items: center; gap: 8px;
|
||||||
|
border-radius: 999px; padding: 8px 13px; background: #fff;
|
||||||
|
border: 1px solid var(--border); box-shadow: 0 4px 16px rgba(0,0,0,.16); cursor: pointer;
|
||||||
|
}
|
||||||
|
#software-job-center.expanded #software-job-toggle { border-radius: 0 0 10px 10px; }
|
||||||
|
#software-job-panel {
|
||||||
|
max-height: min(520px, calc(100vh - 110px)); overflow: auto; background: #fff;
|
||||||
|
border: 1px solid var(--border); border-bottom: 0; border-radius: 10px 10px 0 0;
|
||||||
|
box-shadow: 0 -4px 20px rgba(0,0,0,.16); padding: 8px;
|
||||||
|
}
|
||||||
|
.sj-card { padding: 10px; border-bottom: 1px solid var(--border); }
|
||||||
|
.sj-card:last-child { border-bottom: 0; }
|
||||||
|
.sj-title { display: flex; justify-content: space-between; gap: 10px; }
|
||||||
|
.sj-title span { color: var(--muted); white-space: nowrap; }
|
||||||
|
.sj-card.failed .sj-title span { color: var(--danger); }
|
||||||
|
.sj-card.disconnected .sj-title span, .sj-card.cancelling .sj-title span { color: #a66514; }
|
||||||
|
.sj-sub, .sj-meta { margin-top: 4px; color: var(--muted); font-size: 11px; overflow-wrap: anywhere; }
|
||||||
|
.sj-progress { height: 4px; margin-top: 7px; border-radius: 4px; background: var(--panel-muted); overflow: hidden; }
|
||||||
|
.sj-progress i { display: block; height: 100%; background: var(--accent); transition: width .25s ease; }
|
||||||
|
.sj-actions { display: flex; justify-content: flex-end; gap: 6px; margin-top: 8px; }
|
||||||
|
.sj-empty { padding: 22px; text-align: center; color: var(--muted); }
|
||||||
|
#software-job-center.show ~ #proc-toasts { bottom: 62px; }
|
||||||
/* media tool 摘要 banner(model / size / cost / elapsed,折叠态也可见) */
|
/* media tool 摘要 banner(model / size / cost / elapsed,折叠态也可见) */
|
||||||
.tool-banner {
|
.tool-banner {
|
||||||
display: inline-flex; flex-wrap: wrap; gap: 6px;
|
display: inline-flex; flex-wrap: wrap; gap: 6px;
|
||||||
|
|
@ -2174,6 +2202,13 @@
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div id="software-job-center" aria-live="polite">
|
||||||
|
<div id="software-job-panel" hidden></div>
|
||||||
|
<button id="software-job-toggle" type="button" title="展开专业软件任务">
|
||||||
|
<span>专业软件任务</span><strong id="software-job-count">0 条</strong>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
<script type="module" src="js/main.js"></script>
|
<script type="module" src="js/main.js"></script>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
||||||
|
|
@ -55,7 +55,7 @@ let externalDefinitions = [];
|
||||||
let externalUsers = [];
|
let externalUsers = [];
|
||||||
let externalDefinitionsLoaded = false;
|
let externalDefinitionsLoaded = false;
|
||||||
let externalEditingId = "";
|
let externalEditingId = "";
|
||||||
let computeNodes = [];
|
let softwareNodes = [];
|
||||||
|
|
||||||
// ───── 格式化 ─────
|
// ───── 格式化 ─────
|
||||||
function fmtCNY(n) {
|
function fmtCNY(n) {
|
||||||
|
|
@ -167,12 +167,16 @@ function nodeStatusHTML(status) {
|
||||||
}
|
}
|
||||||
|
|
||||||
function renderWindowsNodes() {
|
function renderWindowsNodes() {
|
||||||
const rows = computeNodes.map(node => {
|
const rows = softwareNodes.map(node => {
|
||||||
const runtime = node.runtime || {};
|
const runtime = node.runtime || {};
|
||||||
|
const origin = runtime.origin || {};
|
||||||
|
const originState = origin.health === "ready" ? "Origin 可用" : "Origin 不可用";
|
||||||
|
const originVersion = origin.software_version ? ` ${origin.software_version}` : "";
|
||||||
const runtimeParts = [
|
const runtimeParts = [
|
||||||
node.os_version || "",
|
node.os_version || "",
|
||||||
runtime.desktop_session === true ? "桌面会话" : "",
|
runtime.desktop_session === true ? "桌面会话" : "",
|
||||||
runtime.available_slots != null ? `可用槽位 ${runtime.available_slots}` : "",
|
runtime.available_slots != null ? `可用槽位 ${runtime.available_slots}` : "",
|
||||||
|
runtime.origin ? `${originState}${originVersion}` : "",
|
||||||
].filter(Boolean);
|
].filter(Boolean);
|
||||||
const lastSeen = node.last_seen_at
|
const lastSeen = node.last_seen_at
|
||||||
? `<span title="${escapeHtml(fmtTime(node.last_seen_at))}">${escapeHtml(fmtTimeAgo(node.last_seen_at))}</span>`
|
? `<span title="${escapeHtml(fmtTime(node.last_seen_at))}">${escapeHtml(fmtTimeAgo(node.last_seen_at))}</span>`
|
||||||
|
|
@ -193,22 +197,22 @@ function renderWindowsNodes() {
|
||||||
}).join("") || `<tr><td colspan="7" class="empty">尚无已注册的 Windows Node</td></tr>`;
|
}).join("") || `<tr><td colspan="7" class="empty">尚无已注册的 Windows Node</td></tr>`;
|
||||||
|
|
||||||
$("s-windows-node").innerHTML = `<div class="card"><div class="card-head">`
|
$("s-windows-node").innerHTML = `<div class="card"><div class="card-head">`
|
||||||
+ `<div><h2>Windows Node(${computeNodes.length})</h2><div class="node-help">查看节点状态;禁用会立即断开节点并拒绝后续连接。</div></div>`
|
+ `<div><h2>Windows Node(${softwareNodes.length})</h2><div class="node-help">查看节点状态;禁用会立即断开节点并拒绝后续连接。</div></div>`
|
||||||
+ `<button id="node-enrollment-open" class="primary" type="button">生成 Windows Node 注册码</button>`
|
+ `<button id="node-enrollment-open" class="primary" type="button">生成 Windows Node 注册码</button>`
|
||||||
+ `</div><div class="scroll-x"><table><thead><tr><th>节点</th><th>状态</th><th>运行环境</th>`
|
+ `</div><div class="scroll-x"><table><thead><tr><th>节点</th><th>状态</th><th>运行环境</th>`
|
||||||
+ `<th>版本</th><th>能力</th><th>最近心跳</th><th>操作</th></tr></thead><tbody>${rows}</tbody></table></div></div>`;
|
+ `<th>版本</th><th>能力</th><th>最近心跳</th><th>操作</th></tr></thead><tbody>${rows}</tbody></table></div></div>`;
|
||||||
$("node-enrollment-open").onclick = openNodeEnrollmentModal;
|
$("node-enrollment-open").onclick = openNodeEnrollmentModal;
|
||||||
$("s-windows-node").querySelectorAll("[data-node-toggle]").forEach(button => {
|
$("s-windows-node").querySelectorAll("[data-node-toggle]").forEach(button => {
|
||||||
button.onclick = () => toggleComputeNode(button);
|
button.onclick = () => toggleSoftwareNode(button);
|
||||||
});
|
});
|
||||||
$("s-windows-node").querySelectorAll("[data-node-delete]").forEach(button => {
|
$("s-windows-node").querySelectorAll("[data-node-delete]").forEach(button => {
|
||||||
button.onclick = () => deleteComputeNode(button);
|
button.onclick = () => deleteSoftwareNode(button);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
async function toggleComputeNode(button) {
|
async function toggleSoftwareNode(button) {
|
||||||
const row = button.closest("tr[data-node-id]");
|
const row = button.closest("tr[data-node-id]");
|
||||||
const node = computeNodes.find(item => item.node_id === row?.dataset.nodeId);
|
const node = softwareNodes.find(item => item.node_id === row?.dataset.nodeId);
|
||||||
if (!node) return;
|
if (!node) return;
|
||||||
const disabling = button.dataset.nodeToggle === "disable";
|
const disabling = button.dataset.nodeToggle === "disable";
|
||||||
const confirmed = await dialogConfirm({
|
const confirmed = await dialogConfirm({
|
||||||
|
|
@ -222,20 +226,20 @@ async function toggleComputeNode(button) {
|
||||||
if (!confirmed) return;
|
if (!confirmed) return;
|
||||||
button.disabled = true;
|
button.disabled = true;
|
||||||
try {
|
try {
|
||||||
await apiSend("PATCH", `/v1/admin/compute-nodes/${node.node_id}`, {
|
await apiSend("PATCH", `/v1/admin/software-nodes/${node.node_id}`, {
|
||||||
disabled: disabling,
|
disabled: disabling,
|
||||||
});
|
});
|
||||||
message(disabling ? "节点已禁用" : "节点已重新启用,请在节点电脑上立即重连", "success", 5000);
|
message(disabling ? "节点已禁用" : "节点已重新启用,请在节点电脑上立即重连", "success", 5000);
|
||||||
await loadComputeNodes();
|
await loadSoftwareNodes();
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (err.code !== "auth") message("更新节点失败:" + (err.message || String(err)), "error", 5000);
|
if (err.code !== "auth") message("更新节点失败:" + (err.message || String(err)), "error", 5000);
|
||||||
button.disabled = false;
|
button.disabled = false;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async function deleteComputeNode(button) {
|
async function deleteSoftwareNode(button) {
|
||||||
const row = button.closest("tr[data-node-id]");
|
const row = button.closest("tr[data-node-id]");
|
||||||
const node = computeNodes.find(item => item.node_id === row?.dataset.nodeId);
|
const node = softwareNodes.find(item => item.node_id === row?.dataset.nodeId);
|
||||||
if (!node) return;
|
if (!node) return;
|
||||||
const confirmed = await dialogConfirm({
|
const confirmed = await dialogConfirm({
|
||||||
title: "删除 Windows Node",
|
title: "删除 Windows Node",
|
||||||
|
|
@ -246,9 +250,9 @@ async function deleteComputeNode(button) {
|
||||||
if (!confirmed) return;
|
if (!confirmed) return;
|
||||||
button.disabled = true;
|
button.disabled = true;
|
||||||
try {
|
try {
|
||||||
await apiSend("DELETE", `/v1/admin/compute-nodes/${node.node_id}`, {});
|
await apiSend("DELETE", `/v1/admin/software-nodes/${node.node_id}`, {});
|
||||||
message("节点已删除,本机需重新注册后才能使用", "success", 5000);
|
message("节点已删除,本机需重新注册后才能使用", "success", 5000);
|
||||||
await loadComputeNodes();
|
await loadSoftwareNodes();
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
if (err.code !== "auth") message("删除节点失败:" + (err.message || String(err)), "error", 5000);
|
if (err.code !== "auth") message("删除节点失败:" + (err.message || String(err)), "error", 5000);
|
||||||
button.disabled = false;
|
button.disabled = false;
|
||||||
|
|
@ -306,7 +310,7 @@ async function createNodeEnrollment(e) {
|
||||||
submit.disabled = true;
|
submit.disabled = true;
|
||||||
submit.textContent = "生成中…";
|
submit.textContent = "生成中…";
|
||||||
try {
|
try {
|
||||||
const result = await apiSend("POST", "/v1/admin/compute-node-enrollments", {
|
const result = await apiSend("POST", "/v1/admin/software-node-enrollments", {
|
||||||
expected_name: $("node-expected-name").value.trim(),
|
expected_name: $("node-expected-name").value.trim(),
|
||||||
capabilities: ["origin.plot@v1"],
|
capabilities: ["origin.plot@v1"],
|
||||||
ttl_seconds: 600,
|
ttl_seconds: 600,
|
||||||
|
|
@ -1031,10 +1035,10 @@ async function loadExternalDefinitions(force = false) {
|
||||||
} catch (e) { /* overview 统一处理鉴权 */ }
|
} catch (e) { /* overview 统一处理鉴权 */ }
|
||||||
}
|
}
|
||||||
|
|
||||||
async function loadComputeNodes() {
|
async function loadSoftwareNodes() {
|
||||||
try {
|
try {
|
||||||
const result = await apiGet("/v1/admin/compute-nodes");
|
const result = await apiGet("/v1/admin/software-nodes");
|
||||||
computeNodes = result.results || [];
|
softwareNodes = result.results || [];
|
||||||
renderWindowsNodes();
|
renderWindowsNodes();
|
||||||
} catch (e) { /* overview 统一处理鉴权 */ }
|
} catch (e) { /* overview 统一处理鉴权 */ }
|
||||||
}
|
}
|
||||||
|
|
@ -1048,7 +1052,7 @@ async function refresh() {
|
||||||
loadModels();
|
loadModels();
|
||||||
loadUserUsage(userPage);
|
loadUserUsage(userPage);
|
||||||
loadStorage(storagePage);
|
loadStorage(storagePage);
|
||||||
loadComputeNodes();
|
loadSoftwareNodes();
|
||||||
loadExternalDefinitions();
|
loadExternalDefinitions();
|
||||||
loadToolFailures();
|
loadToolFailures();
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
|
|
|
||||||
|
|
@ -19,6 +19,7 @@ import { loadFolderSuggestions } from "./newtask.js";
|
||||||
import { embedInit } from "./embed.js";
|
import { embedInit } from "./embed.js";
|
||||||
import { loadTaskList, loadModels, loadChannelCards, selectTask, showNewConversationDraft } from "./chat.js";
|
import { loadTaskList, loadModels, loadChannelCards, selectTask, showNewConversationDraft } from "./chat.js";
|
||||||
import { refreshProcs } from "./procs.js";
|
import { refreshProcs } from "./procs.js";
|
||||||
|
import { initSoftwareJobs } from "./software_jobs.js";
|
||||||
|
|
||||||
// ───── enter app ─────
|
// ───── enter app ─────
|
||||||
export function enterApp(initialTaskId = "") {
|
export function enterApp(initialTaskId = "") {
|
||||||
|
|
@ -40,6 +41,7 @@ export function enterApp(initialTaskId = "") {
|
||||||
loadStorage(); // 顶栏存储用量(后台扫描快照,非实时)
|
loadStorage(); // 顶栏存储用量(后台扫描快照,非实时)
|
||||||
loadRole(); // 拉 /v1/me,admin 才显「管理」入口(/static/admin.html)
|
loadRole(); // 拉 /v1/me,admin 才显「管理」入口(/static/admin.html)
|
||||||
refreshProcs(); // 后台进程(bg proc):上个会话遗留的 running proc 恢复轮询/展示
|
refreshProcs(); // 后台进程(bg proc):上个会话遗留的 running proc 恢复轮询/展示
|
||||||
|
initSoftwareJobs();
|
||||||
}
|
}
|
||||||
|
|
||||||
// 顶栏用户名:默认显 name(兜底 user_name → email → uid8),title 悬浮给完整身份。
|
// 顶栏用户名:默认显 name(兜底 user_name → email → uid8),title 悬浮给完整身份。
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,144 @@
|
||||||
|
// 专业软件任务中心:用户级轮询、跨对话状态、取消与终态通知。
|
||||||
|
import { api } from "./api.js";
|
||||||
|
import { state } from "./state.js";
|
||||||
|
import { $ } from "./dom.js";
|
||||||
|
import { escapeHtml } from "./format.js";
|
||||||
|
import { selectTask } from "./chat.js";
|
||||||
|
import { dialogConfirm, message } from "./dialog.js";
|
||||||
|
|
||||||
|
const ACTIVE = new Set(["queued", "offered", "dispatched", "running", "disconnected", "cancelling"]);
|
||||||
|
const TERMINAL = new Set(["succeeded", "failed", "cancelled"]);
|
||||||
|
const POLL_ACTIVE_MS = 4000;
|
||||||
|
const POLL_IDLE_MS = 30000;
|
||||||
|
let timer = null;
|
||||||
|
let known = new Map();
|
||||||
|
let jobs = [];
|
||||||
|
let expanded = false;
|
||||||
|
let initialized = false;
|
||||||
|
|
||||||
|
const statusLabel = {
|
||||||
|
queued: "等待计算节点", offered: "正在分配节点", dispatched: "节点已接收",
|
||||||
|
running: "正在执行", disconnected: "节点连接中断", cancelling: "正在停止",
|
||||||
|
succeeded: "已完成", failed: "失败", cancelled: "已取消",
|
||||||
|
};
|
||||||
|
const stageLabel = {
|
||||||
|
accepted: "节点已接收", waiting_input: "正在下载输入文件", ready_to_run: "准备软件环境",
|
||||||
|
origin_running: "Origin 正在生成图表", uploading_outputs: "正在上传结果",
|
||||||
|
cancel_requested: "停止请求已发送", terminal: "任务已结束",
|
||||||
|
};
|
||||||
|
|
||||||
|
export function initSoftwareJobs() {
|
||||||
|
if (initialized || !$("software-job-center")) return;
|
||||||
|
initialized = true;
|
||||||
|
$("software-job-toggle").onclick = () => { expanded = !expanded; render(); };
|
||||||
|
document.addEventListener("click", (event) => {
|
||||||
|
const center = $("software-job-center");
|
||||||
|
if (expanded && center && !center.contains(event.target)) { expanded = false; render(); }
|
||||||
|
});
|
||||||
|
refreshSoftwareJobs();
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function refreshSoftwareJobs() {
|
||||||
|
if (!state.token) return;
|
||||||
|
try {
|
||||||
|
const data = await api("GET", "/v1/software-jobs?limit=50");
|
||||||
|
const next = data.results || [];
|
||||||
|
next.forEach((job) => {
|
||||||
|
const previous = known.get(job.job_id);
|
||||||
|
if (previous && ACTIVE.has(previous) && TERMINAL.has(job.status)) notifyTerminal(job);
|
||||||
|
known.set(job.job_id, job.status);
|
||||||
|
});
|
||||||
|
jobs = next;
|
||||||
|
render();
|
||||||
|
} catch (_) { /* 后台刷新失败静默,下轮恢复 */ }
|
||||||
|
schedule();
|
||||||
|
}
|
||||||
|
|
||||||
|
function schedule() {
|
||||||
|
if (timer) clearTimeout(timer);
|
||||||
|
const delay = jobs.some((job) => ACTIVE.has(job.status)) ? POLL_ACTIVE_MS : POLL_IDLE_MS;
|
||||||
|
timer = setTimeout(refreshSoftwareJobs, document.hidden ? Math.max(delay, 30000) : delay);
|
||||||
|
}
|
||||||
|
|
||||||
|
document.addEventListener("visibilitychange", () => {
|
||||||
|
if (!document.hidden && state.token) refreshSoftwareJobs(); else schedule();
|
||||||
|
});
|
||||||
|
|
||||||
|
function render() {
|
||||||
|
const center = $("software-job-center");
|
||||||
|
const panel = $("software-job-panel");
|
||||||
|
const active = jobs.filter((job) => ACTIVE.has(job.status));
|
||||||
|
const recent = jobs.slice(0, 10);
|
||||||
|
center.classList.toggle("show", active.length > 0 || recent.length > 0);
|
||||||
|
center.classList.toggle("expanded", expanded);
|
||||||
|
$("software-job-count").textContent = active.length ? `${active.length} 运行中` : `${jobs.length} 条`;
|
||||||
|
panel.hidden = !expanded;
|
||||||
|
if (!expanded) return;
|
||||||
|
panel.innerHTML = recent.length
|
||||||
|
? recent.map(jobCard).join("")
|
||||||
|
: '<div class="sj-empty">暂无专业软件任务</div>';
|
||||||
|
panel.querySelectorAll("[data-job-action]").forEach((button) => {
|
||||||
|
button.onclick = (event) => handleAction(event, button);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function jobCard(job) {
|
||||||
|
const summary = job.request_summary || {};
|
||||||
|
const input = job.input || {};
|
||||||
|
const active = ACTIVE.has(job.status);
|
||||||
|
const progress = Math.max(0, Math.min(100, Number(job.progress || 0)));
|
||||||
|
const detail = stageLabel[job.stage] || statusLabel[job.status] || "正在处理";
|
||||||
|
const error = job.error && (job.error.detail || job.error.code);
|
||||||
|
return `<article class="sj-card ${escapeHtml(job.status)}" data-job-id="${escapeHtml(job.job_id)}">
|
||||||
|
<div class="sj-title"><strong>${escapeHtml(summary.display_name || job.capability)}</strong>
|
||||||
|
<span>${escapeHtml(statusLabel[job.status] || job.status)}</span></div>
|
||||||
|
<div class="sj-sub">${escapeHtml(detail)}${error ? ` · ${escapeHtml(error)}` : ""}</div>
|
||||||
|
${active ? `<div class="sj-progress"><i style="width:${progress}%"></i></div>` : ""}
|
||||||
|
<div class="sj-meta">${escapeHtml(job.task_name || "未命名对话")}${input.filename ? ` · ${escapeHtml(input.filename)}` : ""}</div>
|
||||||
|
<div class="sj-actions">
|
||||||
|
<button class="small" data-job-action="open" data-task-id="${escapeHtml(job.task_id)}">打开对话</button>
|
||||||
|
${job.status === "succeeded" ? `<button class="small primary" data-job-action="analyze" data-task-id="${escapeHtml(job.task_id)}">分析结果</button>` : ""}
|
||||||
|
${active && job.status !== "cancelling" ? '<button class="small danger" data-job-action="cancel">停止</button>' : ""}
|
||||||
|
</div>
|
||||||
|
</article>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleAction(event, button) {
|
||||||
|
event.stopPropagation();
|
||||||
|
const card = button.closest("[data-job-id]");
|
||||||
|
const jobId = card.dataset.jobId;
|
||||||
|
const action = button.dataset.jobAction;
|
||||||
|
if (action === "cancel") {
|
||||||
|
if (!await dialogConfirm({
|
||||||
|
title: "停止专业软件任务",
|
||||||
|
message: "确定停止这个任务?已经产生但尚未发布的中间输出可能不会保留。",
|
||||||
|
okText: "停止", danger: true,
|
||||||
|
})) return;
|
||||||
|
button.disabled = true;
|
||||||
|
try { await api("POST", `/v1/software-jobs/${jobId}/cancel`); }
|
||||||
|
catch (error) { message(error.message || "停止失败", "error"); }
|
||||||
|
refreshSoftwareJobs();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const taskId = button.dataset.taskId;
|
||||||
|
if (taskId) await selectTask(taskId);
|
||||||
|
expanded = false;
|
||||||
|
render();
|
||||||
|
if (action === "analyze") {
|
||||||
|
setTimeout(() => {
|
||||||
|
const input = $("chat-input");
|
||||||
|
if (!input) return;
|
||||||
|
input.value = `请分析专业软件任务 ${jobId} 的结果,结合输出图表和输入数据总结主要结论。`;
|
||||||
|
input.focus();
|
||||||
|
input.dispatchEvent(new Event("input", { bubbles: true }));
|
||||||
|
}, 400);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function notifyTerminal(job) {
|
||||||
|
const ok = job.status === "succeeded";
|
||||||
|
const label = ok ? "已完成" : (job.status === "cancelled" ? "已取消" : "失败");
|
||||||
|
const summary = job.request_summary || {};
|
||||||
|
message(`${summary.display_name || "专业软件任务"}${label}`, ok ? "success" : "error", 6000);
|
||||||
|
expanded = true;
|
||||||
|
}
|
||||||
|
|
@ -2,7 +2,15 @@
|
||||||
|
|
||||||
内网 MVP 的 Windows 执行节点,目标运行环境为 Windows 11 Enterprise + .NET 10 SDK 10.0.303。仓库根目录 `global.json` 固定 SDK patch;客户端只使用 .NET Windows Desktop Framework,不依赖第三方 NuGet 包。
|
内网 MVP 的 Windows 执行节点,目标运行环境为 Windows 11 Enterprise + .NET 10 SDK 10.0.303。仓库根目录 `global.json` 固定 SDK patch;客户端只使用 .NET Windows Desktop Framework,不依赖第三方 NuGet 包。
|
||||||
|
|
||||||
当前实现托盘状态角标、小型配置窗口、注册、DPAPI/ACL 配置保存、WebSocket `hello`/心跳和退避重连。尚未实现 `compute_jobs`、Origin Worker、任务目录和产物上传。
|
当前实现托盘状态角标、小型配置窗口、注册、DPAPI/ACL 配置保存、WebSocket `hello`/心跳和退避重连,并只读探测 Origin/OriginPro 安装版本、COM 自动化组件与桌面会话状态。Node 可以接收受控的 `origin.plot@v1` offer,在本机任务目录原子保存请求后回报 accept/reject;随后以 Node 身份流式下载任务绑定的 CSV/XLSX/JSON,校验大小与 SHA-256 后原子保存。固定 Origin Worker 独立于单次 WebSocket 执行,断线不终止已启动绘图。成功产物按 manifest 逐项流式上传并由云端复核大小与 SHA-256,全部完成后原子发布到任务工作目录的 `origin/<job_id>/`;中断后按本地 `upload-complete.json` 幂等续传。
|
||||||
|
|
||||||
|
固定 Origin Worker 已支持 `line`、`scatter` 和 `line_scatter`,生成 OPJU、PNG、SVG、PDF、plot spec、provenance 与原子 `terminal.json`。运行时独立于 zcbot 服务端 Python,管理员执行:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\install-origin-runtime.ps1 -BootstrapPython D:\programs\Python312\python.exe
|
||||||
|
```
|
||||||
|
|
||||||
|
默认安装到 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\python.exe`;也可由管理员用绝对路径环境变量 `ZCBOT_ORIGIN_PYTHON` 指向固定解释器。任务请求不能指定解释器、脚本或路径。
|
||||||
|
|
||||||
注册和运行必须使用同一专用 Windows 账号。MVP 通过该账号的登录后计划任务自动启动,不以 Windows Service 在 Session 0 运行。
|
注册和运行必须使用同一专用 Windows 账号。MVP 通过该账号的登录后计划任务自动启动,不以 Windows Service 在 Session 0 运行。
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -2,12 +2,13 @@ namespace Zcbot.WindowsNode;
|
||||||
|
|
||||||
internal sealed class ConfigurationForm : Form
|
internal sealed class ConfigurationForm : Form
|
||||||
{
|
{
|
||||||
private const int ContentWidth = 800;
|
private const int ContentWidth = 760;
|
||||||
|
|
||||||
private readonly TextBox server = CreateTextBox("http://127.0.0.1:8765");
|
private readonly TextBox server = CreateTextBox("http://127.0.0.1:8765");
|
||||||
private readonly TextBox nodeName = CreateTextBox(Environment.MachineName.ToLowerInvariant());
|
private readonly TextBox nodeName = CreateTextBox(Environment.MachineName.ToLowerInvariant());
|
||||||
private readonly TextBox enrollmentCode = CreateTextBox(usePassword: true);
|
private readonly TextBox enrollmentCode = CreateTextBox(usePassword: true);
|
||||||
private readonly Button register = CreateButton("注册并连接", 128, primary: true);
|
private readonly Button register = CreateButton("注册并连接", 128, primary: true);
|
||||||
|
private readonly Button reconnect = CreateButton("立即重连", 112, primary: true);
|
||||||
private readonly Button resetIdentity = CreateButton("清除本机身份并重新注册", 220);
|
private readonly Button resetIdentity = CreateButton("清除本机身份并重新注册", 220);
|
||||||
private readonly CheckBox startAtLogin = new()
|
private readonly CheckBox startAtLogin = new()
|
||||||
{
|
{
|
||||||
|
|
@ -25,18 +26,18 @@ internal sealed class ConfigurationForm : Form
|
||||||
private readonly Label identity = CreateBodyLabel();
|
private readonly Label identity = CreateBodyLabel();
|
||||||
private readonly Label capabilitySummary = CreateBodyLabel();
|
private readonly Label capabilitySummary = CreateBodyLabel();
|
||||||
private readonly TableLayoutPanel registrationCard;
|
private readonly TableLayoutPanel registrationCard;
|
||||||
private readonly TableLayoutPanel registeredActionsCard;
|
|
||||||
private bool changingStartup;
|
private bool changingStartup;
|
||||||
|
|
||||||
internal event Func<EnrollOptions, Task>? RegisterRequested;
|
internal event Func<EnrollOptions, Task>? RegisterRequested;
|
||||||
|
internal event Action? ReconnectRequested;
|
||||||
internal event Action? ResetIdentityRequested;
|
internal event Action? ResetIdentityRequested;
|
||||||
|
|
||||||
internal ConfigurationForm()
|
internal ConfigurationForm()
|
||||||
{
|
{
|
||||||
Text = "zcbot Windows Node";
|
Text = "zcbot Windows Node";
|
||||||
AutoScaleMode = AutoScaleMode.Dpi;
|
AutoScaleMode = AutoScaleMode.Dpi;
|
||||||
ClientSize = new Size(880, 720);
|
ClientSize = new Size(840, 680);
|
||||||
MinimumSize = new Size(850, 680);
|
MinimumSize = new Size(820, 640);
|
||||||
StartPosition = FormStartPosition.CenterScreen;
|
StartPosition = FormStartPosition.CenterScreen;
|
||||||
Font = new Font("Microsoft YaHei UI", 9);
|
Font = new Font("Microsoft YaHei UI", 9);
|
||||||
FormBorderStyle = FormBorderStyle.Sizable;
|
FormBorderStyle = FormBorderStyle.Sizable;
|
||||||
|
|
@ -62,7 +63,7 @@ internal sealed class ConfigurationForm : Form
|
||||||
AutoSizeMode = AutoSizeMode.GrowAndShrink,
|
AutoSizeMode = AutoSizeMode.GrowAndShrink,
|
||||||
Dock = DockStyle.Top,
|
Dock = DockStyle.Top,
|
||||||
ColumnCount = 1,
|
ColumnCount = 1,
|
||||||
RowCount = 6,
|
RowCount = 5,
|
||||||
BackColor = BackColor,
|
BackColor = BackColor,
|
||||||
};
|
};
|
||||||
page.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 100));
|
page.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 100));
|
||||||
|
|
@ -85,33 +86,19 @@ internal sealed class ConfigurationForm : Form
|
||||||
heading.Controls.Add(CreateHint("连接本机科研软件与 zcbot 的受控执行节点"));
|
heading.Controls.Add(CreateHint("连接本机科研软件与 zcbot 的受控执行节点"));
|
||||||
page.Controls.Add(heading);
|
page.Controls.Add(heading);
|
||||||
|
|
||||||
var overview = new TableLayoutPanel
|
|
||||||
{
|
|
||||||
AutoSize = true,
|
|
||||||
Dock = DockStyle.Top,
|
|
||||||
ColumnCount = 2,
|
|
||||||
Margin = new Padding(0, 0, 0, 14),
|
|
||||||
};
|
|
||||||
overview.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 50));
|
|
||||||
overview.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 50));
|
|
||||||
|
|
||||||
var statusCard = CreateCard();
|
var statusCard = CreateCard();
|
||||||
statusCard.Margin = new Padding(0, 0, 7, 0);
|
statusCard.Controls.Add(CreateSectionTitle("节点"));
|
||||||
statusCard.Controls.Add(CreateSectionTitle("节点状态"));
|
|
||||||
statusCard.Controls.Add(state);
|
statusCard.Controls.Add(state);
|
||||||
statusCard.Controls.Add(detail);
|
statusCard.Controls.Add(detail);
|
||||||
statusCard.Controls.Add(identity);
|
statusCard.Controls.Add(identity);
|
||||||
overview.Controls.Add(statusCard, 0, 0);
|
statusCard.Controls.Add(CreateDivider());
|
||||||
|
statusCard.Controls.Add(CreateCapabilityRow("Origin 绘图", "origin.plot@v1"));
|
||||||
var capabilityCard = CreateCard();
|
statusCard.Controls.Add(capabilitySummary);
|
||||||
capabilityCard.Margin = new Padding(7, 0, 0, 0);
|
var resetActions = CreateActions();
|
||||||
capabilityCard.Controls.Add(CreateSectionTitle("节点能力"));
|
resetActions.Controls.Add(reconnect);
|
||||||
capabilityCard.Controls.Add(CreateCapabilityBadge("Origin 绘图", "origin.plot@v1"));
|
resetActions.Controls.Add(resetIdentity);
|
||||||
capabilityCard.Controls.Add(capabilitySummary);
|
statusCard.Controls.Add(resetActions);
|
||||||
capabilityCard.Controls.Add(CreateHint(
|
page.Controls.Add(statusCard);
|
||||||
"当前 MVP 内置声明该协议,不需要手工配置;它不代表已完成 Origin 安装检测。"));
|
|
||||||
overview.Controls.Add(capabilityCard, 1, 0);
|
|
||||||
page.Controls.Add(overview);
|
|
||||||
|
|
||||||
registrationCard = CreateCard();
|
registrationCard = CreateCard();
|
||||||
registrationCard.Controls.Add(CreateSectionTitle("首次注册"));
|
registrationCard.Controls.Add(CreateSectionTitle("首次注册"));
|
||||||
|
|
@ -125,15 +112,6 @@ internal sealed class ConfigurationForm : Form
|
||||||
registrationCard.Controls.Add(registerActions);
|
registrationCard.Controls.Add(registerActions);
|
||||||
page.Controls.Add(registrationCard);
|
page.Controls.Add(registrationCard);
|
||||||
|
|
||||||
registeredActionsCard = CreateCard();
|
|
||||||
registeredActionsCard.Controls.Add(CreateSectionTitle("节点身份"));
|
|
||||||
registeredActionsCard.Controls.Add(CreateHint(
|
|
||||||
"服务地址、节点名称和能力在首次注册时确定。需要修改时,请先在管理后台删除旧节点,再清除本机身份并重新注册。"));
|
|
||||||
var identityActions = CreateActions();
|
|
||||||
identityActions.Controls.Add(resetIdentity);
|
|
||||||
registeredActionsCard.Controls.Add(identityActions);
|
|
||||||
page.Controls.Add(registeredActionsCard);
|
|
||||||
|
|
||||||
var runtimeCard = CreateCard();
|
var runtimeCard = CreateCard();
|
||||||
runtimeCard.Controls.Add(CreateSectionTitle("运行设置"));
|
runtimeCard.Controls.Add(CreateSectionTitle("运行设置"));
|
||||||
runtimeCard.Controls.Add(startAtLogin);
|
runtimeCard.Controls.Add(startAtLogin);
|
||||||
|
|
@ -147,6 +125,7 @@ internal sealed class ConfigurationForm : Form
|
||||||
page.Controls.Add(securityNote);
|
page.Controls.Add(securityNote);
|
||||||
|
|
||||||
register.Click += async (_, _) => await RegisterAsync();
|
register.Click += async (_, _) => await RegisterAsync();
|
||||||
|
reconnect.Click += (_, _) => ReconnectRequested?.Invoke();
|
||||||
resetIdentity.Click += (_, _) => ResetIdentity();
|
resetIdentity.Click += (_, _) => ResetIdentity();
|
||||||
startAtLogin.CheckedChanged += (_, _) => ToggleStartup();
|
startAtLogin.CheckedChanged += (_, _) => ToggleStartup();
|
||||||
FormClosing += (_, eventArgs) =>
|
FormClosing += (_, eventArgs) =>
|
||||||
|
|
@ -185,11 +164,12 @@ internal sealed class ConfigurationForm : Form
|
||||||
: $"节点:{config.NodeName}\nNode ID:{config.NodeId}\n服务:{config.ServerUrl}";
|
: $"节点:{config.NodeName}\nNode ID:{config.NodeId}\n服务:{config.ServerUrl}";
|
||||||
capabilitySummary.Text = config is null
|
capabilitySummary.Text = config is null
|
||||||
? "注册后启用"
|
? "注册后启用"
|
||||||
: $"已声明 {config.Capabilities.Count} 项能力";
|
: FormatOriginStatus(OriginRuntimeProbe.Detect());
|
||||||
|
|
||||||
var registered = config is not null;
|
var registered = config is not null;
|
||||||
registrationCard.Visible = !registered;
|
registrationCard.Visible = !registered;
|
||||||
registeredActionsCard.Visible = registered;
|
reconnect.Visible = registered;
|
||||||
|
resetIdentity.Visible = registered;
|
||||||
register.Enabled = !registered && status.State != NodeState.Connecting;
|
register.Enabled = !registered && status.State != NodeState.Connecting;
|
||||||
if (registered)
|
if (registered)
|
||||||
{
|
{
|
||||||
|
|
@ -199,6 +179,15 @@ internal sealed class ConfigurationForm : Form
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static string FormatOriginStatus(OriginRuntimeStatus origin)
|
||||||
|
{
|
||||||
|
var version = string.IsNullOrWhiteSpace(origin.SoftwareVersion)
|
||||||
|
? "版本未知"
|
||||||
|
: $"版本 {origin.SoftwareVersion}";
|
||||||
|
var state = origin.Health == "ready" ? "可用" : "不可用";
|
||||||
|
return $"{state} · {version}\n{origin.Detail}";
|
||||||
|
}
|
||||||
|
|
||||||
private void ResetIdentity()
|
private void ResetIdentity()
|
||||||
{
|
{
|
||||||
var answer = MessageBox.Show(
|
var answer = MessageBox.Show(
|
||||||
|
|
@ -287,7 +276,7 @@ internal sealed class ConfigurationForm : Form
|
||||||
AutoSize = true,
|
AutoSize = true,
|
||||||
Dock = DockStyle.Fill,
|
Dock = DockStyle.Fill,
|
||||||
ForeColor = Color.FromArgb(71, 85, 105),
|
ForeColor = Color.FromArgb(71, 85, 105),
|
||||||
MaximumSize = new Size(340, 0),
|
MaximumSize = new Size(ContentWidth - 40, 0),
|
||||||
Margin = new Padding(0, 2, 0, 5),
|
Margin = new Padding(0, 2, 0, 5),
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
@ -300,25 +289,33 @@ internal sealed class ConfigurationForm : Form
|
||||||
Margin = new Padding(0, 2, 0, 4),
|
Margin = new Padding(0, 2, 0, 4),
|
||||||
};
|
};
|
||||||
|
|
||||||
private static Panel CreateCapabilityBadge(string title, string protocol)
|
private static Panel CreateDivider() => new()
|
||||||
{
|
{
|
||||||
var badge = new Panel
|
Height = 1,
|
||||||
|
Dock = DockStyle.Top,
|
||||||
|
BackColor = Color.FromArgb(226, 232, 240),
|
||||||
|
Margin = new Padding(0, 10, 0, 12),
|
||||||
|
};
|
||||||
|
|
||||||
|
private static Panel CreateCapabilityRow(string title, string protocol)
|
||||||
|
{
|
||||||
|
var row = new Panel
|
||||||
{
|
{
|
||||||
AutoSize = false,
|
AutoSize = false,
|
||||||
Height = 54,
|
Height = 36,
|
||||||
Dock = DockStyle.Top,
|
Dock = DockStyle.Top,
|
||||||
BackColor = Color.FromArgb(239, 246, 255),
|
BackColor = Color.FromArgb(239, 246, 255),
|
||||||
Margin = new Padding(0, 2, 0, 8),
|
Margin = new Padding(0, 0, 0, 5),
|
||||||
Padding = new Padding(12, 7, 12, 7),
|
Padding = new Padding(11, 7, 11, 7),
|
||||||
};
|
};
|
||||||
badge.Controls.Add(new Label
|
row.Controls.Add(new Label
|
||||||
{
|
{
|
||||||
Text = $"{title}\n协议:{protocol}",
|
Text = $"{title} · {protocol}",
|
||||||
AutoSize = true,
|
AutoSize = true,
|
||||||
Font = new Font("Microsoft YaHei UI", 9, FontStyle.Bold),
|
Font = new Font("Microsoft YaHei UI", 9, FontStyle.Bold),
|
||||||
ForeColor = Color.FromArgb(29, 78, 216),
|
ForeColor = Color.FromArgb(29, 78, 216),
|
||||||
});
|
});
|
||||||
return badge;
|
return row;
|
||||||
}
|
}
|
||||||
|
|
||||||
private static TextBox CreateTextBox(string text = "", bool usePassword = false) => new()
|
private static TextBox CreateTextBox(string text = "", bool usePassword = false) => new()
|
||||||
|
|
|
||||||
|
|
@ -52,7 +52,7 @@ internal static class NodeUri
|
||||||
|
|
||||||
internal static Uri WebSocketEndpoint(Uri serverUrl)
|
internal static Uri WebSocketEndpoint(Uri serverUrl)
|
||||||
{
|
{
|
||||||
var builder = new UriBuilder(new Uri(serverUrl, "v1/compute/nodes/connect"))
|
var builder = new UriBuilder(new Uri(serverUrl, "v1/software-nodes/connect"))
|
||||||
{
|
{
|
||||||
Scheme = serverUrl.Scheme == Uri.UriSchemeHttps ? "wss" : "ws"
|
Scheme = serverUrl.Scheme == Uri.UriSchemeHttps ? "wss" : "ws"
|
||||||
};
|
};
|
||||||
|
|
|
||||||
|
|
@ -28,7 +28,7 @@ internal static class EnrollmentClient
|
||||||
|
|
||||||
using var client = new HttpClient { BaseAddress = options.ServerUrl, Timeout = TimeSpan.FromSeconds(30) };
|
using var client = new HttpClient { BaseAddress = options.ServerUrl, Timeout = TimeSpan.FromSeconds(30) };
|
||||||
using var response = await client.PostAsJsonAsync(
|
using var response = await client.PostAsJsonAsync(
|
||||||
"v1/compute/nodes/enroll", request, cancellationToken);
|
"v1/software-nodes/enroll", request, cancellationToken);
|
||||||
if (!response.IsSuccessStatusCode)
|
if (!response.IsSuccessStatusCode)
|
||||||
{
|
{
|
||||||
var detail = await response.Content.ReadAsStringAsync(cancellationToken);
|
var detail = await response.Content.ReadAsStringAsync(cancellationToken);
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,346 @@
|
||||||
|
using System.Text;
|
||||||
|
using System.Text.Json;
|
||||||
|
|
||||||
|
namespace Zcbot.WindowsNode;
|
||||||
|
|
||||||
|
internal sealed class JobInboxStore(string jobsDirectory)
|
||||||
|
{
|
||||||
|
private static readonly JsonSerializerOptions JsonOptions = new() { WriteIndented = true };
|
||||||
|
private static readonly HashSet<string> PlotTypes =
|
||||||
|
["line", "scatter", "line_scatter"];
|
||||||
|
private static readonly HashSet<string> OutputFormats = ["opju", "png", "svg", "pdf"];
|
||||||
|
|
||||||
|
internal bool HasPendingJobs => Directory.Exists(jobsDirectory)
|
||||||
|
&& ReadRecoverableJobs().Any(item =>
|
||||||
|
item.Terminal is null
|
||||||
|
|| item.Terminal.Value.GetProperty("status").GetString() == "succeeded"
|
||||||
|
&& !item.UploadComplete);
|
||||||
|
|
||||||
|
internal IReadOnlyList<RecoverableJob> ReadRecoverableJobs()
|
||||||
|
{
|
||||||
|
if (!Directory.Exists(jobsDirectory))
|
||||||
|
{
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
var jobs = new List<RecoverableJob>();
|
||||||
|
foreach (var requestPath in Directory.EnumerateFiles(
|
||||||
|
jobsDirectory, "request.json", SearchOption.AllDirectories))
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
using var request = JsonDocument.Parse(File.ReadAllBytes(requestPath));
|
||||||
|
var root = request.RootElement;
|
||||||
|
if (!TryReadGuid(root, "job_id", out var jobId)
|
||||||
|
|| !TryReadGuid(root, "lease_id", out var leaseId)
|
||||||
|
|| !root.TryGetProperty("request_digest", out var digestValue)
|
||||||
|
|| digestValue.GetString() is not { Length: 64 } requestDigest)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
var jobDirectory = Directory.GetParent(Directory.GetParent(requestPath)!.FullName)!.FullName;
|
||||||
|
jobs.Add(new RecoverableJob(
|
||||||
|
jobId,
|
||||||
|
leaseId,
|
||||||
|
requestDigest,
|
||||||
|
root.TryGetProperty("input_transfer", out var transfer)
|
||||||
|
? transfer.Clone() : null,
|
||||||
|
ReadTerminal(Path.Combine(jobDirectory, "terminal.json")),
|
||||||
|
File.Exists(Path.Combine(jobDirectory, "upload-complete.json"))));
|
||||||
|
}
|
||||||
|
catch (Exception exception) when (
|
||||||
|
exception is JsonException or IOException or UnauthorizedAccessException)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return jobs;
|
||||||
|
}
|
||||||
|
|
||||||
|
internal JobOfferResult Accept(JsonElement payload)
|
||||||
|
{
|
||||||
|
if (!TryReadGuid(payload, "job_id", out var jobId)
|
||||||
|
|| !TryReadGuid(payload, "lease_id", out var leaseId)
|
||||||
|
|| !payload.TryGetProperty("request_digest", out var digestValue)
|
||||||
|
|| digestValue.ValueKind != JsonValueKind.String
|
||||||
|
|| digestValue.GetString() is not { Length: 64 } requestDigest
|
||||||
|
|| !payload.TryGetProperty("capability", out var capabilityValue)
|
||||||
|
|| capabilityValue.GetString() != "origin.plot@v1"
|
||||||
|
|| !payload.TryGetProperty("request", out var request)
|
||||||
|
|| request.ValueKind != JsonValueKind.Object
|
||||||
|
|| !payload.TryGetProperty("input_transfer", out var inputTransfer)
|
||||||
|
|| !IsValidInputTransfer(inputTransfer))
|
||||||
|
{
|
||||||
|
return JobOfferResult.Reject("invalid_offer");
|
||||||
|
}
|
||||||
|
if (!IsValidRequest(request))
|
||||||
|
{
|
||||||
|
return JobOfferResult.Reject("unsupported_request");
|
||||||
|
}
|
||||||
|
|
||||||
|
var directory = Path.Combine(jobsDirectory, jobId.ToString("D"));
|
||||||
|
var requestDirectory = Path.Combine(directory, "request");
|
||||||
|
var requestPath = Path.Combine(requestDirectory, "request.json");
|
||||||
|
Directory.CreateDirectory(requestDirectory);
|
||||||
|
if (File.Exists(requestPath))
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
using var existing = JsonDocument.Parse(File.ReadAllBytes(requestPath));
|
||||||
|
var root = existing.RootElement;
|
||||||
|
var sameDigest = root.TryGetProperty("request_digest", out var existingDigest)
|
||||||
|
&& existingDigest.GetString() == requestDigest;
|
||||||
|
if (!sameDigest)
|
||||||
|
{
|
||||||
|
return JobOfferResult.Reject("job_digest_conflict");
|
||||||
|
}
|
||||||
|
if (!TryReadGuid(root, "lease_id", out var existingLease)
|
||||||
|
|| existingLease != leaseId)
|
||||||
|
{
|
||||||
|
var updated = JsonSerializer.SerializeToUtf8Bytes(new
|
||||||
|
{
|
||||||
|
job_id = jobId,
|
||||||
|
lease_id = leaseId,
|
||||||
|
request_digest = requestDigest,
|
||||||
|
capability = "origin.plot@v1",
|
||||||
|
accepted_at = DateTimeOffset.UtcNow,
|
||||||
|
request,
|
||||||
|
input_transfer = inputTransfer,
|
||||||
|
}, JsonOptions);
|
||||||
|
AtomicWrite(requestPath, updated, overwrite: true);
|
||||||
|
}
|
||||||
|
return JobOfferResult.Accept(jobId, leaseId, requestDigest);
|
||||||
|
}
|
||||||
|
catch (JsonException)
|
||||||
|
{
|
||||||
|
return JobOfferResult.Reject("local_job_record_invalid");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var record = JsonSerializer.SerializeToUtf8Bytes(new
|
||||||
|
{
|
||||||
|
job_id = jobId,
|
||||||
|
lease_id = leaseId,
|
||||||
|
request_digest = requestDigest,
|
||||||
|
capability = "origin.plot@v1",
|
||||||
|
accepted_at = DateTimeOffset.UtcNow,
|
||||||
|
request,
|
||||||
|
input_transfer = inputTransfer,
|
||||||
|
}, JsonOptions);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
AtomicWrite(requestPath, record, overwrite: false);
|
||||||
|
return JobOfferResult.Accept(jobId, leaseId, requestDigest);
|
||||||
|
}
|
||||||
|
catch (IOException)
|
||||||
|
{
|
||||||
|
return JobOfferResult.Reject("local_job_persist_failed");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static JsonElement? ReadTerminal(string path)
|
||||||
|
{
|
||||||
|
if (!File.Exists(path))
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
using var document = JsonDocument.Parse(File.ReadAllBytes(path));
|
||||||
|
return document.RootElement.Clone();
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void AtomicWrite(string path, byte[] content, bool overwrite)
|
||||||
|
{
|
||||||
|
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
|
||||||
|
var temporaryPath = path + ".tmp-" + Guid.NewGuid().ToString("N");
|
||||||
|
try
|
||||||
|
{
|
||||||
|
using (var stream = new FileStream(
|
||||||
|
temporaryPath, FileMode.CreateNew, FileAccess.Write, FileShare.None,
|
||||||
|
bufferSize: 4096, FileOptions.WriteThrough))
|
||||||
|
{
|
||||||
|
stream.Write(content);
|
||||||
|
stream.Flush(flushToDisk: true);
|
||||||
|
}
|
||||||
|
File.Move(temporaryPath, path, overwrite);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
if (File.Exists(temporaryPath)) File.Delete(temporaryPath);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool IsValidRequest(JsonElement request) =>
|
||||||
|
HasOnlyProperties(request, "schema_version", "input", "plot", "output")
|
||||||
|
&& request.TryGetProperty("schema_version", out var schemaVersion)
|
||||||
|
&& schemaVersion.TryGetInt32(out var version)
|
||||||
|
&& version == 1
|
||||||
|
&& request.TryGetProperty("input", out var input)
|
||||||
|
&& input.ValueKind == JsonValueKind.Object
|
||||||
|
&& HasOnlyProperties(input, "input_id", "sheet")
|
||||||
|
&& input.TryGetProperty("input_id", out var inputId)
|
||||||
|
&& inputId.ValueKind == JsonValueKind.String
|
||||||
|
&& !string.IsNullOrWhiteSpace(inputId.GetString())
|
||||||
|
&& request.TryGetProperty("plot", out var plot)
|
||||||
|
&& plot.ValueKind == JsonValueKind.Object
|
||||||
|
&& HasOnlyProperties(
|
||||||
|
plot, "type", "x", "y", "template", "title", "x_axis", "y_axis", "legend", "error_bars")
|
||||||
|
&& plot.TryGetProperty("type", out var plotType)
|
||||||
|
&& plotType.ValueKind == JsonValueKind.String
|
||||||
|
&& PlotTypes.Contains(plotType.GetString() ?? "")
|
||||||
|
&& (!plot.TryGetProperty("title", out var title)
|
||||||
|
|| title.ValueKind == JsonValueKind.String && title.GetString()!.Length <= 500)
|
||||||
|
&& plot.TryGetProperty("x", out var x)
|
||||||
|
&& IsColumnName(x)
|
||||||
|
&& plot.TryGetProperty("y", out var y)
|
||||||
|
&& IsValidYColumns(y)
|
||||||
|
&& (!plot.TryGetProperty("template", out var template)
|
||||||
|
|| template.GetString() == "publication_double_column")
|
||||||
|
&& IsValidAxis(plot, "x_axis")
|
||||||
|
&& IsValidAxis(plot, "y_axis")
|
||||||
|
&& IsValidLegend(plot)
|
||||||
|
&& !plot.TryGetProperty("error_bars", out _)
|
||||||
|
&& request.TryGetProperty("output", out var output)
|
||||||
|
&& output.ValueKind == JsonValueKind.Object
|
||||||
|
&& HasOnlyProperties(output, "formats", "dpi", "capture_screenshots", "record_video")
|
||||||
|
&& output.TryGetProperty("formats", out var formats)
|
||||||
|
&& formats.ValueKind == JsonValueKind.Array
|
||||||
|
&& formats.GetArrayLength() > 0
|
||||||
|
&& IsValidFormats(formats)
|
||||||
|
&& (!output.TryGetProperty("dpi", out var dpi)
|
||||||
|
|| dpi.TryGetInt32(out var dpiValue) && dpiValue is >= 72 and <= 1200)
|
||||||
|
&& IsOptionalBoolean(output, "capture_screenshots")
|
||||||
|
&& IsOptionalBoolean(output, "record_video")
|
||||||
|
&& (!output.TryGetProperty("record_video", out var recordVideo)
|
||||||
|
|| recordVideo.ValueKind == JsonValueKind.False);
|
||||||
|
|
||||||
|
private static bool IsColumnName(JsonElement value) =>
|
||||||
|
value.ValueKind == JsonValueKind.String
|
||||||
|
&& value.GetString() is { Length: >= 1 and <= 128 };
|
||||||
|
|
||||||
|
private static bool IsValidFormats(JsonElement formats)
|
||||||
|
{
|
||||||
|
var values = formats.EnumerateArray().ToArray();
|
||||||
|
return values.All(item =>
|
||||||
|
item.ValueKind == JsonValueKind.String
|
||||||
|
&& OutputFormats.Contains(item.GetString() ?? ""))
|
||||||
|
&& values.Select(item => item.GetString()).Distinct(StringComparer.Ordinal).Count()
|
||||||
|
== values.Length;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool IsValidYColumns(JsonElement value)
|
||||||
|
{
|
||||||
|
if (IsColumnName(value)) return true;
|
||||||
|
if (value.ValueKind != JsonValueKind.Array
|
||||||
|
|| value.GetArrayLength() is < 1 or > 16)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
var names = value.EnumerateArray().Select(item => item.GetString()).ToArray();
|
||||||
|
return value.EnumerateArray().All(IsColumnName)
|
||||||
|
&& names.Distinct(StringComparer.Ordinal).Count() == names.Length;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool IsValidAxis(JsonElement plot, string name)
|
||||||
|
{
|
||||||
|
if (!plot.TryGetProperty(name, out var axis)) return true;
|
||||||
|
return axis.ValueKind == JsonValueKind.Object
|
||||||
|
&& HasOnlyProperties(axis, "title", "unit", "scale")
|
||||||
|
&& (!axis.TryGetProperty("title", out var title) || title.ValueKind == JsonValueKind.String)
|
||||||
|
&& (!axis.TryGetProperty("unit", out var unit) || unit.ValueKind == JsonValueKind.String)
|
||||||
|
&& (!axis.TryGetProperty("scale", out var scale) || scale.GetString() == "linear");
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool IsValidLegend(JsonElement plot)
|
||||||
|
{
|
||||||
|
if (!plot.TryGetProperty("legend", out var legend)) return true;
|
||||||
|
return legend.ValueKind == JsonValueKind.Object
|
||||||
|
&& HasOnlyProperties(legend, "enabled", "position")
|
||||||
|
&& (!legend.TryGetProperty("enabled", out var enabled)
|
||||||
|
|| enabled.ValueKind == JsonValueKind.True)
|
||||||
|
&& (!legend.TryGetProperty("position", out var position)
|
||||||
|
|| position.GetString() == "top_right");
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool IsOptionalBoolean(JsonElement value, string name) =>
|
||||||
|
!value.TryGetProperty(name, out var property)
|
||||||
|
|| property.ValueKind is JsonValueKind.True or JsonValueKind.False;
|
||||||
|
|
||||||
|
private static bool IsValidInputTransfer(JsonElement transfer) =>
|
||||||
|
transfer.ValueKind == JsonValueKind.Object
|
||||||
|
&& HasOnlyProperties(transfer, "artifact_id", "filename", "size_bytes", "sha256", "download_path")
|
||||||
|
&& transfer.TryGetProperty("artifact_id", out var artifactId)
|
||||||
|
&& Guid.TryParse(artifactId.GetString(), out _)
|
||||||
|
&& transfer.TryGetProperty("filename", out var filename)
|
||||||
|
&& filename.ValueKind == JsonValueKind.String
|
||||||
|
&& Path.GetFileName(filename.GetString()) == filename.GetString()
|
||||||
|
&& transfer.TryGetProperty("size_bytes", out var size)
|
||||||
|
&& size.TryGetInt64(out var sizeBytes)
|
||||||
|
&& sizeBytes is >= 0 and <= 104_857_600
|
||||||
|
&& transfer.TryGetProperty("sha256", out var sha)
|
||||||
|
&& sha.GetString() is { Length: 64 }
|
||||||
|
&& transfer.TryGetProperty("download_path", out var downloadPath)
|
||||||
|
&& downloadPath.GetString()?.StartsWith("/v1/software-jobs/", StringComparison.Ordinal) == true
|
||||||
|
&& downloadPath.GetString()?.EndsWith("/input", StringComparison.Ordinal) == true;
|
||||||
|
|
||||||
|
internal string InputPath(RecoverableJob job)
|
||||||
|
{
|
||||||
|
if (job.InputTransfer is not JsonElement transfer
|
||||||
|
|| !IsValidInputTransfer(transfer))
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Stored input transfer is invalid.");
|
||||||
|
}
|
||||||
|
var filename = transfer.GetProperty("filename").GetString()!;
|
||||||
|
return Path.Combine(jobsDirectory, job.JobId.ToString("D"), "input", filename);
|
||||||
|
}
|
||||||
|
|
||||||
|
internal void WriteTerminal(
|
||||||
|
RecoverableJob job, string status, string code, string detail)
|
||||||
|
{
|
||||||
|
var path = Path.Combine(jobsDirectory, job.JobId.ToString("D"), "terminal.json");
|
||||||
|
if (File.Exists(path))
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var content = JsonSerializer.SerializeToUtf8Bytes(new
|
||||||
|
{
|
||||||
|
job_id = job.JobId,
|
||||||
|
lease_id = job.LeaseId,
|
||||||
|
request_digest = job.RequestDigest,
|
||||||
|
status,
|
||||||
|
error = new { code, detail },
|
||||||
|
artifact_manifest = Array.Empty<object>(),
|
||||||
|
terminal_at = DateTimeOffset.UtcNow,
|
||||||
|
}, JsonOptions);
|
||||||
|
AtomicWrite(path, content, overwrite: false);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool HasOnlyProperties(JsonElement value, params string[] allowed)
|
||||||
|
{
|
||||||
|
var names = new HashSet<string>(allowed, StringComparer.Ordinal);
|
||||||
|
return value.EnumerateObject().All(item => names.Contains(item.Name));
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool TryReadGuid(JsonElement payload, string name, out Guid value)
|
||||||
|
{
|
||||||
|
value = Guid.Empty;
|
||||||
|
return payload.TryGetProperty(name, out var property)
|
||||||
|
&& property.ValueKind == JsonValueKind.String
|
||||||
|
&& Guid.TryParse(property.GetString(), out value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
internal sealed record RecoverableJob(
|
||||||
|
Guid JobId,
|
||||||
|
Guid LeaseId,
|
||||||
|
string RequestDigest,
|
||||||
|
JsonElement? InputTransfer,
|
||||||
|
JsonElement? Terminal,
|
||||||
|
bool UploadComplete);
|
||||||
|
|
||||||
|
internal sealed record JobOfferResult(
|
||||||
|
bool Accepted, Guid JobId, Guid LeaseId, string RequestDigest, string Reason)
|
||||||
|
{
|
||||||
|
internal static JobOfferResult Accept(Guid jobId, Guid leaseId, string requestDigest) =>
|
||||||
|
new(true, jobId, leaseId, requestDigest, "");
|
||||||
|
|
||||||
|
internal static JobOfferResult Reject(string reason) =>
|
||||||
|
new(false, Guid.Empty, Guid.Empty, "", reason);
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,99 @@
|
||||||
|
using System.Net.Http.Headers;
|
||||||
|
using System.Security.Cryptography;
|
||||||
|
using System.Text.Json;
|
||||||
|
|
||||||
|
namespace Zcbot.WindowsNode;
|
||||||
|
|
||||||
|
internal sealed class JobInputDownloader(NodeConfig config, JobInboxStore inbox)
|
||||||
|
{
|
||||||
|
internal async Task DownloadAsync(RecoverableJob job, CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
if (job.InputTransfer is not JsonElement transfer)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Job input transfer is missing.");
|
||||||
|
}
|
||||||
|
var downloadPath = transfer.GetProperty("download_path").GetString()!;
|
||||||
|
if (!downloadPath.StartsWith("/v1/software-jobs/", StringComparison.Ordinal)
|
||||||
|
|| !downloadPath.EndsWith("/input", StringComparison.Ordinal)
|
||||||
|
|| !Uri.TryCreate(downloadPath, UriKind.Relative, out var relativeUri))
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Job input download path is invalid.");
|
||||||
|
}
|
||||||
|
var expectedSize = transfer.GetProperty("size_bytes").GetInt64();
|
||||||
|
var expectedSha256 = transfer.GetProperty("sha256").GetString()!;
|
||||||
|
var destination = inbox.InputPath(job);
|
||||||
|
if (File.Exists(destination))
|
||||||
|
{
|
||||||
|
await VerifyExistingAsync(destination, expectedSize, expectedSha256, cancellationToken);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
Directory.CreateDirectory(Path.GetDirectoryName(destination)!);
|
||||||
|
var temporaryPath = destination + ".tmp-" + Guid.NewGuid().ToString("N");
|
||||||
|
using var client = new HttpClient { BaseAddress = config.ServerUrl };
|
||||||
|
client.DefaultRequestHeaders.Authorization =
|
||||||
|
new AuthenticationHeaderValue("Bearer", config.NodeToken);
|
||||||
|
client.DefaultRequestHeaders.Add("X-Node-Id", config.NodeId.ToString());
|
||||||
|
try
|
||||||
|
{
|
||||||
|
using var response = await client.GetAsync(
|
||||||
|
relativeUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken);
|
||||||
|
response.EnsureSuccessStatusCode();
|
||||||
|
if (response.Content.Headers.ContentLength is long contentLength
|
||||||
|
&& contentLength != expectedSize)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Job input size header does not match the manifest.");
|
||||||
|
}
|
||||||
|
await using var source = await response.Content.ReadAsStreamAsync(cancellationToken);
|
||||||
|
await using var target = new FileStream(
|
||||||
|
temporaryPath, FileMode.CreateNew, FileAccess.Write, FileShare.None,
|
||||||
|
bufferSize: 64 * 1024, FileOptions.Asynchronous | FileOptions.WriteThrough);
|
||||||
|
using var hash = IncrementalHash.CreateHash(HashAlgorithmName.SHA256);
|
||||||
|
var buffer = new byte[64 * 1024];
|
||||||
|
long total = 0;
|
||||||
|
while (true)
|
||||||
|
{
|
||||||
|
var count = await source.ReadAsync(buffer, cancellationToken);
|
||||||
|
if (count == 0) break;
|
||||||
|
total += count;
|
||||||
|
if (total > expectedSize)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Job input exceeded its declared size.");
|
||||||
|
}
|
||||||
|
hash.AppendData(buffer, 0, count);
|
||||||
|
await target.WriteAsync(buffer.AsMemory(0, count), cancellationToken);
|
||||||
|
}
|
||||||
|
await target.FlushAsync(cancellationToken);
|
||||||
|
target.Flush(flushToDisk: true);
|
||||||
|
var actualSha256 = Convert.ToHexString(hash.GetHashAndReset()).ToLowerInvariant();
|
||||||
|
if (total != expectedSize || actualSha256 != expectedSha256)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Job input digest does not match the manifest.");
|
||||||
|
}
|
||||||
|
target.Close();
|
||||||
|
File.Move(temporaryPath, destination, overwrite: false);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
if (File.Exists(temporaryPath)) File.Delete(temporaryPath);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static async Task VerifyExistingAsync(
|
||||||
|
string path, long expectedSize, string expectedSha256, CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
var info = new FileInfo(path);
|
||||||
|
if (info.Length != expectedSize)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Existing job input size does not match the manifest.");
|
||||||
|
}
|
||||||
|
await using var stream = new FileStream(
|
||||||
|
path, FileMode.Open, FileAccess.Read, FileShare.Read, 64 * 1024, FileOptions.Asynchronous);
|
||||||
|
var digest = Convert.ToHexString(
|
||||||
|
await SHA256.HashDataAsync(stream, cancellationToken)).ToLowerInvariant();
|
||||||
|
if (digest != expectedSha256)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Existing job input digest does not match the manifest.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,100 @@
|
||||||
|
using System.Net.Http.Headers;
|
||||||
|
using System.Security.Cryptography;
|
||||||
|
using System.Text;
|
||||||
|
using System.Text.Json;
|
||||||
|
|
||||||
|
namespace Zcbot.WindowsNode;
|
||||||
|
|
||||||
|
internal sealed class JobOutputUploader(NodeConfig config)
|
||||||
|
{
|
||||||
|
private static readonly JsonSerializerOptions JsonOptions = new() { WriteIndented = true };
|
||||||
|
|
||||||
|
internal async Task UploadAsync(RecoverableJob job)
|
||||||
|
{
|
||||||
|
var jobDirectory = Path.Combine(
|
||||||
|
NodePaths.ForCurrentMachine().JobsDirectory, job.JobId.ToString("D"));
|
||||||
|
var completionPath = Path.Combine(jobDirectory, "upload-complete.json");
|
||||||
|
if (File.Exists(completionPath)) return;
|
||||||
|
var terminalPath = Path.Combine(jobDirectory, "terminal.json");
|
||||||
|
using var terminal = JsonDocument.Parse(await File.ReadAllBytesAsync(terminalPath));
|
||||||
|
if (terminal.RootElement.GetProperty("status").GetString() != "succeeded") return;
|
||||||
|
var manifest = terminal.RootElement.GetProperty("artifact_manifest").Clone();
|
||||||
|
|
||||||
|
using var client = new HttpClient { BaseAddress = config.ServerUrl };
|
||||||
|
client.DefaultRequestHeaders.Authorization =
|
||||||
|
new AuthenticationHeaderValue("Bearer", config.NodeToken);
|
||||||
|
client.DefaultRequestHeaders.Add("X-Node-Id", config.NodeId.ToString());
|
||||||
|
client.DefaultRequestHeaders.Add("X-Lease-Id", job.LeaseId.ToString());
|
||||||
|
client.DefaultRequestHeaders.Add("X-Request-Digest", job.RequestDigest);
|
||||||
|
foreach (var artifact in manifest.EnumerateArray())
|
||||||
|
{
|
||||||
|
var localId = artifact.GetProperty("artifact_id").GetString()!;
|
||||||
|
var filename = artifact.GetProperty("filename").GetString()!;
|
||||||
|
var expectedSize = artifact.GetProperty("size_bytes").GetInt64();
|
||||||
|
var expectedDigest = artifact.GetProperty("sha256").GetString()!;
|
||||||
|
var path = Path.Combine(jobDirectory, "output", filename);
|
||||||
|
var info = new FileInfo(path);
|
||||||
|
if (!info.Exists || info.Length != expectedSize)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException($"Output artifact is missing or changed: {localId}.");
|
||||||
|
}
|
||||||
|
await using (var verify = new FileStream(
|
||||||
|
path, FileMode.Open, FileAccess.Read, FileShare.Read, 64 * 1024,
|
||||||
|
FileOptions.Asynchronous | FileOptions.SequentialScan))
|
||||||
|
{
|
||||||
|
var digest = Convert.ToHexString(
|
||||||
|
await SHA256.HashDataAsync(verify)).ToLowerInvariant();
|
||||||
|
if (digest != expectedDigest)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException($"Output artifact digest changed: {localId}.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await using var stream = new FileStream(
|
||||||
|
path, FileMode.Open, FileAccess.Read, FileShare.Read, 64 * 1024,
|
||||||
|
FileOptions.Asynchronous | FileOptions.SequentialScan);
|
||||||
|
using var content = new StreamContent(stream);
|
||||||
|
content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
|
||||||
|
content.Headers.ContentLength = expectedSize;
|
||||||
|
content.Headers.Add("X-Content-SHA256", expectedDigest);
|
||||||
|
content.Headers.Add("X-Content-Length", expectedSize.ToString());
|
||||||
|
using var response = await client.PutAsync(
|
||||||
|
$"/v1/software-jobs/{job.JobId:D}/outputs/{Uri.EscapeDataString(localId)}",
|
||||||
|
content);
|
||||||
|
response.EnsureSuccessStatusCode();
|
||||||
|
}
|
||||||
|
|
||||||
|
using var completeContent = new StringContent(
|
||||||
|
JsonSerializer.Serialize(new { artifact_manifest = manifest }),
|
||||||
|
Encoding.UTF8,
|
||||||
|
"application/json");
|
||||||
|
using var completeResponse = await client.PostAsync(
|
||||||
|
$"/v1/software-jobs/{job.JobId:D}/outputs/complete", completeContent);
|
||||||
|
completeResponse.EnsureSuccessStatusCode();
|
||||||
|
var responseBody = await completeResponse.Content.ReadAsByteArrayAsync();
|
||||||
|
AtomicWrite(completionPath, responseBody);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void AtomicWrite(string path, byte[] responseBody)
|
||||||
|
{
|
||||||
|
using var response = JsonDocument.Parse(responseBody);
|
||||||
|
var content = JsonSerializer.SerializeToUtf8Bytes(new
|
||||||
|
{
|
||||||
|
completed_at = DateTimeOffset.UtcNow,
|
||||||
|
response = response.RootElement,
|
||||||
|
}, JsonOptions);
|
||||||
|
var temporary = path + ".tmp-" + Guid.NewGuid().ToString("N");
|
||||||
|
try
|
||||||
|
{
|
||||||
|
using var stream = new FileStream(
|
||||||
|
temporary, FileMode.CreateNew, FileAccess.Write, FileShare.None,
|
||||||
|
4096, FileOptions.WriteThrough);
|
||||||
|
stream.Write(content);
|
||||||
|
stream.Flush(flushToDisk: true);
|
||||||
|
File.Move(temporary, path, overwrite: false);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
if (File.Exists(temporary)) File.Delete(temporary);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -4,11 +4,20 @@ using System.Reflection;
|
||||||
using System.Runtime.InteropServices;
|
using System.Runtime.InteropServices;
|
||||||
using System.Text;
|
using System.Text;
|
||||||
using System.Text.Json;
|
using System.Text.Json;
|
||||||
|
using System.Collections.Concurrent;
|
||||||
|
|
||||||
namespace Zcbot.WindowsNode;
|
namespace Zcbot.WindowsNode;
|
||||||
|
|
||||||
internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>? statusChanged = null)
|
internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>? statusChanged = null)
|
||||||
{
|
{
|
||||||
|
private readonly SemaphoreSlim sendLock = new(1, 1);
|
||||||
|
private readonly JobInboxStore jobInbox = new(NodePaths.ForCurrentMachine().JobsDirectory);
|
||||||
|
private readonly JobInputDownloader inputDownloader = new(
|
||||||
|
config, new JobInboxStore(NodePaths.ForCurrentMachine().JobsDirectory));
|
||||||
|
private readonly OriginWorkerRunner workerRunner = new(
|
||||||
|
new JobInboxStore(NodePaths.ForCurrentMachine().JobsDirectory));
|
||||||
|
private readonly JobOutputUploader outputUploader = new(config);
|
||||||
|
private readonly ConcurrentDictionary<Guid, Task> jobPipelines = new();
|
||||||
private static readonly TimeSpan[] Backoff =
|
private static readonly TimeSpan[] Backoff =
|
||||||
[
|
[
|
||||||
TimeSpan.FromSeconds(1),
|
TimeSpan.FromSeconds(1),
|
||||||
|
|
@ -49,6 +58,11 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
||||||
Report(NodeState.Offline, "服务端消息无效,等待重连");
|
Report(NodeState.Offline, "服务端消息无效,等待重连");
|
||||||
Console.Error.WriteLine($"[WARN] Invalid server message: {exception.Message}");
|
Console.Error.WriteLine($"[WARN] Invalid server message: {exception.Message}");
|
||||||
}
|
}
|
||||||
|
catch (NodeEndpointException exception)
|
||||||
|
{
|
||||||
|
Report(NodeState.Offline, "WebSocket 握手被拒绝,请检查服务端或反向代理");
|
||||||
|
Console.Error.WriteLine($"[WARN] WebSocket handshake rejected: {exception.Message}");
|
||||||
|
}
|
||||||
|
|
||||||
var baseDelay = Backoff[Math.Min(attempt, Backoff.Length - 1)];
|
var baseDelay = Backoff[Math.Min(attempt, Backoff.Length - 1)];
|
||||||
attempt++;
|
attempt++;
|
||||||
|
|
@ -73,17 +87,20 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
||||||
{
|
{
|
||||||
await socket.ConnectAsync(endpoint, cancellationToken);
|
await socket.ConnectAsync(endpoint, cancellationToken);
|
||||||
}
|
}
|
||||||
catch (WebSocketException) when (
|
catch (WebSocketException exception) when (
|
||||||
socket.HttpStatusCode is HttpStatusCode.Unauthorized or HttpStatusCode.Forbidden)
|
socket.HttpStatusCode is HttpStatusCode.Unauthorized or HttpStatusCode.Forbidden)
|
||||||
{
|
{
|
||||||
Report(NodeState.AuthenticationRequired, "身份失效,需要重新注册");
|
throw new NodeEndpointException(
|
||||||
throw new NodeConfigurationException(
|
$"WebSocket handshake returned HTTP {(int?)socket.HttpStatusCode}. "
|
||||||
"Node credentials were rejected. Ask an administrator to re-register this node.");
|
+ "Verify that the server includes the current Windows Node routes and that "
|
||||||
|
+ $"the reverse proxy forwards WebSocket Upgrade for {endpoint.AbsolutePath}. "
|
||||||
|
+ exception.Message);
|
||||||
}
|
}
|
||||||
Console.WriteLine("[OK] Node connected.");
|
Console.WriteLine("[OK] Node connected.");
|
||||||
Report(NodeState.Online, "已连接");
|
Report(NodeState.Online, "已连接");
|
||||||
|
|
||||||
await SendAsync(socket, "hello", RuntimePayload(), cancellationToken);
|
await SendAsync(socket, "hello", RuntimePayload(), cancellationToken);
|
||||||
|
await ReportRecoverableJobsAsync(socket, cancellationToken);
|
||||||
using var heartbeatStop = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
|
using var heartbeatStop = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
|
||||||
var heartbeat = HeartbeatLoopAsync(socket, heartbeatStop.Token);
|
var heartbeat = HeartbeatLoopAsync(socket, heartbeatStop.Token);
|
||||||
try
|
try
|
||||||
|
|
@ -109,10 +126,52 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
||||||
while (await timer.WaitForNextTickAsync(cancellationToken))
|
while (await timer.WaitForNextTickAsync(cancellationToken))
|
||||||
{
|
{
|
||||||
await SendAsync(socket, "heartbeat", RuntimePayload(), cancellationToken);
|
await SendAsync(socket, "heartbeat", RuntimePayload(), cancellationToken);
|
||||||
|
foreach (var job in jobInbox.ReadRecoverableJobs())
|
||||||
|
{
|
||||||
|
if (job.Terminal is JsonElement terminal
|
||||||
|
&& terminal.GetProperty("status").GetString() == "succeeded"
|
||||||
|
&& !job.UploadComplete)
|
||||||
|
{
|
||||||
|
StartJobPipeline(socket, job);
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private static async Task ReceiveLoopAsync(
|
private async Task ReportRecoverableJobsAsync(
|
||||||
|
ClientWebSocket socket, CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
foreach (var job in jobInbox.ReadRecoverableJobs())
|
||||||
|
{
|
||||||
|
if (job.Terminal is JsonElement terminal)
|
||||||
|
{
|
||||||
|
if (terminal.GetProperty("status").GetString() == "succeeded")
|
||||||
|
{
|
||||||
|
if (!job.UploadComplete)
|
||||||
|
{
|
||||||
|
StartJobPipeline(socket, job);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
await SendAsync(socket, "job_terminal", terminal, cancellationToken);
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
await SendAsync(socket, "job_state", new
|
||||||
|
{
|
||||||
|
job_id = job.JobId,
|
||||||
|
lease_id = job.LeaseId,
|
||||||
|
request_digest = job.RequestDigest,
|
||||||
|
stage = "waiting_input",
|
||||||
|
progress = 0,
|
||||||
|
metrics = new { },
|
||||||
|
}, cancellationToken);
|
||||||
|
StartJobPipeline(socket, job);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task ReceiveLoopAsync(
|
||||||
ClientWebSocket socket, CancellationToken cancellationToken)
|
ClientWebSocket socket, CancellationToken cancellationToken)
|
||||||
{
|
{
|
||||||
var buffer = new byte[16 * 1024];
|
var buffer = new byte[16 * 1024];
|
||||||
|
|
@ -122,10 +181,17 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
||||||
var result = await socket.ReceiveAsync(buffer, cancellationToken);
|
var result = await socket.ReceiveAsync(buffer, cancellationToken);
|
||||||
if (result.MessageType == WebSocketMessageType.Close)
|
if (result.MessageType == WebSocketMessageType.Close)
|
||||||
{
|
{
|
||||||
|
if ((int?)result.CloseStatus == 4003)
|
||||||
|
{
|
||||||
|
throw new NodeConfigurationException(
|
||||||
|
"节点身份已被服务端拒绝。请在管理后台确认该 Node ID 未被禁用或删除;"
|
||||||
|
+ "若记录不存在或身份已撤销,请清除本机身份并使用新注册码重新注册。"
|
||||||
|
+ $" 服务端信息:{result.CloseStatusDescription}");
|
||||||
|
}
|
||||||
if (result.CloseStatus == WebSocketCloseStatus.PolicyViolation)
|
if (result.CloseStatus == WebSocketCloseStatus.PolicyViolation)
|
||||||
{
|
{
|
||||||
throw new NodeConfigurationException(
|
throw new NodeConfigurationException(
|
||||||
$"Node credentials or install identity were rejected: {result.CloseStatusDescription}");
|
$"节点上报被服务端拒绝:{result.CloseStatusDescription}");
|
||||||
}
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
@ -146,11 +212,222 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
||||||
if (document.RootElement.TryGetProperty("type", out var type))
|
if (document.RootElement.TryGetProperty("type", out var type))
|
||||||
{
|
{
|
||||||
Console.WriteLine($"[INFO] Server message: {type.GetString()}.");
|
Console.WriteLine($"[INFO] Server message: {type.GetString()}.");
|
||||||
|
if (type.GetString() == "job_offer"
|
||||||
|
&& document.RootElement.TryGetProperty("payload", out var payload))
|
||||||
|
{
|
||||||
|
var offerResult = jobInbox.Accept(payload);
|
||||||
|
await SendAsync(
|
||||||
|
socket,
|
||||||
|
offerResult.Accepted ? "job_accept" : "job_reject",
|
||||||
|
offerResult.Accepted
|
||||||
|
? new
|
||||||
|
{
|
||||||
|
job_id = offerResult.JobId,
|
||||||
|
lease_id = offerResult.LeaseId,
|
||||||
|
request_digest = offerResult.RequestDigest,
|
||||||
|
}
|
||||||
|
: new
|
||||||
|
{
|
||||||
|
job_id = payload.TryGetProperty("job_id", out var jobId)
|
||||||
|
? jobId.GetString() : "",
|
||||||
|
lease_id = payload.TryGetProperty("lease_id", out var leaseId)
|
||||||
|
? leaseId.GetString() : "",
|
||||||
|
reason = offerResult.Reason,
|
||||||
|
},
|
||||||
|
cancellationToken);
|
||||||
|
if (offerResult.Accepted)
|
||||||
|
{
|
||||||
|
await SendAsync(socket, "job_state", new
|
||||||
|
{
|
||||||
|
job_id = offerResult.JobId,
|
||||||
|
lease_id = offerResult.LeaseId,
|
||||||
|
request_digest = offerResult.RequestDigest,
|
||||||
|
stage = "waiting_input",
|
||||||
|
progress = 0,
|
||||||
|
metrics = new { },
|
||||||
|
}, cancellationToken);
|
||||||
|
var acceptedJob = jobInbox.ReadRecoverableJobs()
|
||||||
|
.Single(item => item.JobId == offerResult.JobId);
|
||||||
|
StartJobPipeline(socket, acceptedJob);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else if (type.GetString() == "job_cancel"
|
||||||
|
&& document.RootElement.TryGetProperty("payload", out var cancelPayload)
|
||||||
|
&& TryCancelJob(cancelPayload, out var cancelledJob))
|
||||||
|
{
|
||||||
|
workerRunner.Cancel(cancelledJob.JobId);
|
||||||
|
jobInbox.WriteTerminal(
|
||||||
|
cancelledJob, "cancelled", "USER_CANCELLED", "Cancelled by user.");
|
||||||
|
await SendAsync(socket, "job_terminal", new
|
||||||
|
{
|
||||||
|
job_id = cancelledJob.JobId,
|
||||||
|
lease_id = cancelledJob.LeaseId,
|
||||||
|
request_digest = cancelledJob.RequestDigest,
|
||||||
|
status = "cancelled",
|
||||||
|
error = new { code = "USER_CANCELLED", detail = "Cancelled by user." },
|
||||||
|
artifact_manifest = Array.Empty<object>(),
|
||||||
|
}, cancellationToken);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
message.SetLength(0);
|
message.SetLength(0);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private bool TryCancelJob(JsonElement payload, out RecoverableJob job)
|
||||||
|
{
|
||||||
|
job = null!;
|
||||||
|
if (!payload.TryGetProperty("job_id", out var jobIdValue)
|
||||||
|
|| !Guid.TryParse(jobIdValue.GetString(), out var jobId)
|
||||||
|
|| !payload.TryGetProperty("lease_id", out var leaseIdValue)
|
||||||
|
|| !Guid.TryParse(leaseIdValue.GetString(), out var leaseId)
|
||||||
|
|| !payload.TryGetProperty("request_digest", out var digestValue))
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
job = jobInbox.ReadRecoverableJobs().SingleOrDefault(item =>
|
||||||
|
item.JobId == jobId
|
||||||
|
&& item.LeaseId == leaseId
|
||||||
|
&& item.RequestDigest == digestValue.GetString()
|
||||||
|
&& item.Terminal is null)!;
|
||||||
|
return job is not null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void StartJobPipeline(ClientWebSocket socket, RecoverableJob job)
|
||||||
|
{
|
||||||
|
var completion = new TaskCompletionSource(
|
||||||
|
TaskCreationOptions.RunContinuationsAsynchronously);
|
||||||
|
if (jobPipelines.TryAdd(job.JobId, completion.Task))
|
||||||
|
{
|
||||||
|
_ = RunJobPipelineAndReleaseAsync(socket, job, completion);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task RunJobPipelineAndReleaseAsync(
|
||||||
|
ClientWebSocket socket, RecoverableJob job, TaskCompletionSource completion)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await RunJobPipelineAsync(socket, job);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
completion.TrySetResult();
|
||||||
|
jobPipelines.TryRemove(job.JobId, out _);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task RunJobPipelineAsync(ClientWebSocket socket, RecoverableJob job)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (job.Terminal is null)
|
||||||
|
{
|
||||||
|
await inputDownloader.DownloadAsync(job, CancellationToken.None);
|
||||||
|
var afterDownload = jobInbox.ReadRecoverableJobs()
|
||||||
|
.Single(item => item.JobId == job.JobId);
|
||||||
|
if (afterDownload.Terminal is JsonElement cancelledTerminal)
|
||||||
|
{
|
||||||
|
await TrySendAsync(socket, "job_terminal", cancelledTerminal);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await TrySendAsync(socket, "job_state", new
|
||||||
|
{
|
||||||
|
job_id = job.JobId,
|
||||||
|
lease_id = job.LeaseId,
|
||||||
|
request_digest = job.RequestDigest,
|
||||||
|
stage = "ready_to_run",
|
||||||
|
progress = 5,
|
||||||
|
metrics = new { input_bytes = job.InputTransfer?.GetProperty("size_bytes").GetInt64() },
|
||||||
|
});
|
||||||
|
await TrySendAsync(socket, "job_state", new
|
||||||
|
{
|
||||||
|
job_id = job.JobId,
|
||||||
|
lease_id = job.LeaseId,
|
||||||
|
request_digest = job.RequestDigest,
|
||||||
|
stage = "origin_running",
|
||||||
|
progress = 10,
|
||||||
|
metrics = new { },
|
||||||
|
});
|
||||||
|
await workerRunner.RunAsync(job);
|
||||||
|
}
|
||||||
|
var refreshed = jobInbox.ReadRecoverableJobs()
|
||||||
|
.Single(item => item.JobId == job.JobId);
|
||||||
|
if (refreshed.Terminal is not JsonElement terminal)
|
||||||
|
{
|
||||||
|
throw new InvalidDataException("Origin worker did not create a terminal record.");
|
||||||
|
}
|
||||||
|
if (terminal.GetProperty("status").GetString() != "succeeded")
|
||||||
|
{
|
||||||
|
await TrySendAsync(socket, "job_terminal", terminal);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await TrySendAsync(socket, "job_state", new
|
||||||
|
{
|
||||||
|
job_id = job.JobId,
|
||||||
|
lease_id = job.LeaseId,
|
||||||
|
request_digest = job.RequestDigest,
|
||||||
|
stage = "uploading_outputs",
|
||||||
|
progress = 90,
|
||||||
|
metrics = new { },
|
||||||
|
});
|
||||||
|
await outputUploader.UploadAsync(refreshed);
|
||||||
|
}
|
||||||
|
catch (Exception exception) when (
|
||||||
|
exception is HttpRequestException
|
||||||
|
or IOException
|
||||||
|
or JsonException
|
||||||
|
or UnauthorizedAccessException
|
||||||
|
or InvalidDataException)
|
||||||
|
{
|
||||||
|
var current = jobInbox.ReadRecoverableJobs()
|
||||||
|
.Single(item => item.JobId == job.JobId);
|
||||||
|
if (current.Terminal is JsonElement terminal
|
||||||
|
&& terminal.GetProperty("status").GetString() == "succeeded")
|
||||||
|
{
|
||||||
|
Console.Error.WriteLine($"[WARN] Output upload deferred: {exception.Message}");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
jobInbox.WriteTerminal(
|
||||||
|
job,
|
||||||
|
"failed",
|
||||||
|
"INPUT_DOWNLOAD_FAILED",
|
||||||
|
exception.Message[..Math.Min(exception.Message.Length, 500)]);
|
||||||
|
var failedTerminal = jobInbox.ReadRecoverableJobs()
|
||||||
|
.Single(item => item.JobId == job.JobId).Terminal;
|
||||||
|
if (failedTerminal is JsonElement payload)
|
||||||
|
{
|
||||||
|
await TrySendAsync(socket, "job_terminal", payload);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task TrySendTerminalAsync(ClientWebSocket socket, RecoverableJob job)
|
||||||
|
{
|
||||||
|
var terminal = jobInbox.ReadRecoverableJobs()
|
||||||
|
.Single(item => item.JobId == job.JobId).Terminal;
|
||||||
|
if (terminal is JsonElement terminalPayload)
|
||||||
|
{
|
||||||
|
await TrySendAsync(socket, "job_terminal", terminalPayload);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task TrySendAsync(ClientWebSocket socket, string type, object payload)
|
||||||
|
{
|
||||||
|
if (socket.State != WebSocketState.Open)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await SendAsync(socket, type, payload, CancellationToken.None);
|
||||||
|
}
|
||||||
|
catch (Exception exception) when (
|
||||||
|
exception is WebSocketException or IOException or ObjectDisposedException)
|
||||||
|
{
|
||||||
|
Console.Error.WriteLine($"[WARN] Job report deferred: {exception.Message}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private void Report(NodeState state, string message) =>
|
private void Report(NodeState state, string message) =>
|
||||||
statusChanged?.Invoke(NodeStatus.Create(state, message));
|
statusChanged?.Invoke(NodeStatus.Create(state, message));
|
||||||
|
|
||||||
|
|
@ -165,13 +442,22 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
||||||
sent_at = DateTimeOffset.UtcNow,
|
sent_at = DateTimeOffset.UtcNow,
|
||||||
payload,
|
payload,
|
||||||
});
|
});
|
||||||
await socket.SendAsync(
|
await sendLock.WaitAsync(cancellationToken);
|
||||||
envelope, WebSocketMessageType.Text, endOfMessage: true, cancellationToken);
|
try
|
||||||
|
{
|
||||||
|
await socket.SendAsync(
|
||||||
|
envelope, WebSocketMessageType.Text, endOfMessage: true, cancellationToken);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
sendLock.Release();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private object RuntimePayload()
|
private object RuntimePayload()
|
||||||
{
|
{
|
||||||
var root = Path.GetPathRoot(Environment.SystemDirectory) ?? "C:\\";
|
var root = Path.GetPathRoot(Environment.SystemDirectory) ?? "C:\\";
|
||||||
|
var origin = OriginRuntimeProbe.Detect();
|
||||||
return new
|
return new
|
||||||
{
|
{
|
||||||
install_id = config.InstallId,
|
install_id = config.InstallId,
|
||||||
|
|
@ -179,9 +465,17 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
||||||
node_version = Assembly.GetExecutingAssembly().GetName().Version?.ToString(3) ?? "0.1.0",
|
node_version = Assembly.GetExecutingAssembly().GetName().Version?.ToString(3) ?? "0.1.0",
|
||||||
os_version = RuntimeInformation.OSDescription,
|
os_version = RuntimeInformation.OSDescription,
|
||||||
capabilities = config.Capabilities,
|
capabilities = config.Capabilities,
|
||||||
available_slots = 1,
|
available_slots = origin.Health == "ready" && !jobInbox.HasPendingJobs ? 1 : 0,
|
||||||
disk_free_bytes = new DriveInfo(root).AvailableFreeSpace,
|
disk_free_bytes = new DriveInfo(root).AvailableFreeSpace,
|
||||||
desktop_session = Environment.UserInteractive,
|
desktop_session = Environment.UserInteractive,
|
||||||
|
origin = new
|
||||||
|
{
|
||||||
|
software = origin.Software,
|
||||||
|
software_version = origin.SoftwareVersion,
|
||||||
|
adapter_version = origin.AdapterVersion,
|
||||||
|
health = origin.Health,
|
||||||
|
detail = origin.Detail,
|
||||||
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -34,15 +34,18 @@ internal sealed record EnrollResponse(
|
||||||
[property: JsonPropertyName("heartbeat_seconds")] int HeartbeatSeconds,
|
[property: JsonPropertyName("heartbeat_seconds")] int HeartbeatSeconds,
|
||||||
[property: JsonPropertyName("max_concurrency")] int MaxConcurrency);
|
[property: JsonPropertyName("max_concurrency")] int MaxConcurrency);
|
||||||
|
|
||||||
internal sealed record NodePaths(string RootDirectory, string ConfigPath)
|
internal sealed record NodePaths(string RootDirectory, string ConfigPath, string JobsDirectory)
|
||||||
{
|
{
|
||||||
internal static NodePaths ForCurrentMachine()
|
internal static NodePaths ForCurrentMachine()
|
||||||
{
|
{
|
||||||
var root = Path.Combine(
|
var root = Path.Combine(
|
||||||
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
|
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
|
||||||
"Zcbot", "WindowsNode");
|
"Zcbot", "WindowsNode");
|
||||||
return new NodePaths(root, Path.Combine(root, "node.json"));
|
return new NodePaths(
|
||||||
|
root, Path.Combine(root, "node.json"), Path.Combine(root, "jobs"));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
internal sealed class NodeConfigurationException(string message) : Exception(message);
|
internal sealed class NodeConfigurationException(string message) : Exception(message);
|
||||||
|
|
||||||
|
internal sealed class NodeEndpointException(string message) : Exception(message);
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,102 @@
|
||||||
|
using Microsoft.Win32;
|
||||||
|
using System.Security;
|
||||||
|
|
||||||
|
namespace Zcbot.WindowsNode;
|
||||||
|
|
||||||
|
internal sealed record OriginRuntimeStatus(
|
||||||
|
string Software,
|
||||||
|
string? SoftwareVersion,
|
||||||
|
string AdapterVersion,
|
||||||
|
string Health,
|
||||||
|
string Detail);
|
||||||
|
|
||||||
|
internal static class OriginRuntimeProbe
|
||||||
|
{
|
||||||
|
private static readonly Lazy<OriginRuntimeStatus> Current = new(DetectCore);
|
||||||
|
private const string AutomationProgId = @"Origin.ApplicationSI\CLSID";
|
||||||
|
|
||||||
|
internal static OriginRuntimeStatus Detect() => Current.Value;
|
||||||
|
|
||||||
|
private static OriginRuntimeStatus DetectCore()
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var version = FindInstalledVersion();
|
||||||
|
using var automationKey = Registry.ClassesRoot.OpenSubKey(AutomationProgId);
|
||||||
|
var automationRegistered = automationKey is not null;
|
||||||
|
if (version is null && !automationRegistered)
|
||||||
|
{
|
||||||
|
return Status(null, "unavailable", "未检测到 Origin/OriginPro 安装");
|
||||||
|
}
|
||||||
|
if (!automationRegistered)
|
||||||
|
{
|
||||||
|
return Status(version, "unavailable", "已检测到 Origin,但 COM 自动化组件未注册");
|
||||||
|
}
|
||||||
|
if (!Environment.UserInteractive)
|
||||||
|
{
|
||||||
|
return Status(version, "unavailable", "Origin 需要交互式 Windows 桌面会话");
|
||||||
|
}
|
||||||
|
var interpreter = OriginWorkerRuntime.ResolveInterpreter();
|
||||||
|
if (interpreter is null)
|
||||||
|
{
|
||||||
|
return Status(
|
||||||
|
version,
|
||||||
|
"unavailable",
|
||||||
|
"Origin 可用,但固定 Python 运行时缺失;请配置 ZCBOT_ORIGIN_PYTHON");
|
||||||
|
}
|
||||||
|
return Status(version, "ready", $"Origin COM 与固定 Python 运行时可用({interpreter})");
|
||||||
|
}
|
||||||
|
catch (Exception exception) when (
|
||||||
|
exception is SecurityException or UnauthorizedAccessException or IOException)
|
||||||
|
{
|
||||||
|
return Status(null, "unavailable", $"Origin 运行时探测失败:{exception.Message}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static OriginRuntimeStatus Status(string? version, string health, string detail) =>
|
||||||
|
new("OriginPro", version, "0.2.0", health, detail);
|
||||||
|
|
||||||
|
private static string? FindInstalledVersion()
|
||||||
|
{
|
||||||
|
var candidates = new List<string>();
|
||||||
|
foreach (var hive in new[] { RegistryHive.LocalMachine, RegistryHive.CurrentUser })
|
||||||
|
{
|
||||||
|
foreach (var view in new[] { RegistryView.Registry64, RegistryView.Registry32 })
|
||||||
|
{
|
||||||
|
using var baseKey = RegistryKey.OpenBaseKey(hive, view);
|
||||||
|
using var uninstall = baseKey.OpenSubKey(
|
||||||
|
@"SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall");
|
||||||
|
if (uninstall is null)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
foreach (var keyName in uninstall.GetSubKeyNames())
|
||||||
|
{
|
||||||
|
using var product = uninstall.OpenSubKey(keyName);
|
||||||
|
var name = product?.GetValue("DisplayName") as string;
|
||||||
|
var publisher = product?.GetValue("Publisher") as string;
|
||||||
|
if (!IsOriginProduct(name, publisher))
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
var version = product?.GetValue("DisplayVersion") as string;
|
||||||
|
if (!string.IsNullOrWhiteSpace(version))
|
||||||
|
{
|
||||||
|
candidates.Add(version.Trim());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return candidates.OrderByDescending(ParseVersion).ThenByDescending(x => x).FirstOrDefault();
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool IsOriginProduct(string? name, string? publisher) =>
|
||||||
|
!string.IsNullOrWhiteSpace(name)
|
||||||
|
&& (name.Equals("Origin", StringComparison.OrdinalIgnoreCase)
|
||||||
|
|| name.StartsWith("Origin ", StringComparison.OrdinalIgnoreCase)
|
||||||
|
|| name.StartsWith("OriginPro", StringComparison.OrdinalIgnoreCase))
|
||||||
|
&& (publisher?.Contains("OriginLab", StringComparison.OrdinalIgnoreCase) ?? false);
|
||||||
|
|
||||||
|
private static Version ParseVersion(string value) =>
|
||||||
|
Version.TryParse(value, out var version) ? version : new Version(0, 0);
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,164 @@
|
||||||
|
using System.Collections.Concurrent;
|
||||||
|
using System.Diagnostics;
|
||||||
|
using System.Text;
|
||||||
|
using System.Text.Json;
|
||||||
|
|
||||||
|
namespace Zcbot.WindowsNode;
|
||||||
|
|
||||||
|
internal sealed class OriginWorkerRunner(JobInboxStore inbox)
|
||||||
|
{
|
||||||
|
private static readonly TimeSpan WorkerTimeout = TimeSpan.FromMinutes(30);
|
||||||
|
private readonly ConcurrentDictionary<Guid, Task> active = new();
|
||||||
|
private readonly ConcurrentDictionary<Guid, CancellationTokenSource> cancellations = new();
|
||||||
|
|
||||||
|
internal Task RunAsync(RecoverableJob job) =>
|
||||||
|
active.GetOrAdd(job.JobId, _ => RunOnceAsync(job, CancellationFor(job.JobId).Token));
|
||||||
|
|
||||||
|
internal void Cancel(Guid jobId)
|
||||||
|
{
|
||||||
|
CancellationFor(jobId).Cancel();
|
||||||
|
}
|
||||||
|
|
||||||
|
private CancellationTokenSource CancellationFor(Guid jobId) =>
|
||||||
|
cancellations.GetOrAdd(jobId, _ => new CancellationTokenSource());
|
||||||
|
|
||||||
|
private async Task RunOnceAsync(RecoverableJob job, CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var paths = NodePaths.ForCurrentMachine();
|
||||||
|
var jobDirectory = Path.Combine(paths.JobsDirectory, job.JobId.ToString("D"));
|
||||||
|
var terminalPath = Path.Combine(jobDirectory, "terminal.json");
|
||||||
|
if (File.Exists(terminalPath)) return;
|
||||||
|
|
||||||
|
var markerPath = Path.Combine(jobDirectory, "worker-started.json");
|
||||||
|
if (File.Exists(markerPath))
|
||||||
|
{
|
||||||
|
inbox.WriteTerminal(
|
||||||
|
job,
|
||||||
|
"failed",
|
||||||
|
"NODE_RESTARTED_DURING_JOB",
|
||||||
|
"The node restarted after Origin execution began and cannot prove the prior worker state.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
var interpreter = OriginWorkerRuntime.ResolveInterpreter()
|
||||||
|
?? throw new InvalidOperationException("The fixed Origin Python interpreter is unavailable.");
|
||||||
|
var workerScript = Path.GetFullPath(
|
||||||
|
Path.Combine(AppContext.BaseDirectory, "origin-worker", "worker.py"));
|
||||||
|
if (!File.Exists(workerScript))
|
||||||
|
{
|
||||||
|
throw new FileNotFoundException("The fixed Origin worker script is missing.", workerScript);
|
||||||
|
}
|
||||||
|
WriteMarker(markerPath, interpreter, workerScript);
|
||||||
|
|
||||||
|
var startInfo = new ProcessStartInfo
|
||||||
|
{
|
||||||
|
FileName = interpreter,
|
||||||
|
WorkingDirectory = jobDirectory,
|
||||||
|
UseShellExecute = false,
|
||||||
|
CreateNoWindow = true,
|
||||||
|
RedirectStandardOutput = true,
|
||||||
|
RedirectStandardError = true,
|
||||||
|
StandardOutputEncoding = Encoding.UTF8,
|
||||||
|
StandardErrorEncoding = Encoding.UTF8,
|
||||||
|
};
|
||||||
|
startInfo.ArgumentList.Add(workerScript);
|
||||||
|
startInfo.ArgumentList.Add(jobDirectory);
|
||||||
|
using var process = Process.Start(startInfo)
|
||||||
|
?? throw new InvalidOperationException("The fixed Origin worker did not start.");
|
||||||
|
var stdout = process.StandardOutput.ReadToEndAsync();
|
||||||
|
var stderr = process.StandardError.ReadToEndAsync();
|
||||||
|
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
|
||||||
|
timeout.CancelAfter(WorkerTimeout);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await process.WaitForExitAsync(timeout.Token);
|
||||||
|
}
|
||||||
|
catch (OperationCanceledException)
|
||||||
|
{
|
||||||
|
process.Kill(entireProcessTree: true);
|
||||||
|
if (cancellationToken.IsCancellationRequested)
|
||||||
|
{
|
||||||
|
inbox.WriteTerminal(job, "cancelled", "USER_CANCELLED", "Cancelled by user.");
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
inbox.WriteTerminal(job, "failed", "ORIGIN_WORKER_TIMEOUT", "Origin worker exceeded 30 minutes.");
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var output = await stdout;
|
||||||
|
var error = await stderr;
|
||||||
|
WriteDiagnostic(jobDirectory, output, error, process.ExitCode);
|
||||||
|
if (!File.Exists(terminalPath))
|
||||||
|
{
|
||||||
|
inbox.WriteTerminal(
|
||||||
|
job,
|
||||||
|
"failed",
|
||||||
|
"ORIGIN_WORKER_NO_TERMINAL",
|
||||||
|
$"Origin worker exited with code {process.ExitCode} without terminal.json.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (Exception exception) when (
|
||||||
|
exception is IOException
|
||||||
|
or JsonException
|
||||||
|
or UnauthorizedAccessException
|
||||||
|
or InvalidOperationException)
|
||||||
|
{
|
||||||
|
inbox.WriteTerminal(job, "failed", "ORIGIN_WORKER_START_FAILED", exception.Message[..Math.Min(500, exception.Message.Length)]);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
active.TryRemove(job.JobId, out _);
|
||||||
|
if (cancellations.TryRemove(job.JobId, out var cancellation)) cancellation.Dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void WriteMarker(string path, string interpreter, string workerScript)
|
||||||
|
{
|
||||||
|
var value = JsonSerializer.SerializeToUtf8Bytes(new
|
||||||
|
{
|
||||||
|
started_at = DateTimeOffset.UtcNow,
|
||||||
|
node_pid = Environment.ProcessId,
|
||||||
|
interpreter,
|
||||||
|
worker_script = workerScript,
|
||||||
|
});
|
||||||
|
using var stream = new FileStream(
|
||||||
|
path, FileMode.CreateNew, FileAccess.Write, FileShare.None,
|
||||||
|
bufferSize: 4096, FileOptions.WriteThrough);
|
||||||
|
stream.Write(value);
|
||||||
|
stream.Flush(flushToDisk: true);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void WriteDiagnostic(string jobDirectory, string output, string error, int exitCode)
|
||||||
|
{
|
||||||
|
var logs = Path.Combine(jobDirectory, "logs");
|
||||||
|
Directory.CreateDirectory(logs);
|
||||||
|
var value = JsonSerializer.Serialize(new
|
||||||
|
{
|
||||||
|
exit_code = exitCode,
|
||||||
|
stdout = output[..Math.Min(output.Length, 16 * 1024)],
|
||||||
|
stderr = error[..Math.Min(error.Length, 16 * 1024)],
|
||||||
|
});
|
||||||
|
File.WriteAllText(Path.Combine(logs, "worker-process.json"), value, Encoding.UTF8);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static class OriginWorkerRuntime
|
||||||
|
{
|
||||||
|
internal static string? ResolveInterpreter()
|
||||||
|
{
|
||||||
|
var paths = NodePaths.ForCurrentMachine();
|
||||||
|
var configured = Environment.GetEnvironmentVariable("ZCBOT_ORIGIN_PYTHON");
|
||||||
|
var candidate = string.IsNullOrWhiteSpace(configured)
|
||||||
|
? Path.Combine(paths.RootDirectory, "runtimes", "origin", "python.exe")
|
||||||
|
: configured;
|
||||||
|
if (!Path.IsPathFullyQualified(candidate)) return null;
|
||||||
|
var resolved = Path.GetFullPath(candidate);
|
||||||
|
return File.Exists(resolved)
|
||||||
|
&& Path.GetFileName(resolved).Equals("python.exe", StringComparison.OrdinalIgnoreCase)
|
||||||
|
? resolved
|
||||||
|
: null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -16,6 +16,7 @@ internal sealed class TrayApplicationContext : ApplicationContext
|
||||||
this.store = store;
|
this.store = store;
|
||||||
form = new ConfigurationForm();
|
form = new ConfigurationForm();
|
||||||
form.RegisterRequested += RegisterAsync;
|
form.RegisterRequested += RegisterAsync;
|
||||||
|
form.ReconnectRequested += RestartConnection;
|
||||||
form.ResetIdentityRequested += ResetIdentity;
|
form.ResetIdentityRequested += ResetIdentity;
|
||||||
|
|
||||||
statusItem = new ToolStripMenuItem("尚未注册") { Enabled = false };
|
statusItem = new ToolStripMenuItem("尚未注册") { Enabled = false };
|
||||||
|
|
|
||||||
|
|
@ -11,4 +11,10 @@
|
||||||
<RootNamespace>Zcbot.WindowsNode</RootNamespace>
|
<RootNamespace>Zcbot.WindowsNode</RootNamespace>
|
||||||
<Version>0.1.0</Version>
|
<Version>0.1.0</Version>
|
||||||
</PropertyGroup>
|
</PropertyGroup>
|
||||||
|
<ItemGroup>
|
||||||
|
<Content Include="..\origin-worker\worker.py">
|
||||||
|
<Link>origin-worker\worker.py</Link>
|
||||||
|
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||||
|
</Content>
|
||||||
|
</ItemGroup>
|
||||||
</Project>
|
</Project>
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,32 @@
|
||||||
|
param(
|
||||||
|
[Parameter(Mandatory = $true)]
|
||||||
|
[string]$BootstrapPython,
|
||||||
|
[string]$RuntimeDirectory = "$env:ProgramData\Zcbot\WindowsNode\runtimes\origin"
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = "Stop"
|
||||||
|
$python = (Resolve-Path -LiteralPath $BootstrapPython).Path
|
||||||
|
if ([IO.Path]::GetFileName($python) -ne "python.exe") {
|
||||||
|
throw "BootstrapPython must point to python.exe."
|
||||||
|
}
|
||||||
|
$requirements = Join-Path $PSScriptRoot "origin-worker\requirements.txt"
|
||||||
|
if (-not (Test-Path -LiteralPath $requirements -PathType Leaf)) {
|
||||||
|
throw "Pinned Origin worker requirements are missing."
|
||||||
|
}
|
||||||
|
$runtime = [IO.Path]::GetFullPath($RuntimeDirectory)
|
||||||
|
if ($runtime -eq [IO.Path]::GetPathRoot($runtime)) {
|
||||||
|
throw "RuntimeDirectory cannot be a drive root."
|
||||||
|
}
|
||||||
|
|
||||||
|
if (-not (Test-Path -LiteralPath (Join-Path $runtime "python.exe"))) {
|
||||||
|
& $python -m venv $runtime
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw "Failed to create the Origin runtime." }
|
||||||
|
}
|
||||||
|
$runtimePython = Join-Path $runtime "python.exe"
|
||||||
|
& $runtimePython -m pip install --requirement $requirements
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw "Failed to install the pinned Origin runtime packages." }
|
||||||
|
& $runtimePython -c "import originpro, openpyxl; print('[OK] Origin worker Python packages are available.')"
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw "Origin runtime import verification failed." }
|
||||||
|
|
||||||
|
Write-Output "[OK] Fixed Origin runtime installed: $runtimePython"
|
||||||
|
Write-Output "[INFO] Restart zcbot Windows Node to refresh runtime health."
|
||||||
|
|
@ -0,0 +1,2 @@
|
||||||
|
originpro==1.1.15
|
||||||
|
openpyxl==3.1.5
|
||||||
|
|
@ -0,0 +1,252 @@
|
||||||
|
"""Fixed Origin adapter for origin.plot@v1.
|
||||||
|
|
||||||
|
This process accepts exactly one argument: a Node-created job directory. It never
|
||||||
|
installs packages, evaluates user code, downloads data, or resolves paths from the
|
||||||
|
request. terminal.json is its only terminal-state contract.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import csv
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from importlib.metadata import PackageNotFoundError, version
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
PLOT_TYPES = {"line": "l", "scatter": "s", "line_scatter": "y"}
|
||||||
|
FORMATS = {"opju", "png", "svg", "pdf"}
|
||||||
|
|
||||||
|
|
||||||
|
def _atomic_json(path: Path, value: Any) -> None:
|
||||||
|
temporary = path.with_name(path.name + ".tmp-" + os.urandom(8).hex())
|
||||||
|
try:
|
||||||
|
with temporary.open("w", encoding="utf-8", newline="\n") as handle:
|
||||||
|
json.dump(value, handle, ensure_ascii=False, indent=2)
|
||||||
|
handle.flush()
|
||||||
|
os.fsync(handle.fileno())
|
||||||
|
os.replace(temporary, path)
|
||||||
|
finally:
|
||||||
|
temporary.unlink(missing_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
def _read_rows(path: Path, sheet: str | None) -> tuple[list[str], list[list[Any]]]:
|
||||||
|
suffix = path.suffix.lower()
|
||||||
|
if suffix == ".csv":
|
||||||
|
with path.open("r", encoding="utf-8-sig", newline="") as handle:
|
||||||
|
rows = list(csv.reader(handle))
|
||||||
|
if len(rows) < 2:
|
||||||
|
raise ValueError("CSV_INPUT_EMPTY")
|
||||||
|
return [str(item) for item in rows[0]], rows[1:]
|
||||||
|
if suffix == ".json":
|
||||||
|
value = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
if isinstance(value, list) and value and all(isinstance(item, dict) for item in value):
|
||||||
|
headers = list(value[0])
|
||||||
|
return headers, [[item.get(name) for name in headers] for item in value]
|
||||||
|
if isinstance(value, dict) and value and all(isinstance(item, list) for item in value.values()):
|
||||||
|
headers = list(value)
|
||||||
|
length = max(len(value[name]) for name in headers)
|
||||||
|
return headers, [[value[name][index] if index < len(value[name]) else None for name in headers] for index in range(length)]
|
||||||
|
raise ValueError("JSON_INPUT_SHAPE_UNSUPPORTED")
|
||||||
|
if suffix == ".xlsx":
|
||||||
|
from openpyxl import load_workbook
|
||||||
|
|
||||||
|
workbook = load_workbook(path, read_only=True, data_only=True)
|
||||||
|
try:
|
||||||
|
worksheet = workbook[sheet] if sheet else workbook.active
|
||||||
|
rows = list(worksheet.iter_rows(values_only=True))
|
||||||
|
finally:
|
||||||
|
workbook.close()
|
||||||
|
if len(rows) < 2:
|
||||||
|
raise ValueError("XLSX_INPUT_EMPTY")
|
||||||
|
return [str(item or "") for item in rows[0]], [list(row) for row in rows[1:]]
|
||||||
|
raise ValueError("INPUT_TYPE_UNSUPPORTED")
|
||||||
|
|
||||||
|
|
||||||
|
def _column_index(headers: list[str], value: Any, field: str) -> int:
|
||||||
|
if not isinstance(value, str) or value not in headers:
|
||||||
|
raise ValueError(f"{field.upper()}_COLUMN_NOT_FOUND")
|
||||||
|
return headers.index(value)
|
||||||
|
|
||||||
|
|
||||||
|
def _manifest(path: Path, media_type: str) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"artifact_id": {
|
||||||
|
"project.opju": "project",
|
||||||
|
"figure.png": "figure_png",
|
||||||
|
"figure.svg": "figure_svg",
|
||||||
|
"figure.pdf": "figure_pdf",
|
||||||
|
"plot-spec.json": "plot_spec",
|
||||||
|
"provenance.json": "provenance",
|
||||||
|
}[path.name],
|
||||||
|
"filename": path.name,
|
||||||
|
"media_type": media_type,
|
||||||
|
"size_bytes": path.stat().st_size,
|
||||||
|
"sha256": _file_sha256(path),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _file_sha256(path: Path) -> str:
|
||||||
|
digest = hashlib.sha256()
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||||
|
digest.update(chunk)
|
||||||
|
return digest.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _axis_title(axis: Any, fallback: str) -> str:
|
||||||
|
if not isinstance(axis, dict):
|
||||||
|
return fallback
|
||||||
|
title = str(axis.get("title") or fallback)
|
||||||
|
unit = str(axis.get("unit") or "")
|
||||||
|
return f"{title} ({unit})" if unit else title
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_artifact(path: Path, extension: str) -> None:
|
||||||
|
if not path.is_file() or path.stat().st_size == 0:
|
||||||
|
raise RuntimeError(f"{extension.upper()}_EXPORT_EMPTY")
|
||||||
|
head = path.read_bytes()[:1024]
|
||||||
|
if extension == "png" and not head.startswith(b"\x89PNG\r\n\x1a\n"):
|
||||||
|
raise RuntimeError("PNG_EXPORT_INVALID")
|
||||||
|
if extension == "pdf" and not head.startswith(b"%PDF-"):
|
||||||
|
raise RuntimeError("PDF_EXPORT_INVALID")
|
||||||
|
if extension == "svg" and b"<svg" not in head.lower():
|
||||||
|
raise RuntimeError("SVG_EXPORT_INVALID")
|
||||||
|
if extension == "opju" and len(head) < 64:
|
||||||
|
raise RuntimeError("OPJU_EXPORT_INVALID")
|
||||||
|
|
||||||
|
|
||||||
|
def run(job_dir: Path) -> list[dict[str, Any]]:
|
||||||
|
job_dir = job_dir.resolve(strict=True)
|
||||||
|
request_record = json.loads((job_dir / "request" / "request.json").read_text(encoding="utf-8"))
|
||||||
|
request = request_record["request"]
|
||||||
|
input_files = [path for path in (job_dir / "input").iterdir() if path.is_file() and not path.name.startswith(".")]
|
||||||
|
if len(input_files) != 1:
|
||||||
|
raise ValueError("INPUT_FILE_COUNT_INVALID")
|
||||||
|
headers, rows = _read_rows(input_files[0], request["input"].get("sheet"))
|
||||||
|
plot_spec = request["plot"]
|
||||||
|
plot_type = plot_spec["type"]
|
||||||
|
if plot_type not in PLOT_TYPES:
|
||||||
|
raise ValueError("PLOT_TYPE_NOT_IMPLEMENTED")
|
||||||
|
x_index = _column_index(headers, plot_spec.get("x"), "x")
|
||||||
|
y_names = plot_spec.get("y")
|
||||||
|
if isinstance(y_names, str):
|
||||||
|
y_names = [y_names]
|
||||||
|
if not isinstance(y_names, list) or not y_names:
|
||||||
|
raise ValueError("Y_COLUMNS_REQUIRED")
|
||||||
|
y_indexes = [_column_index(headers, name, "y") for name in y_names]
|
||||||
|
|
||||||
|
import originpro as op
|
||||||
|
|
||||||
|
output = job_dir / "output"
|
||||||
|
output.mkdir(exist_ok=True)
|
||||||
|
op.set_show(False)
|
||||||
|
try:
|
||||||
|
op.new()
|
||||||
|
worksheet = op.new_sheet("w", lname="Data")
|
||||||
|
for index, header in enumerate(headers):
|
||||||
|
worksheet.from_list(index, [row[index] if index < len(row) else None for row in rows], lname=header)
|
||||||
|
graph = op.new_graph(template={"line": "line", "scatter": "scatter", "line_scatter": "linesymb"}[plot_type])
|
||||||
|
layer = graph[0]
|
||||||
|
for y_index in y_indexes:
|
||||||
|
layer.add_plot(worksheet, coly=y_index, colx=x_index, type=PLOT_TYPES[plot_type])
|
||||||
|
layer.rescale()
|
||||||
|
layer.axis("x").title = _axis_title(plot_spec.get("x_axis"), str(plot_spec.get("x") or "X"))
|
||||||
|
layer.axis("y").title = _axis_title(plot_spec.get("y_axis"), "Y")
|
||||||
|
if plot_spec.get("title"):
|
||||||
|
title = layer.add_label(str(plot_spec["title"]))
|
||||||
|
title.set_int("fsize", 18)
|
||||||
|
title.set_int("left", 2200)
|
||||||
|
title.set_int("top", 120)
|
||||||
|
formats = request["output"]["formats"]
|
||||||
|
if any(item not in FORMATS for item in formats):
|
||||||
|
raise ValueError("OUTPUT_FORMAT_UNSUPPORTED")
|
||||||
|
artifacts: list[dict[str, Any]] = []
|
||||||
|
if "opju" in formats:
|
||||||
|
project = output / "project.opju"
|
||||||
|
op.save(str(project))
|
||||||
|
_validate_artifact(project, "opju")
|
||||||
|
artifacts.append(_manifest(project, "application/x-origin-project"))
|
||||||
|
media = {"png": "image/png", "svg": "image/svg+xml", "pdf": "application/pdf"}
|
||||||
|
dpi = request["output"].get("dpi", 300)
|
||||||
|
if not isinstance(dpi, int) or isinstance(dpi, bool) or not 72 <= dpi <= 1200:
|
||||||
|
raise ValueError("OUTPUT_DPI_INVALID")
|
||||||
|
pixel_width = round(dpi * 160 / 25.4)
|
||||||
|
for extension in ("png", "svg", "pdf"):
|
||||||
|
if extension in formats:
|
||||||
|
target = output / f"figure.{extension}"
|
||||||
|
exported = Path(graph.save_fig(
|
||||||
|
str(target),
|
||||||
|
type=extension,
|
||||||
|
width=pixel_width if extension == "png" else 0,
|
||||||
|
ratio=100 if extension in {"svg", "pdf"} else 0,
|
||||||
|
)).resolve()
|
||||||
|
if exported != target.resolve() or not target.is_file():
|
||||||
|
raise RuntimeError(f"{extension.upper()}_EXPORT_FAILED")
|
||||||
|
_validate_artifact(target, extension)
|
||||||
|
artifacts.append(_manifest(target, media[extension]))
|
||||||
|
try:
|
||||||
|
originpro_version = version("originpro")
|
||||||
|
except PackageNotFoundError:
|
||||||
|
originpro_version = "embedded"
|
||||||
|
provenance = {
|
||||||
|
"adapter_version": "0.2.0",
|
||||||
|
"originpro_version": originpro_version,
|
||||||
|
"request_digest": request_record["request_digest"],
|
||||||
|
"input_sha256": _file_sha256(input_files[0]),
|
||||||
|
"requested_dpi": dpi,
|
||||||
|
"png_pixel_width": pixel_width,
|
||||||
|
}
|
||||||
|
plot_spec_path = output / "plot-spec.json"
|
||||||
|
provenance_path = output / "provenance.json"
|
||||||
|
_atomic_json(plot_spec_path, request)
|
||||||
|
_atomic_json(provenance_path, provenance)
|
||||||
|
artifacts.append(_manifest(plot_spec_path, "application/json"))
|
||||||
|
artifacts.append(_manifest(provenance_path, "application/json"))
|
||||||
|
return artifacts
|
||||||
|
finally:
|
||||||
|
if op.oext:
|
||||||
|
op.exit()
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
if len(sys.argv) != 2:
|
||||||
|
print("[ERR] Usage: worker.py <job-directory>", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
job_dir = Path(sys.argv[1])
|
||||||
|
request_record: dict[str, Any] = {}
|
||||||
|
try:
|
||||||
|
request_record = json.loads((job_dir / "request" / "request.json").read_text(encoding="utf-8"))
|
||||||
|
artifacts = run(job_dir)
|
||||||
|
terminal = {
|
||||||
|
"job_id": request_record["job_id"],
|
||||||
|
"lease_id": request_record["lease_id"],
|
||||||
|
"request_digest": request_record["request_digest"],
|
||||||
|
"status": "succeeded",
|
||||||
|
"error": {},
|
||||||
|
"artifact_manifest": artifacts,
|
||||||
|
"terminal_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
}
|
||||||
|
_atomic_json(job_dir / "artifacts.json", artifacts)
|
||||||
|
_atomic_json(job_dir / "terminal.json", terminal)
|
||||||
|
print("[OK] Origin job completed.")
|
||||||
|
return 0
|
||||||
|
except Exception as exception:
|
||||||
|
terminal = {
|
||||||
|
"job_id": request_record.get("job_id", ""),
|
||||||
|
"lease_id": request_record.get("lease_id", ""),
|
||||||
|
"request_digest": request_record.get("request_digest", ""),
|
||||||
|
"status": "failed",
|
||||||
|
"error": {"code": type(exception).__name__, "detail": str(exception)[:500]},
|
||||||
|
"artifact_manifest": [],
|
||||||
|
"terminal_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
}
|
||||||
|
_atomic_json(job_dir / "terminal.json", terminal)
|
||||||
|
print(f"[ERR] {type(exception).__name__}: {exception}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
Loading…
Reference in New Issue