Compare commits

..

5 Commits

48 changed files with 4126 additions and 429 deletions

View File

@ -36,8 +36,11 @@
## 文档与版本
- 版本号、`CHANGELOG.md`、`PROGRESS.md` 在 push 前统一更新一次,不按每个 commit 更新。
- push 前更新 `PROGRESS.md`:补“已完成关键能力”条目,状态表变化随之更新,新增或删除模块时同步文件清单。
- 开发 push 与正式发布分离:功能开发期间允许持续 commit / push不因此提升版本号也不新增已发布的 `CHANGELOG.md` 数字版本条目。
- 尚未发布但需要预先整理的用户文案写在 `CHANGELOG.md` 顶部 `## Unreleased`;该区不会被前端更新日志接口解析。正式发布时再把它改成 `## <版本> — <日期>`
- 版本号与用户版 `CHANGELOG.md` 只在功能稳定、准备上线时通过单独的 release commit 统一更新;同一次发布中校准 `PROGRESS.md`,不按每个开发 commit 更新。
- 阶段性成果可随开发更新 `PROGRESS.md`;正式发布前补“已完成关键能力”条目,状态表变化随之更新,新增或删除模块时同步文件清单。
- 未完成且不能让线上用户接触的功能,应在独立功能分支开发、待稳定后合并生产分支;若必须提前合并或部署,则使用默认关闭的 feature flag并限制为管理员或测试账号启用。
- 版本号唯一事实源是 `core/__init__.py::__version__`
- patchbug 修复、重构、调参、新 skill、样式
- minor成批新功能或明显对外行为变化
@ -59,4 +62,3 @@
## 领域语境
主要使用方是中国建筑材料科学研究总院核心语境是无机非金属材料研发与生产包括水泥、混凝土、玻璃、陶瓷、耐火材料和新型建材。典型任务是配方研发、性能测试、XRD/SEM/热分析、实验数据建模,以及申报书、调研报告、专利和论文写作。默认按材料研发而非建筑施工、结构计算或 BIM 理解需求。

View File

@ -3,7 +3,13 @@
> 面向用户的更新说明,由 `GET /v1/changelog` 解析后在前端「点版本号」弹层展示。
> 维护口径:只记**用户可感知**的变化("能干什么了"而不是"改了哪个模块"),纯内部重构 / 修复不记,
> 所以不是每个版本号都有条目。条目格式 `## <版本> — <日期>`,新条目加在最上面。
> 开发中的用户文案可先写入 `## Unreleased`;该区不会被前端解析,正式发布时再替换为数字版本和日期。
> 工程口径的完整记录见 `PROGRESS.md` / git log。
## Unreleased
- 新增专业软件 Job 中心Agent 可提交、查询和停止 Windows Node 上的受控软件任务;用户可在右下角跨对话查看进度、收到完成通知,并回到原对话分析结果。
## 0.65.2 — 2026-08-13
- 粘贴、拖拽或选择附件发送后,对话中的用户消息会保留附件卡片;图片同时显示可点击放大的缩略图,刷新历史对话后也不会消失。

View File

@ -460,11 +460,21 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
第一阶段以 `docs/windows-node-mvp-intranet.md` 为实现契约Windows Node 只作为受控执行节点,通过出站 HTTP/WS 主动连接 zcbot首批能力固定为 `origin.plot@v1`。长期方案中的 mTLS、Service/DesktopRunner 双进程、完整租约与多节点调度暂不进入 MVP但 URL path、Node ID、Bearer Header 和任务协议保留原位升级空间。
云端控制面使用独立的 `compute_node_enrollments` 与 `compute_nodes`,不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵,数据库只保存 SHA-256 摘要;节点注册在行锁事务中校验有效期、预期名称和允许能力,成功后原子消费。每个节点获得独立高熵 Token数据库只保存 bcrypt 强哈希,明文仅在注册响应出现一次。
云端控制面使用独立的 `software_node_enrollments` 与 `software_nodes`,不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵,数据库只保存 SHA-256 摘要;节点注册在行锁事务中校验有效期、预期名称和允许能力,成功后原子消费。每个节点获得独立高熵 Token数据库只保存 bcrypt 强哈希,明文仅在注册响应出现一次。
Node 通过 `Authorization: Bearer``X-Node-Id` 建立 `/v1/compute/nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活,新连接关闭旧连接;`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态,再关闭现有连接;断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。
Node 通过 `Authorization: Bearer``X-Node-Id` 建立 `/v1/software-nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活,新连接关闭旧连接;`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态,再关闭现有连接;断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。
首批只落注册、认证、心跳、状态与禁用基础链路。`compute_jobs`、任务 offer/accept、Origin Worker、输入输出传输、重连对账和 Token 轮换属于后续垂直闭环,不以任意命令或脚本接口临时代替。
第二阶段已增加 `software_jobs`(专业软件任务)账本与 `origin.plot@v1` 的 offer/accept 骨架。用户只能在本人 task 下以幂等键提交固定 schema云端规范化请求并记录 SHA-256按当前进程真实在线、能力匹配、健康且有空闲 slot 的 Node 创建短期 offer。Node 再次校验 schema、图形类型和输出格式使用 write-through、flush 与原子 rename 先落本机任务目录,再回 `job_accept`;重复 job 只有 digest 一致才接受。过期或发送失败的 offer 回到队列lease、Node 和 digest 不匹配的响应被拒绝。Node 接收后云端进入 `dispatched` 而非 `running`,并将 slot 降为 0只有固定 Worker 真正启动后才进入 `origin_running`
第三阶段补齐输入下载与恢复状态协议:`input_id` 固定为用户已有 artifact UUID提交时快照文件名、大小和 SHA-256只允许 CSV/XLSX/JSON 且不超过 100 MiB。Node 以自身 Bearer 身份访问任务绑定的只读下载端点,流式写入本 job 的 `input/`,同时限制声明大小并校验 SHA-256完成后原子 rename不暴露工作区路径。Node 会原子读取/补报 `terminal.json`,断线后云端把活动任务标记 `disconnected` 并保留 Node/lease重连按 job、lease、digest 恢复下载或幂等补报终态,不自动重派。
第四阶段落地固定 Origin WorkerNode 仅从管理员安装的固定 Python 运行时启动随程序发布的 `worker.py`,参数只有本机 job 目录请求不能指定脚本、解释器或文件路径。Worker 使用 `originpro` 生成 OPJU、PNG、SVG、PDF、plot spec 和 provenance校验产物签名并原子写入终态当前受控图形仅含 line、scatter、line_scatter 和双栏出版布局。进程内 pipeline 按 job 去重,并脱离单次 WebSocket 的取消令牌运行;连接中断只延迟状态/终态上报。Node 进程若在 Worker 启动后重启,则保守失败而不重复驱动 Origin避免无法证明的双执行。
第五阶段完成输出上传与发布Node 只按固定 manifest ID 逐项流式 PUT并携带 Node、lease、request digest 与内容摘要;云端重新绑定任务身份,不信 Node 提供的路径或媒体类型。文件先进入用户根下隐藏暂存区,固定文件名、单文件/总大小和 SHA-256 全部验证后,目录级原子移动到 `<working_dir>/origin/<job_id>/`,再登记平台 artifact UUID 并写成功终态。重复 PUT、complete 和重连均按摘要幂等;部分上传不可见,只有完整集合才能发布。
第六阶段增加用户级 Job 中心与 Agent typed tools。`software_capability_list` 只暴露固定能力及当前在线空闲节点数,`software_job_submit/status/cancel` 在构造时绑定当前 user/task模型不能跨用户或跨对话指定归属。右下角 Job 中心按用户聚合各对话任务,活动期短轮询、空闲期降频;终态变化通知用户,成功任务可回到原对话发起分析。取消采用协作协议:未派发任务直接终止,已派发任务先进入 `cancelling`,云端通过 WebSocket 发送并在心跳时重放 `job_cancel`Node 杀死固定 Worker 进程树后回报 `cancelled`;终态写入仍由云端账本裁决。
后续仍需实现 Token 轮换;不得以任意命令或脚本接口临时代替。当前 Job 中心采用轮询而非用户事件推送,单活与 offer 选择仍只覆盖单 Web 进程;生产启用多实例前必须增加 Redis/PG fencing 或固定路由到单一控制面实例。
---

View File

@ -2,7 +2,7 @@
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`
最后更新:2026-08-13(用户消息结构化附件与图片预览,bump 0.65.2)
最后更新:2026-08-13(专业软件 Job 中心开发中,未发版)
---
@ -22,6 +22,8 @@
### 2026-08-13
- **08-13 / Unreleased / 专业软件 Job 中心**:Agent 新增固定能力发现、提交、状态和取消工具后端提供用户级跨对话任务列表与协作取消Windows Node 可终止固定 Worker 进程树并幂等回报取消终态Web 右下角展示活动/最近任务、进度、完成通知、停止与回到原对话分析入口。相关 Python 78 项、JavaScript 语法、Python 编译、.NET build 与 diff 检查通过;数据库保持在 0031未连库、未执行 migration。
- **08-13 / 0.65.2 / 用户消息结构化附件 + 对话内图片预览**:新增 0031 `messages.attachment_refs`,新客户端将附件作为结构化字段发送,数据库正文仅保留用户自然语言;后端按 task working_dir 校验路径并在内存模型上下文中补附件提示,旧客户端与历史正文标记继续兼容。用户消息即时态和历史态统一展示附件 chip图片额外显示可点击缩略图相关 Python 34 项、全部前端 Node 26 项、Python/JavaScript 语法、Alembic 单 head 与 diff 检查通过,未连接或迁移生产 DB。
### 2026-08-12
@ -341,7 +343,7 @@ core/llm_transport.py 438 ← wire 层健壮性:畸形/吐空检测+留
core/tool_registry.py 264 ← 声明式工具注册表((组名,gate,factory);secret/host 工具按实际能力 gate)
core/context.py 95 ← LLM 调用前压缩旧 tool / load_skill 消息(带压力门槛),保 tool_call 协议字段
core/external_systems/*.py ← 外部系统目录/用户授权/凭据加密 + 通用 OpenAPI/MCP connector
core/compute_nodes.py ← Windows Node 注册码、身份认证与运行状态
core/software_nodes.py ← Windows Node 注册码、身份认证与运行状态
core/sinks.py 101
core/paths.py 50 ← task_dir db form 归一
core/probe.py 243
@ -363,7 +365,7 @@ tools/{base,output,fs,shell,run_python,skill_tool,skill_authoring,media_common,s
main.py ~210 ← 入口:web / db / probe / user / sandbox check
db/migrations/versions/ 0001-0030
web/app.py ~210 ← 工厂 + lifespan 编排(07-23 拆分;路由在 routers/,协程在 background 等)
web/routers/*.py ← 含 external_systems 用户连接与 compute_nodes 节点路由
web/routers/*.py ← 含 external_systems 用户连接与 software_nodes 节点路由
web/{background,scheduler_runner,wechat_runner}.py ← lifespan 后台协程按域析出
web/{runs,common,schemas,model_gate,userfiles}.py ← BG worker/共享 helper/请求体/档位门控/路径安全
web/auth.py ~190 ← 邮箱密码 + platform_key → JWT

36
RUN.md
View File

@ -538,6 +538,18 @@ sudo bash deploy/update_bluegreen.sh
> 蓝绿部署机(生产,已 bootstrap)一律用 `sudo bash deploy/update_bluegreen.sh`(见上节)。
> 下面的 `update.sh` 是单实例版:host backend 机 / 内网机 / 未切蓝绿的部署继续用。
开发 push 与正式发布是两个动作:开发期间可以持续 push但不提升
`core/__init__.py::__version__`,也不新增 `CHANGELOG.md` 的数字版本条目。需要提前整理的
用户文案写在 `## Unreleased`,前端不会展示。功能稳定后创建单独的 release commit统一完成
1. 将 `## Unreleased` 改为 `## <版本> — <日期>`,只保留用户可感知的变化;
2. 更新 `core/__init__.py::__version__`
3. 校准 `PROGRESS.md`,并按变更类型同步本文件或 `DESIGN.md`
4. 部署并验证 `/healthz``/v1/changelog`
未完成且不能向线上用户开放的功能应留在功能分支,稳定后再合并生产分支;必须提前部署时,
使用默认关闭且仅向管理员或测试账号开放的 feature flag。
**一把梭(单实例):`deploy/update.sh`**
```bash
@ -1042,14 +1054,16 @@ sudo xfs_quota -x -c "limit -p bhard=10g zcbot_<user_uuid>" /opt
### Windows Node 内网 MVP开发中
先执行 `alembic upgrade head` 创建 `compute_node_enrollments` 和 `compute_nodes`。不要在未确认目标数据库时运行迁移;本机 `.env``ZCBOT_DB_URL` 可能是生产隧道。
先执行 `alembic upgrade head` 创建 `software_node_enrollments`、`software_nodes` 和 `software_jobs`。不要在未确认目标数据库时运行迁移;本机 `.env``ZCBOT_DB_URL` 可能是生产隧道。
云端当前提供:
- 管理员 `POST /v1/admin/compute-node-enrollments` 创建一次性注册码;
- Node `POST /v1/compute/nodes/enroll` 注册并一次性取得 `node_id`、`node_token`
- Node 携带 `Authorization: Bearer <node_token>``X-Node-Id` 连接 `WS /v1/compute/nodes/connect`
- 管理员 `GET /v1/admin/compute-nodes` 查看节点,`PATCH /v1/admin/compute-nodes/{node_id}` 启停节点,`DELETE /v1/admin/compute-nodes/{node_id}` 永久删除节点身份。
- 管理员 `POST /v1/admin/software-node-enrollments` 创建一次性注册码;
- Node `POST /v1/software-nodes/enroll` 注册并一次性取得 `node_id`、`node_token`
- Node 携带 `Authorization: Bearer <node_token>``X-Node-Id` 连接 `WS /v1/software-nodes/connect`
- 用户或 Agent 通过 `POST /v1/tasks/{task_id}/software-jobs` 提交专业软件任务;
- 用户通过 `GET /v1/software-jobs` 查看本人跨对话任务,可用 `task_id`、`active_only` 和 `limit` 筛选,`POST /v1/software-jobs/{job_id}/cancel` 请求停止;
- 管理员 `GET /v1/admin/software-nodes` 查看节点,`PATCH /v1/admin/software-nodes/{node_id}` 启停节点,`DELETE /v1/admin/software-nodes/{node_id}` 永久删除节点身份。
Node API 只能绑定受控内网地址并由安全组限制来源 IP。当前 HTTP/WS 链路不加密;跨安全域、公网或不可信终端接入前,必须先升级 HTTPS/WSS。多 Web 实例部署时Node API 暂时固定路由到单一实例,直至 Connection Manager 增加跨实例 fencing。
@ -1064,11 +1078,21 @@ windows-node/Zcbot.WindowsNode/bin/Debug/net10.0-windows/Zcbot.WindowsNode.exe e
windows-node/Zcbot.WindowsNode/bin/Debug/net10.0-windows/Zcbot.WindowsNode.exe
```
Origin Worker 使用独立固定 Python 运行时。先确认该交互式 Windows 账号已安装并可启动 Origin/OriginPro再由管理员安装运行时不要复用服务端 `.venv`
```powershell
windows-node\install-origin-runtime.ps1 -BootstrapPython D:\programs\Python312\python.exe
```
默认解释器为 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\python.exe`。如需放在其他受管目录,设置机器级 `ZCBOT_ORIGIN_PYTHON` 为绝对 `python.exe` 路径后重启 Node。运行时固定依赖见 `windows-node/origin-worker/requirements.txt`;任务请求无权选择解释器、脚本或路径。当前 Worker 支持 CSV/XLSX/JSON 输入,`line`、`scatter`、`line_scatter` 与 OPJU/PNG/SVG/PDF 输出。成功产物由 Node 流式上传,全部校验通过后发布到任务工作目录 `origin/<job_id>/`;上传中断会在重连时幂等续传。
Web 用户登录后,右下角 Job 中心会聚合本人最近任务。活动任务约 4 秒刷新一次,空闲时降为约 30 秒停止已派发任务是协作取消状态先显示“正在停止”Node 在线时立即接收断线后在下次连接或心跳时重放。Agent 可调用 `software_capability_list`、`software_job_submit`、`software_job_status` 和 `software_job_cancel`;提交工具只创建固定 schema 的持久任务,不会阻塞当前对话等待完成。
注册配置写入 `%ProgramData%\Zcbot\WindowsNode\node.json`Token 使用 DPAPI `LocalMachine` 加密ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号注册并运行 Node。当前 MVP 以该账号的登录后计划任务启动,不安装 Windows Service。
直接双击 EXE 启动托盘 UI红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开配置窗。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless`
经 nginx 反代时,`/v1/compute/nodes/connect` 必须单独透传 WebSocket Upgrade/Connection 头并设置长连接超时,配置见 `deploy/nginx/zcbot.conf.example`。若注册成功后节点持续显示“连接中断,等待重连”,先用 WebSocket 握手检查该路径;返回普通 HTTP 404 通常表示请求落入了清空 `Connection` 头的默认 location。
经 nginx 反代时,`/v1/software-nodes/connect` 必须单独透传 WebSocket Upgrade/Connection 头并设置长连接超时,配置见 `deploy/nginx/zcbot.conf.example`。若注册成功后节点持续显示“连接中断,等待重连”,先用 WebSocket 握手检查该路径;返回普通 HTTP 404 通常表示请求落入了清空 `Connection` 头的默认 location。
- **入口**:`main.py`(`web / db / probe / user`)→ `core/agent_builder.py::build_agent`

View File

@ -59,7 +59,7 @@ def register_published_artifacts(
continue
current_path = _rel(root, path)
label = str(ref.get("label") or "")
media_type = mimetypes.guess_type(path.name)[0]
media_type = str(ref.get("media_type") or "") or mimetypes.guess_type(path.name)[0]
size_bytes = path.stat().st_size
content_sha256 = _hash_file(path)
statement = pg_insert(Artifact).values(

757
core/software_jobs.py Normal file
View File

@ -0,0 +1,757 @@
"""专业软件任务的校验、幂等持久化和 offer 状态机。"""
from __future__ import annotations
import json
import re
from datetime import datetime, timedelta, timezone
from hashlib import sha256
from uuid import UUID, uuid4
from sqlalchemy import desc, select
from sqlalchemy.exc import IntegrityError
from core.software_nodes import SUPPORTED_CAPABILITIES
from core.storage.engine import session_scope
from core.storage.models import Artifact, SoftwareJob, SoftwareNode, Task
OFFER_SECONDS = 60
ALLOWED_PLOT_TYPES = frozenset(
{"line", "scatter", "line_scatter"}
)
ALLOWED_OUTPUT_FORMATS = frozenset({"opju", "png", "svg", "pdf"})
ALLOWED_INPUT_SUFFIXES = frozenset({".csv", ".xlsx", ".json"})
MAX_INPUT_BYTES = 100 * 1024 * 1024
MAX_OUTPUT_ARTIFACT_BYTES = 256 * 1024 * 1024
class SoftwareJobError(Exception):
pass
MAX_OUTPUT_TOTAL_BYTES = 512 * 1024 * 1024
OUTPUT_ARTIFACTS = {
"project": ("project.opju", "application/x-origin-project", "opju"),
"figure_png": ("figure.png", "image/png", "png"),
"figure_svg": ("figure.svg", "image/svg+xml", "svg"),
"figure_pdf": ("figure.pdf", "application/pdf", "pdf"),
"plot_spec": ("plot-spec.json", "application/json", None),
"provenance": ("provenance.json", "application/json", None),
}
def _has_only(value: dict, fields: set[str]) -> bool:
return set(value).issubset(fields)
def _canonical_request(request: dict) -> tuple[dict, str]:
if not isinstance(request, dict) or set(request) != {"schema_version", "input", "plot", "output"}:
raise SoftwareJobError("invalid origin plot request fields")
if request.get("schema_version") != 1:
raise SoftwareJobError("unsupported origin plot schema version")
input_spec = request.get("input")
plot = request.get("plot")
output = request.get("output")
if not all(isinstance(item, dict) for item in (input_spec, plot, output)):
raise SoftwareJobError("origin plot request sections must be objects")
if not _has_only(input_spec, {"input_id", "sheet"}):
raise SoftwareJobError("unsupported origin input fields")
try:
UUID(str(input_spec.get("input_id") or ""))
except ValueError as exc:
raise SoftwareJobError("input.input_id must be an artifact UUID") from exc
if "sheet" in input_spec and (
not isinstance(input_spec["sheet"], str) or not 1 <= len(input_spec["sheet"]) <= 128
):
raise SoftwareJobError("input.sheet must be a string")
if not _has_only(
plot,
{"type", "x", "y", "template", "title", "x_axis", "y_axis", "legend", "error_bars"},
):
raise SoftwareJobError("unsupported origin plot fields")
if plot.get("type") not in ALLOWED_PLOT_TYPES:
raise SoftwareJobError("unsupported origin plot type")
if "title" in plot and (
not isinstance(plot["title"], str) or len(plot["title"]) > 500
):
raise SoftwareJobError("plot.title must be a string")
if plot.get("template", "publication_double_column") != "publication_double_column":
raise SoftwareJobError("unsupported origin plot template")
x_column = plot.get("x")
y_columns = plot.get("y")
if not isinstance(x_column, str) or not 1 <= len(x_column) <= 128:
raise SoftwareJobError("plot.x must be a column name")
if isinstance(y_columns, str):
y_columns = [y_columns]
if (
not isinstance(y_columns, list)
or not 1 <= len(y_columns) <= 16
or len(y_columns) != len(set(y_columns))
or any(not isinstance(item, str) or not 1 <= len(item) <= 128 for item in y_columns)
):
raise SoftwareJobError("plot.y must contain 1 to 16 unique column names")
for axis_name in ("x_axis", "y_axis"):
axis = plot.get(axis_name)
if axis is not None and (
not isinstance(axis, dict)
or not _has_only(axis, {"title", "unit", "scale"})
or axis.get("scale", "linear") != "linear"
or any(
name in axis and not isinstance(axis[name], str)
for name in ("title", "unit")
)
):
raise SoftwareJobError(f"invalid {axis_name}")
legend = plot.get("legend")
if legend is not None and (
not isinstance(legend, dict)
or not _has_only(legend, {"enabled", "position"})
or ("enabled" in legend and not isinstance(legend["enabled"], bool))
or legend.get("enabled", True) is not True
or legend.get("position", "top_right") != "top_right"
):
raise SoftwareJobError("invalid plot.legend")
if plot.get("error_bars") is not None:
raise SoftwareJobError("error bars are not supported in origin.plot@v1")
if not _has_only(output, {"formats", "dpi", "capture_screenshots", "record_video"}):
raise SoftwareJobError("unsupported origin output fields")
if any(
name in output and not isinstance(output[name], bool)
for name in ("capture_screenshots", "record_video")
):
raise SoftwareJobError("origin output capture flags must be boolean")
if output.get("record_video", False):
raise SoftwareJobError("origin video recording is not supported")
dpi = output.get("dpi", 300)
if not isinstance(dpi, int) or isinstance(dpi, bool) or not 72 <= dpi <= 1200:
raise SoftwareJobError("output.dpi must be between 72 and 1200")
formats = output.get("formats")
if (
not isinstance(formats, list)
or not formats
or len(formats) != len(set(formats))
or any(item not in ALLOWED_OUTPUT_FORMATS for item in formats)
):
raise SoftwareJobError("output.formats contains unsupported values")
encoded = json.dumps(request, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
if len(encoded.encode("utf-8")) > 256 * 1024:
raise SoftwareJobError("origin plot request is too large")
normalized = json.loads(encoded)
return normalized, sha256(encoded.encode("utf-8")).hexdigest()
def _job_dict(row: SoftwareJob) -> dict:
return {
"job_id": str(row.job_id),
"task_id": str(row.task_id),
"capability": row.capability,
"request_digest": row.request_digest,
"node_id": str(row.node_id) if row.node_id else None,
"status": row.status,
"stage": row.stage,
"progress": row.progress,
"metrics": row.metrics,
"error": row.error,
"artifact_manifest": row.artifact_manifest,
"created_at": row.created_at.isoformat() if row.created_at else None,
"started_at": row.started_at.isoformat() if row.started_at else None,
"terminal_at": row.terminal_at.isoformat() if row.terminal_at else None,
}
def list_jobs(
user_id: UUID,
*,
task_id: UUID | None = None,
active_only: bool = False,
limit: int = 50,
) -> list[dict]:
"""列出用户的软件任务;用于全局 Job 中心和 Agent 查询。"""
limit = max(1, min(int(limit), 100))
with session_scope() as session:
statement = (
select(SoftwareJob, Task.name, SoftwareNode.name)
.join(Task, Task.task_id == SoftwareJob.task_id)
.outerjoin(SoftwareNode, SoftwareNode.node_id == SoftwareJob.node_id)
.where(SoftwareJob.user_id == user_id)
)
if task_id is not None:
statement = statement.where(SoftwareJob.task_id == task_id)
if active_only:
statement = statement.where(
SoftwareJob.status.in_(
{
"queued",
"offered",
"dispatched",
"running",
"disconnected",
"cancelling",
}
)
)
rows = session.execute(
statement.order_by(
desc(SoftwareJob.created_at), desc(SoftwareJob.job_id)
).limit(limit)
).all()
results: list[dict] = []
for job, task_name, node_name in rows:
item = _job_dict(job)
item.update(
{
"task_name": task_name,
"node_name": node_name,
"input": job.input_manifest,
"request_summary": _request_summary(job),
}
)
results.append(item)
return results
def _request_summary(job: SoftwareJob) -> dict:
plot = job.request.get("plot") or {}
output = job.request.get("output") or {}
return {
"display_name": (
"Origin 科研绘图"
if job.capability == "origin.plot@v1"
else job.capability
),
"title": str(plot.get("title") or ""),
"formats": list(output.get("formats") or []),
}
def request_job_cancel(user_id: UUID, job_id: UUID) -> tuple[dict, dict | None]:
"""持久化取消意图queued 直接终止,已分派任务返回 Node 消息。"""
now = datetime.now(timezone.utc)
with session_scope() as session:
job = session.execute(
select(SoftwareJob).where(
SoftwareJob.job_id == job_id,
SoftwareJob.user_id == user_id,
).with_for_update()
).scalar_one_or_none()
if job is None:
raise SoftwareJobError("job not found")
if job.status in {"succeeded", "failed", "cancelled"}:
return _job_dict(job), None
if job.status == "queued" or job.node_id is None or job.lease_id is None:
job.status = "cancelled"
job.stage = "terminal"
job.error = {"code": "USER_CANCELLED", "detail": "Cancelled before dispatch."}
job.terminal_at = now
return _job_dict(job), None
job.status = "cancelling"
job.stage = "cancel_requested"
payload = {
"job_id": str(job.job_id),
"lease_id": str(job.lease_id),
"request_digest": job.request_digest,
}
return _job_dict(job), {"node_id": job.node_id, "payload": payload}
def pending_node_cancellations(node_id: UUID) -> list[dict]:
"""节点重连或心跳时重放未确认的取消意图。"""
with session_scope() as session:
rows = session.execute(
select(SoftwareJob).where(
SoftwareJob.node_id == node_id,
SoftwareJob.status == "cancelling",
)
).scalars()
return [
{
"job_id": str(job.job_id),
"lease_id": str(job.lease_id),
"request_digest": job.request_digest,
}
for job in rows
if job.lease_id is not None
]
def create_job(
user_id: UUID,
task_id: UUID,
*,
idempotency_key: str,
capability: str,
request: dict,
) -> tuple[dict, bool]:
key = idempotency_key.strip()
if not key or len(key) > 200:
raise SoftwareJobError("idempotency_key must contain 1 to 200 characters")
if capability not in SUPPORTED_CAPABILITIES:
raise SoftwareJobError("unsupported capability")
normalized, digest = _canonical_request(request)
with session_scope() as session:
task = session.execute(
select(Task.task_id).where(Task.task_id == task_id, Task.user_id == user_id)
).first()
if task is None:
raise SoftwareJobError("task not found")
artifact_id = UUID(normalized["input"]["input_id"])
artifact = session.execute(
select(Artifact).where(
Artifact.artifact_id == artifact_id,
Artifact.user_id == user_id,
Artifact.status == "active",
)
).scalar_one_or_none()
if artifact is None:
raise SoftwareJobError("input artifact not found")
suffix = "." + artifact.current_path.rsplit(".", 1)[-1].lower() if "." in artifact.current_path else ""
if suffix not in ALLOWED_INPUT_SUFFIXES:
raise SoftwareJobError("input artifact type is not supported")
if (
artifact.size_bytes is None
or artifact.size_bytes < 0
or artifact.size_bytes > MAX_INPUT_BYTES
or not artifact.content_sha256
or len(artifact.content_sha256) != 64
):
raise SoftwareJobError("input artifact metadata is incomplete or too large")
input_manifest = {
"artifact_id": str(artifact.artifact_id),
"filename": artifact.current_path.replace("\\", "/").rsplit("/", 1)[-1],
"size_bytes": artifact.size_bytes,
"sha256": artifact.content_sha256,
}
existing = session.execute(
select(SoftwareJob).where(
SoftwareJob.user_id == user_id,
SoftwareJob.idempotency_key == key,
)
).scalar_one_or_none()
if existing is not None:
if (
existing.task_id != task_id
or existing.capability != capability
or existing.request_digest != digest
):
raise SoftwareJobError("idempotency key was already used for a different request")
return _job_dict(existing), False
row = SoftwareJob(
job_id=uuid4(),
user_id=user_id,
task_id=task_id,
idempotency_key=key,
capability=capability,
request=normalized,
request_digest=digest,
input_manifest=input_manifest,
status="queued",
stage="",
metrics={},
error={},
artifact_manifest=[],
)
try:
with session.begin_nested():
session.add(row)
session.flush()
return _job_dict(row), True
except IntegrityError:
existing = session.execute(
select(SoftwareJob).where(
SoftwareJob.user_id == user_id,
SoftwareJob.idempotency_key == key,
)
).scalar_one()
if (
existing.task_id != task_id
or existing.capability != capability
or existing.request_digest != digest
):
raise SoftwareJobError(
"idempotency key was already used for a different request"
)
return _job_dict(existing), False
def get_job(user_id: UUID, job_id: UUID) -> dict | None:
with session_scope() as session:
row = session.execute(
select(SoftwareJob).where(SoftwareJob.job_id == job_id, SoftwareJob.user_id == user_id)
).scalar_one_or_none()
return _job_dict(row) if row else None
def offer_next_job(node_ids: set[UUID]) -> dict | None:
"""从当前进程实际在线的节点中选择一个,为最早 queued job 创建短租约。"""
if not node_ids:
return None
now = datetime.now(timezone.utc)
with session_scope() as session:
expired = session.execute(
select(SoftwareJob)
.where(
SoftwareJob.status == "offered",
SoftwareJob.lease_expires_at <= now,
)
.with_for_update(skip_locked=True)
).scalars()
for item in expired:
item.status = "queued"
item.node_id = None
item.lease_id = None
item.lease_expires_at = None
job = session.execute(
select(SoftwareJob)
.where(SoftwareJob.status == "queued")
.order_by(SoftwareJob.created_at, SoftwareJob.job_id)
.with_for_update(skip_locked=True)
.limit(1)
).scalar_one_or_none()
if job is None:
return None
busy_node_ids = set(
session.execute(
select(SoftwareJob.node_id).where(
SoftwareJob.node_id.is_not(None),
SoftwareJob.status.in_({"offered", "dispatched", "running"}),
)
).scalars()
)
nodes = session.execute(
select(SoftwareNode)
.where(SoftwareNode.node_id.in_(node_ids), SoftwareNode.status == "online")
.order_by(SoftwareNode.last_seen_at.desc())
).scalars()
node = next(
(
item
for item in nodes
if item.node_id not in busy_node_ids
and job.capability in item.capabilities
and int((item.runtime or {}).get("available_slots") or 0) > 0
),
None,
)
if node is None:
return None
lease_id = uuid4()
expires_at = now + timedelta(seconds=OFFER_SECONDS)
job.node_id = node.node_id
job.lease_id = lease_id
job.lease_expires_at = expires_at
job.status = "offered"
return {
"node_id": node.node_id,
"payload": {
"job_id": str(job.job_id),
"lease_id": str(lease_id),
"lease_expires_at": expires_at.isoformat(),
"capability": job.capability,
"request_digest": job.request_digest,
"request": job.request,
"input_transfer": {
**job.input_manifest,
"download_path": f"/v1/software-jobs/{job.job_id}/input",
},
},
}
def get_job_input(node_id: UUID, job_id: UUID) -> dict | None:
"""返回任务绑定的 artifact 定位信息;调用方仍需在 user_root 内安全解析。"""
with session_scope() as session:
job = session.execute(
select(SoftwareJob).where(
SoftwareJob.job_id == job_id,
SoftwareJob.node_id == node_id,
SoftwareJob.status.in_({"offered", "dispatched", "running", "disconnected"}),
)
).scalar_one_or_none()
if job is None:
return None
artifact_id = UUID(job.input_manifest["artifact_id"])
artifact = session.execute(
select(Artifact).where(
Artifact.artifact_id == artifact_id,
Artifact.user_id == job.user_id,
Artifact.status == "active",
)
).scalar_one_or_none()
if artifact is None:
return None
return {
"user_id": job.user_id,
"current_path": artifact.current_path,
**job.input_manifest,
}
def get_job_output_context(node_id: UUID, job_id: UUID, lease_id: UUID, digest: str) -> dict | None:
"""返回 Node 输出上传上下文,不向 Node 暴露任何云端文件路径。"""
with session_scope() as session:
row = session.execute(
select(SoftwareJob, Task.working_dir)
.join(Task, Task.task_id == SoftwareJob.task_id)
.where(SoftwareJob.job_id == job_id)
).one_or_none()
if row is None:
return None
job, working_dir = row
if (
job.node_id != node_id
or job.lease_id != lease_id
or job.request_digest != digest
or job.status not in {"dispatched", "running", "disconnected", "succeeded"}
):
return None
return {
"user_id": job.user_id,
"task_id": job.task_id,
"working_dir": working_dir,
"request": job.request,
"status": job.status,
"artifact_manifest": job.artifact_manifest,
}
def validate_output_manifest(request: dict, manifest: object) -> list[dict]:
if not isinstance(manifest, list):
raise SoftwareJobError("job artifact manifest must be a list")
requested_formats = set(request.get("output", {}).get("formats") or [])
expected_ids = {"plot_spec", "provenance"}
expected_ids.update(
artifact_id
for artifact_id, (_, _, output_format) in OUTPUT_ARTIFACTS.items()
if output_format in requested_formats
)
if len(manifest) != len(expected_ids):
raise SoftwareJobError("job artifact manifest is incomplete")
normalized: list[dict] = []
seen: set[str] = set()
total = 0
for raw in manifest:
if not isinstance(raw, dict) or set(raw) != {
"artifact_id", "filename", "media_type", "size_bytes", "sha256"
}:
raise SoftwareJobError("job artifact manifest entry is invalid")
local_id = raw.get("artifact_id")
if local_id not in expected_ids or local_id in seen:
raise SoftwareJobError("job artifact manifest identity is invalid")
filename, media_type, _ = OUTPUT_ARTIFACTS[local_id]
size = raw.get("size_bytes")
digest = raw.get("sha256")
if raw.get("filename") != filename or raw.get("media_type") != media_type:
raise SoftwareJobError("job artifact manifest metadata does not match its identity")
if not isinstance(size, int) or isinstance(size, bool) or not 1 <= size <= MAX_OUTPUT_ARTIFACT_BYTES:
raise SoftwareJobError("job output artifact size is invalid")
if not isinstance(digest, str) or not re.fullmatch(r"[0-9a-f]{64}", digest):
raise SoftwareJobError("job output artifact digest is invalid")
total += size
seen.add(local_id)
normalized.append(dict(raw))
if seen != expected_ids or total > MAX_OUTPUT_TOTAL_BYTES:
raise SoftwareJobError("job artifact manifest is incomplete or too large")
return normalized
def abandon_offer(node_id: UUID, payload: dict) -> None:
"""WebSocket 发送失败时只回滚仍属于该连接租约的 offer。"""
try:
job_id = UUID(str(payload.get("job_id", "")))
lease_id = UUID(str(payload.get("lease_id", "")))
except ValueError:
return
with session_scope() as session:
job = session.execute(
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
).scalar_one_or_none()
if (
job is not None
and job.status == "offered"
and job.node_id == node_id
and job.lease_id == lease_id
):
job.status = "queued"
job.node_id = None
job.lease_id = None
job.lease_expires_at = None
def respond_to_offer(node_id: UUID, *, accepted: bool, payload: dict) -> None:
try:
job_id = UUID(str(payload.get("job_id", "")))
lease_id = UUID(str(payload.get("lease_id", "")))
except ValueError as exc:
raise SoftwareJobError("invalid job offer response identity") from exc
now = datetime.now(timezone.utc)
with session_scope() as session:
job = session.execute(
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
).scalar_one_or_none()
if job is None or job.node_id != node_id or job.lease_id != lease_id:
raise SoftwareJobError("job offer is stale or does not belong to this node")
if (
accepted
and job.status in {"dispatched", "running", "cancelling", "succeeded", "failed", "cancelled"}
and payload.get("request_digest") == job.request_digest
):
return
if job.status != "offered":
raise SoftwareJobError("job offer is stale or does not belong to this node")
if job.lease_expires_at is None or job.lease_expires_at <= now:
job.status = "queued"
job.node_id = None
job.lease_id = None
job.lease_expires_at = None
raise SoftwareJobError("job offer has expired")
if accepted:
if payload.get("request_digest") != job.request_digest:
raise SoftwareJobError("job request digest mismatch")
job.status = "dispatched"
job.stage = "accepted"
job.error = {}
else:
job.status = "queued"
job.node_id = None
job.lease_id = None
job.lease_expires_at = None
job.error = {"code": "node_rejected", "detail": str(payload.get("reason") or "")[:500]}
def update_job_state(node_id: UUID, payload: dict) -> None:
job_id, lease_id, digest = _message_identity(payload)
stage = str(payload.get("stage") or "")
progress = payload.get("progress")
metrics = payload.get("metrics") or {}
if not stage or len(stage) > 100:
raise SoftwareJobError("job stage is required")
if not isinstance(progress, int) or isinstance(progress, bool) or not 0 <= progress <= 100:
raise SoftwareJobError("job progress must be between 0 and 100")
if not isinstance(metrics, dict) or len(json.dumps(metrics, ensure_ascii=False)) > 64 * 1024:
raise SoftwareJobError("job metrics are invalid")
now = datetime.now(timezone.utc)
with session_scope() as session:
job = session.execute(
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
).scalar_one_or_none()
_assert_job_message(job, node_id, lease_id, digest)
if job.status in {"succeeded", "failed", "cancelled", "cancelling"}:
return
if not _can_accept_state(job.status):
raise SoftwareJobError("job state cannot advance from its current status")
job.status = (
"dispatched"
if stage in {"accepted", "waiting_input", "ready_to_run"}
else "running"
)
job.stage = stage
job.progress = progress
job.metrics = metrics
if job.status == "running" and job.started_at is None:
job.started_at = now
def record_job_terminal(node_id: UUID, payload: dict) -> None:
job_id, lease_id, digest = _message_identity(payload)
terminal_status = payload.get("status")
if terminal_status not in {"succeeded", "failed", "cancelled"}:
raise SoftwareJobError("invalid job terminal status")
error = payload.get("error") or {}
manifest = payload.get("artifact_manifest") or []
if not isinstance(error, dict) or len(json.dumps(error, ensure_ascii=False)) > 64 * 1024:
raise SoftwareJobError("job terminal error is invalid")
if not isinstance(manifest, list) or len(json.dumps(manifest, ensure_ascii=False)) > 256 * 1024:
raise SoftwareJobError("job artifact manifest is invalid")
now = datetime.now(timezone.utc)
with session_scope() as session:
job = session.execute(
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
).scalar_one_or_none()
_assert_job_message(job, node_id, lease_id, digest)
if terminal_status == "succeeded":
expected = validate_output_manifest(
job.request,
[
{
"artifact_id": item.get("source_artifact_id"),
"filename": item.get("filename"),
"media_type": item.get("media_type"),
"size_bytes": item.get("size_bytes"),
"sha256": item.get("sha256"),
}
for item in manifest
if isinstance(item, dict)
],
)
if len(expected) != len(manifest) or any(
not isinstance(item.get("artifact_id"), str)
or not _is_uuid(item["artifact_id"])
or not isinstance(item.get("path"), str)
or not item["path"].startswith(f"origin/{job.job_id}/")
for item in manifest
):
raise SoftwareJobError("successful job artifacts have not been published")
if job.status in {"succeeded", "failed", "cancelled"}:
if job.status != terminal_status:
raise SoftwareJobError("job terminal status conflicts with existing terminal")
return
if job.status not in {"offered", "dispatched", "running", "disconnected", "cancelling"}:
raise SoftwareJobError("job terminal cannot advance from its current status")
job.status = terminal_status
job.stage = "terminal"
job.progress = 100 if terminal_status == "succeeded" else job.progress
job.error = error
job.artifact_manifest = manifest
job.terminal_at = now
def _is_uuid(value: str) -> bool:
try:
UUID(value)
except ValueError:
return False
return True
def mark_node_jobs_disconnected(node_id: UUID) -> None:
"""连接丢失后保留 Node 归属和 lease禁止任务被自动重派。"""
with session_scope() as session:
jobs = session.execute(
select(SoftwareJob)
.where(
SoftwareJob.node_id == node_id,
SoftwareJob.status.in_({"dispatched", "running"}),
)
.with_for_update()
).scalars()
for job in jobs:
job.status = "disconnected"
def _can_accept_state(status: str) -> bool:
return status in {"offered", "dispatched", "running", "disconnected"}
def _message_identity(payload: dict) -> tuple[UUID, UUID, str]:
try:
job_id = UUID(str(payload.get("job_id", "")))
lease_id = UUID(str(payload.get("lease_id", "")))
except ValueError as exc:
raise SoftwareJobError("invalid job message identity") from exc
digest = str(payload.get("request_digest") or "")
if len(digest) != 64:
raise SoftwareJobError("invalid job request digest")
return job_id, lease_id, digest
def _assert_job_message(
job: SoftwareJob | None,
node_id: UUID,
lease_id: UUID,
digest: str,
) -> None:
if (
job is None
or job.node_id != node_id
or job.lease_id != lease_id
or job.request_digest != digest
):
raise SoftwareJobError("job message does not belong to this node or lease")

View File

@ -11,13 +11,13 @@ import bcrypt
from sqlalchemy import select
from core.storage.engine import session_scope
from core.storage.models import ComputeNode, ComputeNodeEnrollment
from core.storage.models import SoftwareNode, SoftwareNodeEnrollment
SUPPORTED_CAPABILITIES = frozenset({"origin.plot@v1"})
MAX_ENROLLMENT_FAILURES = 5
class ComputeNodeError(Exception):
class SoftwareNodeError(Exception):
pass
@ -46,12 +46,12 @@ def create_enrollment(
) -> dict:
allowed = list(dict.fromkeys(capabilities or ["origin.plot@v1"]))
if not allowed or any(item not in SUPPORTED_CAPABILITIES for item in allowed):
raise ComputeNodeError("unsupported capability")
raise SoftwareNodeError("unsupported capability")
if not 60 <= ttl_seconds <= 3600:
raise ComputeNodeError("ttl_seconds must be between 60 and 3600")
raise SoftwareNodeError("ttl_seconds must be between 60 and 3600")
code = "ZCN-" + secrets.token_hex(16).upper()
expires_at = datetime.now(timezone.utc) + timedelta(seconds=ttl_seconds)
row = ComputeNodeEnrollment(
row = SoftwareNodeEnrollment(
enrollment_id=uuid4(),
code_hash=_enrollment_digest(code),
expected_name=(expected_name or "").strip() or None,
@ -80,22 +80,22 @@ def enroll_node(
name = node_name.strip()
requested = list(dict.fromkeys(capabilities))
if not name or not requested:
raise ComputeNodeError("node_name and capabilities are required")
raise SoftwareNodeError("node_name and capabilities are required")
now = datetime.now(timezone.utc)
error: str | None = None
with session_scope() as session:
enrollment = session.execute(
select(ComputeNodeEnrollment)
select(SoftwareNodeEnrollment)
.where(
ComputeNodeEnrollment.code_hash == _enrollment_digest(enrollment_code),
ComputeNodeEnrollment.consumed_at.is_(None),
ComputeNodeEnrollment.expires_at > now,
ComputeNodeEnrollment.failed_attempts < MAX_ENROLLMENT_FAILURES,
SoftwareNodeEnrollment.code_hash == _enrollment_digest(enrollment_code),
SoftwareNodeEnrollment.consumed_at.is_(None),
SoftwareNodeEnrollment.expires_at > now,
SoftwareNodeEnrollment.failed_attempts < MAX_ENROLLMENT_FAILURES,
)
.with_for_update()
).scalar_one_or_none()
if enrollment is None:
raise ComputeNodeError("invalid or expired enrollment code")
raise SoftwareNodeError("invalid or expired enrollment code")
enrollment.failed_attempts += 1
if enrollment.expected_name and enrollment.expected_name != name:
error = "node name does not match enrollment"
@ -103,7 +103,7 @@ def enroll_node(
error = "capability is not allowed by enrollment"
else:
existing = session.execute(
select(ComputeNode.node_id).where(ComputeNode.install_id == install_id)
select(SoftwareNode.node_id).where(SoftwareNode.install_id == install_id)
).first()
if existing is not None:
error = "install is already enrolled"
@ -111,7 +111,7 @@ def enroll_node(
token = secrets.token_urlsafe(48)
node_id = uuid4()
session.add(
ComputeNode(
SoftwareNode(
node_id=node_id,
name=name,
install_id=install_id,
@ -125,7 +125,7 @@ def enroll_node(
)
enrollment.consumed_at = now
if error is not None:
raise ComputeNodeError(error)
raise SoftwareNodeError(error)
return {
"node_id": str(node_id),
"node_token": token,
@ -136,13 +136,13 @@ def enroll_node(
def authenticate_node(node_id: UUID, token: str) -> dict:
with session_scope() as session:
node = session.get(ComputeNode, node_id)
node = session.get(SoftwareNode, node_id)
if (
node is None
or node.status == "disabled"
or not _verify_secret(token, node.token_hash)
):
raise ComputeNodeError("invalid node credentials")
raise SoftwareNodeError("invalid node credentials")
return {
"node_id": node.node_id,
"install_id": node.install_id,
@ -152,9 +152,9 @@ def authenticate_node(node_id: UUID, token: str) -> dict:
def update_node_runtime(node_id: UUID, *, status: str, runtime: dict) -> None:
with session_scope() as session:
node = session.get(ComputeNode, node_id)
node = session.get(SoftwareNode, node_id)
if node is None or node.status == "disabled":
raise ComputeNodeError("node is disabled or missing")
raise SoftwareNodeError("node is disabled or missing")
node.status = status
node.runtime = runtime
node.last_seen_at = datetime.now(timezone.utc)
@ -163,14 +163,14 @@ def update_node_runtime(node_id: UUID, *, status: str, runtime: dict) -> None:
def mark_node_offline(node_id: UUID) -> None:
"""仅把活动节点转离线;管理员禁用态不可被断线收尾覆盖。"""
with session_scope() as session:
node = session.get(ComputeNode, node_id)
node = session.get(SoftwareNode, node_id)
if node is not None and node.status != "disabled":
node.status = "offline"
def set_node_disabled(node_id: UUID, disabled: bool) -> bool:
with session_scope() as session:
node = session.get(ComputeNode, node_id)
node = session.get(SoftwareNode, node_id)
if node is None:
return False
node.status = "disabled" if disabled else "offline"
@ -180,7 +180,7 @@ def set_node_disabled(node_id: UUID, disabled: bool) -> bool:
def delete_node(node_id: UUID) -> bool:
"""撤销并物理删除节点身份;当前节点表没有任务历史外键。"""
with session_scope() as session:
node = session.get(ComputeNode, node_id)
node = session.get(SoftwareNode, node_id)
if node is None:
return False
session.delete(node)
@ -190,7 +190,7 @@ def delete_node(node_id: UUID) -> bool:
def list_nodes() -> list[dict]:
with session_scope() as session:
rows = (
session.execute(select(ComputeNode).order_by(ComputeNode.created_at))
session.execute(select(SoftwareNode).order_by(SoftwareNode.created_at))
.scalars()
.all()
)

View File

@ -424,10 +424,10 @@ class ChannelBinding(Base):
)
class ComputeNodeEnrollment(Base):
class SoftwareNodeEnrollment(Base):
"""Windows Node 一次性注册码;数据库只保存不可逆摘要。"""
__tablename__ = "compute_node_enrollments"
__tablename__ = "software_node_enrollments"
enrollment_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
code_hash: Mapped[str] = mapped_column(Text, nullable=False, unique=True)
expected_name: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
@ -443,10 +443,10 @@ class ComputeNodeEnrollment(Base):
)
class ComputeNode(Base):
class SoftwareNode(Base):
"""平台托管的 Windows 执行节点身份与最后一次运行态。"""
__tablename__ = "compute_nodes"
__tablename__ = "software_nodes"
node_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
name: Mapped[str] = mapped_column(Text, nullable=False)
install_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), nullable=False, unique=True)
@ -465,6 +465,49 @@ class ComputeNode(Base):
)
class SoftwareJob(Base):
"""专业软件任务账本;请求只保存规范化参数和输入引用。"""
__tablename__ = "software_jobs"
__table_args__ = (
UniqueConstraint("user_id", "idempotency_key", name="uq_software_jobs_user_idempotency"),
Index("ix_software_jobs_status_created", "status", "created_at"),
Index("ix_software_jobs_node_status", "node_id", "status"),
)
job_id: Mapped[UUID] = mapped_column(PG_UUID(as_uuid=True), primary_key=True, default=uuid4)
user_id: Mapped[UUID] = mapped_column(
PG_UUID(as_uuid=True), ForeignKey("users.user_id", ondelete="CASCADE"), nullable=False
)
task_id: Mapped[UUID] = mapped_column(
PG_UUID(as_uuid=True), ForeignKey("tasks.task_id", ondelete="CASCADE"), nullable=False
)
idempotency_key: Mapped[str] = mapped_column(Text, nullable=False)
capability: Mapped[str] = mapped_column(Text, nullable=False)
request: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
request_digest: Mapped[str] = mapped_column(Text, nullable=False)
input_manifest: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
node_id: Mapped[Optional[UUID]] = mapped_column(
PG_UUID(as_uuid=True), ForeignKey("software_nodes.node_id", ondelete="SET NULL"), nullable=True
)
lease_id: Mapped[Optional[UUID]] = mapped_column(PG_UUID(as_uuid=True), nullable=True)
lease_expires_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
status: Mapped[str] = mapped_column(Text, nullable=False, default="queued", server_default="queued")
stage: Mapped[str] = mapped_column(Text, nullable=False, default="", server_default="")
progress: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
metrics: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
error: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
artifact_manifest: Mapped[list[Any]] = mapped_column(JSONB, nullable=False, default=list)
started_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
terminal_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), onupdate=func.now(), nullable=False
)
class ExternalSystemDefinition(Base):
"""管理员维护的可信外部系统目录;不含任何用户凭据。"""

View File

@ -47,6 +47,12 @@ from tools.run_python import RunPythonTool
from tools.schedule import (
ScheduleCancelTool, ScheduleCreateTool, ScheduleListTool, ScheduleUpdateTool,
)
from tools.software_jobs import (
SoftwareCapabilityListTool,
SoftwareJobCancelTool,
SoftwareJobStatusTool,
SoftwareJobSubmitTool,
)
from tools.seedance import SeedanceTool
from tools.seedream import SeedreamTool
from tools.send_email import SendEmailTool, smtp_configured
@ -207,6 +213,14 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
ScheduleCancelTool(ctx.uid, **base),
]
def _software_jobs() -> list:
return [
SoftwareCapabilityListTool(ctx.uid, ctx.task_id, **base),
SoftwareJobSubmitTool(ctx.uid, ctx.task_id, **base),
SoftwareJobStatusTool(ctx.uid, ctx.task_id, **base),
SoftwareJobCancelTool(ctx.uid, ctx.task_id, **base),
]
def _send_email() -> list:
# base_dir 用 working_dir_path(宿主 task 目录):send_email 在宿主进程读附件,
# docker 下 agent 给的相对路径相对容器 workdir=task_dir,翻回宿主即 working_dir_path;
@ -294,6 +308,7 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
("skill_authoring", lambda: True, _skill_authoring),
# 定时 run 内不挂 schedule_*(防任务造任务自我繁殖);仅交互对话可建/管 job。
("schedules", lambda: not ctx.scheduled_run, _schedules),
("software_jobs", lambda: not ctx.scheduled_run, _software_jobs),
("send_email", smtp_configured, _send_email),
("wechat_push", wechat_push_available, _wechat_push),
("run_python", lambda: ctx.caps.enable_run_python, _run_python),

View File

@ -0,0 +1,102 @@
"""Rename software nodes and add the professional software job ledger.
Revision ID: 0032
Revises: 0031
Create Date: 2026-08-13
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0032"
down_revision: str | None = "0031"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.rename_table("compute_node_enrollments", "software_node_enrollments")
op.rename_table("compute_nodes", "software_nodes")
op.execute(
"ALTER INDEX ix_compute_nodes_status RENAME TO ix_software_nodes_status"
)
op.execute(
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
"compute_node_enrollments_pkey TO software_node_enrollments_pkey"
)
op.execute(
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
"compute_node_enrollments_code_hash_key TO software_node_enrollments_code_hash_key"
)
op.execute(
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
"compute_node_enrollments_created_by_fkey TO software_node_enrollments_created_by_fkey"
)
op.execute(
"ALTER TABLE software_nodes RENAME CONSTRAINT "
"compute_nodes_pkey TO software_nodes_pkey"
)
op.execute(
"ALTER TABLE software_nodes RENAME CONSTRAINT "
"compute_nodes_install_id_key TO software_nodes_install_id_key"
)
op.create_table(
"software_jobs",
sa.Column("job_id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.user_id", ondelete="CASCADE"), nullable=False),
sa.Column("task_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("tasks.task_id", ondelete="CASCADE"), nullable=False),
sa.Column("idempotency_key", sa.Text(), nullable=False),
sa.Column("capability", sa.Text(), nullable=False),
sa.Column("request", postgresql.JSONB(), nullable=False),
sa.Column("request_digest", sa.Text(), nullable=False),
sa.Column("input_manifest", postgresql.JSONB(), nullable=False),
sa.Column("node_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("software_nodes.node_id", ondelete="SET NULL"), nullable=True),
sa.Column("lease_id", postgresql.UUID(as_uuid=True), nullable=True),
sa.Column("lease_expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("status", sa.Text(), server_default="queued", nullable=False),
sa.Column("stage", sa.Text(), server_default="", nullable=False),
sa.Column("progress", sa.Integer(), server_default="0", nullable=False),
sa.Column("metrics", postgresql.JSONB(), nullable=False),
sa.Column("error", postgresql.JSONB(), nullable=False),
sa.Column("artifact_manifest", postgresql.JSONB(), nullable=False),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("terminal_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.UniqueConstraint("user_id", "idempotency_key", name="uq_software_jobs_user_idempotency"),
)
op.create_index("ix_software_jobs_status_created", "software_jobs", ["status", "created_at"])
op.create_index("ix_software_jobs_node_status", "software_jobs", ["node_id", "status"])
def downgrade() -> None:
op.drop_index("ix_software_jobs_node_status", table_name="software_jobs")
op.drop_index("ix_software_jobs_status_created", table_name="software_jobs")
op.drop_table("software_jobs")
op.execute(
"ALTER TABLE software_nodes RENAME CONSTRAINT "
"software_nodes_install_id_key TO compute_nodes_install_id_key"
)
op.execute(
"ALTER TABLE software_nodes RENAME CONSTRAINT "
"software_nodes_pkey TO compute_nodes_pkey"
)
op.execute(
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
"software_node_enrollments_created_by_fkey TO compute_node_enrollments_created_by_fkey"
)
op.execute(
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
"software_node_enrollments_code_hash_key TO compute_node_enrollments_code_hash_key"
)
op.execute(
"ALTER TABLE software_node_enrollments RENAME CONSTRAINT "
"software_node_enrollments_pkey TO compute_node_enrollments_pkey"
)
op.execute(
"ALTER INDEX ix_software_nodes_status RENAME TO ix_compute_nodes_status"
)
op.rename_table("software_nodes", "compute_nodes")
op.rename_table("software_node_enrollments", "compute_node_enrollments")

View File

@ -60,7 +60,7 @@ server {
# ★ Windows Node 长连接:注册走普通 POST,注册后的节点控制通道走 WebSocket。
# 必须独立于下面会清空 Connection 头的默认 location15s 应用心跳保持链路活跃。
location = /v1/compute/nodes/connect {
location = /v1/software-nodes/connect {
proxy_pass http://zcbot_backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;

View File

@ -34,7 +34,7 @@ Windows Node 不是完整的本地 zcbot
### 2.1 目标
- 云端 zcbot 可以发现节点能力、容量、软件版本和在线状态。
- 用户可以提交、查询、取消长时间 Windows 计算任务。
- 用户可以提交、查询、取消长时间运行的专业软件任务。
- 网络中断、云端重启或节点重启后,任务可以确定性对账。
- 节点可以上报阶段、进度、结构化指标、日志摘要和事件截图。
- 中间产物和最终产物支持校验、断点上传与按需导入工作目录。
@ -79,7 +79,7 @@ flowchart LR
|---|---|
| `NodeRegistry` | 节点注册、证书指纹、启停、能力和管理员标签 |
| `NodeConnectionManager` | WSS 连接、心跳、消息 ACK、同节点单活连接 |
| `ComputeJobService` | 用户授权、幂等提交、节点选择、租约、取消和终态 |
| `SoftwareJobService` | 用户授权、幂等提交、节点选择、租约、取消和终态 |
| `ComputeTransferService` | 输入下载凭证、分块上传、SHA-256、容量与保留期 |
| `ComputeBroker` | 把任务事件推送到 Web UI不承担持久化事实源 |
| `ComputeTools` | agent 可调用的能力发现、提交、查询、取消、产物导入工具 |
@ -157,7 +157,7 @@ sequenceDiagram
participant Node as Windows Node
Admin->>Cloud: 创建一次性 enrollment token
Node->>Node: 生成设备密钥对
Node->>Cloud: POST /v1/compute/nodes/enroll
Node->>Cloud: POST /v1/software-nodes/enroll
Cloud->>Cloud: 消耗 token创建 node_id
Cloud-->>Node: 客户端证书、CA、云端地址
Node->>Node: 私钥写入 Windows Certificate Store
@ -178,7 +178,7 @@ sequenceDiagram
节点连接:
```text
WSS /v1/compute/nodes/connect
WSS /v1/software-nodes/connect
```
统一消息 envelope
@ -252,13 +252,13 @@ Node 在 `hello` 和心跳中声明由本机可信配置生成的能力:
建议新增三张表,不复用外部系统连接表:
```text
compute_nodes(
software_nodes(
node_id pk, name, cert_fingerprint, status,
labels jsonb, capabilities jsonb, config_revision,
last_seen_at, disabled_at, created_at, updated_at
)
compute_jobs(
software_jobs(
job_id pk, user_id fk, task_id fk, tool_call_id,
capability, schema_version, request jsonb,
idempotency_key, request_digest,
@ -268,7 +268,7 @@ compute_jobs(
created_at, started_at, terminal_at, updated_at
)
compute_job_events(
software_job_events(
event_id pk, job_id fk, sequence,
kind, level, payload jsonb, created_at
)
@ -352,7 +352,7 @@ Node 不访问整个用户 workspace。云端只为显式引用的文件创建
大产物不通过 WSS 消息传输。Node 使用 HTTPS 分块上传:
```text
POST /v1/compute/jobs/{job_id}/artifacts/upload-session
POST /v1/software-jobs/{job_id}/artifacts/upload-session
PUT /v1/compute/transfers/{transfer_id}/parts/{part_number}
POST /v1/compute/transfers/{transfer_id}/complete
```
@ -364,7 +364,7 @@ POST /v1/compute/transfers/{transfer_id}/complete
Node 上传完成后先进入:
```text
<user_root>/.zcbot_cache/<task_id>/compute_jobs/<job_id>/
<user_root>/.zcbot_cache/<task_id>/software_jobs/<job_id>/
```
该目录默认隐藏且有 TTL。用户或 agent 明确导入后复制到:
@ -379,10 +379,10 @@ Node 上传完成后先进入:
```text
compute_capability_list
compute_job_submit
compute_job_status
compute_job_cancel
compute_job_artifact_import
software_job_submit
software_job_status
software_job_cancel
software_job_artifact_import
```
- capability list 只返回用户有权使用且有健康节点承载的能力;
@ -418,7 +418,7 @@ process.cancel_adapter
- 优先捕获目标窗口;
- 敏感信息上传前遮罩;
- 视频默认关闭,显式启用时建议 1280×720、510 FPS、H.264 分段;
- 采集失败不得使计算任务失败;
- 采集失败不得使专业软件任务失败;
- 模型只按需读取关键帧,不持续消费完整视频。
## 12. Origin 首批适配器

View File

@ -60,7 +60,7 @@ sequenceDiagram
participant N as Windows Node
A->>Z: 创建一次性注册码
A->>N: 输入内网地址和注册码
N->>Z: HTTP POST /v1/compute/nodes/enroll
N->>Z: HTTP POST /v1/software-nodes/enroll
Z->>Z: 校验并原子消费注册码
Z-->>N: node_id + node_token + 配置
N->>N: DPAPI 加密保存 node_token
@ -71,7 +71,7 @@ sequenceDiagram
注册请求:
```http
POST http://zcbot.internal:8765/v1/compute/nodes/enroll
POST http://zcbot.internal:8765/v1/software-nodes/enroll
Content-Type: application/json
```
@ -116,7 +116,7 @@ Content-Type: application/json
### 4.1 WS 连接
```http
GET ws://zcbot.internal:8765/v1/compute/nodes/connect
GET ws://zcbot.internal:8765/v1/software-nodes/connect
Authorization: Bearer <node_token>
X-Node-Id: <node_id>
Upgrade: websocket
@ -170,13 +170,13 @@ RDP不向公网开放使用 VPN、堡垒机或云安全登录
云端首期只增加:
```text
compute_nodes(
software_nodes(
node_id pk, name, install_id, token_hash, status,
capabilities jsonb, last_seen_at,
created_at, updated_at
)
compute_jobs(
software_jobs(
job_id pk, user_id fk, task_id fk,
idempotency_key, capability, request jsonb,
node_id fk, status, progress,
@ -200,6 +200,8 @@ Node 断线且本地任务可能仍在执行时标记 `disconnected`,不得自
## 7. Origin 任务闭环
当前实现进度:云端任务账本、幂等提交、短期 offer、Node 本地原子保存与 accept/reject 已落地。输入以任务绑定的 artifact UUID 下载Node 流式校验大小和 SHA-256 后原子保存。固定 Worker 使用管理员安装的隔离 Python 运行时与随程序发布的 `worker.py` 驱动 Origin生成 OPJU、PNG、SVG、PDF、plot spec、provenance 和原子 `terminal.json`;运行不绑定单次 WebSocket断线后继续执行。同一进程按 job 去重Node 重启后不重复启动已留启动标记但无可信终态的任务。成功产物逐项流式上传到云端隐藏暂存区,云端复核任务身份、固定文件名、大小和 SHA-256 后,一次性发布到 `<working_dir>/origin/<job_id>/` 并登记平台 artifact UUIDNode 以 `upload-complete.json` 恢复中断上传。
```text
用户上传 CSV/XLSX
→ zcbot 生成受控 plot spec

View File

@ -8,6 +8,10 @@ SAMPLE = """# 更新日志
> 导语在首个条目前,应被丢弃
## Unreleased
- 尚未发布,不能出现在用户版接口中
## 0.55.0 — 2026-07-09
- 条目 A
@ -36,6 +40,12 @@ class ChangelogParseTests(unittest.TestCase):
def test_preamble_before_first_entry_is_dropped(self) -> None:
es = _parse_changelog(SAMPLE)
self.assertNotIn("导语", es[0]["body_md"])
self.assertNotIn("尚未发布", es[0]["body_md"])
def test_unreleased_section_is_not_returned(self) -> None:
es = _parse_changelog(SAMPLE)
self.assertNotIn("Unreleased", [e["version"] for e in es])
self.assertEqual([e["version"] for e in es], ["0.55.0", "0.54.1", "0.53.0"])
def test_real_changelog_file_parses(self) -> None:
# 仓库里的真文件:格式坏了(条目头打错 / 空正文)在 CI 就翻车,不等用户点开才发现

View File

@ -1,109 +0,0 @@
from __future__ import annotations
import importlib
import unittest
from unittest.mock import AsyncMock, patch
from uuid import uuid4
from alembic.migration import MigrationContext
from alembic.operations import Operations
from sqlalchemy import create_mock_engine
from sqlalchemy.dialects import postgresql
from core.compute_nodes import (
_enrollment_digest,
_hash_secret,
_verify_secret,
delete_node,
)
from web.routers.compute_nodes import NodeConnectionManager, _bearer
class ComputeNodeSecurityTests(unittest.TestCase):
def test_secret_hash_is_salted_and_verifiable(self) -> None:
first = _hash_secret("node-secret")
second = _hash_secret("node-secret")
self.assertNotEqual(first, second)
self.assertNotIn("node-secret", first)
self.assertTrue(_verify_secret("node-secret", first))
self.assertFalse(_verify_secret("wrong", first))
def test_bearer_parser_rejects_query_style_or_missing_token(self) -> None:
self.assertEqual(_bearer("Bearer abc"), "abc")
with self.assertRaisesRegex(Exception, "missing node bearer token"):
_bearer(None)
def test_enrollment_digest_does_not_store_plaintext(self) -> None:
digest = _enrollment_digest("ZCN-ABC")
self.assertEqual(len(digest), 64)
self.assertNotIn("ZCN-ABC", digest)
class ComputeNodeConnectionTests(unittest.IsolatedAsyncioTestCase):
async def test_new_connection_replaces_old_without_removing_new(self) -> None:
manager = NodeConnectionManager()
node_id = uuid4()
old = AsyncMock()
new = AsyncMock()
await manager.activate(node_id, old)
await manager.activate(node_id, new)
old.close.assert_awaited_once_with(
code=4001, reason="replaced by a newer connection"
)
self.assertFalse(await manager.remove(node_id, old))
self.assertTrue(await manager.remove(node_id, new))
async def test_admin_close_removes_and_closes_connection(self) -> None:
manager = NodeConnectionManager()
node_id = uuid4()
websocket = AsyncMock()
await manager.activate(node_id, websocket)
await manager.close(node_id)
websocket.close.assert_awaited_once_with(code=4003, reason="node disabled")
self.assertFalse(await manager.remove(node_id, websocket))
class ComputeNodeMigrationTests(unittest.TestCase):
def test_0030_upgrade_compiles_as_postgresql_ddl(self) -> None:
statements: list[str] = []
def capture(sql, *multiparams, **params):
statements.append(str(sql.compile(dialect=postgresql.dialect())))
engine = create_mock_engine("postgresql+psycopg://", capture)
operations = Operations(MigrationContext.configure(engine.connect()))
migration = importlib.import_module(
"db.migrations.versions.20260812_2000_0030_compute_nodes"
)
with patch.object(migration, "op", operations):
migration.upgrade()
rendered = "\n".join(statements)
self.assertIn("compute_node_enrollments", rendered)
self.assertIn("compute_nodes", rendered)
self.assertIn("ix_compute_nodes_status", rendered)
class ComputeNodeDeleteTests(unittest.TestCase):
@patch("core.compute_nodes.session_scope")
def test_delete_node_removes_existing_identity(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
node = object()
session.get.return_value = node
self.assertTrue(delete_node(uuid4()))
session.delete.assert_called_once_with(node)
@patch("core.compute_nodes.session_scope")
def test_delete_node_reports_missing_identity(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
session.get.return_value = None
self.assertFalse(delete_node(uuid4()))
session.delete.assert_not_called()
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,46 @@
from __future__ import annotations
import importlib.util
import json
import tempfile
import unittest
from pathlib import Path
WORKER_PATH = (
Path(__file__).resolve().parents[1] / "windows-node" / "origin-worker" / "worker.py"
)
SPEC = importlib.util.spec_from_file_location("zcbot_origin_worker", WORKER_PATH)
assert SPEC and SPEC.loader
worker = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(worker)
class OriginWorkerUnitTests(unittest.TestCase):
def test_csv_and_json_inputs_are_read_without_origin(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
csv_path = root / "input.csv"
csv_path.write_text("x,y\n1,2\n3,4\n", encoding="utf-8")
self.assertEqual(worker._read_rows(csv_path, None), (["x", "y"], [["1", "2"], ["3", "4"]]))
json_path = root / "input.json"
json_path.write_text(json.dumps([{"x": 1, "y": 2}, {"x": 3, "y": 4}]), encoding="utf-8")
self.assertEqual(worker._read_rows(json_path, None), (["x", "y"], [[1, 2], [3, 4]]))
def test_manifest_uses_stable_id_and_streaming_digest(self) -> None:
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "plot-spec.json"
path.write_text("{}", encoding="utf-8")
manifest = worker._manifest(path, "application/json")
self.assertEqual(manifest["artifact_id"], "plot_spec")
self.assertEqual(manifest["sha256"], worker._file_sha256(path))
self.assertEqual(manifest["size_bytes"], 2)
def test_axis_title_includes_units(self) -> None:
self.assertEqual(worker._axis_title({"title": "Stress", "unit": "MPa"}, "Y"), "Stress (MPa)")
self.assertEqual(worker._axis_title(None, "Time"), "Time")
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,80 @@
from __future__ import annotations
import json
import unittest
from unittest.mock import patch
from uuid import uuid4
from tools.software_jobs import (
SoftwareCapabilityListTool,
SoftwareJobCancelTool,
SoftwareJobStatusTool,
SoftwareJobSubmitTool,
)
class SoftwareJobToolTests(unittest.TestCase):
def setUp(self):
self.user_id = uuid4()
self.task_id = uuid4()
def test_capability_list_reports_current_capacity(self):
nodes = [
{
"status": "online",
"capabilities": ["origin.plot@v1"],
"runtime": {"available_slots": 1},
},
{
"status": "offline",
"capabilities": ["origin.plot@v1"],
"runtime": {"available_slots": 1},
},
]
with patch("tools.software_jobs.list_nodes", return_value=nodes):
result = json.loads(
SoftwareCapabilityListTool(self.user_id, self.task_id).execute()
)
self.assertEqual(result["capabilities"][0]["available_nodes"], 1)
def test_submit_injects_current_user_and_task(self):
created = {"job_id": str(uuid4()), "status": "queued"}
tool = SoftwareJobSubmitTool(self.user_id, self.task_id)
with patch("tools.software_jobs.create_job", return_value=(created, True)) as create:
result = json.loads(tool.execute("origin.plot@v1", {"input_id": str(uuid4())}))
self.assertTrue(result["created"])
self.assertEqual(create.call_args.args[:2], (self.user_id, self.task_id))
self.assertEqual(create.call_args.kwargs["capability"], "origin.plot@v1")
def test_status_and_cancel_reject_cross_task_job(self):
foreign = {"job_id": str(uuid4()), "task_id": str(uuid4())}
with patch("tools.software_jobs.get_job", return_value=foreign):
status = SoftwareJobStatusTool(self.user_id, self.task_id).execute(
foreign["job_id"]
)
cancel = SoftwareJobCancelTool(self.user_id, self.task_id).execute(
foreign["job_id"]
)
self.assertIn("not found", status)
self.assertIn("not found", cancel)
def test_cancel_uses_user_scoped_service(self):
job_id = uuid4()
current = {"job_id": str(job_id), "task_id": str(self.task_id)}
cancelled = {**current, "status": "cancelled"}
with (
patch("tools.software_jobs.get_job", return_value=current),
patch(
"tools.software_jobs.request_job_cancel",
return_value=(cancelled, None),
) as request_cancel,
):
result = json.loads(
SoftwareJobCancelTool(self.user_id, self.task_id).execute(str(job_id))
)
self.assertEqual(result["status"], "cancelled")
request_cancel.assert_called_once_with(self.user_id, job_id)
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,379 @@
from __future__ import annotations
import importlib
import unittest
from pathlib import Path
from unittest.mock import AsyncMock, patch
from uuid import uuid4
from alembic.migration import MigrationContext
from alembic.operations import Operations
from sqlalchemy import create_mock_engine
from sqlalchemy.dialects import postgresql
from core.software_nodes import (
_enrollment_digest,
_hash_secret,
_verify_secret,
delete_node,
)
from core.software_jobs import (
_canonical_request,
abandon_offer,
mark_node_jobs_disconnected,
record_job_terminal,
respond_to_offer,
update_job_state,
validate_output_manifest,
list_jobs,
request_job_cancel,
)
from web.routers.software_nodes import NodeConnectionManager, _bearer
class SoftwareNodeSecurityTests(unittest.TestCase):
def test_secret_hash_is_salted_and_verifiable(self) -> None:
first = _hash_secret("node-secret")
second = _hash_secret("node-secret")
self.assertNotEqual(first, second)
self.assertNotIn("node-secret", first)
self.assertTrue(_verify_secret("node-secret", first))
self.assertFalse(_verify_secret("wrong", first))
def test_bearer_parser_rejects_query_style_or_missing_token(self) -> None:
self.assertEqual(_bearer("Bearer abc"), "abc")
with self.assertRaisesRegex(Exception, "missing node bearer token"):
_bearer(None)
def test_enrollment_digest_does_not_store_plaintext(self) -> None:
digest = _enrollment_digest("ZCN-ABC")
self.assertEqual(len(digest), 64)
self.assertNotIn("ZCN-ABC", digest)
def test_websocket_auth_rejection_uses_explicit_application_close_code(self) -> None:
source = (
Path(__file__).resolve().parents[1] / "web" / "routers" / "software_nodes.py"
).read_text(encoding="utf-8")
rejection = source.split("except (ValueError, SoftwareNodeError):", 1)[1].split(
"await node_connections.activate", 1
)[0]
self.assertLess(
rejection.index("await websocket.accept()"), rejection.index("await websocket.close")
)
self.assertIn('code=4003, reason="invalid node credentials"', rejection)
class SoftwareNodeConnectionTests(unittest.IsolatedAsyncioTestCase):
async def test_new_connection_replaces_old_without_removing_new(self) -> None:
manager = NodeConnectionManager()
node_id = uuid4()
old = AsyncMock()
new = AsyncMock()
await manager.activate(node_id, old)
await manager.activate(node_id, new)
old.close.assert_awaited_once_with(
code=4001, reason="replaced by a newer connection"
)
self.assertFalse(await manager.remove(node_id, old))
self.assertTrue(await manager.remove(node_id, new))
async def test_admin_close_removes_and_closes_connection(self) -> None:
manager = NodeConnectionManager()
node_id = uuid4()
websocket = AsyncMock()
await manager.activate(node_id, websocket)
await manager.close(node_id)
websocket.close.assert_awaited_once_with(code=4003, reason="node disabled")
self.assertFalse(await manager.remove(node_id, websocket))
class SoftwareNodeMigrationTests(unittest.TestCase):
def test_0030_upgrade_compiles_as_postgresql_ddl(self) -> None:
statements: list[str] = []
def capture(sql, *multiparams, **params):
statements.append(str(sql.compile(dialect=postgresql.dialect())))
engine = create_mock_engine("postgresql+psycopg://", capture)
operations = Operations(MigrationContext.configure(engine.connect()))
migration = importlib.import_module(
"db.migrations.versions.20260812_2000_0030_compute_nodes"
)
with patch.object(migration, "op", operations):
migration.upgrade()
rendered = "\n".join(statements)
self.assertIn("compute_node_enrollments", rendered)
self.assertIn("compute_nodes", rendered)
self.assertIn("ix_compute_nodes_status", rendered)
def test_0032_upgrade_compiles_as_postgresql_ddl(self) -> None:
statements: list[str] = []
def capture(sql, *multiparams, **params):
statements.append(str(sql.compile(dialect=postgresql.dialect())))
engine = create_mock_engine("postgresql+psycopg://", capture)
operations = Operations(MigrationContext.configure(engine.connect()))
migration = importlib.import_module(
"db.migrations.versions.20260813_1600_0032_software_jobs"
)
with patch.object(migration, "op", operations):
migration.upgrade()
rendered = "\n".join(statements)
self.assertIn("ALTER TABLE compute_node_enrollments RENAME TO software_node_enrollments", rendered)
self.assertIn("ALTER TABLE compute_nodes RENAME TO software_nodes", rendered)
self.assertIn("software_jobs", rendered)
self.assertIn("uq_software_jobs_user_idempotency", rendered)
self.assertIn("ix_software_jobs_status_created", rendered)
class SoftwareJobProtocolTests(unittest.TestCase):
@patch("core.software_jobs.session_scope")
def test_queued_job_cancels_without_node_message(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
job = type("Job", (), {})()
job.job_id = uuid4(); job.user_id = uuid4(); job.task_id = uuid4()
job.capability = "origin.plot@v1"; job.request_digest = "a" * 64
job.node_id = None; job.lease_id = None; job.status = "queued"; job.stage = ""
job.progress = 0; job.metrics = {}; job.error = {}; job.artifact_manifest = []
job.created_at = None; job.started_at = None; job.terminal_at = None
session.execute.return_value.scalar_one_or_none.return_value = job
result, node_message = request_job_cancel(job.user_id, job.job_id)
self.assertEqual(result["status"], "cancelled")
self.assertIsNone(node_message)
self.assertEqual(job.error["code"], "USER_CANCELLED")
@patch("core.software_jobs.session_scope")
def test_running_job_persists_cancel_before_sending(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
job = type("Job", (), {})()
job.job_id = uuid4(); job.user_id = uuid4(); job.task_id = uuid4()
job.capability = "origin.plot@v1"; job.request_digest = "b" * 64
job.node_id = uuid4(); job.lease_id = uuid4(); job.status = "running"
job.stage = "origin_running"; job.progress = 10; job.metrics = {}; job.error = {}
job.artifact_manifest = []; job.created_at = None; job.started_at = None; job.terminal_at = None
session.execute.return_value.scalar_one_or_none.return_value = job
result, node_message = request_job_cancel(job.user_id, job.job_id)
self.assertEqual(result["status"], "cancelling")
self.assertEqual(node_message["node_id"], job.node_id)
self.assertEqual(node_message["payload"]["lease_id"], str(job.lease_id))
@patch("core.software_jobs.session_scope")
def test_job_list_is_enriched_for_job_center(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
job = type("Job", (), {})()
job.job_id = uuid4(); job.task_id = uuid4(); job.capability = "origin.plot@v1"
job.request_digest = "c" * 64; job.node_id = uuid4(); job.status = "running"
job.stage = "origin_running"; job.progress = 20; job.metrics = {}; job.error = {}
job.artifact_manifest = []; job.input_manifest = {"filename": "input.xlsx"}
job.request = {"plot": {"title": "Test"}, "output": {"formats": ["png"]}}
job.created_at = None; job.started_at = None; job.terminal_at = None
session.execute.return_value.all.return_value = [(job, "材料仿真", "LAB-01")]
results = list_jobs(uuid4(), limit=10)
self.assertEqual(results[0]["task_name"], "材料仿真")
self.assertEqual(results[0]["node_name"], "LAB-01")
self.assertEqual(results[0]["request_summary"]["display_name"], "Origin 科研绘图")
def test_origin_request_is_canonical_and_rejects_extra_fields(self) -> None:
request = {
"schema_version": 1,
"input": {"input_id": str(uuid4()), "sheet": "Sheet1"},
"plot": {"type": "line", "x": "x", "y": ["y"]},
"output": {"formats": ["png", "opju"]},
}
normalized, digest = _canonical_request(request)
self.assertEqual(normalized, request)
self.assertEqual(len(digest), 64)
with self.assertRaisesRegex(Exception, "invalid origin plot request fields"):
_canonical_request({**request, "script": "anything"})
with self.assertRaisesRegex(Exception, "unsupported origin plot fields"):
_canonical_request({**request, "plot": {**request["plot"], "script": "anything"}})
with self.assertRaisesRegex(Exception, "input.input_id must be an artifact UUID"):
_canonical_request({**request, "input": {"input_id": "C:\\data.csv"}})
def test_origin_request_rejects_unimplemented_plot_semantics(self) -> None:
request = {
"schema_version": 1,
"input": {"input_id": str(uuid4())},
"plot": {"type": "scatter", "x": "time", "y": ["a", "b"]},
"output": {"formats": ["png"], "dpi": 300},
}
for plot, message in (
({**request["plot"], "template": "custom"}, "unsupported origin plot template"),
({**request["plot"], "x_axis": {"scale": "log10"}}, "invalid x_axis"),
({**request["plot"], "legend": {"enabled": False}}, "invalid plot.legend"),
({**request["plot"], "y": ["a", "a"]}, "plot.y must contain"),
):
with self.subTest(message=message), self.assertRaisesRegex(Exception, message):
_canonical_request({**request, "plot": plot})
with self.assertRaisesRegex(Exception, "video recording is not supported"):
_canonical_request(
{**request, "output": {"formats": ["png"], "record_video": True}}
)
def test_output_manifest_matches_exact_requested_formats(self) -> None:
request = {"output": {"formats": ["opju", "png"]}}
manifest = [
{"artifact_id": "project", "filename": "project.opju", "media_type": "application/x-origin-project", "size_bytes": 10, "sha256": "a" * 64},
{"artifact_id": "figure_png", "filename": "figure.png", "media_type": "image/png", "size_bytes": 20, "sha256": "b" * 64},
{"artifact_id": "plot_spec", "filename": "plot-spec.json", "media_type": "application/json", "size_bytes": 30, "sha256": "c" * 64},
{"artifact_id": "provenance", "filename": "provenance.json", "media_type": "application/json", "size_bytes": 40, "sha256": "d" * 64},
]
self.assertEqual(validate_output_manifest(request, manifest), manifest)
with self.assertRaisesRegex(Exception, "incomplete"):
validate_output_manifest(request, manifest[:-1])
with self.assertRaisesRegex(Exception, "metadata"):
validate_output_manifest(
request,
[{**manifest[0], "filename": "anything.opju"}, *manifest[1:]],
)
@patch("core.software_jobs.session_scope")
def test_stale_offer_cannot_be_accepted_by_another_node(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
job = type("Job", (), {})()
job.node_id = uuid4()
job.lease_id = uuid4()
job.status = "offered"
session.execute.return_value.scalar_one_or_none.return_value = job
with self.assertRaisesRegex(Exception, "stale or does not belong"):
respond_to_offer(
uuid4(),
accepted=True,
payload={"job_id": str(uuid4()), "lease_id": str(job.lease_id)},
)
@patch("core.software_jobs.session_scope")
def test_failed_delivery_only_abandons_matching_offer(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
node_id = uuid4()
lease_id = uuid4()
job = type("Job", (), {})()
job.node_id = node_id
job.lease_id = lease_id
job.status = "offered"
session.execute.return_value.scalar_one_or_none.return_value = job
abandon_offer(
node_id,
{"job_id": str(uuid4()), "lease_id": str(lease_id)},
)
self.assertEqual(job.status, "queued")
self.assertIsNone(job.node_id)
def test_dispatcher_excludes_nodes_with_active_jobs(self) -> None:
source = (
Path(__file__).resolve().parents[1] / "core" / "software_jobs.py"
).read_text(encoding="utf-8")
self.assertIn('{"offered", "dispatched", "running"}', source)
self.assertIn("item.node_id not in busy_node_ids", source)
def test_input_download_rechecks_file_digest(self) -> None:
source = (
Path(__file__).resolve().parents[1]
/ "web" / "routers" / "software_nodes.py"
).read_text(encoding="utf-8")
self.assertIn("digest = sha256()", source)
self.assertIn('digest.hexdigest() != item["sha256"]', source)
@patch("core.software_jobs.session_scope")
def test_job_state_restores_disconnected_job(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
node_id = uuid4()
lease_id = uuid4()
digest = "a" * 64
job = type("Job", (), {})()
job.node_id = node_id
job.lease_id = lease_id
job.request_digest = digest
job.status = "disconnected"
job.started_at = None
session.execute.return_value.scalar_one_or_none.return_value = job
update_job_state(node_id, {
"job_id": str(uuid4()),
"lease_id": str(lease_id),
"request_digest": digest,
"stage": "waiting_input",
"progress": 0,
"metrics": {},
})
self.assertEqual(job.status, "dispatched")
self.assertEqual(job.stage, "waiting_input")
@patch("core.software_jobs.session_scope")
def test_ready_to_run_is_not_reported_as_running(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
node_id = uuid4()
lease_id = uuid4()
digest = "c" * 64
job = type("Job", (), {})()
job.node_id = node_id
job.lease_id = lease_id
job.request_digest = digest
job.status = "dispatched"
job.started_at = None
session.execute.return_value.scalar_one_or_none.return_value = job
update_job_state(node_id, {
"job_id": str(uuid4()), "lease_id": str(lease_id),
"request_digest": digest, "stage": "ready_to_run",
"progress": 5, "metrics": {"input_bytes": 10},
})
self.assertEqual(job.status, "dispatched")
@patch("core.software_jobs.session_scope")
def test_terminal_replay_is_idempotent(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
node_id = uuid4()
lease_id = uuid4()
digest = "b" * 64
job = type("Job", (), {})()
job.node_id = node_id
job.lease_id = lease_id
job.request_digest = digest
job.status = "failed"
session.execute.return_value.scalar_one_or_none.return_value = job
record_job_terminal(node_id, {
"job_id": str(uuid4()),
"lease_id": str(lease_id),
"request_digest": digest,
"status": "failed",
"error": {"code": "TEST"},
"artifact_manifest": [],
})
self.assertEqual(job.status, "failed")
@patch("core.software_jobs.session_scope")
def test_disconnect_does_not_requeue_active_jobs(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
first = type("Job", (), {"status": "running"})()
second = type("Job", (), {"status": "dispatched"})()
session.execute.return_value.scalars.return_value = [first, second]
mark_node_jobs_disconnected(uuid4())
self.assertEqual(first.status, "disconnected")
self.assertEqual(second.status, "disconnected")
class SoftwareNodeDeleteTests(unittest.TestCase):
@patch("core.software_nodes.session_scope")
def test_delete_node_removes_existing_identity(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
node = object()
session.get.return_value = node
self.assertTrue(delete_node(uuid4()))
session.delete.assert_called_once_with(node)
@patch("core.software_nodes.session_scope")
def test_delete_node_reports_missing_identity(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
session.get.return_value = None
self.assertFalse(delete_node(uuid4()))
session.delete.assert_not_called()
if __name__ == "__main__":
unittest.main()

View File

@ -0,0 +1,65 @@
from __future__ import annotations
import hashlib
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from uuid import uuid4
from web.routers.software_nodes import _publish_software_job_outputs
class SoftwareOutputPublishTests(unittest.TestCase):
def test_complete_set_moves_atomically_and_can_be_replayed(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
job_id = uuid4()
working_dir = root / "research"
staging = root / ".zcbot_software_job_staging" / str(job_id)
staging.mkdir(parents=True)
working_dir.mkdir()
content = b"origin-result"
(staging / "figure.png").write_bytes(content)
manifest = [{
"artifact_id": "figure_png",
"filename": "figure.png",
"media_type": "image/png",
"size_bytes": len(content),
"sha256": hashlib.sha256(content).hexdigest(),
}]
context = {
"user_id": uuid4(),
"task_id": uuid4(),
"working_dir": "research",
}
def register(**kwargs):
return tuple({
"version": 2,
"scope": "working_dir",
"path": ref["path"],
"label": ref["label"],
"artifact_id": str(uuid4()),
} for ref in kwargs["refs"])
with (
patch("web.routers.software_nodes.load_user_root", return_value=root),
patch(
"web.routers.software_nodes.register_published_artifacts",
side_effect=register,
),
):
first = _publish_software_job_outputs(job_id, context, manifest)
second = _publish_software_job_outputs(job_id, context, manifest)
published = working_dir / "origin" / str(job_id) / "figure.png"
self.assertEqual(published.read_bytes(), content)
self.assertFalse(staging.exists())
self.assertEqual(first[0]["source_artifact_id"], "figure_png")
self.assertEqual(first[0]["path"], f"origin/{job_id}/figure.png")
self.assertEqual(second[0]["source_artifact_id"], "figure_png")
if __name__ == "__main__":
unittest.main()

View File

@ -26,19 +26,29 @@ def _frontend_source() -> str:
class StaticVendorTests(unittest.TestCase):
def test_dev_console_has_software_job_center(self) -> None:
html = DEV_HTML.read_text(encoding="utf-8")
source = (JS_DIR / "software_jobs.js").read_text(encoding="utf-8")
self.assertIn('id="software-job-center"', html)
self.assertIn('id="software-job-toggle"', html)
self.assertIn('"/v1/software-jobs?limit=50"', source)
self.assertIn("/cancel`", source)
self.assertIn("分析结果", source)
def test_admin_can_create_windows_node_enrollment_code(self) -> None:
html = ADMIN_HTML.read_text(encoding="utf-8")
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")
self.assertIn('id="node-enrollment-modal" class="modal"', html)
self.assertIn("生成 Windows Node 注册码", html)
self.assertIn('"/v1/admin/compute-node-enrollments"', admin_js)
self.assertIn('"/v1/admin/software-node-enrollments"', admin_js)
self.assertIn('capabilities: ["origin.plot@v1"]', admin_js)
self.assertIn('origin.health === "ready"', admin_js)
self.assertIn("ttl_seconds: 600", admin_js)
self.assertIn("navigator.clipboard.writeText(value)", admin_js)
self.assertIn('apiGet("/v1/admin/compute-nodes")', admin_js)
self.assertIn('apiSend("PATCH", `/v1/admin/compute-nodes/${node.node_id}`', admin_js)
self.assertIn('apiSend("DELETE", `/v1/admin/compute-nodes/${node.node_id}`', admin_js)
self.assertIn('apiGet("/v1/admin/software-nodes")', admin_js)
self.assertIn('apiSend("PATCH", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
self.assertIn('apiSend("DELETE", `/v1/admin/software-nodes/${node.node_id}`', admin_js)
self.assertIn("最近心跳", admin_js)
self.assertIn("重新启用", admin_js)
self.assertIn("永久删除", admin_js)

View File

@ -110,8 +110,10 @@ class AuthGateTests(unittest.TestCase):
("POST", "/v1/tasks"),
("POST", "/v1/asr/transcribe"),
("GET", "/v1/admin/overview"),
("GET", "/v1/admin/compute-nodes"),
("DELETE", "/v1/admin/compute-nodes/00000000-0000-0000-0000-000000000000"),
("GET", "/v1/admin/software-nodes"),
("GET", "/v1/software-jobs"),
("POST", "/v1/software-jobs/00000000-0000-0000-0000-000000000000/cancel"),
("DELETE", "/v1/admin/software-nodes/00000000-0000-0000-0000-000000000000"),
("GET", "/v1/admin/tool-wire-health"),
("GET", "/v1/admin/external-system-definitions"),
("GET", "/v1/admin/external-system-users"),

View File

@ -20,8 +20,8 @@ class WindowsNodeSourceTests(unittest.TestCase):
def test_node_protocol_and_secret_storage_markers_are_present(self) -> None:
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs"))
for marker in (
"v1/compute/nodes/enroll",
"v1/compute/nodes/connect",
"v1/software-nodes/enroll",
"v1/software-nodes/connect",
'SetRequestHeader("Authorization"',
'SetRequestHeader("X-Node-Id"',
"DataProtectionScope.LocalMachine",
@ -35,7 +35,11 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertIn(marker, source)
def test_node_does_not_expose_arbitrary_execution_primitives(self) -> None:
source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs"))
source = "\n".join(
path.read_text(encoding="utf-8")
for path in PROJECT.glob("*.cs")
if path.name != "OriginWorkerRunner.cs"
)
for forbidden in ("Process.Start", "cmd.exe", "powershell.exe", "LabTalk"):
self.assertNotIn(forbidden, source)
@ -56,7 +60,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
def test_configuration_window_is_resizable_and_dpi_safe(self) -> None:
form = (PROJECT / "ConfigurationForm.cs").read_text(encoding="utf-8")
self.assertIn("ClientSize = new Size(880, 720)", form)
self.assertIn("ClientSize = new Size(840, 680)", form)
self.assertIn("FormBorderStyle.Sizable", form)
self.assertIn("AutoScaleMode.Dpi", form)
self.assertIn("AutoScroll = true", form)
@ -65,12 +69,17 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertIn("成功注册一次后立即失效", form)
self.assertIn("CreateCard", form)
self.assertIn("注册并连接", form)
self.assertIn("ContentWidth = 800", form)
self.assertIn("节点能力", form)
self.assertIn("ContentWidth = 760", form)
self.assertIn("Origin 绘图", form)
self.assertIn("当前 MVP 内置声明该协议,不需要手工配置", form)
self.assertIn('CreateCapabilityRow("Origin 绘图", "origin.plot@v1")', form)
self.assertIn('CreateButton("立即重连", 112, primary: true)', form)
self.assertIn("ReconnectRequested?.Invoke()", form)
self.assertIn("registrationCard.Visible = !registered", form)
self.assertIn("registeredActionsCard.Visible = registered", form)
self.assertIn("reconnect.Visible = registered", form)
self.assertIn("resetIdentity.Visible = registered", form)
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
self.assertIn("form.ReconnectRequested += RestartConnection", tray)
def test_startup_task_is_login_scoped_and_runs_the_fixed_node_executable(self) -> None:
script = (ROOT / "install-startup.ps1").read_text(encoding="utf-8")
@ -80,6 +89,109 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertIn("-RunLevel Limited", script)
self.assertNotIn("-RunLevel Highest", script)
def test_auth_rejection_is_distinct_from_http_websocket_handshake_failure(self) -> None:
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
self.assertIn('socket.HttpStatusCode is HttpStatusCode.Unauthorized', connection)
self.assertIn('or HttpStatusCode.Forbidden', connection)
self.assertIn("WebSocket 握手被拒绝,请检查服务端或反向代理", connection)
self.assertIn("catch (NodeEndpointException exception)", connection)
self.assertIn("throw new NodeEndpointException", connection)
self.assertIn("(int?)result.CloseStatus == 4003", connection)
self.assertIn("节点身份已被服务端拒绝", connection)
self.assertNotIn("Node credentials were rejected", connection)
def test_origin_runtime_probe_is_read_only_and_reported(self) -> None:
probe = (PROJECT / "OriginRuntimeProbe.cs").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
self.assertIn('AutomationProgId = @"Origin.ApplicationSI\\CLSID"', probe)
self.assertIn("RegistryHive.LocalMachine", probe)
self.assertIn("RegistryHive.CurrentUser", probe)
self.assertIn('new("OriginPro", version, "0.2.0", health, detail)', probe)
self.assertIn(
'available_slots = origin.Health == "ready" && !jobInbox.HasPendingJobs ? 1 : 0',
connection,
)
self.assertNotIn("CreateInstance", probe)
self.assertNotIn("Process.Start", probe)
for marker in (
"software_version = origin.SoftwareVersion",
"adapter_version = origin.AdapterVersion",
"health = origin.Health",
"detail = origin.Detail",
):
self.assertIn(marker, connection)
def test_job_offer_is_persisted_before_acceptance(self) -> None:
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
self.assertIn('capabilityValue.GetString() != "origin.plot@v1"', inbox)
self.assertIn("PlotTypes.Contains", inbox)
self.assertIn("OutputFormats.Contains", inbox)
self.assertIn("FileOptions.WriteThrough", inbox)
self.assertIn("stream.Flush(flushToDisk: true)", inbox)
new_record = inbox.split("var record =", 1)[1].split("private static JsonElement?", 1)[0]
self.assertLess(
new_record.index("AtomicWrite(requestPath, record"),
new_record.index("JobOfferResult.Accept"),
)
self.assertIn('offerResult.Accepted ? "job_accept" : "job_reject"', connection)
self.assertIn("sendLock.WaitAsync", connection)
self.assertIn("!jobInbox.HasPendingJobs ? 1 : 0", connection)
self.assertIn("ReportRecoverableJobsAsync", connection)
self.assertIn("ConcurrentDictionary<Guid, Task> jobPipelines", connection)
self.assertIn("StartJobPipeline(socket, acceptedJob)", connection)
self.assertIn("inputDownloader.DownloadAsync(job, CancellationToken.None)", connection)
self.assertIn('stage = "uploading_outputs"', connection)
self.assertIn("&& !job.UploadComplete", connection)
self.assertIn("StartJobPipeline(socket, job)", connection)
self.assertIn('stage = "waiting_input"', connection)
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', inbox)
self.assertIn("AtomicWrite(requestPath, updated, overwrite: true)", inbox)
downloader = (PROJECT / "JobInputDownloader.cs").read_text(encoding="utf-8")
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', downloader)
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', downloader)
self.assertIn("HttpCompletionOption.ResponseHeadersRead", downloader)
self.assertIn("IncrementalHash.CreateHash", downloader)
self.assertIn("total > expectedSize", downloader)
self.assertIn("File.Move(temporaryPath, destination, overwrite: false)", downloader)
self.assertNotIn("Process.Start", downloader)
def test_origin_worker_launch_is_fixed_and_terminal_driven(self) -> None:
runner = (PROJECT / "OriginWorkerRunner.cs").read_text(encoding="utf-8")
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
project = (PROJECT / "Zcbot.WindowsNode.csproj").read_text(encoding="utf-8")
worker = (ROOT / "origin-worker" / "worker.py").read_text(encoding="utf-8")
self.assertIn('Environment.GetEnvironmentVariable("ZCBOT_ORIGIN_PYTHON")', runner)
self.assertIn('Path.Combine(paths.RootDirectory, "runtimes", "origin", "python.exe")', runner)
self.assertIn("UseShellExecute = false", runner)
self.assertIn("startInfo.ArgumentList.Add(workerScript)", runner)
self.assertIn("startInfo.ArgumentList.Add(jobDirectory)", runner)
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', runner)
self.assertIn('"NODE_RESTARTED_DURING_JOB"', runner)
self.assertIn("CancellationTokenSource.CreateLinkedTokenSource", runner)
self.assertIn("process.Kill(entireProcessTree: true)", runner)
self.assertIn('type.GetString() == "job_cancel"', connection)
self.assertIn('"cancelled", "USER_CANCELLED"', connection)
self.assertIn("origin-worker\\worker.py", project)
self.assertIn("if op.oext:", worker)
self.assertIn("op.exit()", worker)
self.assertIn("op.new_graph", worker)
self.assertIn("layer.add_plot", worker)
self.assertIn("op.save", worker)
self.assertIn("graph.save_fig", worker)
self.assertIn('_atomic_json(job_dir / "terminal.json"', worker)
for forbidden in ("subprocess", "eval(", "exec(", "os.system"):
self.assertNotIn(forbidden, worker)
uploader = (PROJECT / "JobOutputUploader.cs").read_text(encoding="utf-8")
self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', uploader)
self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', uploader)
self.assertIn('DefaultRequestHeaders.Add("X-Lease-Id"', uploader)
self.assertIn("SHA256.HashDataAsync", uploader)
self.assertIn("upload-complete.json", connection + uploader)
self.assertNotIn("Process.Start", uploader)
if __name__ == "__main__":
unittest.main()

122
tools/software_jobs.py Normal file
View File

@ -0,0 +1,122 @@
"""Agent 可调用的专业软件任务工具。"""
from __future__ import annotations
import json
from uuid import UUID, uuid4
from core.software_jobs import (
SoftwareJobError,
create_job,
get_job,
list_jobs,
request_job_cancel,
)
from core.software_nodes import SUPPORTED_CAPABILITIES, list_nodes
from .base import Tool
class _SoftwareJobTool(Tool):
def __init__(self, user_id: UUID, task_id: UUID, **kwargs) -> None:
super().__init__(**kwargs)
self.user_id = user_id
self.task_id = task_id
class SoftwareCapabilityListTool(_SoftwareJobTool):
name = "software_capability_list"
description = "List professional software capabilities available through managed Windows nodes."
parameters = {"type": "object", "properties": {}, "additionalProperties": False}
def execute(self) -> str:
nodes = list_nodes()
items = [{
"capability": item,
"display_name": "Origin 科研绘图" if item == "origin.plot@v1" else item,
"available_nodes": sum(
1
for node in nodes
if node["status"] == "online"
and item in (node.get("capabilities") or [])
and (node.get("runtime") or {}).get("available_slots", 0) > 0
),
} for item in sorted(SUPPORTED_CAPABILITIES)]
return json.dumps({"capabilities": items}, ensure_ascii=False)
class SoftwareJobSubmitTool(_SoftwareJobTool):
name = "software_job_submit"
description = (
"Submit a persistent professional software job for the current task. "
"Return immediately with job_id; do not poll continuously or wait for completion."
)
parameters = {
"type": "object",
"properties": {
"capability": {"type": "string", "enum": sorted(SUPPORTED_CAPABILITIES)},
"request": {"type": "object"},
"idempotency_key": {
"type": "string",
"description": "Stable unique key for this exact submission; omit to generate one.",
},
},
"required": ["capability", "request"],
"additionalProperties": False,
}
def execute(self, capability: str, request: dict, idempotency_key: str = "") -> str:
try:
job, created = create_job(
self.user_id,
self.task_id,
idempotency_key=idempotency_key.strip() or str(uuid4()),
capability=capability,
request=request,
)
return json.dumps({**job, "created": created}, ensure_ascii=False)
except SoftwareJobError as exc:
return f"[Error] {exc}"
class SoftwareJobStatusTool(_SoftwareJobTool):
name = "software_job_status"
description = "Check one software job, or list recent jobs in the current task when job_id is omitted."
parameters = {
"type": "object",
"properties": {"job_id": {"type": "string"}},
"additionalProperties": False,
}
def execute(self, job_id: str = "") -> str:
try:
if job_id.strip():
item = get_job(self.user_id, UUID(job_id.strip()))
if item is None or item["task_id"] != str(self.task_id):
return "[Error] software job not found"
return json.dumps(item, ensure_ascii=False)
return json.dumps(
{"results": list_jobs(self.user_id, task_id=self.task_id, limit=20)},
ensure_ascii=False,
)
except ValueError:
return "[Error] invalid job_id"
class SoftwareJobCancelTool(_SoftwareJobTool):
name = "software_job_cancel"
description = "Request cancellation of a software job in the current task after the user asks to stop it."
parameters = {
"type": "object",
"properties": {"job_id": {"type": "string"}},
"required": ["job_id"],
"additionalProperties": False,
}
def execute(self, job_id: str) -> str:
try:
item = get_job(self.user_id, UUID(job_id.strip()))
if item is None or item["task_id"] != str(self.task_id):
return "[Error] software job not found"
job, _ = request_job_cancel(self.user_id, UUID(job_id.strip()))
return json.dumps(job, ensure_ascii=False)
except (ValueError, SoftwareJobError) as exc:
return f"[Error] {exc}"

View File

@ -49,7 +49,7 @@ from .background import (
from .broker import broker
from .routers.asr import register_asr_routes
from .routers.authroutes import register_auth_routes
from .routers.compute_nodes import register_compute_node_routes
from .routers.software_nodes import register_software_node_routes
from .routers.external_systems import register_external_system_routes
from .routers.files import register_file_routes
from .routers.kb import register_kb_routes
@ -206,7 +206,9 @@ def create_app() -> FastAPI:
register_asr_routes(app, require_user=require_user, auth_cfg=auth_cfg)
register_task_routes(app, require_user=require_user)
register_message_routes(app, require_user=require_user)
register_compute_node_routes(app, require_admin=require_admin)
register_software_node_routes(
app, require_user=require_user, require_admin=require_admin
)
# ───────────── 管理后台(admin-only)─────────────
register_admin_routes(app, require_admin)

View File

@ -1,158 +0,0 @@
"""Windows Node MVP 的注册、管理与长连接端点。"""
from __future__ import annotations
import asyncio
from uuid import UUID
from fastapi import Depends, HTTPException, WebSocket, WebSocketDisconnect, status
from core.compute_nodes import (
ComputeNodeError,
authenticate_node,
create_enrollment,
delete_node,
enroll_node,
list_nodes,
mark_node_offline,
set_node_disabled,
update_node_runtime,
)
from web.schemas import (
ComputeEnrollmentCreateRequest,
ComputeNodeDisableRequest,
ComputeNodeEnrollRequest,
)
class NodeConnectionManager:
def __init__(self) -> None:
self._connections: dict[UUID, WebSocket] = {}
self._lock = asyncio.Lock()
async def activate(self, node_id: UUID, websocket: WebSocket) -> None:
async with self._lock:
old = self._connections.get(node_id)
self._connections[node_id] = websocket
if old is not None and old is not websocket:
await old.close(code=4001, reason="replaced by a newer connection")
async def remove(self, node_id: UUID, websocket: WebSocket) -> bool:
async with self._lock:
if self._connections.get(node_id) is websocket:
self._connections.pop(node_id, None)
return True
return False
async def close(self, node_id: UUID) -> None:
async with self._lock:
websocket = self._connections.pop(node_id, None)
if websocket is not None:
await websocket.close(code=4003, reason="node disabled")
node_connections = NodeConnectionManager()
def _bearer(authorization: str | None) -> str:
scheme, _, token = (authorization or "").partition(" ")
if scheme.lower() != "bearer" or not token:
raise ComputeNodeError("missing node bearer token")
return token
def register_compute_node_routes(app, *, require_admin) -> None:
@app.post(
"/v1/compute/nodes/enroll",
tags=["compute-nodes"],
status_code=status.HTTP_201_CREATED,
)
def node_enroll(body: ComputeNodeEnrollRequest):
try:
return enroll_node(**body.model_dump())
except ComputeNodeError as exc:
raise HTTPException(400, str(exc)) from exc
@app.websocket("/v1/compute/nodes/connect")
async def node_connect(websocket: WebSocket):
try:
node_id = UUID(websocket.headers.get("x-node-id", ""))
token = _bearer(websocket.headers.get("authorization"))
identity = await asyncio.to_thread(authenticate_node, node_id, token)
except (ValueError, ComputeNodeError):
await websocket.close(code=1008, reason="invalid node credentials")
return
await websocket.accept()
await node_connections.activate(node_id, websocket)
try:
await websocket.send_json({"type": "connected", "heartbeat_seconds": 15})
while True:
message = await websocket.receive_json()
message_type = message.get("type")
payload = message.get("payload") or {}
if message_type not in {"hello", "heartbeat"} or not isinstance(
payload, dict
):
await websocket.send_json(
{"type": "error", "code": "unsupported_message"}
)
continue
if payload.get("install_id") and payload["install_id"] != str(
identity["install_id"]
):
await websocket.close(code=1008, reason="install identity mismatch")
return
await asyncio.to_thread(
update_node_runtime,
node_id,
status="online",
runtime=payload,
)
await websocket.send_json(
{"type": "ack", "message_id": message.get("message_id")}
)
except (ComputeNodeError, WebSocketDisconnect, RuntimeError, ValueError):
pass
finally:
if await node_connections.remove(node_id, websocket):
await asyncio.to_thread(mark_node_offline, node_id)
@app.post("/v1/admin/compute-node-enrollments", tags=["admin"])
def admin_create_compute_enrollment(
body: ComputeEnrollmentCreateRequest,
user_id: UUID = Depends(require_admin), # noqa: B008
):
try:
return create_enrollment(user_id, **body.model_dump())
except ComputeNodeError as exc:
raise HTTPException(400, str(exc)) from exc
@app.get("/v1/admin/compute-nodes", tags=["admin"])
def admin_compute_nodes(user_id: UUID = Depends(require_admin)): # noqa: B008
return {"results": list_nodes()}
@app.patch("/v1/admin/compute-nodes/{node_id}", tags=["admin"])
async def admin_disable_compute_node(
node_id: UUID,
body: ComputeNodeDisableRequest,
user_id: UUID = Depends(require_admin), # noqa: B008
):
if not await asyncio.to_thread(set_node_disabled, node_id, body.disabled):
raise HTTPException(404, "compute node not found")
if body.disabled:
await node_connections.close(node_id)
return {
"node_id": str(node_id),
"status": "disabled" if body.disabled else "offline",
}
@app.delete("/v1/admin/compute-nodes/{node_id}", tags=["admin"])
async def admin_delete_compute_node(
node_id: UUID,
user_id: UUID = Depends(require_admin), # noqa: B008
):
# 先撤掉在线连接,避免删除后的旧 socket 继续上报运行态。
await node_connections.close(node_id)
if not await asyncio.to_thread(delete_node, node_id):
raise HTTPException(404, "compute node not found")
return {"node_id": str(node_id), "status": "deleted"}

View File

@ -0,0 +1,576 @@
"""Windows Node MVP 的注册、管理与长连接端点。"""
from __future__ import annotations
import asyncio
import os
from hashlib import sha256
from pathlib import Path
from uuid import UUID
from fastapi import Depends, Header, HTTPException, Request, WebSocket, WebSocketDisconnect, status
from fastapi.responses import FileResponse
from core.software_nodes import (
SoftwareNodeError,
authenticate_node,
create_enrollment,
delete_node,
enroll_node,
list_nodes,
mark_node_offline,
set_node_disabled,
update_node_runtime,
)
from core.software_jobs import (
MAX_OUTPUT_ARTIFACT_BYTES,
MAX_OUTPUT_TOTAL_BYTES,
OUTPUT_ARTIFACTS,
abandon_offer,
create_job,
get_job,
get_job_input,
get_job_output_context,
mark_node_jobs_disconnected,
offer_next_job,
pending_node_cancellations,
record_job_terminal,
list_jobs,
request_job_cancel,
respond_to_offer,
update_job_state,
validate_output_manifest,
SoftwareJobError,
)
from core.artifact_lifecycle import register_published_artifacts
from web.schemas import (
SoftwareEnrollmentCreateRequest,
SoftwareJobCreateRequest,
SoftwareNodeDisableRequest,
SoftwareNodeEnrollRequest,
)
from web.userfiles import load_user_root, safe_join
class NodeConnectionManager:
def __init__(self) -> None:
self._connections: dict[UUID, WebSocket] = {}
self._send_locks: dict[UUID, asyncio.Lock] = {}
self._lock = asyncio.Lock()
async def activate(self, node_id: UUID, websocket: WebSocket) -> None:
async with self._lock:
old = self._connections.get(node_id)
self._connections[node_id] = websocket
self._send_locks.setdefault(node_id, asyncio.Lock())
if old is not None and old is not websocket:
await old.close(code=4001, reason="replaced by a newer connection")
async def remove(self, node_id: UUID, websocket: WebSocket) -> bool:
async with self._lock:
if self._connections.get(node_id) is websocket:
self._connections.pop(node_id, None)
self._send_locks.pop(node_id, None)
return True
return False
async def close(self, node_id: UUID) -> None:
async with self._lock:
websocket = self._connections.pop(node_id, None)
self._send_locks.pop(node_id, None)
if websocket is not None:
await websocket.close(code=4003, reason="node disabled")
async def node_ids(self) -> set[UUID]:
async with self._lock:
return set(self._connections)
async def send(self, node_id: UUID, message: dict) -> bool:
async with self._lock:
websocket = self._connections.get(node_id)
send_lock = self._send_locks.get(node_id)
if websocket is None or send_lock is None:
return False
async with send_lock:
await websocket.send_json(message)
return True
async def send_on(self, node_id: UUID, websocket: WebSocket, message: dict) -> bool:
async with self._lock:
current = self._connections.get(node_id)
send_lock = self._send_locks.get(node_id)
if current is not websocket or send_lock is None:
return False
async with send_lock:
await websocket.send_json(message)
return True
node_connections = NodeConnectionManager()
def _bearer(authorization: str | None) -> str:
scheme, _, token = (authorization or "").partition(" ")
if scheme.lower() != "bearer" or not token:
raise SoftwareNodeError("missing node bearer token")
return token
def _authenticate_output_request(
job_id: UUID,
authorization: str | None,
x_node_id: str,
x_lease_id: str,
x_request_digest: str,
) -> tuple[UUID, UUID, dict]:
try:
node_id = UUID(x_node_id)
lease_id = UUID(x_lease_id)
authenticate_node(node_id, _bearer(authorization))
except (ValueError, SoftwareNodeError) as exc:
raise HTTPException(401, "invalid node credentials or job identity") from exc
context = get_job_output_context(node_id, job_id, lease_id, x_request_digest)
if context is None:
raise HTTPException(404, "software job output target not found")
return node_id, lease_id, context
def _hash_file(path: Path) -> str:
digest = sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _reject_symlink_path(root: Path, target: Path) -> None:
root = root.resolve()
current = root
for part in target.relative_to(root).parts:
current = current / part
if current.is_symlink():
raise HTTPException(409, "software job output path contains a symbolic link")
def _publish_software_job_outputs(job_id: UUID, context: dict, manifest: list[dict]) -> list[dict]:
root = load_user_root(context["user_id"])
working_dir = safe_join(root, context["working_dir"])
staging = safe_join(root, f".zcbot_software_job_staging/{job_id}")
relative_output = Path("origin") / str(job_id)
destination = safe_join(working_dir, relative_output.as_posix())
source = staging if staging.is_dir() else destination
_reject_symlink_path(root, source)
_reject_symlink_path(root, destination)
for item in manifest:
path = source / item["filename"]
if (
not path.is_file()
or path.stat().st_size != item["size_bytes"]
or _hash_file(path) != item["sha256"]
):
raise SoftwareJobError(f"uploaded artifact is missing or invalid: {item['artifact_id']}")
if source == staging:
destination.parent.mkdir(parents=True, exist_ok=True)
if destination.exists():
raise SoftwareJobError("software job output destination already exists unexpectedly")
os.replace(staging, destination)
try:
staging.parent.rmdir()
except OSError:
pass
refs = tuple({
"path": (relative_output / item["filename"]).as_posix(),
"label": item["filename"],
"media_type": item["media_type"],
} for item in manifest)
published_refs = register_published_artifacts(
user_id=context["user_id"],
task_id=context["task_id"],
user_root=root,
working_dir=working_dir,
refs=refs,
)
refs_by_path = {item["path"]: item for item in published_refs}
return [
{
**item,
"source_artifact_id": item["artifact_id"],
"artifact_id": refs_by_path[(relative_output / item["filename"]).as_posix()]["artifact_id"],
"path": (relative_output / item["filename"]).as_posix(),
}
for item in manifest
]
def register_software_node_routes(app, *, require_user, require_admin) -> None:
@app.post(
"/v1/software-nodes/enroll",
tags=["software-nodes"],
status_code=status.HTTP_201_CREATED,
)
def node_enroll(body: SoftwareNodeEnrollRequest):
try:
return enroll_node(**body.model_dump())
except SoftwareNodeError as exc:
raise HTTPException(400, str(exc)) from exc
@app.get("/v1/software-jobs/{job_id}/input", tags=["software-nodes"])
def download_software_job_input(
job_id: UUID,
authorization: str | None = Header(default=None),
x_node_id: str = Header(default=""),
):
try:
node_id = UUID(x_node_id)
authenticate_node(node_id, _bearer(authorization))
except (ValueError, SoftwareNodeError) as exc:
raise HTTPException(401, "invalid node credentials") from exc
item = get_job_input(node_id, job_id)
if item is None:
raise HTTPException(404, "software job input not found")
target = safe_join(load_user_root(item["user_id"]), item["current_path"])
if not target.is_file():
raise HTTPException(404, "software job input file not found")
stat = target.stat()
if stat.st_size != item["size_bytes"]:
raise HTTPException(409, "software job input changed after submission")
digest = sha256()
with target.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
if digest.hexdigest() != item["sha256"]:
raise HTTPException(409, "software job input changed after submission")
return FileResponse(
path=str(target),
filename=item["filename"],
media_type="application/octet-stream",
headers={
"Cache-Control": "no-store",
"X-Content-SHA256": item["sha256"],
},
)
@app.put(
"/v1/software-jobs/{job_id}/outputs/{artifact_id}",
tags=["software-nodes"],
status_code=status.HTTP_204_NO_CONTENT,
)
async def upload_software_job_output(
job_id: UUID,
artifact_id: str,
request: Request,
authorization: str | None = Header(default=None),
x_node_id: str = Header(default=""),
x_lease_id: str = Header(default=""),
x_request_digest: str = Header(default=""),
x_content_sha256: str = Header(default=""),
x_content_length: int = Header(default=-1),
):
_, _, context = await asyncio.to_thread(
_authenticate_output_request,
job_id, authorization, x_node_id, x_lease_id, x_request_digest,
)
metadata = OUTPUT_ARTIFACTS.get(artifact_id)
if metadata is None:
raise HTTPException(400, "unsupported output artifact identity")
filename, _, output_format = metadata
requested_formats = set(context["request"].get("output", {}).get("formats") or [])
if output_format is not None and output_format not in requested_formats:
raise HTTPException(400, "output artifact was not requested")
if not 1 <= x_content_length <= MAX_OUTPUT_ARTIFACT_BYTES:
raise HTTPException(400, "output artifact size is invalid")
if len(x_content_sha256) != 64 or any(c not in "0123456789abcdef" for c in x_content_sha256):
raise HTTPException(400, "output artifact digest is invalid")
root = load_user_root(context["user_id"])
published = safe_join(
safe_join(root, context["working_dir"]),
f"origin/{job_id}/{filename}",
)
if published.is_file():
if published.stat().st_size == x_content_length and _hash_file(published) == x_content_sha256:
return None
raise HTTPException(409, "published output conflicts with uploaded artifact")
staging = safe_join(root, f".zcbot_software_job_staging/{job_id}")
_reject_symlink_path(root, staging)
staging.mkdir(parents=True, exist_ok=True)
destination = staging / filename
if destination.is_file():
if destination.stat().st_size == x_content_length and _hash_file(destination) == x_content_sha256:
return None
raise HTTPException(409, "uploaded output conflicts with existing staging file")
staged_total = sum(
item.stat().st_size for item in staging.iterdir() if item.is_file()
)
if staged_total + x_content_length > MAX_OUTPUT_TOTAL_BYTES:
raise HTTPException(413, "software job outputs exceed the total size limit")
temporary = destination.with_name(destination.name + ".tmp-" + os.urandom(8).hex())
digest = sha256()
total = 0
try:
with temporary.open("xb") as handle:
async for chunk in request.stream():
total += len(chunk)
if total > x_content_length or total > MAX_OUTPUT_ARTIFACT_BYTES:
raise HTTPException(413, "output artifact exceeded declared size")
digest.update(chunk)
handle.write(chunk)
handle.flush()
os.fsync(handle.fileno())
if total != x_content_length or digest.hexdigest() != x_content_sha256:
raise HTTPException(400, "output artifact did not match declared metadata")
os.replace(temporary, destination)
finally:
temporary.unlink(missing_ok=True)
return None
@app.post("/v1/software-jobs/{job_id}/outputs/complete", tags=["software-nodes"])
async def complete_software_job_outputs(
job_id: UUID,
request: Request,
authorization: str | None = Header(default=None),
x_node_id: str = Header(default=""),
x_lease_id: str = Header(default=""),
x_request_digest: str = Header(default=""),
):
node_id, lease_id, context = await asyncio.to_thread(
_authenticate_output_request,
job_id, authorization, x_node_id, x_lease_id, x_request_digest,
)
body = await request.json()
if not isinstance(body, dict):
raise HTTPException(400, "output completion body must be an object")
try:
manifest = validate_output_manifest(context["request"], body.get("artifact_manifest"))
published = await asyncio.to_thread(
_publish_software_job_outputs, job_id, context, manifest
)
terminal = {
"job_id": str(job_id),
"lease_id": str(lease_id),
"request_digest": x_request_digest,
"status": "succeeded",
"error": {},
"artifact_manifest": published,
}
await asyncio.to_thread(record_job_terminal, node_id, terminal)
except (SoftwareJobError, KeyError, TypeError) as exc:
raise HTTPException(409, str(exc)) from exc
return {"status": "succeeded", "artifact_manifest": published}
@app.websocket("/v1/software-nodes/connect")
async def node_connect(websocket: WebSocket):
try:
node_id = UUID(websocket.headers.get("x-node-id", ""))
token = _bearer(websocket.headers.get("authorization"))
identity = await asyncio.to_thread(authenticate_node, node_id, token)
except (ValueError, SoftwareNodeError):
# 握手前 close 会被 ASGI 统一表现为 HTTP 403客户端无法区分
# “凭据无效”和“代理/路由没有正确转发 WebSocket”。先升级再用
# 应用关闭码给已持有 Node ID/Token 的节点返回明确诊断。
await websocket.accept()
await websocket.close(code=4003, reason="invalid node credentials")
return
await websocket.accept()
await node_connections.activate(node_id, websocket)
try:
await node_connections.send_on(
node_id,
websocket,
{"type": "connected", "heartbeat_seconds": 15},
)
for cancel in await asyncio.to_thread(pending_node_cancellations, node_id):
await node_connections.send_on(
node_id, websocket, {"type": "job_cancel", "payload": cancel}
)
while True:
message = await websocket.receive_json()
message_type = message.get("type")
payload = message.get("payload") or {}
if not isinstance(payload, dict):
await node_connections.send_on(node_id, websocket,
{"type": "error", "code": "unsupported_message"}
)
continue
if message_type in {"job_accept", "job_reject"}:
await asyncio.to_thread(
respond_to_offer,
node_id,
accepted=message_type == "job_accept",
payload=payload,
)
await node_connections.send_on(node_id, websocket,
{"type": "ack", "message_id": message.get("message_id")}
)
continue
if message_type in {"job_state", "job_terminal"}:
if message_type == "job_terminal" and payload.get("status") == "succeeded":
await node_connections.send_on(node_id, websocket,
{
"type": "error",
"code": "outputs_not_published",
"message_id": message.get("message_id"),
}
)
continue
handler = (
update_job_state
if message_type == "job_state"
else record_job_terminal
)
await asyncio.to_thread(handler, node_id, payload)
await node_connections.send_on(node_id, websocket,
{"type": "ack", "message_id": message.get("message_id")}
)
continue
if message_type not in {"hello", "heartbeat"}:
await node_connections.send_on(node_id, websocket,
{"type": "error", "code": "unsupported_message"}
)
continue
if payload.get("install_id") and payload["install_id"] != str(
identity["install_id"]
):
await websocket.close(code=1008, reason="install identity mismatch")
return
await asyncio.to_thread(
update_node_runtime,
node_id,
status="online",
runtime=payload,
)
await node_connections.send_on(node_id, websocket,
{"type": "ack", "message_id": message.get("message_id")}
)
for cancel in await asyncio.to_thread(pending_node_cancellations, node_id):
await node_connections.send_on(
node_id, websocket, {"type": "job_cancel", "payload": cancel}
)
offer = await asyncio.to_thread(
offer_next_job, await node_connections.node_ids()
)
if offer is not None:
delivered = await node_connections.send(
offer["node_id"],
{"type": "job_offer", "payload": offer["payload"]},
)
if not delivered:
await asyncio.to_thread(
abandon_offer, offer["node_id"], offer["payload"]
)
except (
SoftwareJobError,
SoftwareNodeError,
WebSocketDisconnect,
RuntimeError,
ValueError,
):
pass
finally:
if await node_connections.remove(node_id, websocket):
await asyncio.to_thread(mark_node_offline, node_id)
await asyncio.to_thread(mark_node_jobs_disconnected, node_id)
@app.post("/v1/tasks/{task_id}/software-jobs", tags=["software-jobs"])
async def submit_software_job(
task_id: UUID,
body: SoftwareJobCreateRequest,
user_id: UUID = Depends(require_user), # noqa: B008
):
try:
job, created = await asyncio.to_thread(
create_job, user_id, task_id, **body.model_dump()
)
offer = await asyncio.to_thread(
offer_next_job, await node_connections.node_ids()
)
if offer is not None:
delivered = await node_connections.send(
offer["node_id"], {"type": "job_offer", "payload": offer["payload"]}
)
if not delivered:
await asyncio.to_thread(
abandon_offer, offer["node_id"], offer["payload"]
)
return {**job, "created": created}
except SoftwareJobError as exc:
detail = str(exc)
raise HTTPException(404 if detail == "task not found" else 400, detail) from exc
@app.get("/v1/software-jobs", tags=["software-jobs"])
def read_software_jobs(
task_id: UUID | None = None,
active_only: bool = False,
limit: int = 50,
user_id: UUID = Depends(require_user), # noqa: B008
):
return {
"results": list_jobs(
user_id, task_id=task_id, active_only=active_only, limit=limit
)
}
@app.post("/v1/software-jobs/{job_id}/cancel", tags=["software-jobs"])
async def cancel_software_job(
job_id: UUID,
user_id: UUID = Depends(require_user), # noqa: B008
):
try:
job, message = await asyncio.to_thread(request_job_cancel, user_id, job_id)
except SoftwareJobError as exc:
raise HTTPException(404, str(exc)) from exc
if message is not None:
await node_connections.send(
message["node_id"],
{"type": "job_cancel", "payload": message["payload"]},
)
return job
@app.get("/v1/software-jobs/{job_id}", tags=["software-jobs"])
def read_software_job(
job_id: UUID,
user_id: UUID = Depends(require_user), # noqa: B008
):
job = get_job(user_id, job_id)
if job is None:
raise HTTPException(404, "software job not found")
return job
@app.post("/v1/admin/software-node-enrollments", tags=["admin"])
def admin_create_software_enrollment(
body: SoftwareEnrollmentCreateRequest,
user_id: UUID = Depends(require_admin), # noqa: B008
):
try:
return create_enrollment(user_id, **body.model_dump())
except SoftwareNodeError as exc:
raise HTTPException(400, str(exc)) from exc
@app.get("/v1/admin/software-nodes", tags=["admin"])
def admin_software_nodes(user_id: UUID = Depends(require_admin)): # noqa: B008
return {"results": list_nodes()}
@app.patch("/v1/admin/software-nodes/{node_id}", tags=["admin"])
async def admin_disable_software_node(
node_id: UUID,
body: SoftwareNodeDisableRequest,
user_id: UUID = Depends(require_admin), # noqa: B008
):
if not await asyncio.to_thread(set_node_disabled, node_id, body.disabled):
raise HTTPException(404, "software node not found")
if body.disabled:
await node_connections.close(node_id)
return {
"node_id": str(node_id),
"status": "disabled" if body.disabled else "offline",
}
@app.delete("/v1/admin/software-nodes/{node_id}", tags=["admin"])
async def admin_delete_software_node(
node_id: UUID,
user_id: UUID = Depends(require_admin), # noqa: B008
):
# 先撤掉在线连接,避免删除后的旧 socket 继续上报运行态。
await node_connections.close(node_id)
if not await asyncio.to_thread(delete_node, node_id):
raise HTTPException(404, "software node not found")
return {"node_id": str(node_id), "status": "deleted"}

View File

@ -116,13 +116,13 @@ class ExternalSystemCredentialsRequest(BaseModel):
credentials: dict[str, str] = Field(default_factory=dict)
class ComputeEnrollmentCreateRequest(BaseModel):
class SoftwareEnrollmentCreateRequest(BaseModel):
expected_name: str = ""
capabilities: list[str] = Field(default_factory=lambda: ["origin.plot@v1"])
ttl_seconds: int = 600
class ComputeNodeEnrollRequest(BaseModel):
class SoftwareNodeEnrollRequest(BaseModel):
enrollment_code: str
node_name: str
install_id: UUID
@ -131,5 +131,11 @@ class ComputeNodeEnrollRequest(BaseModel):
capabilities: list[str]
class ComputeNodeDisableRequest(BaseModel):
class SoftwareNodeDisableRequest(BaseModel):
disabled: bool = True
class SoftwareJobCreateRequest(BaseModel):
idempotency_key: str
capability: str = "origin.plot@v1"
request: dict = Field(default_factory=dict)

View File

@ -1069,6 +1069,34 @@
overflow: hidden; text-overflow: ellipsis; white-space: nowrap; margin-top: 2px;
}
.proc-toast.fail { border-color: rgba(192,57,43,0.5); }
#software-job-center {
position: fixed; right: 16px; bottom: 16px; z-index: 115; display: none;
width: min(380px, calc(100vw - 24px)); font-size: 13px;
}
#software-job-center.show { display: block; }
#software-job-toggle {
margin-left: auto; display: flex; align-items: center; gap: 8px;
border-radius: 999px; padding: 8px 13px; background: #fff;
border: 1px solid var(--border); box-shadow: 0 4px 16px rgba(0,0,0,.16); cursor: pointer;
}
#software-job-center.expanded #software-job-toggle { border-radius: 0 0 10px 10px; }
#software-job-panel {
max-height: min(520px, calc(100vh - 110px)); overflow: auto; background: #fff;
border: 1px solid var(--border); border-bottom: 0; border-radius: 10px 10px 0 0;
box-shadow: 0 -4px 20px rgba(0,0,0,.16); padding: 8px;
}
.sj-card { padding: 10px; border-bottom: 1px solid var(--border); }
.sj-card:last-child { border-bottom: 0; }
.sj-title { display: flex; justify-content: space-between; gap: 10px; }
.sj-title span { color: var(--muted); white-space: nowrap; }
.sj-card.failed .sj-title span { color: var(--danger); }
.sj-card.disconnected .sj-title span, .sj-card.cancelling .sj-title span { color: #a66514; }
.sj-sub, .sj-meta { margin-top: 4px; color: var(--muted); font-size: 11px; overflow-wrap: anywhere; }
.sj-progress { height: 4px; margin-top: 7px; border-radius: 4px; background: var(--panel-muted); overflow: hidden; }
.sj-progress i { display: block; height: 100%; background: var(--accent); transition: width .25s ease; }
.sj-actions { display: flex; justify-content: flex-end; gap: 6px; margin-top: 8px; }
.sj-empty { padding: 22px; text-align: center; color: var(--muted); }
#software-job-center.show ~ #proc-toasts { bottom: 62px; }
/* media tool 摘要 banner(model / size / cost / elapsed,折叠态也可见) */
.tool-banner {
display: inline-flex; flex-wrap: wrap; gap: 6px;
@ -2174,6 +2202,13 @@
</div>
</div>
<div id="software-job-center" aria-live="polite">
<div id="software-job-panel" hidden></div>
<button id="software-job-toggle" type="button" title="展开专业软件任务">
<span>专业软件任务</span><strong id="software-job-count">0 条</strong>
</button>
</div>
<script type="module" src="js/main.js"></script>
</body>
</html>

View File

@ -55,7 +55,7 @@ let externalDefinitions = [];
let externalUsers = [];
let externalDefinitionsLoaded = false;
let externalEditingId = "";
let computeNodes = [];
let softwareNodes = [];
// ───── 格式化 ─────
function fmtCNY(n) {
@ -167,12 +167,16 @@ function nodeStatusHTML(status) {
}
function renderWindowsNodes() {
const rows = computeNodes.map(node => {
const rows = softwareNodes.map(node => {
const runtime = node.runtime || {};
const origin = runtime.origin || {};
const originState = origin.health === "ready" ? "Origin 可用" : "Origin 不可用";
const originVersion = origin.software_version ? ` ${origin.software_version}` : "";
const runtimeParts = [
node.os_version || "",
runtime.desktop_session === true ? "桌面会话" : "",
runtime.available_slots != null ? `可用槽位 ${runtime.available_slots}` : "",
runtime.origin ? `${originState}${originVersion}` : "",
].filter(Boolean);
const lastSeen = node.last_seen_at
? `<span title="${escapeHtml(fmtTime(node.last_seen_at))}">${escapeHtml(fmtTimeAgo(node.last_seen_at))}</span>`
@ -193,22 +197,22 @@ function renderWindowsNodes() {
}).join("") || `<tr><td colspan="7" class="empty">尚无已注册的 Windows Node</td></tr>`;
$("s-windows-node").innerHTML = `<div class="card"><div class="card-head">`
+ `<div><h2>Windows Node${computeNodes.length}</h2><div class="node-help">查看节点状态;禁用会立即断开节点并拒绝后续连接。</div></div>`
+ `<div><h2>Windows Node${softwareNodes.length}</h2><div class="node-help">查看节点状态;禁用会立即断开节点并拒绝后续连接。</div></div>`
+ `<button id="node-enrollment-open" class="primary" type="button">生成 Windows Node 注册码</button>`
+ `</div><div class="scroll-x"><table><thead><tr><th>节点</th><th>状态</th><th>运行环境</th>`
+ `<th>版本</th><th>能力</th><th>最近心跳</th><th>操作</th></tr></thead><tbody>${rows}</tbody></table></div></div>`;
$("node-enrollment-open").onclick = openNodeEnrollmentModal;
$("s-windows-node").querySelectorAll("[data-node-toggle]").forEach(button => {
button.onclick = () => toggleComputeNode(button);
button.onclick = () => toggleSoftwareNode(button);
});
$("s-windows-node").querySelectorAll("[data-node-delete]").forEach(button => {
button.onclick = () => deleteComputeNode(button);
button.onclick = () => deleteSoftwareNode(button);
});
}
async function toggleComputeNode(button) {
async function toggleSoftwareNode(button) {
const row = button.closest("tr[data-node-id]");
const node = computeNodes.find(item => item.node_id === row?.dataset.nodeId);
const node = softwareNodes.find(item => item.node_id === row?.dataset.nodeId);
if (!node) return;
const disabling = button.dataset.nodeToggle === "disable";
const confirmed = await dialogConfirm({
@ -222,20 +226,20 @@ async function toggleComputeNode(button) {
if (!confirmed) return;
button.disabled = true;
try {
await apiSend("PATCH", `/v1/admin/compute-nodes/${node.node_id}`, {
await apiSend("PATCH", `/v1/admin/software-nodes/${node.node_id}`, {
disabled: disabling,
});
message(disabling ? "节点已禁用" : "节点已重新启用,请在节点电脑上立即重连", "success", 5000);
await loadComputeNodes();
await loadSoftwareNodes();
} catch (err) {
if (err.code !== "auth") message("更新节点失败:" + (err.message || String(err)), "error", 5000);
button.disabled = false;
}
}
async function deleteComputeNode(button) {
async function deleteSoftwareNode(button) {
const row = button.closest("tr[data-node-id]");
const node = computeNodes.find(item => item.node_id === row?.dataset.nodeId);
const node = softwareNodes.find(item => item.node_id === row?.dataset.nodeId);
if (!node) return;
const confirmed = await dialogConfirm({
title: "删除 Windows Node",
@ -246,9 +250,9 @@ async function deleteComputeNode(button) {
if (!confirmed) return;
button.disabled = true;
try {
await apiSend("DELETE", `/v1/admin/compute-nodes/${node.node_id}`, {});
await apiSend("DELETE", `/v1/admin/software-nodes/${node.node_id}`, {});
message("节点已删除,本机需重新注册后才能使用", "success", 5000);
await loadComputeNodes();
await loadSoftwareNodes();
} catch (err) {
if (err.code !== "auth") message("删除节点失败:" + (err.message || String(err)), "error", 5000);
button.disabled = false;
@ -306,7 +310,7 @@ async function createNodeEnrollment(e) {
submit.disabled = true;
submit.textContent = "生成中…";
try {
const result = await apiSend("POST", "/v1/admin/compute-node-enrollments", {
const result = await apiSend("POST", "/v1/admin/software-node-enrollments", {
expected_name: $("node-expected-name").value.trim(),
capabilities: ["origin.plot@v1"],
ttl_seconds: 600,
@ -1031,10 +1035,10 @@ async function loadExternalDefinitions(force = false) {
} catch (e) { /* overview 统一处理鉴权 */ }
}
async function loadComputeNodes() {
async function loadSoftwareNodes() {
try {
const result = await apiGet("/v1/admin/compute-nodes");
computeNodes = result.results || [];
const result = await apiGet("/v1/admin/software-nodes");
softwareNodes = result.results || [];
renderWindowsNodes();
} catch (e) { /* overview 统一处理鉴权 */ }
}
@ -1048,7 +1052,7 @@ async function refresh() {
loadModels();
loadUserUsage(userPage);
loadStorage(storagePage);
loadComputeNodes();
loadSoftwareNodes();
loadExternalDefinitions();
loadToolFailures();
} catch (e) {

View File

@ -19,6 +19,7 @@ import { loadFolderSuggestions } from "./newtask.js";
import { embedInit } from "./embed.js";
import { loadTaskList, loadModels, loadChannelCards, selectTask, showNewConversationDraft } from "./chat.js";
import { refreshProcs } from "./procs.js";
import { initSoftwareJobs } from "./software_jobs.js";
// ───── enter app ─────
export function enterApp(initialTaskId = "") {
@ -40,6 +41,7 @@ export function enterApp(initialTaskId = "") {
loadStorage(); // 顶栏存储用量(后台扫描快照,非实时)
loadRole(); // 拉 /v1/me,admin 才显「管理」入口(/static/admin.html)
refreshProcs(); // 后台进程(bg proc):上个会话遗留的 running proc 恢复轮询/展示
initSoftwareJobs();
}
// 顶栏用户名:默认显 name(兜底 user_name → email → uid8),title 悬浮给完整身份。

View File

@ -0,0 +1,144 @@
// 专业软件任务中心:用户级轮询、跨对话状态、取消与终态通知。
import { api } from "./api.js";
import { state } from "./state.js";
import { $ } from "./dom.js";
import { escapeHtml } from "./format.js";
import { selectTask } from "./chat.js";
import { dialogConfirm, message } from "./dialog.js";
const ACTIVE = new Set(["queued", "offered", "dispatched", "running", "disconnected", "cancelling"]);
const TERMINAL = new Set(["succeeded", "failed", "cancelled"]);
const POLL_ACTIVE_MS = 4000;
const POLL_IDLE_MS = 30000;
let timer = null;
let known = new Map();
let jobs = [];
let expanded = false;
let initialized = false;
const statusLabel = {
queued: "等待计算节点", offered: "正在分配节点", dispatched: "节点已接收",
running: "正在执行", disconnected: "节点连接中断", cancelling: "正在停止",
succeeded: "已完成", failed: "失败", cancelled: "已取消",
};
const stageLabel = {
accepted: "节点已接收", waiting_input: "正在下载输入文件", ready_to_run: "准备软件环境",
origin_running: "Origin 正在生成图表", uploading_outputs: "正在上传结果",
cancel_requested: "停止请求已发送", terminal: "任务已结束",
};
export function initSoftwareJobs() {
if (initialized || !$("software-job-center")) return;
initialized = true;
$("software-job-toggle").onclick = () => { expanded = !expanded; render(); };
document.addEventListener("click", (event) => {
const center = $("software-job-center");
if (expanded && center && !center.contains(event.target)) { expanded = false; render(); }
});
refreshSoftwareJobs();
}
export async function refreshSoftwareJobs() {
if (!state.token) return;
try {
const data = await api("GET", "/v1/software-jobs?limit=50");
const next = data.results || [];
next.forEach((job) => {
const previous = known.get(job.job_id);
if (previous && ACTIVE.has(previous) && TERMINAL.has(job.status)) notifyTerminal(job);
known.set(job.job_id, job.status);
});
jobs = next;
render();
} catch (_) { /* 后台刷新失败静默,下轮恢复 */ }
schedule();
}
function schedule() {
if (timer) clearTimeout(timer);
const delay = jobs.some((job) => ACTIVE.has(job.status)) ? POLL_ACTIVE_MS : POLL_IDLE_MS;
timer = setTimeout(refreshSoftwareJobs, document.hidden ? Math.max(delay, 30000) : delay);
}
document.addEventListener("visibilitychange", () => {
if (!document.hidden && state.token) refreshSoftwareJobs(); else schedule();
});
function render() {
const center = $("software-job-center");
const panel = $("software-job-panel");
const active = jobs.filter((job) => ACTIVE.has(job.status));
const recent = jobs.slice(0, 10);
center.classList.toggle("show", active.length > 0 || recent.length > 0);
center.classList.toggle("expanded", expanded);
$("software-job-count").textContent = active.length ? `${active.length} 运行中` : `${jobs.length}`;
panel.hidden = !expanded;
if (!expanded) return;
panel.innerHTML = recent.length
? recent.map(jobCard).join("")
: '<div class="sj-empty">暂无专业软件任务</div>';
panel.querySelectorAll("[data-job-action]").forEach((button) => {
button.onclick = (event) => handleAction(event, button);
});
}
function jobCard(job) {
const summary = job.request_summary || {};
const input = job.input || {};
const active = ACTIVE.has(job.status);
const progress = Math.max(0, Math.min(100, Number(job.progress || 0)));
const detail = stageLabel[job.stage] || statusLabel[job.status] || "正在处理";
const error = job.error && (job.error.detail || job.error.code);
return `<article class="sj-card ${escapeHtml(job.status)}" data-job-id="${escapeHtml(job.job_id)}">
<div class="sj-title"><strong>${escapeHtml(summary.display_name || job.capability)}</strong>
<span>${escapeHtml(statusLabel[job.status] || job.status)}</span></div>
<div class="sj-sub">${escapeHtml(detail)}${error ? ` · ${escapeHtml(error)}` : ""}</div>
${active ? `<div class="sj-progress"><i style="width:${progress}%"></i></div>` : ""}
<div class="sj-meta">${escapeHtml(job.task_name || "未命名对话")}${input.filename ? ` · ${escapeHtml(input.filename)}` : ""}</div>
<div class="sj-actions">
<button class="small" data-job-action="open" data-task-id="${escapeHtml(job.task_id)}">打开对话</button>
${job.status === "succeeded" ? `<button class="small primary" data-job-action="analyze" data-task-id="${escapeHtml(job.task_id)}">分析结果</button>` : ""}
${active && job.status !== "cancelling" ? '<button class="small danger" data-job-action="cancel">停止</button>' : ""}
</div>
</article>`;
}
async function handleAction(event, button) {
event.stopPropagation();
const card = button.closest("[data-job-id]");
const jobId = card.dataset.jobId;
const action = button.dataset.jobAction;
if (action === "cancel") {
if (!await dialogConfirm({
title: "停止专业软件任务",
message: "确定停止这个任务?已经产生但尚未发布的中间输出可能不会保留。",
okText: "停止", danger: true,
})) return;
button.disabled = true;
try { await api("POST", `/v1/software-jobs/${jobId}/cancel`); }
catch (error) { message(error.message || "停止失败", "error"); }
refreshSoftwareJobs();
return;
}
const taskId = button.dataset.taskId;
if (taskId) await selectTask(taskId);
expanded = false;
render();
if (action === "analyze") {
setTimeout(() => {
const input = $("chat-input");
if (!input) return;
input.value = `请分析专业软件任务 ${jobId} 的结果,结合输出图表和输入数据总结主要结论。`;
input.focus();
input.dispatchEvent(new Event("input", { bubbles: true }));
}, 400);
}
}
function notifyTerminal(job) {
const ok = job.status === "succeeded";
const label = ok ? "已完成" : (job.status === "cancelled" ? "已取消" : "失败");
const summary = job.request_summary || {};
message(`${summary.display_name || "专业软件任务"}${label}`, ok ? "success" : "error", 6000);
expanded = true;
}

View File

@ -2,7 +2,15 @@
内网 MVP 的 Windows 执行节点,目标运行环境为 Windows 11 Enterprise + .NET 10 SDK 10.0.303。仓库根目录 `global.json` 固定 SDK patch客户端只使用 .NET Windows Desktop Framework不依赖第三方 NuGet 包。
当前实现托盘状态角标、小型配置窗口、注册、DPAPI/ACL 配置保存、WebSocket `hello`/心跳和退避重连。尚未实现 `compute_jobs`、Origin Worker、任务目录和产物上传。
当前实现托盘状态角标、小型配置窗口、注册、DPAPI/ACL 配置保存、WebSocket `hello`/心跳和退避重连,并只读探测 Origin/OriginPro 安装版本、COM 自动化组件与桌面会话状态。Node 可以接收受控的 `origin.plot@v1` offer在本机任务目录原子保存请求后回报 accept/reject随后以 Node 身份流式下载任务绑定的 CSV/XLSX/JSON校验大小与 SHA-256 后原子保存。固定 Origin Worker 独立于单次 WebSocket 执行,断线不终止已启动绘图。成功产物按 manifest 逐项流式上传并由云端复核大小与 SHA-256全部完成后原子发布到任务工作目录的 `origin/<job_id>/`;中断后按本地 `upload-complete.json` 幂等续传。
固定 Origin Worker 已支持 `line`、`scatter` 和 `line_scatter`,生成 OPJU、PNG、SVG、PDF、plot spec、provenance 与原子 `terminal.json`。运行时独立于 zcbot 服务端 Python管理员执行
```powershell
.\install-origin-runtime.ps1 -BootstrapPython D:\programs\Python312\python.exe
```
默认安装到 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\python.exe`;也可由管理员用绝对路径环境变量 `ZCBOT_ORIGIN_PYTHON` 指向固定解释器。任务请求不能指定解释器、脚本或路径。
注册和运行必须使用同一专用 Windows 账号。MVP 通过该账号的登录后计划任务自动启动,不以 Windows Service 在 Session 0 运行。

View File

@ -2,12 +2,13 @@ namespace Zcbot.WindowsNode;
internal sealed class ConfigurationForm : Form
{
private const int ContentWidth = 800;
private const int ContentWidth = 760;
private readonly TextBox server = CreateTextBox("http://127.0.0.1:8765");
private readonly TextBox nodeName = CreateTextBox(Environment.MachineName.ToLowerInvariant());
private readonly TextBox enrollmentCode = CreateTextBox(usePassword: true);
private readonly Button register = CreateButton("注册并连接", 128, primary: true);
private readonly Button reconnect = CreateButton("立即重连", 112, primary: true);
private readonly Button resetIdentity = CreateButton("清除本机身份并重新注册", 220);
private readonly CheckBox startAtLogin = new()
{
@ -25,18 +26,18 @@ internal sealed class ConfigurationForm : Form
private readonly Label identity = CreateBodyLabel();
private readonly Label capabilitySummary = CreateBodyLabel();
private readonly TableLayoutPanel registrationCard;
private readonly TableLayoutPanel registeredActionsCard;
private bool changingStartup;
internal event Func<EnrollOptions, Task>? RegisterRequested;
internal event Action? ReconnectRequested;
internal event Action? ResetIdentityRequested;
internal ConfigurationForm()
{
Text = "zcbot Windows Node";
AutoScaleMode = AutoScaleMode.Dpi;
ClientSize = new Size(880, 720);
MinimumSize = new Size(850, 680);
ClientSize = new Size(840, 680);
MinimumSize = new Size(820, 640);
StartPosition = FormStartPosition.CenterScreen;
Font = new Font("Microsoft YaHei UI", 9);
FormBorderStyle = FormBorderStyle.Sizable;
@ -62,7 +63,7 @@ internal sealed class ConfigurationForm : Form
AutoSizeMode = AutoSizeMode.GrowAndShrink,
Dock = DockStyle.Top,
ColumnCount = 1,
RowCount = 6,
RowCount = 5,
BackColor = BackColor,
};
page.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 100));
@ -85,33 +86,19 @@ internal sealed class ConfigurationForm : Form
heading.Controls.Add(CreateHint("连接本机科研软件与 zcbot 的受控执行节点"));
page.Controls.Add(heading);
var overview = new TableLayoutPanel
{
AutoSize = true,
Dock = DockStyle.Top,
ColumnCount = 2,
Margin = new Padding(0, 0, 0, 14),
};
overview.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 50));
overview.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 50));
var statusCard = CreateCard();
statusCard.Margin = new Padding(0, 0, 7, 0);
statusCard.Controls.Add(CreateSectionTitle("节点状态"));
statusCard.Controls.Add(CreateSectionTitle("节点"));
statusCard.Controls.Add(state);
statusCard.Controls.Add(detail);
statusCard.Controls.Add(identity);
overview.Controls.Add(statusCard, 0, 0);
var capabilityCard = CreateCard();
capabilityCard.Margin = new Padding(7, 0, 0, 0);
capabilityCard.Controls.Add(CreateSectionTitle("节点能力"));
capabilityCard.Controls.Add(CreateCapabilityBadge("Origin 绘图", "origin.plot@v1"));
capabilityCard.Controls.Add(capabilitySummary);
capabilityCard.Controls.Add(CreateHint(
"当前 MVP 内置声明该协议,不需要手工配置;它不代表已完成 Origin 安装检测。"));
overview.Controls.Add(capabilityCard, 1, 0);
page.Controls.Add(overview);
statusCard.Controls.Add(CreateDivider());
statusCard.Controls.Add(CreateCapabilityRow("Origin 绘图", "origin.plot@v1"));
statusCard.Controls.Add(capabilitySummary);
var resetActions = CreateActions();
resetActions.Controls.Add(reconnect);
resetActions.Controls.Add(resetIdentity);
statusCard.Controls.Add(resetActions);
page.Controls.Add(statusCard);
registrationCard = CreateCard();
registrationCard.Controls.Add(CreateSectionTitle("首次注册"));
@ -125,15 +112,6 @@ internal sealed class ConfigurationForm : Form
registrationCard.Controls.Add(registerActions);
page.Controls.Add(registrationCard);
registeredActionsCard = CreateCard();
registeredActionsCard.Controls.Add(CreateSectionTitle("节点身份"));
registeredActionsCard.Controls.Add(CreateHint(
"服务地址、节点名称和能力在首次注册时确定。需要修改时,请先在管理后台删除旧节点,再清除本机身份并重新注册。"));
var identityActions = CreateActions();
identityActions.Controls.Add(resetIdentity);
registeredActionsCard.Controls.Add(identityActions);
page.Controls.Add(registeredActionsCard);
var runtimeCard = CreateCard();
runtimeCard.Controls.Add(CreateSectionTitle("运行设置"));
runtimeCard.Controls.Add(startAtLogin);
@ -147,6 +125,7 @@ internal sealed class ConfigurationForm : Form
page.Controls.Add(securityNote);
register.Click += async (_, _) => await RegisterAsync();
reconnect.Click += (_, _) => ReconnectRequested?.Invoke();
resetIdentity.Click += (_, _) => ResetIdentity();
startAtLogin.CheckedChanged += (_, _) => ToggleStartup();
FormClosing += (_, eventArgs) =>
@ -185,11 +164,12 @@ internal sealed class ConfigurationForm : Form
: $"节点:{config.NodeName}\nNode ID{config.NodeId}\n服务{config.ServerUrl}";
capabilitySummary.Text = config is null
? "注册后启用"
: $"已声明 {config.Capabilities.Count} 项能力";
: FormatOriginStatus(OriginRuntimeProbe.Detect());
var registered = config is not null;
registrationCard.Visible = !registered;
registeredActionsCard.Visible = registered;
reconnect.Visible = registered;
resetIdentity.Visible = registered;
register.Enabled = !registered && status.State != NodeState.Connecting;
if (registered)
{
@ -199,6 +179,15 @@ internal sealed class ConfigurationForm : Form
}
}
private static string FormatOriginStatus(OriginRuntimeStatus origin)
{
var version = string.IsNullOrWhiteSpace(origin.SoftwareVersion)
? "版本未知"
: $"版本 {origin.SoftwareVersion}";
var state = origin.Health == "ready" ? "可用" : "不可用";
return $"{state} · {version}\n{origin.Detail}";
}
private void ResetIdentity()
{
var answer = MessageBox.Show(
@ -287,7 +276,7 @@ internal sealed class ConfigurationForm : Form
AutoSize = true,
Dock = DockStyle.Fill,
ForeColor = Color.FromArgb(71, 85, 105),
MaximumSize = new Size(340, 0),
MaximumSize = new Size(ContentWidth - 40, 0),
Margin = new Padding(0, 2, 0, 5),
};
@ -300,25 +289,33 @@ internal sealed class ConfigurationForm : Form
Margin = new Padding(0, 2, 0, 4),
};
private static Panel CreateCapabilityBadge(string title, string protocol)
private static Panel CreateDivider() => new()
{
var badge = new Panel
Height = 1,
Dock = DockStyle.Top,
BackColor = Color.FromArgb(226, 232, 240),
Margin = new Padding(0, 10, 0, 12),
};
private static Panel CreateCapabilityRow(string title, string protocol)
{
var row = new Panel
{
AutoSize = false,
Height = 54,
Height = 36,
Dock = DockStyle.Top,
BackColor = Color.FromArgb(239, 246, 255),
Margin = new Padding(0, 2, 0, 8),
Padding = new Padding(12, 7, 12, 7),
Margin = new Padding(0, 0, 0, 5),
Padding = new Padding(11, 7, 11, 7),
};
badge.Controls.Add(new Label
row.Controls.Add(new Label
{
Text = $"{title}\n协议{protocol}",
Text = $"{title} · {protocol}",
AutoSize = true,
Font = new Font("Microsoft YaHei UI", 9, FontStyle.Bold),
ForeColor = Color.FromArgb(29, 78, 216),
});
return badge;
return row;
}
private static TextBox CreateTextBox(string text = "", bool usePassword = false) => new()

View File

@ -52,7 +52,7 @@ internal static class NodeUri
internal static Uri WebSocketEndpoint(Uri serverUrl)
{
var builder = new UriBuilder(new Uri(serverUrl, "v1/compute/nodes/connect"))
var builder = new UriBuilder(new Uri(serverUrl, "v1/software-nodes/connect"))
{
Scheme = serverUrl.Scheme == Uri.UriSchemeHttps ? "wss" : "ws"
};

View File

@ -28,7 +28,7 @@ internal static class EnrollmentClient
using var client = new HttpClient { BaseAddress = options.ServerUrl, Timeout = TimeSpan.FromSeconds(30) };
using var response = await client.PostAsJsonAsync(
"v1/compute/nodes/enroll", request, cancellationToken);
"v1/software-nodes/enroll", request, cancellationToken);
if (!response.IsSuccessStatusCode)
{
var detail = await response.Content.ReadAsStringAsync(cancellationToken);

View File

@ -0,0 +1,346 @@
using System.Text;
using System.Text.Json;
namespace Zcbot.WindowsNode;
internal sealed class JobInboxStore(string jobsDirectory)
{
private static readonly JsonSerializerOptions JsonOptions = new() { WriteIndented = true };
private static readonly HashSet<string> PlotTypes =
["line", "scatter", "line_scatter"];
private static readonly HashSet<string> OutputFormats = ["opju", "png", "svg", "pdf"];
internal bool HasPendingJobs => Directory.Exists(jobsDirectory)
&& ReadRecoverableJobs().Any(item =>
item.Terminal is null
|| item.Terminal.Value.GetProperty("status").GetString() == "succeeded"
&& !item.UploadComplete);
internal IReadOnlyList<RecoverableJob> ReadRecoverableJobs()
{
if (!Directory.Exists(jobsDirectory))
{
return [];
}
var jobs = new List<RecoverableJob>();
foreach (var requestPath in Directory.EnumerateFiles(
jobsDirectory, "request.json", SearchOption.AllDirectories))
{
try
{
using var request = JsonDocument.Parse(File.ReadAllBytes(requestPath));
var root = request.RootElement;
if (!TryReadGuid(root, "job_id", out var jobId)
|| !TryReadGuid(root, "lease_id", out var leaseId)
|| !root.TryGetProperty("request_digest", out var digestValue)
|| digestValue.GetString() is not { Length: 64 } requestDigest)
{
continue;
}
var jobDirectory = Directory.GetParent(Directory.GetParent(requestPath)!.FullName)!.FullName;
jobs.Add(new RecoverableJob(
jobId,
leaseId,
requestDigest,
root.TryGetProperty("input_transfer", out var transfer)
? transfer.Clone() : null,
ReadTerminal(Path.Combine(jobDirectory, "terminal.json")),
File.Exists(Path.Combine(jobDirectory, "upload-complete.json"))));
}
catch (Exception exception) when (
exception is JsonException or IOException or UnauthorizedAccessException)
{
}
}
return jobs;
}
internal JobOfferResult Accept(JsonElement payload)
{
if (!TryReadGuid(payload, "job_id", out var jobId)
|| !TryReadGuid(payload, "lease_id", out var leaseId)
|| !payload.TryGetProperty("request_digest", out var digestValue)
|| digestValue.ValueKind != JsonValueKind.String
|| digestValue.GetString() is not { Length: 64 } requestDigest
|| !payload.TryGetProperty("capability", out var capabilityValue)
|| capabilityValue.GetString() != "origin.plot@v1"
|| !payload.TryGetProperty("request", out var request)
|| request.ValueKind != JsonValueKind.Object
|| !payload.TryGetProperty("input_transfer", out var inputTransfer)
|| !IsValidInputTransfer(inputTransfer))
{
return JobOfferResult.Reject("invalid_offer");
}
if (!IsValidRequest(request))
{
return JobOfferResult.Reject("unsupported_request");
}
var directory = Path.Combine(jobsDirectory, jobId.ToString("D"));
var requestDirectory = Path.Combine(directory, "request");
var requestPath = Path.Combine(requestDirectory, "request.json");
Directory.CreateDirectory(requestDirectory);
if (File.Exists(requestPath))
{
try
{
using var existing = JsonDocument.Parse(File.ReadAllBytes(requestPath));
var root = existing.RootElement;
var sameDigest = root.TryGetProperty("request_digest", out var existingDigest)
&& existingDigest.GetString() == requestDigest;
if (!sameDigest)
{
return JobOfferResult.Reject("job_digest_conflict");
}
if (!TryReadGuid(root, "lease_id", out var existingLease)
|| existingLease != leaseId)
{
var updated = JsonSerializer.SerializeToUtf8Bytes(new
{
job_id = jobId,
lease_id = leaseId,
request_digest = requestDigest,
capability = "origin.plot@v1",
accepted_at = DateTimeOffset.UtcNow,
request,
input_transfer = inputTransfer,
}, JsonOptions);
AtomicWrite(requestPath, updated, overwrite: true);
}
return JobOfferResult.Accept(jobId, leaseId, requestDigest);
}
catch (JsonException)
{
return JobOfferResult.Reject("local_job_record_invalid");
}
}
var record = JsonSerializer.SerializeToUtf8Bytes(new
{
job_id = jobId,
lease_id = leaseId,
request_digest = requestDigest,
capability = "origin.plot@v1",
accepted_at = DateTimeOffset.UtcNow,
request,
input_transfer = inputTransfer,
}, JsonOptions);
try
{
AtomicWrite(requestPath, record, overwrite: false);
return JobOfferResult.Accept(jobId, leaseId, requestDigest);
}
catch (IOException)
{
return JobOfferResult.Reject("local_job_persist_failed");
}
}
private static JsonElement? ReadTerminal(string path)
{
if (!File.Exists(path))
{
return null;
}
using var document = JsonDocument.Parse(File.ReadAllBytes(path));
return document.RootElement.Clone();
}
private static void AtomicWrite(string path, byte[] content, bool overwrite)
{
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
var temporaryPath = path + ".tmp-" + Guid.NewGuid().ToString("N");
try
{
using (var stream = new FileStream(
temporaryPath, FileMode.CreateNew, FileAccess.Write, FileShare.None,
bufferSize: 4096, FileOptions.WriteThrough))
{
stream.Write(content);
stream.Flush(flushToDisk: true);
}
File.Move(temporaryPath, path, overwrite);
}
finally
{
if (File.Exists(temporaryPath)) File.Delete(temporaryPath);
}
}
private static bool IsValidRequest(JsonElement request) =>
HasOnlyProperties(request, "schema_version", "input", "plot", "output")
&& request.TryGetProperty("schema_version", out var schemaVersion)
&& schemaVersion.TryGetInt32(out var version)
&& version == 1
&& request.TryGetProperty("input", out var input)
&& input.ValueKind == JsonValueKind.Object
&& HasOnlyProperties(input, "input_id", "sheet")
&& input.TryGetProperty("input_id", out var inputId)
&& inputId.ValueKind == JsonValueKind.String
&& !string.IsNullOrWhiteSpace(inputId.GetString())
&& request.TryGetProperty("plot", out var plot)
&& plot.ValueKind == JsonValueKind.Object
&& HasOnlyProperties(
plot, "type", "x", "y", "template", "title", "x_axis", "y_axis", "legend", "error_bars")
&& plot.TryGetProperty("type", out var plotType)
&& plotType.ValueKind == JsonValueKind.String
&& PlotTypes.Contains(plotType.GetString() ?? "")
&& (!plot.TryGetProperty("title", out var title)
|| title.ValueKind == JsonValueKind.String && title.GetString()!.Length <= 500)
&& plot.TryGetProperty("x", out var x)
&& IsColumnName(x)
&& plot.TryGetProperty("y", out var y)
&& IsValidYColumns(y)
&& (!plot.TryGetProperty("template", out var template)
|| template.GetString() == "publication_double_column")
&& IsValidAxis(plot, "x_axis")
&& IsValidAxis(plot, "y_axis")
&& IsValidLegend(plot)
&& !plot.TryGetProperty("error_bars", out _)
&& request.TryGetProperty("output", out var output)
&& output.ValueKind == JsonValueKind.Object
&& HasOnlyProperties(output, "formats", "dpi", "capture_screenshots", "record_video")
&& output.TryGetProperty("formats", out var formats)
&& formats.ValueKind == JsonValueKind.Array
&& formats.GetArrayLength() > 0
&& IsValidFormats(formats)
&& (!output.TryGetProperty("dpi", out var dpi)
|| dpi.TryGetInt32(out var dpiValue) && dpiValue is >= 72 and <= 1200)
&& IsOptionalBoolean(output, "capture_screenshots")
&& IsOptionalBoolean(output, "record_video")
&& (!output.TryGetProperty("record_video", out var recordVideo)
|| recordVideo.ValueKind == JsonValueKind.False);
private static bool IsColumnName(JsonElement value) =>
value.ValueKind == JsonValueKind.String
&& value.GetString() is { Length: >= 1 and <= 128 };
private static bool IsValidFormats(JsonElement formats)
{
var values = formats.EnumerateArray().ToArray();
return values.All(item =>
item.ValueKind == JsonValueKind.String
&& OutputFormats.Contains(item.GetString() ?? ""))
&& values.Select(item => item.GetString()).Distinct(StringComparer.Ordinal).Count()
== values.Length;
}
private static bool IsValidYColumns(JsonElement value)
{
if (IsColumnName(value)) return true;
if (value.ValueKind != JsonValueKind.Array
|| value.GetArrayLength() is < 1 or > 16)
{
return false;
}
var names = value.EnumerateArray().Select(item => item.GetString()).ToArray();
return value.EnumerateArray().All(IsColumnName)
&& names.Distinct(StringComparer.Ordinal).Count() == names.Length;
}
private static bool IsValidAxis(JsonElement plot, string name)
{
if (!plot.TryGetProperty(name, out var axis)) return true;
return axis.ValueKind == JsonValueKind.Object
&& HasOnlyProperties(axis, "title", "unit", "scale")
&& (!axis.TryGetProperty("title", out var title) || title.ValueKind == JsonValueKind.String)
&& (!axis.TryGetProperty("unit", out var unit) || unit.ValueKind == JsonValueKind.String)
&& (!axis.TryGetProperty("scale", out var scale) || scale.GetString() == "linear");
}
private static bool IsValidLegend(JsonElement plot)
{
if (!plot.TryGetProperty("legend", out var legend)) return true;
return legend.ValueKind == JsonValueKind.Object
&& HasOnlyProperties(legend, "enabled", "position")
&& (!legend.TryGetProperty("enabled", out var enabled)
|| enabled.ValueKind == JsonValueKind.True)
&& (!legend.TryGetProperty("position", out var position)
|| position.GetString() == "top_right");
}
private static bool IsOptionalBoolean(JsonElement value, string name) =>
!value.TryGetProperty(name, out var property)
|| property.ValueKind is JsonValueKind.True or JsonValueKind.False;
private static bool IsValidInputTransfer(JsonElement transfer) =>
transfer.ValueKind == JsonValueKind.Object
&& HasOnlyProperties(transfer, "artifact_id", "filename", "size_bytes", "sha256", "download_path")
&& transfer.TryGetProperty("artifact_id", out var artifactId)
&& Guid.TryParse(artifactId.GetString(), out _)
&& transfer.TryGetProperty("filename", out var filename)
&& filename.ValueKind == JsonValueKind.String
&& Path.GetFileName(filename.GetString()) == filename.GetString()
&& transfer.TryGetProperty("size_bytes", out var size)
&& size.TryGetInt64(out var sizeBytes)
&& sizeBytes is >= 0 and <= 104_857_600
&& transfer.TryGetProperty("sha256", out var sha)
&& sha.GetString() is { Length: 64 }
&& transfer.TryGetProperty("download_path", out var downloadPath)
&& downloadPath.GetString()?.StartsWith("/v1/software-jobs/", StringComparison.Ordinal) == true
&& downloadPath.GetString()?.EndsWith("/input", StringComparison.Ordinal) == true;
internal string InputPath(RecoverableJob job)
{
if (job.InputTransfer is not JsonElement transfer
|| !IsValidInputTransfer(transfer))
{
throw new InvalidDataException("Stored input transfer is invalid.");
}
var filename = transfer.GetProperty("filename").GetString()!;
return Path.Combine(jobsDirectory, job.JobId.ToString("D"), "input", filename);
}
internal void WriteTerminal(
RecoverableJob job, string status, string code, string detail)
{
var path = Path.Combine(jobsDirectory, job.JobId.ToString("D"), "terminal.json");
if (File.Exists(path))
{
return;
}
var content = JsonSerializer.SerializeToUtf8Bytes(new
{
job_id = job.JobId,
lease_id = job.LeaseId,
request_digest = job.RequestDigest,
status,
error = new { code, detail },
artifact_manifest = Array.Empty<object>(),
terminal_at = DateTimeOffset.UtcNow,
}, JsonOptions);
AtomicWrite(path, content, overwrite: false);
}
private static bool HasOnlyProperties(JsonElement value, params string[] allowed)
{
var names = new HashSet<string>(allowed, StringComparer.Ordinal);
return value.EnumerateObject().All(item => names.Contains(item.Name));
}
private static bool TryReadGuid(JsonElement payload, string name, out Guid value)
{
value = Guid.Empty;
return payload.TryGetProperty(name, out var property)
&& property.ValueKind == JsonValueKind.String
&& Guid.TryParse(property.GetString(), out value);
}
}
internal sealed record RecoverableJob(
Guid JobId,
Guid LeaseId,
string RequestDigest,
JsonElement? InputTransfer,
JsonElement? Terminal,
bool UploadComplete);
internal sealed record JobOfferResult(
bool Accepted, Guid JobId, Guid LeaseId, string RequestDigest, string Reason)
{
internal static JobOfferResult Accept(Guid jobId, Guid leaseId, string requestDigest) =>
new(true, jobId, leaseId, requestDigest, "");
internal static JobOfferResult Reject(string reason) =>
new(false, Guid.Empty, Guid.Empty, "", reason);
}

View File

@ -0,0 +1,99 @@
using System.Net.Http.Headers;
using System.Security.Cryptography;
using System.Text.Json;
namespace Zcbot.WindowsNode;
internal sealed class JobInputDownloader(NodeConfig config, JobInboxStore inbox)
{
internal async Task DownloadAsync(RecoverableJob job, CancellationToken cancellationToken)
{
if (job.InputTransfer is not JsonElement transfer)
{
throw new InvalidDataException("Job input transfer is missing.");
}
var downloadPath = transfer.GetProperty("download_path").GetString()!;
if (!downloadPath.StartsWith("/v1/software-jobs/", StringComparison.Ordinal)
|| !downloadPath.EndsWith("/input", StringComparison.Ordinal)
|| !Uri.TryCreate(downloadPath, UriKind.Relative, out var relativeUri))
{
throw new InvalidDataException("Job input download path is invalid.");
}
var expectedSize = transfer.GetProperty("size_bytes").GetInt64();
var expectedSha256 = transfer.GetProperty("sha256").GetString()!;
var destination = inbox.InputPath(job);
if (File.Exists(destination))
{
await VerifyExistingAsync(destination, expectedSize, expectedSha256, cancellationToken);
return;
}
Directory.CreateDirectory(Path.GetDirectoryName(destination)!);
var temporaryPath = destination + ".tmp-" + Guid.NewGuid().ToString("N");
using var client = new HttpClient { BaseAddress = config.ServerUrl };
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", config.NodeToken);
client.DefaultRequestHeaders.Add("X-Node-Id", config.NodeId.ToString());
try
{
using var response = await client.GetAsync(
relativeUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken);
response.EnsureSuccessStatusCode();
if (response.Content.Headers.ContentLength is long contentLength
&& contentLength != expectedSize)
{
throw new InvalidDataException("Job input size header does not match the manifest.");
}
await using var source = await response.Content.ReadAsStreamAsync(cancellationToken);
await using var target = new FileStream(
temporaryPath, FileMode.CreateNew, FileAccess.Write, FileShare.None,
bufferSize: 64 * 1024, FileOptions.Asynchronous | FileOptions.WriteThrough);
using var hash = IncrementalHash.CreateHash(HashAlgorithmName.SHA256);
var buffer = new byte[64 * 1024];
long total = 0;
while (true)
{
var count = await source.ReadAsync(buffer, cancellationToken);
if (count == 0) break;
total += count;
if (total > expectedSize)
{
throw new InvalidDataException("Job input exceeded its declared size.");
}
hash.AppendData(buffer, 0, count);
await target.WriteAsync(buffer.AsMemory(0, count), cancellationToken);
}
await target.FlushAsync(cancellationToken);
target.Flush(flushToDisk: true);
var actualSha256 = Convert.ToHexString(hash.GetHashAndReset()).ToLowerInvariant();
if (total != expectedSize || actualSha256 != expectedSha256)
{
throw new InvalidDataException("Job input digest does not match the manifest.");
}
target.Close();
File.Move(temporaryPath, destination, overwrite: false);
}
finally
{
if (File.Exists(temporaryPath)) File.Delete(temporaryPath);
}
}
private static async Task VerifyExistingAsync(
string path, long expectedSize, string expectedSha256, CancellationToken cancellationToken)
{
var info = new FileInfo(path);
if (info.Length != expectedSize)
{
throw new InvalidDataException("Existing job input size does not match the manifest.");
}
await using var stream = new FileStream(
path, FileMode.Open, FileAccess.Read, FileShare.Read, 64 * 1024, FileOptions.Asynchronous);
var digest = Convert.ToHexString(
await SHA256.HashDataAsync(stream, cancellationToken)).ToLowerInvariant();
if (digest != expectedSha256)
{
throw new InvalidDataException("Existing job input digest does not match the manifest.");
}
}
}

View File

@ -0,0 +1,100 @@
using System.Net.Http.Headers;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
namespace Zcbot.WindowsNode;
internal sealed class JobOutputUploader(NodeConfig config)
{
private static readonly JsonSerializerOptions JsonOptions = new() { WriteIndented = true };
internal async Task UploadAsync(RecoverableJob job)
{
var jobDirectory = Path.Combine(
NodePaths.ForCurrentMachine().JobsDirectory, job.JobId.ToString("D"));
var completionPath = Path.Combine(jobDirectory, "upload-complete.json");
if (File.Exists(completionPath)) return;
var terminalPath = Path.Combine(jobDirectory, "terminal.json");
using var terminal = JsonDocument.Parse(await File.ReadAllBytesAsync(terminalPath));
if (terminal.RootElement.GetProperty("status").GetString() != "succeeded") return;
var manifest = terminal.RootElement.GetProperty("artifact_manifest").Clone();
using var client = new HttpClient { BaseAddress = config.ServerUrl };
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", config.NodeToken);
client.DefaultRequestHeaders.Add("X-Node-Id", config.NodeId.ToString());
client.DefaultRequestHeaders.Add("X-Lease-Id", job.LeaseId.ToString());
client.DefaultRequestHeaders.Add("X-Request-Digest", job.RequestDigest);
foreach (var artifact in manifest.EnumerateArray())
{
var localId = artifact.GetProperty("artifact_id").GetString()!;
var filename = artifact.GetProperty("filename").GetString()!;
var expectedSize = artifact.GetProperty("size_bytes").GetInt64();
var expectedDigest = artifact.GetProperty("sha256").GetString()!;
var path = Path.Combine(jobDirectory, "output", filename);
var info = new FileInfo(path);
if (!info.Exists || info.Length != expectedSize)
{
throw new InvalidDataException($"Output artifact is missing or changed: {localId}.");
}
await using (var verify = new FileStream(
path, FileMode.Open, FileAccess.Read, FileShare.Read, 64 * 1024,
FileOptions.Asynchronous | FileOptions.SequentialScan))
{
var digest = Convert.ToHexString(
await SHA256.HashDataAsync(verify)).ToLowerInvariant();
if (digest != expectedDigest)
{
throw new InvalidDataException($"Output artifact digest changed: {localId}.");
}
}
await using var stream = new FileStream(
path, FileMode.Open, FileAccess.Read, FileShare.Read, 64 * 1024,
FileOptions.Asynchronous | FileOptions.SequentialScan);
using var content = new StreamContent(stream);
content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
content.Headers.ContentLength = expectedSize;
content.Headers.Add("X-Content-SHA256", expectedDigest);
content.Headers.Add("X-Content-Length", expectedSize.ToString());
using var response = await client.PutAsync(
$"/v1/software-jobs/{job.JobId:D}/outputs/{Uri.EscapeDataString(localId)}",
content);
response.EnsureSuccessStatusCode();
}
using var completeContent = new StringContent(
JsonSerializer.Serialize(new { artifact_manifest = manifest }),
Encoding.UTF8,
"application/json");
using var completeResponse = await client.PostAsync(
$"/v1/software-jobs/{job.JobId:D}/outputs/complete", completeContent);
completeResponse.EnsureSuccessStatusCode();
var responseBody = await completeResponse.Content.ReadAsByteArrayAsync();
AtomicWrite(completionPath, responseBody);
}
private static void AtomicWrite(string path, byte[] responseBody)
{
using var response = JsonDocument.Parse(responseBody);
var content = JsonSerializer.SerializeToUtf8Bytes(new
{
completed_at = DateTimeOffset.UtcNow,
response = response.RootElement,
}, JsonOptions);
var temporary = path + ".tmp-" + Guid.NewGuid().ToString("N");
try
{
using var stream = new FileStream(
temporary, FileMode.CreateNew, FileAccess.Write, FileShare.None,
4096, FileOptions.WriteThrough);
stream.Write(content);
stream.Flush(flushToDisk: true);
File.Move(temporary, path, overwrite: false);
}
finally
{
if (File.Exists(temporary)) File.Delete(temporary);
}
}
}

View File

@ -4,11 +4,20 @@ using System.Reflection;
using System.Runtime.InteropServices;
using System.Text;
using System.Text.Json;
using System.Collections.Concurrent;
namespace Zcbot.WindowsNode;
internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>? statusChanged = null)
{
private readonly SemaphoreSlim sendLock = new(1, 1);
private readonly JobInboxStore jobInbox = new(NodePaths.ForCurrentMachine().JobsDirectory);
private readonly JobInputDownloader inputDownloader = new(
config, new JobInboxStore(NodePaths.ForCurrentMachine().JobsDirectory));
private readonly OriginWorkerRunner workerRunner = new(
new JobInboxStore(NodePaths.ForCurrentMachine().JobsDirectory));
private readonly JobOutputUploader outputUploader = new(config);
private readonly ConcurrentDictionary<Guid, Task> jobPipelines = new();
private static readonly TimeSpan[] Backoff =
[
TimeSpan.FromSeconds(1),
@ -49,6 +58,11 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
Report(NodeState.Offline, "服务端消息无效,等待重连");
Console.Error.WriteLine($"[WARN] Invalid server message: {exception.Message}");
}
catch (NodeEndpointException exception)
{
Report(NodeState.Offline, "WebSocket 握手被拒绝,请检查服务端或反向代理");
Console.Error.WriteLine($"[WARN] WebSocket handshake rejected: {exception.Message}");
}
var baseDelay = Backoff[Math.Min(attempt, Backoff.Length - 1)];
attempt++;
@ -73,17 +87,20 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
{
await socket.ConnectAsync(endpoint, cancellationToken);
}
catch (WebSocketException) when (
catch (WebSocketException exception) when (
socket.HttpStatusCode is HttpStatusCode.Unauthorized or HttpStatusCode.Forbidden)
{
Report(NodeState.AuthenticationRequired, "身份失效,需要重新注册");
throw new NodeConfigurationException(
"Node credentials were rejected. Ask an administrator to re-register this node.");
throw new NodeEndpointException(
$"WebSocket handshake returned HTTP {(int?)socket.HttpStatusCode}. "
+ "Verify that the server includes the current Windows Node routes and that "
+ $"the reverse proxy forwards WebSocket Upgrade for {endpoint.AbsolutePath}. "
+ exception.Message);
}
Console.WriteLine("[OK] Node connected.");
Report(NodeState.Online, "已连接");
await SendAsync(socket, "hello", RuntimePayload(), cancellationToken);
await ReportRecoverableJobsAsync(socket, cancellationToken);
using var heartbeatStop = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
var heartbeat = HeartbeatLoopAsync(socket, heartbeatStop.Token);
try
@ -109,10 +126,52 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
while (await timer.WaitForNextTickAsync(cancellationToken))
{
await SendAsync(socket, "heartbeat", RuntimePayload(), cancellationToken);
foreach (var job in jobInbox.ReadRecoverableJobs())
{
if (job.Terminal is JsonElement terminal
&& terminal.GetProperty("status").GetString() == "succeeded"
&& !job.UploadComplete)
{
StartJobPipeline(socket, job);
}
}
}
}
private static async Task ReceiveLoopAsync(
private async Task ReportRecoverableJobsAsync(
ClientWebSocket socket, CancellationToken cancellationToken)
{
foreach (var job in jobInbox.ReadRecoverableJobs())
{
if (job.Terminal is JsonElement terminal)
{
if (terminal.GetProperty("status").GetString() == "succeeded")
{
if (!job.UploadComplete)
{
StartJobPipeline(socket, job);
}
}
else
{
await SendAsync(socket, "job_terminal", terminal, cancellationToken);
}
continue;
}
await SendAsync(socket, "job_state", new
{
job_id = job.JobId,
lease_id = job.LeaseId,
request_digest = job.RequestDigest,
stage = "waiting_input",
progress = 0,
metrics = new { },
}, cancellationToken);
StartJobPipeline(socket, job);
}
}
private async Task ReceiveLoopAsync(
ClientWebSocket socket, CancellationToken cancellationToken)
{
var buffer = new byte[16 * 1024];
@ -122,10 +181,17 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
var result = await socket.ReceiveAsync(buffer, cancellationToken);
if (result.MessageType == WebSocketMessageType.Close)
{
if ((int?)result.CloseStatus == 4003)
{
throw new NodeConfigurationException(
"节点身份已被服务端拒绝。请在管理后台确认该 Node ID 未被禁用或删除;"
+ "若记录不存在或身份已撤销,请清除本机身份并使用新注册码重新注册。"
+ $" 服务端信息:{result.CloseStatusDescription}");
}
if (result.CloseStatus == WebSocketCloseStatus.PolicyViolation)
{
throw new NodeConfigurationException(
$"Node credentials or install identity were rejected: {result.CloseStatusDescription}");
$"节点上报被服务端拒绝:{result.CloseStatusDescription}");
}
return;
}
@ -146,11 +212,222 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
if (document.RootElement.TryGetProperty("type", out var type))
{
Console.WriteLine($"[INFO] Server message: {type.GetString()}.");
if (type.GetString() == "job_offer"
&& document.RootElement.TryGetProperty("payload", out var payload))
{
var offerResult = jobInbox.Accept(payload);
await SendAsync(
socket,
offerResult.Accepted ? "job_accept" : "job_reject",
offerResult.Accepted
? new
{
job_id = offerResult.JobId,
lease_id = offerResult.LeaseId,
request_digest = offerResult.RequestDigest,
}
: new
{
job_id = payload.TryGetProperty("job_id", out var jobId)
? jobId.GetString() : "",
lease_id = payload.TryGetProperty("lease_id", out var leaseId)
? leaseId.GetString() : "",
reason = offerResult.Reason,
},
cancellationToken);
if (offerResult.Accepted)
{
await SendAsync(socket, "job_state", new
{
job_id = offerResult.JobId,
lease_id = offerResult.LeaseId,
request_digest = offerResult.RequestDigest,
stage = "waiting_input",
progress = 0,
metrics = new { },
}, cancellationToken);
var acceptedJob = jobInbox.ReadRecoverableJobs()
.Single(item => item.JobId == offerResult.JobId);
StartJobPipeline(socket, acceptedJob);
}
}
else if (type.GetString() == "job_cancel"
&& document.RootElement.TryGetProperty("payload", out var cancelPayload)
&& TryCancelJob(cancelPayload, out var cancelledJob))
{
workerRunner.Cancel(cancelledJob.JobId);
jobInbox.WriteTerminal(
cancelledJob, "cancelled", "USER_CANCELLED", "Cancelled by user.");
await SendAsync(socket, "job_terminal", new
{
job_id = cancelledJob.JobId,
lease_id = cancelledJob.LeaseId,
request_digest = cancelledJob.RequestDigest,
status = "cancelled",
error = new { code = "USER_CANCELLED", detail = "Cancelled by user." },
artifact_manifest = Array.Empty<object>(),
}, cancellationToken);
}
}
message.SetLength(0);
}
}
private bool TryCancelJob(JsonElement payload, out RecoverableJob job)
{
job = null!;
if (!payload.TryGetProperty("job_id", out var jobIdValue)
|| !Guid.TryParse(jobIdValue.GetString(), out var jobId)
|| !payload.TryGetProperty("lease_id", out var leaseIdValue)
|| !Guid.TryParse(leaseIdValue.GetString(), out var leaseId)
|| !payload.TryGetProperty("request_digest", out var digestValue))
{
return false;
}
job = jobInbox.ReadRecoverableJobs().SingleOrDefault(item =>
item.JobId == jobId
&& item.LeaseId == leaseId
&& item.RequestDigest == digestValue.GetString()
&& item.Terminal is null)!;
return job is not null;
}
private void StartJobPipeline(ClientWebSocket socket, RecoverableJob job)
{
var completion = new TaskCompletionSource(
TaskCreationOptions.RunContinuationsAsynchronously);
if (jobPipelines.TryAdd(job.JobId, completion.Task))
{
_ = RunJobPipelineAndReleaseAsync(socket, job, completion);
}
}
private async Task RunJobPipelineAndReleaseAsync(
ClientWebSocket socket, RecoverableJob job, TaskCompletionSource completion)
{
try
{
await RunJobPipelineAsync(socket, job);
}
finally
{
completion.TrySetResult();
jobPipelines.TryRemove(job.JobId, out _);
}
}
private async Task RunJobPipelineAsync(ClientWebSocket socket, RecoverableJob job)
{
try
{
if (job.Terminal is null)
{
await inputDownloader.DownloadAsync(job, CancellationToken.None);
var afterDownload = jobInbox.ReadRecoverableJobs()
.Single(item => item.JobId == job.JobId);
if (afterDownload.Terminal is JsonElement cancelledTerminal)
{
await TrySendAsync(socket, "job_terminal", cancelledTerminal);
return;
}
await TrySendAsync(socket, "job_state", new
{
job_id = job.JobId,
lease_id = job.LeaseId,
request_digest = job.RequestDigest,
stage = "ready_to_run",
progress = 5,
metrics = new { input_bytes = job.InputTransfer?.GetProperty("size_bytes").GetInt64() },
});
await TrySendAsync(socket, "job_state", new
{
job_id = job.JobId,
lease_id = job.LeaseId,
request_digest = job.RequestDigest,
stage = "origin_running",
progress = 10,
metrics = new { },
});
await workerRunner.RunAsync(job);
}
var refreshed = jobInbox.ReadRecoverableJobs()
.Single(item => item.JobId == job.JobId);
if (refreshed.Terminal is not JsonElement terminal)
{
throw new InvalidDataException("Origin worker did not create a terminal record.");
}
if (terminal.GetProperty("status").GetString() != "succeeded")
{
await TrySendAsync(socket, "job_terminal", terminal);
return;
}
await TrySendAsync(socket, "job_state", new
{
job_id = job.JobId,
lease_id = job.LeaseId,
request_digest = job.RequestDigest,
stage = "uploading_outputs",
progress = 90,
metrics = new { },
});
await outputUploader.UploadAsync(refreshed);
}
catch (Exception exception) when (
exception is HttpRequestException
or IOException
or JsonException
or UnauthorizedAccessException
or InvalidDataException)
{
var current = jobInbox.ReadRecoverableJobs()
.Single(item => item.JobId == job.JobId);
if (current.Terminal is JsonElement terminal
&& terminal.GetProperty("status").GetString() == "succeeded")
{
Console.Error.WriteLine($"[WARN] Output upload deferred: {exception.Message}");
return;
}
jobInbox.WriteTerminal(
job,
"failed",
"INPUT_DOWNLOAD_FAILED",
exception.Message[..Math.Min(exception.Message.Length, 500)]);
var failedTerminal = jobInbox.ReadRecoverableJobs()
.Single(item => item.JobId == job.JobId).Terminal;
if (failedTerminal is JsonElement payload)
{
await TrySendAsync(socket, "job_terminal", payload);
}
}
}
private async Task TrySendTerminalAsync(ClientWebSocket socket, RecoverableJob job)
{
var terminal = jobInbox.ReadRecoverableJobs()
.Single(item => item.JobId == job.JobId).Terminal;
if (terminal is JsonElement terminalPayload)
{
await TrySendAsync(socket, "job_terminal", terminalPayload);
}
}
private async Task TrySendAsync(ClientWebSocket socket, string type, object payload)
{
if (socket.State != WebSocketState.Open)
{
return;
}
try
{
await SendAsync(socket, type, payload, CancellationToken.None);
}
catch (Exception exception) when (
exception is WebSocketException or IOException or ObjectDisposedException)
{
Console.Error.WriteLine($"[WARN] Job report deferred: {exception.Message}");
}
}
private void Report(NodeState state, string message) =>
statusChanged?.Invoke(NodeStatus.Create(state, message));
@ -165,13 +442,22 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
sent_at = DateTimeOffset.UtcNow,
payload,
});
await socket.SendAsync(
envelope, WebSocketMessageType.Text, endOfMessage: true, cancellationToken);
await sendLock.WaitAsync(cancellationToken);
try
{
await socket.SendAsync(
envelope, WebSocketMessageType.Text, endOfMessage: true, cancellationToken);
}
finally
{
sendLock.Release();
}
}
private object RuntimePayload()
{
var root = Path.GetPathRoot(Environment.SystemDirectory) ?? "C:\\";
var origin = OriginRuntimeProbe.Detect();
return new
{
install_id = config.InstallId,
@ -179,9 +465,17 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
node_version = Assembly.GetExecutingAssembly().GetName().Version?.ToString(3) ?? "0.1.0",
os_version = RuntimeInformation.OSDescription,
capabilities = config.Capabilities,
available_slots = 1,
available_slots = origin.Health == "ready" && !jobInbox.HasPendingJobs ? 1 : 0,
disk_free_bytes = new DriveInfo(root).AvailableFreeSpace,
desktop_session = Environment.UserInteractive,
origin = new
{
software = origin.Software,
software_version = origin.SoftwareVersion,
adapter_version = origin.AdapterVersion,
health = origin.Health,
detail = origin.Detail,
},
};
}
}

View File

@ -34,15 +34,18 @@ internal sealed record EnrollResponse(
[property: JsonPropertyName("heartbeat_seconds")] int HeartbeatSeconds,
[property: JsonPropertyName("max_concurrency")] int MaxConcurrency);
internal sealed record NodePaths(string RootDirectory, string ConfigPath)
internal sealed record NodePaths(string RootDirectory, string ConfigPath, string JobsDirectory)
{
internal static NodePaths ForCurrentMachine()
{
var root = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
"Zcbot", "WindowsNode");
return new NodePaths(root, Path.Combine(root, "node.json"));
return new NodePaths(
root, Path.Combine(root, "node.json"), Path.Combine(root, "jobs"));
}
}
internal sealed class NodeConfigurationException(string message) : Exception(message);
internal sealed class NodeEndpointException(string message) : Exception(message);

View File

@ -0,0 +1,102 @@
using Microsoft.Win32;
using System.Security;
namespace Zcbot.WindowsNode;
internal sealed record OriginRuntimeStatus(
string Software,
string? SoftwareVersion,
string AdapterVersion,
string Health,
string Detail);
internal static class OriginRuntimeProbe
{
private static readonly Lazy<OriginRuntimeStatus> Current = new(DetectCore);
private const string AutomationProgId = @"Origin.ApplicationSI\CLSID";
internal static OriginRuntimeStatus Detect() => Current.Value;
private static OriginRuntimeStatus DetectCore()
{
try
{
var version = FindInstalledVersion();
using var automationKey = Registry.ClassesRoot.OpenSubKey(AutomationProgId);
var automationRegistered = automationKey is not null;
if (version is null && !automationRegistered)
{
return Status(null, "unavailable", "未检测到 Origin/OriginPro 安装");
}
if (!automationRegistered)
{
return Status(version, "unavailable", "已检测到 Origin但 COM 自动化组件未注册");
}
if (!Environment.UserInteractive)
{
return Status(version, "unavailable", "Origin 需要交互式 Windows 桌面会话");
}
var interpreter = OriginWorkerRuntime.ResolveInterpreter();
if (interpreter is null)
{
return Status(
version,
"unavailable",
"Origin 可用,但固定 Python 运行时缺失;请配置 ZCBOT_ORIGIN_PYTHON");
}
return Status(version, "ready", $"Origin COM 与固定 Python 运行时可用({interpreter}");
}
catch (Exception exception) when (
exception is SecurityException or UnauthorizedAccessException or IOException)
{
return Status(null, "unavailable", $"Origin 运行时探测失败:{exception.Message}");
}
}
private static OriginRuntimeStatus Status(string? version, string health, string detail) =>
new("OriginPro", version, "0.2.0", health, detail);
private static string? FindInstalledVersion()
{
var candidates = new List<string>();
foreach (var hive in new[] { RegistryHive.LocalMachine, RegistryHive.CurrentUser })
{
foreach (var view in new[] { RegistryView.Registry64, RegistryView.Registry32 })
{
using var baseKey = RegistryKey.OpenBaseKey(hive, view);
using var uninstall = baseKey.OpenSubKey(
@"SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall");
if (uninstall is null)
{
continue;
}
foreach (var keyName in uninstall.GetSubKeyNames())
{
using var product = uninstall.OpenSubKey(keyName);
var name = product?.GetValue("DisplayName") as string;
var publisher = product?.GetValue("Publisher") as string;
if (!IsOriginProduct(name, publisher))
{
continue;
}
var version = product?.GetValue("DisplayVersion") as string;
if (!string.IsNullOrWhiteSpace(version))
{
candidates.Add(version.Trim());
}
}
}
}
return candidates.OrderByDescending(ParseVersion).ThenByDescending(x => x).FirstOrDefault();
}
private static bool IsOriginProduct(string? name, string? publisher) =>
!string.IsNullOrWhiteSpace(name)
&& (name.Equals("Origin", StringComparison.OrdinalIgnoreCase)
|| name.StartsWith("Origin ", StringComparison.OrdinalIgnoreCase)
|| name.StartsWith("OriginPro", StringComparison.OrdinalIgnoreCase))
&& (publisher?.Contains("OriginLab", StringComparison.OrdinalIgnoreCase) ?? false);
private static Version ParseVersion(string value) =>
Version.TryParse(value, out var version) ? version : new Version(0, 0);
}

View File

@ -0,0 +1,164 @@
using System.Collections.Concurrent;
using System.Diagnostics;
using System.Text;
using System.Text.Json;
namespace Zcbot.WindowsNode;
internal sealed class OriginWorkerRunner(JobInboxStore inbox)
{
private static readonly TimeSpan WorkerTimeout = TimeSpan.FromMinutes(30);
private readonly ConcurrentDictionary<Guid, Task> active = new();
private readonly ConcurrentDictionary<Guid, CancellationTokenSource> cancellations = new();
internal Task RunAsync(RecoverableJob job) =>
active.GetOrAdd(job.JobId, _ => RunOnceAsync(job, CancellationFor(job.JobId).Token));
internal void Cancel(Guid jobId)
{
CancellationFor(jobId).Cancel();
}
private CancellationTokenSource CancellationFor(Guid jobId) =>
cancellations.GetOrAdd(jobId, _ => new CancellationTokenSource());
private async Task RunOnceAsync(RecoverableJob job, CancellationToken cancellationToken)
{
try
{
var paths = NodePaths.ForCurrentMachine();
var jobDirectory = Path.Combine(paths.JobsDirectory, job.JobId.ToString("D"));
var terminalPath = Path.Combine(jobDirectory, "terminal.json");
if (File.Exists(terminalPath)) return;
var markerPath = Path.Combine(jobDirectory, "worker-started.json");
if (File.Exists(markerPath))
{
inbox.WriteTerminal(
job,
"failed",
"NODE_RESTARTED_DURING_JOB",
"The node restarted after Origin execution began and cannot prove the prior worker state.");
return;
}
var interpreter = OriginWorkerRuntime.ResolveInterpreter()
?? throw new InvalidOperationException("The fixed Origin Python interpreter is unavailable.");
var workerScript = Path.GetFullPath(
Path.Combine(AppContext.BaseDirectory, "origin-worker", "worker.py"));
if (!File.Exists(workerScript))
{
throw new FileNotFoundException("The fixed Origin worker script is missing.", workerScript);
}
WriteMarker(markerPath, interpreter, workerScript);
var startInfo = new ProcessStartInfo
{
FileName = interpreter,
WorkingDirectory = jobDirectory,
UseShellExecute = false,
CreateNoWindow = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
StandardOutputEncoding = Encoding.UTF8,
StandardErrorEncoding = Encoding.UTF8,
};
startInfo.ArgumentList.Add(workerScript);
startInfo.ArgumentList.Add(jobDirectory);
using var process = Process.Start(startInfo)
?? throw new InvalidOperationException("The fixed Origin worker did not start.");
var stdout = process.StandardOutput.ReadToEndAsync();
var stderr = process.StandardError.ReadToEndAsync();
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeout.CancelAfter(WorkerTimeout);
try
{
await process.WaitForExitAsync(timeout.Token);
}
catch (OperationCanceledException)
{
process.Kill(entireProcessTree: true);
if (cancellationToken.IsCancellationRequested)
{
inbox.WriteTerminal(job, "cancelled", "USER_CANCELLED", "Cancelled by user.");
}
else
{
inbox.WriteTerminal(job, "failed", "ORIGIN_WORKER_TIMEOUT", "Origin worker exceeded 30 minutes.");
}
return;
}
var output = await stdout;
var error = await stderr;
WriteDiagnostic(jobDirectory, output, error, process.ExitCode);
if (!File.Exists(terminalPath))
{
inbox.WriteTerminal(
job,
"failed",
"ORIGIN_WORKER_NO_TERMINAL",
$"Origin worker exited with code {process.ExitCode} without terminal.json.");
}
}
catch (Exception exception) when (
exception is IOException
or JsonException
or UnauthorizedAccessException
or InvalidOperationException)
{
inbox.WriteTerminal(job, "failed", "ORIGIN_WORKER_START_FAILED", exception.Message[..Math.Min(500, exception.Message.Length)]);
}
finally
{
active.TryRemove(job.JobId, out _);
if (cancellations.TryRemove(job.JobId, out var cancellation)) cancellation.Dispose();
}
}
private static void WriteMarker(string path, string interpreter, string workerScript)
{
var value = JsonSerializer.SerializeToUtf8Bytes(new
{
started_at = DateTimeOffset.UtcNow,
node_pid = Environment.ProcessId,
interpreter,
worker_script = workerScript,
});
using var stream = new FileStream(
path, FileMode.CreateNew, FileAccess.Write, FileShare.None,
bufferSize: 4096, FileOptions.WriteThrough);
stream.Write(value);
stream.Flush(flushToDisk: true);
}
private static void WriteDiagnostic(string jobDirectory, string output, string error, int exitCode)
{
var logs = Path.Combine(jobDirectory, "logs");
Directory.CreateDirectory(logs);
var value = JsonSerializer.Serialize(new
{
exit_code = exitCode,
stdout = output[..Math.Min(output.Length, 16 * 1024)],
stderr = error[..Math.Min(error.Length, 16 * 1024)],
});
File.WriteAllText(Path.Combine(logs, "worker-process.json"), value, Encoding.UTF8);
}
}
internal static class OriginWorkerRuntime
{
internal static string? ResolveInterpreter()
{
var paths = NodePaths.ForCurrentMachine();
var configured = Environment.GetEnvironmentVariable("ZCBOT_ORIGIN_PYTHON");
var candidate = string.IsNullOrWhiteSpace(configured)
? Path.Combine(paths.RootDirectory, "runtimes", "origin", "python.exe")
: configured;
if (!Path.IsPathFullyQualified(candidate)) return null;
var resolved = Path.GetFullPath(candidate);
return File.Exists(resolved)
&& Path.GetFileName(resolved).Equals("python.exe", StringComparison.OrdinalIgnoreCase)
? resolved
: null;
}
}

View File

@ -16,6 +16,7 @@ internal sealed class TrayApplicationContext : ApplicationContext
this.store = store;
form = new ConfigurationForm();
form.RegisterRequested += RegisterAsync;
form.ReconnectRequested += RestartConnection;
form.ResetIdentityRequested += ResetIdentity;
statusItem = new ToolStripMenuItem("尚未注册") { Enabled = false };

View File

@ -11,4 +11,10 @@
<RootNamespace>Zcbot.WindowsNode</RootNamespace>
<Version>0.1.0</Version>
</PropertyGroup>
<ItemGroup>
<Content Include="..\origin-worker\worker.py">
<Link>origin-worker\worker.py</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
</Project>

View File

@ -0,0 +1,32 @@
param(
[Parameter(Mandatory = $true)]
[string]$BootstrapPython,
[string]$RuntimeDirectory = "$env:ProgramData\Zcbot\WindowsNode\runtimes\origin"
)
$ErrorActionPreference = "Stop"
$python = (Resolve-Path -LiteralPath $BootstrapPython).Path
if ([IO.Path]::GetFileName($python) -ne "python.exe") {
throw "BootstrapPython must point to python.exe."
}
$requirements = Join-Path $PSScriptRoot "origin-worker\requirements.txt"
if (-not (Test-Path -LiteralPath $requirements -PathType Leaf)) {
throw "Pinned Origin worker requirements are missing."
}
$runtime = [IO.Path]::GetFullPath($RuntimeDirectory)
if ($runtime -eq [IO.Path]::GetPathRoot($runtime)) {
throw "RuntimeDirectory cannot be a drive root."
}
if (-not (Test-Path -LiteralPath (Join-Path $runtime "python.exe"))) {
& $python -m venv $runtime
if ($LASTEXITCODE -ne 0) { throw "Failed to create the Origin runtime." }
}
$runtimePython = Join-Path $runtime "python.exe"
& $runtimePython -m pip install --requirement $requirements
if ($LASTEXITCODE -ne 0) { throw "Failed to install the pinned Origin runtime packages." }
& $runtimePython -c "import originpro, openpyxl; print('[OK] Origin worker Python packages are available.')"
if ($LASTEXITCODE -ne 0) { throw "Origin runtime import verification failed." }
Write-Output "[OK] Fixed Origin runtime installed: $runtimePython"
Write-Output "[INFO] Restart zcbot Windows Node to refresh runtime health."

View File

@ -0,0 +1,2 @@
originpro==1.1.15
openpyxl==3.1.5

View File

@ -0,0 +1,252 @@
"""Fixed Origin adapter for origin.plot@v1.
This process accepts exactly one argument: a Node-created job directory. It never
installs packages, evaluates user code, downloads data, or resolves paths from the
request. terminal.json is its only terminal-state contract.
"""
from __future__ import annotations
import csv
import hashlib
import json
import os
import sys
from datetime import datetime, timezone
from importlib.metadata import PackageNotFoundError, version
from pathlib import Path
from typing import Any
PLOT_TYPES = {"line": "l", "scatter": "s", "line_scatter": "y"}
FORMATS = {"opju", "png", "svg", "pdf"}
def _atomic_json(path: Path, value: Any) -> None:
temporary = path.with_name(path.name + ".tmp-" + os.urandom(8).hex())
try:
with temporary.open("w", encoding="utf-8", newline="\n") as handle:
json.dump(value, handle, ensure_ascii=False, indent=2)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
finally:
temporary.unlink(missing_ok=True)
def _read_rows(path: Path, sheet: str | None) -> tuple[list[str], list[list[Any]]]:
suffix = path.suffix.lower()
if suffix == ".csv":
with path.open("r", encoding="utf-8-sig", newline="") as handle:
rows = list(csv.reader(handle))
if len(rows) < 2:
raise ValueError("CSV_INPUT_EMPTY")
return [str(item) for item in rows[0]], rows[1:]
if suffix == ".json":
value = json.loads(path.read_text(encoding="utf-8"))
if isinstance(value, list) and value and all(isinstance(item, dict) for item in value):
headers = list(value[0])
return headers, [[item.get(name) for name in headers] for item in value]
if isinstance(value, dict) and value and all(isinstance(item, list) for item in value.values()):
headers = list(value)
length = max(len(value[name]) for name in headers)
return headers, [[value[name][index] if index < len(value[name]) else None for name in headers] for index in range(length)]
raise ValueError("JSON_INPUT_SHAPE_UNSUPPORTED")
if suffix == ".xlsx":
from openpyxl import load_workbook
workbook = load_workbook(path, read_only=True, data_only=True)
try:
worksheet = workbook[sheet] if sheet else workbook.active
rows = list(worksheet.iter_rows(values_only=True))
finally:
workbook.close()
if len(rows) < 2:
raise ValueError("XLSX_INPUT_EMPTY")
return [str(item or "") for item in rows[0]], [list(row) for row in rows[1:]]
raise ValueError("INPUT_TYPE_UNSUPPORTED")
def _column_index(headers: list[str], value: Any, field: str) -> int:
if not isinstance(value, str) or value not in headers:
raise ValueError(f"{field.upper()}_COLUMN_NOT_FOUND")
return headers.index(value)
def _manifest(path: Path, media_type: str) -> dict[str, Any]:
return {
"artifact_id": {
"project.opju": "project",
"figure.png": "figure_png",
"figure.svg": "figure_svg",
"figure.pdf": "figure_pdf",
"plot-spec.json": "plot_spec",
"provenance.json": "provenance",
}[path.name],
"filename": path.name,
"media_type": media_type,
"size_bytes": path.stat().st_size,
"sha256": _file_sha256(path),
}
def _file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _axis_title(axis: Any, fallback: str) -> str:
if not isinstance(axis, dict):
return fallback
title = str(axis.get("title") or fallback)
unit = str(axis.get("unit") or "")
return f"{title} ({unit})" if unit else title
def _validate_artifact(path: Path, extension: str) -> None:
if not path.is_file() or path.stat().st_size == 0:
raise RuntimeError(f"{extension.upper()}_EXPORT_EMPTY")
head = path.read_bytes()[:1024]
if extension == "png" and not head.startswith(b"\x89PNG\r\n\x1a\n"):
raise RuntimeError("PNG_EXPORT_INVALID")
if extension == "pdf" and not head.startswith(b"%PDF-"):
raise RuntimeError("PDF_EXPORT_INVALID")
if extension == "svg" and b"<svg" not in head.lower():
raise RuntimeError("SVG_EXPORT_INVALID")
if extension == "opju" and len(head) < 64:
raise RuntimeError("OPJU_EXPORT_INVALID")
def run(job_dir: Path) -> list[dict[str, Any]]:
job_dir = job_dir.resolve(strict=True)
request_record = json.loads((job_dir / "request" / "request.json").read_text(encoding="utf-8"))
request = request_record["request"]
input_files = [path for path in (job_dir / "input").iterdir() if path.is_file() and not path.name.startswith(".")]
if len(input_files) != 1:
raise ValueError("INPUT_FILE_COUNT_INVALID")
headers, rows = _read_rows(input_files[0], request["input"].get("sheet"))
plot_spec = request["plot"]
plot_type = plot_spec["type"]
if plot_type not in PLOT_TYPES:
raise ValueError("PLOT_TYPE_NOT_IMPLEMENTED")
x_index = _column_index(headers, plot_spec.get("x"), "x")
y_names = plot_spec.get("y")
if isinstance(y_names, str):
y_names = [y_names]
if not isinstance(y_names, list) or not y_names:
raise ValueError("Y_COLUMNS_REQUIRED")
y_indexes = [_column_index(headers, name, "y") for name in y_names]
import originpro as op
output = job_dir / "output"
output.mkdir(exist_ok=True)
op.set_show(False)
try:
op.new()
worksheet = op.new_sheet("w", lname="Data")
for index, header in enumerate(headers):
worksheet.from_list(index, [row[index] if index < len(row) else None for row in rows], lname=header)
graph = op.new_graph(template={"line": "line", "scatter": "scatter", "line_scatter": "linesymb"}[plot_type])
layer = graph[0]
for y_index in y_indexes:
layer.add_plot(worksheet, coly=y_index, colx=x_index, type=PLOT_TYPES[plot_type])
layer.rescale()
layer.axis("x").title = _axis_title(plot_spec.get("x_axis"), str(plot_spec.get("x") or "X"))
layer.axis("y").title = _axis_title(plot_spec.get("y_axis"), "Y")
if plot_spec.get("title"):
title = layer.add_label(str(plot_spec["title"]))
title.set_int("fsize", 18)
title.set_int("left", 2200)
title.set_int("top", 120)
formats = request["output"]["formats"]
if any(item not in FORMATS for item in formats):
raise ValueError("OUTPUT_FORMAT_UNSUPPORTED")
artifacts: list[dict[str, Any]] = []
if "opju" in formats:
project = output / "project.opju"
op.save(str(project))
_validate_artifact(project, "opju")
artifacts.append(_manifest(project, "application/x-origin-project"))
media = {"png": "image/png", "svg": "image/svg+xml", "pdf": "application/pdf"}
dpi = request["output"].get("dpi", 300)
if not isinstance(dpi, int) or isinstance(dpi, bool) or not 72 <= dpi <= 1200:
raise ValueError("OUTPUT_DPI_INVALID")
pixel_width = round(dpi * 160 / 25.4)
for extension in ("png", "svg", "pdf"):
if extension in formats:
target = output / f"figure.{extension}"
exported = Path(graph.save_fig(
str(target),
type=extension,
width=pixel_width if extension == "png" else 0,
ratio=100 if extension in {"svg", "pdf"} else 0,
)).resolve()
if exported != target.resolve() or not target.is_file():
raise RuntimeError(f"{extension.upper()}_EXPORT_FAILED")
_validate_artifact(target, extension)
artifacts.append(_manifest(target, media[extension]))
try:
originpro_version = version("originpro")
except PackageNotFoundError:
originpro_version = "embedded"
provenance = {
"adapter_version": "0.2.0",
"originpro_version": originpro_version,
"request_digest": request_record["request_digest"],
"input_sha256": _file_sha256(input_files[0]),
"requested_dpi": dpi,
"png_pixel_width": pixel_width,
}
plot_spec_path = output / "plot-spec.json"
provenance_path = output / "provenance.json"
_atomic_json(plot_spec_path, request)
_atomic_json(provenance_path, provenance)
artifacts.append(_manifest(plot_spec_path, "application/json"))
artifacts.append(_manifest(provenance_path, "application/json"))
return artifacts
finally:
if op.oext:
op.exit()
def main() -> int:
if len(sys.argv) != 2:
print("[ERR] Usage: worker.py <job-directory>", file=sys.stderr)
return 2
job_dir = Path(sys.argv[1])
request_record: dict[str, Any] = {}
try:
request_record = json.loads((job_dir / "request" / "request.json").read_text(encoding="utf-8"))
artifacts = run(job_dir)
terminal = {
"job_id": request_record["job_id"],
"lease_id": request_record["lease_id"],
"request_digest": request_record["request_digest"],
"status": "succeeded",
"error": {},
"artifact_manifest": artifacts,
"terminal_at": datetime.now(timezone.utc).isoformat(),
}
_atomic_json(job_dir / "artifacts.json", artifacts)
_atomic_json(job_dir / "terminal.json", terminal)
print("[OK] Origin job completed.")
return 0
except Exception as exception:
terminal = {
"job_id": request_record.get("job_id", ""),
"lease_id": request_record.get("lease_id", ""),
"request_digest": request_record.get("request_digest", ""),
"status": "failed",
"error": {"code": type(exception).__name__, "detail": str(exception)[:500]},
"artifact_manifest": [],
"terminal_at": datetime.now(timezone.utc).isoformat(),
}
_atomic_json(job_dir / "terminal.json", terminal)
print(f"[ERR] {type(exception).__name__}: {exception}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())