feat(software): add local monitoring and multi-artifact jobs
This commit is contained in:
parent
89dac0d119
commit
13ba6b1a61
|
|
@ -12,6 +12,8 @@
|
|||
- 专业软件任务完成后，输出文件会立即显示在当前对话的文件面板；任务中心收起时也不会再遮挡发送按钮。
|
||||
- 专业软件生成的正式文件会保存在当前对话的正确目录并记录来源；运行参数和溯源信息收纳到隐藏元数据目录，减少产物列表噪声。
|
||||
- 专业软件任务即使在完成回执时短暂断线，也不会持续占用 Windows Node；后续任务可继续执行，已完成结果会在后台自动确认。
|
||||
- Windows Node 窗口新增本机任务列表，可查看任务标题、输入文件、执行阶段、进度、时间、Job ID 和失败详情；任务状态会在本机持久保存，断线或重启后仍可查看。
|
||||
- Origin 绘图任务支持同时绑定多个 CSV、XLSX 或 JSON 输入、从不同输入中选择数据系列，并显式选择需要生成的工程文件和多种图形格式。
|
||||
|
||||
## 0.65.2 — 2026-08-13
|
||||
|
||||
|
|
|
|||
|
|
@ -458,21 +458,21 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB)
|
|||
|
||||
### 8.16 Windows Node 内网 MVP（implementation，2026-08-12）
|
||||
|
||||
第一阶段以 `docs/windows-node-mvp-intranet.md` 为实现契约：Windows Node 只作为受控执行节点，通过出站 HTTP/WS 主动连接 zcbot；首批能力固定为 `origin.plot@v1`。长期方案中的 mTLS、Service/DesktopRunner 双进程、完整租约与多节点调度暂不进入 MVP，但 URL path、Node ID、Bearer Header 和任务协议保留原位升级空间。
|
||||
第一阶段以 `docs/windows-node-mvp-intranet.md` 为实现契约：Windows Node 只作为受控执行节点，通过出站 HTTP/WS 主动连接 zcbot；首批能力固定为 Origin 绘图。长期方案中的 mTLS、Service/DesktopRunner 双进程、完整租约与多节点调度暂不进入 MVP，但 URL path、Node ID、Bearer Header 和任务协议保留原位升级空间。
|
||||
|
||||
云端控制面使用独立的 `software_node_enrollments` 与 `software_nodes`，不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵，数据库只保存 SHA-256 摘要；节点注册在行锁事务中校验有效期、预期名称和允许能力，成功后原子消费。每个节点获得独立高熵 Token，数据库只保存 bcrypt 强哈希，明文仅在注册响应出现一次。
|
||||
|
||||
Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/software-nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活，新连接关闭旧连接；`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态，再关闭现有连接；断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程，生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。
|
||||
|
||||
第二阶段已增加 `software_jobs`（专业软件任务）账本与 `origin.plot@v1` 的 offer/accept 骨架。用户只能在本人 task 下以幂等键提交固定 schema；云端规范化请求并记录 SHA-256，按当前进程真实在线、能力匹配、健康且有空闲 slot 的 Node 创建短期 offer。Node 再次校验 schema、图形类型和输出格式，使用 write-through、flush 与原子 rename 先落本机任务目录，再回 `job_accept`；重复 job 只有 digest 一致才接受。过期或发送失败的 offer 回到队列，lease、Node 和 digest 不匹配的响应被拒绝。Node 接收后云端进入 `dispatched` 而非 `running`，并将 slot 降为 0；只有固定 Worker 真正启动后才进入 `origin_running`。
|
||||
第二阶段已增加 `software_jobs`（专业软件任务）账本与 `origin.plot@v2` 的 offer/accept 骨架。用户只能在本人 task 下以幂等键提交固定 schema；云端规范化请求并记录 SHA-256，按当前进程真实在线、能力匹配、健康且有空闲 slot 的 Node 创建短期 offer。Node 再次校验 schema、图形类型和输出格式，使用 write-through、flush 与原子 rename 先落本机任务目录，再回 `job_accept`；重复 job 只有 digest 一致才接受。过期或发送失败的 offer 回到队列，lease、Node 和 digest 不匹配的响应被拒绝。Node 接收后云端进入 `dispatched` 而非 `running`，并将 slot 降为 0；只有固定 Worker 真正启动后才进入软件无关的 `software_running`，具体软件和操作由 capability/request 表达。
|
||||
|
||||
第三阶段补齐输入下载与恢复状态协议：`input_id` 固定为 artifact UUID。已有 artifact 可直接提交；普通 task 文件先调用 `register_artifact(path)` 登记稳定身份，该动作不把文件发布为聊天交付物。提交时快照文件名、大小和 SHA-256，只允许 CSV/XLSX/JSON 且不超过 100 MiB。Node 以自身 Bearer 身份访问任务绑定的只读下载端点，流式写入本 job 的 `input/`，同时限制声明大小并校验 SHA-256，完成后原子 rename；不暴露工作区路径。Node 会原子读取/补报 `terminal.json`，断线后云端把活动任务标记 `disconnected` 并保留 Node/lease，重连按 job、lease、digest 恢复下载或幂等补报终态，不自动重派。
|
||||
第三阶段补齐多输入下载与恢复状态协议：请求使用通用 `inputs[]` 绑定 1–16 个 artifact，并由 `operation.plot.series[]` 以输入 key 引用各自的 X/Y 列；单文件不超过 100 MiB、总量不超过 512 MiB。已有 artifact 可直接提交；普通 task 文件先逐个调用 `register_artifact(path)` 登记稳定身份。Node 以自身 Bearer 身份访问每个任务绑定的只读下载端点，流式写入本 job 的 `input/<key>/<filename>`，同时校验大小与 SHA-256；不暴露工作区路径。Node 会原子读取/补报 `terminal.json`，断线后云端把活动任务标记 `disconnected` 并保留 Node/lease，重连按 job、lease、digest 恢复下载或幂等补报终态，不自动重派。Node UI 的“本机任务”只读取已经 accept 到本机的任务目录，不查询云端未派发 Job；每个任务以原子 `state.json` 持久化 `accepted/downloading_inputs/ready_to_run/software_running/uploading_outputs/succeeded/failed/cancelled` 通用阶段，窗口再与 request、terminal、upload-complete 合并成可恢复视图。
|
||||
|
||||
第四阶段落地固定 Origin Worker：Node 仅从管理员安装的固定 Python 运行时启动随程序发布的 `worker.py`，参数只有本机 job 目录；请求不能指定脚本、解释器或文件路径。Worker 使用 `originpro` 生成 OPJU、PNG、SVG、PDF、plot spec 和 provenance，校验产物签名并原子写入终态；当前受控图形仅含 line、scatter、line_scatter 和双栏出版布局。进程内 pipeline 按 job 去重，并脱离单次 WebSocket 的取消令牌运行；连接中断只延迟状态/终态上报。Node 进程若在 Worker 启动后重启，则保守失败而不重复驱动 Origin，避免无法证明的双执行。
|
||||
|
||||
第五阶段完成输出上传与发布：Node 只按固定 manifest ID 逐项流式 PUT，并携带 Node、lease、request digest 与内容摘要；云端重新绑定任务身份，不信 Node 提供的路径或媒体类型。文件先进入用户根下隐藏暂存区，固定文件名、单文件/总大小和 SHA-256 全部验证后，把 plot spec、provenance 整理进 `.meta/`，再将完整目录原子移动到 `<working_dir>/origin/<job_id>/`。PNG/SVG/PDF/OPJU 等正式输出登记平台 artifact UUID 和 `software_job_id`，`.meta/` 只落真实文件；成功状态返回 task-relative `output_dir`，Agent 以该目录为起点按需搜索。重复 PUT、complete 和重连均按摘要幂等；部分上传不可见，只有完整集合才能发布。Origin 执行槽与上传确认是两个正交状态：本地已有终态且固定 Worker 已退出时即释放软件执行槽，成功但尚无 `upload-complete.json` 的任务继续后台补传；若云端已经是 succeeded，重复 PUT/complete 必须按数据库持久化 manifest 校验并直接确认，不得按新版本目录规则重新发布旧 Job。
|
||||
|
||||
第六阶段增加用户级 Job 中心与 Agent typed tools。`software_capability_list` 只暴露固定能力及当前在线空闲节点数，`software_job_submit/status/cancel` 在构造时绑定当前 user/task，模型不能跨用户或跨对话指定归属。`software_job_submit` 的模型可见唯一入口为 `input_id + plot + output`；`register_artifact` 是普通文件获得输入身份的唯一入口，并明确返回 UUID。内部完整 `request` 形式仅保留执行层兼容，不进入工具 schema。右下角 Job 中心按用户聚合各对话任务，活动期短轮询、空闲期降频；终态变化通知用户，成功任务可回到原对话发起分析。取消采用协作协议：未派发任务直接终止，已派发任务先进入 `cancelling`，云端通过 WebSocket 发送并在心跳时重放 `job_cancel`，Node 杀死固定 Worker 进程树后回报 `cancelled`；终态写入仍由云端账本裁决。
|
||||
第六阶段增加用户级 Job 中心与 Agent typed tools。`software_capability_list` 只暴露固定能力及当前在线空闲节点数，`software_job_submit/status/cancel` 在构造时绑定当前 user/task，模型不能跨用户或跨对话指定归属。`software_job_submit` 的唯一入口为通用 `inputs[] + operation + outputs[]`；输入和输出使用任务内稳定 key，具体 selector、type、format 和 options 由 capability 校验。`register_artifact` 是普通文件获得输入身份的唯一入口，并明确返回 UUID。右下角 Job 中心按用户聚合各对话任务，活动期短轮询、空闲期降频；终态变化通知用户，成功任务可回到原对话发起分析。取消采用协作协议：未派发任务直接终止，已派发任务先进入 `cancelling`，云端通过 WebSocket 发送并在心跳时重放 `job_cancel`，Node 杀死固定 Worker 进程树后回报 `cancelled`；终态写入仍由云端账本裁决。
|
||||
|
||||
后续仍需实现 Token 轮换；不得以任意命令或脚本接口临时代替。当前 Job 中心采用轮询而非用户事件推送，单活与 offer 选择仍只覆盖单 Web 进程；生产启用多实例前必须增加 Redis/PG fencing 或固定路由到单一控制面实例。
|
||||
|
||||
|
|
|
|||
|
|
@ -22,6 +22,8 @@
|
|||
|
||||
### 2026-08-14
|
||||
|
||||
- **08-14 / Unreleased / Windows Node 本机任务监控 + Origin 多输入多输出 v2**:配置窗口新增只读本机任务列表与详情，每秒从 `%ProgramData%/Zcbot/WindowsNode/jobs` 合并 request/state/terminal/upload-complete，展示标题、全部输入、阶段、执行耗时、时间、Job ID 与错误，重启后可恢复且不查询云端未派发 Job。按用户明确授权直接切换到 `origin.plot@v2`：通用 `inputs[] + operation + outputs[]` 支持 1–16 个输入、跨输入系列和多个显式产物，Node 下载到 keyed 目录，Origin Worker 建立多工作表后合并绘图；本地阶段统一为 `downloading_inputs/software_running`，不兼容 v1 请求或旧本地任务。相关专项 71 项 unittest、Python/JavaScript 语法、Ruff 致命规则、.NET build 与 diff 检查通过；完整测试 593 项中非数据库测试通过，3 个数据库集成模块因显式测试库未迁移、缺少 `users` 表而未通过，未连接或写入生产 DB。
|
||||
|
||||
- **08-14 / Unreleased / 专业软件执行槽与上传恢复解耦**:修正服务端已完成但 Node 本地 `upload-complete.json` 未落盘时整节点长期上报 0 slot 的活性故障；Origin 终态且 Worker 退出后即释放执行槽，成功输出继续后台重传，服务端对 succeeded Job 的重复 PUT/complete 以已持久化 manifest 幂等确认并兼容旧目录布局。相关 43 项 unittest 与 .NET build 通过；只读核查生产任务状态，未写入生产 DB。
|
||||
|
||||
- **08-14 / Unreleased / 专业软件输出目录归一 + artifact 来源**:修正 `tasks.working_dir` 被二次拼入 user root 导致 Job 输出错位的问题，正式输出原子发布到 `<working_dir>/origin/<job_id>/` 并以 nullable `software_job_id` 记录来源；plot spec/provenance 改入 `.meta/` 且不再登记 artifact。新增 0033 migration 和只认显式迁移库地址、默认 dry-run、冲突即停的存量目录修复脚本；相关 Python 63 项、Node 前端 26 项、编译、Alembic 单 head、Ruff 致命规则及 diff 检查通过，未连接或写入生产 DB。
|
||||
|
|
|
|||
2
RUN.md
2
RUN.md
|
|
@ -1100,7 +1100,7 @@ install-windows-node.bat
|
|||
|
||||
若 Python 未加入 PATH，可把绝对路径作为第一个参数，例如 `install-windows-node.bat "C:\Python312\python.exe"`。默认解释器为 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\Scripts\python.exe`。如需使用其他受管解释器，设置机器级 `ZCBOT_ORIGIN_PYTHON` 为绝对 `python.exe` 路径后重启 Node。`node.json`、可恢复任务和 runtime 集中保存在 `%ProgramData%\Zcbot\WindowsNode\`，不会因替换程序目录而丢失。运行时固定依赖见发布目录的 `origin-worker/requirements.txt`；任务请求无权选择解释器、脚本或路径。当前 Worker 支持 CSV/XLSX/JSON 输入，`line`、`scatter`、`line_scatter` 与 OPJU/PNG/SVG/PDF 输出。成功产物由 Node 流式上传，全部校验通过后发布到任务工作目录 `origin/<job_id>/`，plot spec 与 provenance 位于其 `.meta/`；上传中断会在重连时幂等续传。
|
||||
|
||||
Web 用户登录后，文件栏 Job 中心会聚合本人最近任务。活动任务约 4 秒刷新一次，空闲时降为约 30 秒；停止已派发任务是协作取消，状态先显示“正在停止”，Node 在线时立即接收，断线后在下次连接或心跳时重放。Agent 可调用 `software_capability_list`、`register_artifact`、`software_job_submit`、`software_job_status` 和 `software_job_cancel`。Origin 输入必须是 artifact：已有 UUID 可直接提交，普通 task 文件先用相对路径登记；登记不会发布聊天交付卡片。提交工具接收 `input_id`、`plot`、`output`，只创建固定 schema 的持久任务，不会阻塞当前对话等待完成。成功状态提供 `output_dir`，Agent可在该目录内搜索并分析；正式输出的 artifact 带 `software_job_id`，供结果卡和产物详情展示来源。
|
||||
Web 用户登录后，文件栏 Job 中心会聚合本人最近任务。活动任务约 4 秒刷新一次，空闲时降为约 30 秒；停止已派发任务是协作取消，状态先显示“正在停止”，Node 在线时立即接收，断线后在下次连接或心跳时重放。Agent 可调用 `software_capability_list`、`register_artifact`、`software_job_submit`、`software_job_status` 和 `software_job_cancel`。Origin 输入必须是 artifact：已有 UUID 可直接提交，普通 task 文件先逐个用相对路径登记；登记不会发布聊天交付卡片。提交工具接收 `inputs`、`operation`、`outputs`，支持 1–16 个输入、跨输入系列和多个显式输出，只创建固定 v2 schema 的持久任务，不会阻塞当前对话等待完成。成功状态提供 `output_dir`，Agent可在该目录内搜索并分析；正式输出的 artifact 带 `software_job_id`，供结果卡和产物详情展示来源。
|
||||
|
||||
注册配置写入 `%ProgramData%\Zcbot\WindowsNode\node.json`；Token 使用 DPAPI `LocalMachine` 加密，ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号执行统一安装器、注册并运行 Node。当前 MVP 以该账号的登录后计划任务启动，不安装 Windows Service。
|
||||
|
||||
|
|
|
|||
|
|
@ -19,9 +19,10 @@ OFFER_SECONDS = 60
|
|||
ALLOWED_PLOT_TYPES = frozenset(
|
||||
{"line", "scatter", "line_scatter"}
|
||||
)
|
||||
ALLOWED_OUTPUT_FORMATS = frozenset({"opju", "png", "svg", "pdf"})
|
||||
ALLOWED_INPUT_SUFFIXES = frozenset({".csv", ".xlsx", ".json"})
|
||||
MAX_INPUT_BYTES = 100 * 1024 * 1024
|
||||
MAX_INPUTS = 16
|
||||
MAX_INPUT_TOTAL_BYTES = 512 * 1024 * 1024
|
||||
MAX_OUTPUT_ARTIFACT_BYTES = 256 * 1024 * 1024
|
||||
class SoftwareJobError(Exception):
|
||||
pass
|
||||
|
|
@ -37,6 +38,12 @@ OUTPUT_ARTIFACTS = {
|
|||
"provenance": ("provenance.json", "application/json", None),
|
||||
}
|
||||
SOFTWARE_JOB_METADATA_IDS = frozenset({"plot_spec", "provenance"})
|
||||
ORIGIN_OUTPUT_IDENTITIES = {
|
||||
("project", "opju"): "project",
|
||||
("figure", "png"): "figure_png",
|
||||
("figure", "svg"): "figure_svg",
|
||||
("figure", "pdf"): "figure_pdf",
|
||||
}
|
||||
|
||||
|
||||
def software_job_output_path(output_id: str) -> str:
|
||||
|
|
@ -52,29 +59,50 @@ def _has_only(value: dict, fields: set[str]) -> bool:
|
|||
return set(value).issubset(fields)
|
||||
|
||||
|
||||
def _canonical_request(request: dict) -> tuple[dict, str]:
|
||||
if not isinstance(request, dict) or set(request) != {"schema_version", "input", "plot", "output"}:
|
||||
def _canonical_origin_plot_request(request: dict) -> tuple[dict, str]:
|
||||
if not isinstance(request, dict) or set(request) != {
|
||||
"schema_version", "inputs", "operation", "outputs"
|
||||
}:
|
||||
raise SoftwareJobError("invalid origin plot request fields")
|
||||
if request.get("schema_version") != 1:
|
||||
if request.get("schema_version") != 2:
|
||||
raise SoftwareJobError("unsupported origin plot schema version")
|
||||
input_spec = request.get("input")
|
||||
plot = request.get("plot")
|
||||
output = request.get("output")
|
||||
if not all(isinstance(item, dict) for item in (input_spec, plot, output)):
|
||||
inputs = request.get("inputs")
|
||||
operation = request.get("operation")
|
||||
outputs = request.get("outputs")
|
||||
if not isinstance(inputs, list) or not 1 <= len(inputs) <= MAX_INPUTS:
|
||||
raise SoftwareJobError("inputs must contain 1 to 16 artifact bindings")
|
||||
if not isinstance(operation, dict) or set(operation) != {"plot"}:
|
||||
raise SoftwareJobError("origin operation must contain exactly plot")
|
||||
plot = operation.get("plot")
|
||||
if not isinstance(plot, dict):
|
||||
raise SoftwareJobError("origin plot request sections must be objects")
|
||||
if not _has_only(input_spec, {"input_id", "sheet"}):
|
||||
raise SoftwareJobError("unsupported origin input fields")
|
||||
try:
|
||||
UUID(str(input_spec.get("input_id") or ""))
|
||||
except ValueError as exc:
|
||||
raise SoftwareJobError("input.input_id must be an artifact UUID") from exc
|
||||
if "sheet" in input_spec and (
|
||||
not isinstance(input_spec["sheet"], str) or not 1 <= len(input_spec["sheet"]) <= 128
|
||||
):
|
||||
raise SoftwareJobError("input.sheet must be a string")
|
||||
input_keys: list[str] = []
|
||||
for input_spec in inputs:
|
||||
if not isinstance(input_spec, dict) or set(input_spec) not in (
|
||||
{"key", "artifact_id"}, {"key", "artifact_id", "selector"}
|
||||
):
|
||||
raise SoftwareJobError("invalid origin input binding fields")
|
||||
input_key = input_spec.get("key")
|
||||
if not isinstance(input_key, str) or not re.fullmatch(r"[a-z][a-z0-9_]{0,31}", input_key):
|
||||
raise SoftwareJobError("input key must match [a-z][a-z0-9_]{0,31}")
|
||||
try:
|
||||
UUID(str(input_spec.get("artifact_id") or ""))
|
||||
except ValueError as exc:
|
||||
raise SoftwareJobError("inputs[].artifact_id must be an artifact UUID") from exc
|
||||
selector = input_spec.get("selector")
|
||||
if selector is not None and (
|
||||
not isinstance(selector, dict)
|
||||
or set(selector) != {"sheet"}
|
||||
or not isinstance(selector.get("sheet"), str)
|
||||
or not 1 <= len(selector["sheet"]) <= 128
|
||||
):
|
||||
raise SoftwareJobError("origin input selector must contain a valid sheet")
|
||||
input_keys.append(input_key)
|
||||
if len(input_keys) != len(set(input_keys)):
|
||||
raise SoftwareJobError("input keys must be unique")
|
||||
if not _has_only(
|
||||
plot,
|
||||
{"type", "x", "y", "template", "title", "x_axis", "y_axis", "legend", "error_bars"},
|
||||
{"type", "series", "template", "title", "x_axis", "y_axis", "legend", "error_bars"},
|
||||
):
|
||||
raise SoftwareJobError("unsupported origin plot fields")
|
||||
if plot.get("type") not in ALLOWED_PLOT_TYPES:
|
||||
|
|
@ -85,19 +113,42 @@ def _canonical_request(request: dict) -> tuple[dict, str]:
|
|||
raise SoftwareJobError("plot.title must be a string")
|
||||
if plot.get("template", "publication_double_column") != "publication_double_column":
|
||||
raise SoftwareJobError("unsupported origin plot template")
|
||||
x_column = plot.get("x")
|
||||
y_columns = plot.get("y")
|
||||
if not isinstance(x_column, str) or not 1 <= len(x_column) <= 128:
|
||||
raise SoftwareJobError("plot.x must be a column name")
|
||||
if isinstance(y_columns, str):
|
||||
y_columns = [y_columns]
|
||||
if (
|
||||
not isinstance(y_columns, list)
|
||||
or not 1 <= len(y_columns) <= 16
|
||||
or len(y_columns) != len(set(y_columns))
|
||||
or any(not isinstance(item, str) or not 1 <= len(item) <= 128 for item in y_columns)
|
||||
):
|
||||
raise SoftwareJobError("plot.y must contain 1 to 16 unique column names")
|
||||
series = plot.get("series")
|
||||
if not isinstance(series, list) or not 1 <= len(series) <= 16:
|
||||
raise SoftwareJobError("plot.series must contain 1 to 16 series")
|
||||
identities: list[tuple[str, str, str]] = []
|
||||
used_input_keys: set[str] = set()
|
||||
series_labels: dict[tuple[str, str], str] = {}
|
||||
for item in series:
|
||||
if not isinstance(item, dict) or not _has_only(item, {"input", "x", "y", "label"}):
|
||||
raise SoftwareJobError("invalid plot series fields")
|
||||
if not {"input", "x", "y"}.issubset(item):
|
||||
raise SoftwareJobError("plot series requires input, x, and y")
|
||||
input_key = item.get("input")
|
||||
x_column = item.get("x")
|
||||
y_column = item.get("y")
|
||||
if input_key not in input_keys:
|
||||
raise SoftwareJobError("plot series references an unknown input")
|
||||
if any(
|
||||
not isinstance(value, str) or not 1 <= len(value) <= 128
|
||||
for value in (x_column, y_column)
|
||||
):
|
||||
raise SoftwareJobError("plot series x and y must be column names")
|
||||
if "label" in item and (
|
||||
not isinstance(item["label"], str) or not 1 <= len(item["label"]) <= 200
|
||||
):
|
||||
raise SoftwareJobError("plot series label must be a string")
|
||||
label_key = (input_key, y_column)
|
||||
effective_label = item.get("label", y_column)
|
||||
if label_key in series_labels and series_labels[label_key] != effective_label:
|
||||
raise SoftwareJobError("series sharing an input Y column must use one label")
|
||||
series_labels[label_key] = effective_label
|
||||
used_input_keys.add(input_key)
|
||||
identities.append((input_key, x_column, y_column))
|
||||
if len(identities) != len(set(identities)):
|
||||
raise SoftwareJobError("plot series must be unique")
|
||||
if used_input_keys != set(input_keys):
|
||||
raise SoftwareJobError("every input must be referenced by a plot series")
|
||||
for axis_name in ("x_axis", "y_axis"):
|
||||
axis = plot.get(axis_name)
|
||||
if axis is not None and (
|
||||
|
|
@ -120,27 +171,37 @@ def _canonical_request(request: dict) -> tuple[dict, str]:
|
|||
):
|
||||
raise SoftwareJobError("invalid plot.legend")
|
||||
if plot.get("error_bars") is not None:
|
||||
raise SoftwareJobError("error bars are not supported in origin.plot@v1")
|
||||
if not _has_only(output, {"formats", "dpi", "capture_screenshots", "record_video"}):
|
||||
raise SoftwareJobError("unsupported origin output fields")
|
||||
if any(
|
||||
name in output and not isinstance(output[name], bool)
|
||||
for name in ("capture_screenshots", "record_video")
|
||||
):
|
||||
raise SoftwareJobError("origin output capture flags must be boolean")
|
||||
if output.get("record_video", False):
|
||||
raise SoftwareJobError("origin video recording is not supported")
|
||||
dpi = output.get("dpi", 300)
|
||||
if not isinstance(dpi, int) or isinstance(dpi, bool) or not 72 <= dpi <= 1200:
|
||||
raise SoftwareJobError("output.dpi must be between 72 and 1200")
|
||||
formats = output.get("formats")
|
||||
if (
|
||||
not isinstance(formats, list)
|
||||
or not formats
|
||||
or len(formats) != len(set(formats))
|
||||
or any(item not in ALLOWED_OUTPUT_FORMATS for item in formats)
|
||||
):
|
||||
raise SoftwareJobError("output.formats contains unsupported values")
|
||||
raise SoftwareJobError("error bars are not supported")
|
||||
if not isinstance(outputs, list) or not 1 <= len(outputs) <= 16:
|
||||
raise SoftwareJobError("outputs must contain 1 to 16 declarations")
|
||||
output_keys: list[str] = []
|
||||
output_identities: list[tuple[str, str]] = []
|
||||
for output in outputs:
|
||||
if not isinstance(output, dict) or set(output) not in (
|
||||
{"key", "type", "format"}, {"key", "type", "format", "options"}
|
||||
):
|
||||
raise SoftwareJobError("invalid origin output declaration fields")
|
||||
output_type = output.get("type")
|
||||
output_format = output.get("format")
|
||||
expected_key = ORIGIN_OUTPUT_IDENTITIES.get((output_type, output_format))
|
||||
if output.get("key") != expected_key:
|
||||
raise SoftwareJobError("origin output key, type, and format do not match")
|
||||
options = output.get("options")
|
||||
if output_format == "png":
|
||||
if options is not None and (
|
||||
not isinstance(options, dict)
|
||||
or set(options) != {"dpi"}
|
||||
or not isinstance(options.get("dpi"), int)
|
||||
or isinstance(options.get("dpi"), bool)
|
||||
or not 72 <= options["dpi"] <= 1200
|
||||
):
|
||||
raise SoftwareJobError("PNG output options must contain a valid dpi")
|
||||
elif options is not None:
|
||||
raise SoftwareJobError("output options are only supported for PNG")
|
||||
output_keys.append(expected_key)
|
||||
output_identities.append((output_type, output_format))
|
||||
if len(output_keys) != len(set(output_keys)) or len(output_identities) != len(set(output_identities)):
|
||||
raise SoftwareJobError("outputs must be unique")
|
||||
encoded = json.dumps(request, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
||||
if len(encoded.encode("utf-8")) > 256 * 1024:
|
||||
raise SoftwareJobError("origin plot request is too large")
|
||||
|
|
@ -148,6 +209,18 @@ def _canonical_request(request: dict) -> tuple[dict, str]:
|
|||
return normalized, sha256(encoded.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
REQUEST_VALIDATORS = {
|
||||
"origin.plot@v2": _canonical_origin_plot_request,
|
||||
}
|
||||
|
||||
|
||||
def _canonical_request(capability: str, request: dict) -> tuple[dict, str]:
|
||||
validator = REQUEST_VALIDATORS.get(capability)
|
||||
if validator is None:
|
||||
raise SoftwareJobError("unsupported capability")
|
||||
return validator(request)
|
||||
|
||||
|
||||
def _job_dict(row: SoftwareJob) -> dict:
|
||||
return {
|
||||
"job_id": str(row.job_id),
|
||||
|
|
@ -233,16 +306,16 @@ def list_jobs(
|
|||
|
||||
|
||||
def _request_summary(job: SoftwareJob) -> dict:
|
||||
plot = job.request.get("plot") or {}
|
||||
output = job.request.get("output") or {}
|
||||
plot = (job.request.get("operation") or {}).get("plot") or {}
|
||||
outputs = job.request.get("outputs") or []
|
||||
return {
|
||||
"display_name": (
|
||||
"Origin 科研绘图"
|
||||
if job.capability == "origin.plot@v1"
|
||||
if job.capability == "origin.plot@v2"
|
||||
else job.capability
|
||||
),
|
||||
"title": str(plot.get("title") or ""),
|
||||
"formats": list(output.get("formats") or []),
|
||||
"formats": [item.get("format") for item in outputs if isinstance(item, dict)],
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -309,40 +382,55 @@ def create_job(
|
|||
raise SoftwareJobError("idempotency_key must contain 1 to 200 characters")
|
||||
if capability not in SUPPORTED_CAPABILITIES:
|
||||
raise SoftwareJobError("unsupported capability")
|
||||
normalized, digest = _canonical_request(request)
|
||||
normalized, digest = _canonical_request(capability, request)
|
||||
with session_scope() as session:
|
||||
task = session.execute(
|
||||
select(Task.task_id).where(Task.task_id == task_id, Task.user_id == user_id)
|
||||
).first()
|
||||
if task is None:
|
||||
raise SoftwareJobError("task not found")
|
||||
artifact_id = UUID(normalized["input"]["input_id"])
|
||||
artifact = session.execute(
|
||||
artifact_ids = [UUID(item["artifact_id"]) for item in normalized["inputs"]]
|
||||
artifacts = session.execute(
|
||||
select(Artifact).where(
|
||||
Artifact.artifact_id == artifact_id,
|
||||
Artifact.artifact_id.in_(set(artifact_ids)),
|
||||
Artifact.user_id == user_id,
|
||||
Artifact.status == "active",
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if artifact is None:
|
||||
raise SoftwareJobError("input artifact not found")
|
||||
suffix = "." + artifact.current_path.rsplit(".", 1)[-1].lower() if "." in artifact.current_path else ""
|
||||
if suffix not in ALLOWED_INPUT_SUFFIXES:
|
||||
raise SoftwareJobError("input artifact type is not supported")
|
||||
if (
|
||||
artifact.size_bytes is None
|
||||
or artifact.size_bytes < 0
|
||||
or artifact.size_bytes > MAX_INPUT_BYTES
|
||||
or not artifact.content_sha256
|
||||
or len(artifact.content_sha256) != 64
|
||||
):
|
||||
raise SoftwareJobError("input artifact metadata is incomplete or too large")
|
||||
input_manifest = {
|
||||
"artifact_id": str(artifact.artifact_id),
|
||||
"filename": artifact.current_path.replace("\\", "/").rsplit("/", 1)[-1],
|
||||
"size_bytes": artifact.size_bytes,
|
||||
"sha256": artifact.content_sha256,
|
||||
}
|
||||
).scalars().all()
|
||||
artifacts_by_id = {artifact.artifact_id: artifact for artifact in artifacts}
|
||||
input_manifest: list[dict] = []
|
||||
total_input_bytes = 0
|
||||
for binding, artifact_id in zip(normalized["inputs"], artifact_ids, strict=True):
|
||||
artifact = artifacts_by_id.get(artifact_id)
|
||||
if artifact is None:
|
||||
raise SoftwareJobError("input artifact not found")
|
||||
suffix = (
|
||||
"." + artifact.current_path.rsplit(".", 1)[-1].lower()
|
||||
if "." in artifact.current_path else ""
|
||||
)
|
||||
if suffix not in ALLOWED_INPUT_SUFFIXES:
|
||||
raise SoftwareJobError("input artifact type is not supported")
|
||||
if (
|
||||
artifact.size_bytes is None
|
||||
or artifact.size_bytes < 0
|
||||
or artifact.size_bytes > MAX_INPUT_BYTES
|
||||
or not artifact.content_sha256
|
||||
or len(artifact.content_sha256) != 64
|
||||
):
|
||||
raise SoftwareJobError("input artifact metadata is incomplete or too large")
|
||||
total_input_bytes += artifact.size_bytes
|
||||
item = {
|
||||
"key": binding["key"],
|
||||
"artifact_id": str(artifact.artifact_id),
|
||||
"filename": artifact.current_path.replace("\\", "/").rsplit("/", 1)[-1],
|
||||
"size_bytes": artifact.size_bytes,
|
||||
"sha256": artifact.content_sha256,
|
||||
}
|
||||
if binding.get("selector") is not None:
|
||||
item["selector"] = binding["selector"]
|
||||
input_manifest.append(item)
|
||||
if total_input_bytes > MAX_INPUT_TOTAL_BYTES:
|
||||
raise SoftwareJobError("job inputs exceed the total size limit")
|
||||
existing = session.execute(
|
||||
select(SoftwareJob).where(
|
||||
SoftwareJob.user_id == user_id,
|
||||
|
|
@ -471,15 +559,20 @@ def offer_next_job(node_ids: set[UUID]) -> dict | None:
|
|||
"capability": job.capability,
|
||||
"request_digest": job.request_digest,
|
||||
"request": job.request,
|
||||
"input_transfer": {
|
||||
**job.input_manifest,
|
||||
"download_path": f"/v1/software-jobs/{job.job_id}/input",
|
||||
},
|
||||
"input_transfers": [
|
||||
{
|
||||
**item,
|
||||
"download_path": (
|
||||
f"/v1/software-jobs/{job.job_id}/inputs/{item['key']}"
|
||||
),
|
||||
}
|
||||
for item in job.input_manifest
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_job_input(node_id: UUID, job_id: UUID) -> dict | None:
|
||||
def get_job_input(node_id: UUID, job_id: UUID, input_key: str) -> dict | None:
|
||||
"""返回任务绑定的 artifact 定位信息；调用方仍需在 user_root 内安全解析。"""
|
||||
with session_scope() as session:
|
||||
job = session.execute(
|
||||
|
|
@ -491,7 +584,13 @@ def get_job_input(node_id: UUID, job_id: UUID) -> dict | None:
|
|||
).scalar_one_or_none()
|
||||
if job is None:
|
||||
return None
|
||||
artifact_id = UUID(job.input_manifest["artifact_id"])
|
||||
manifest = next(
|
||||
(item for item in job.input_manifest if item.get("key") == input_key),
|
||||
None,
|
||||
)
|
||||
if manifest is None:
|
||||
return None
|
||||
artifact_id = UUID(manifest["artifact_id"])
|
||||
artifact = session.execute(
|
||||
select(Artifact).where(
|
||||
Artifact.artifact_id == artifact_id,
|
||||
|
|
@ -504,7 +603,7 @@ def get_job_input(node_id: UUID, job_id: UUID) -> dict | None:
|
|||
return {
|
||||
"user_id": job.user_id,
|
||||
"current_path": artifact.current_path,
|
||||
**job.input_manifest,
|
||||
**manifest,
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -539,12 +638,11 @@ def get_job_output_context(node_id: UUID, job_id: UUID, lease_id: UUID, digest:
|
|||
def validate_output_manifest(request: dict, manifest: object) -> list[dict]:
|
||||
if not isinstance(manifest, list):
|
||||
raise SoftwareJobError("job artifact manifest must be a list")
|
||||
requested_formats = set(request.get("output", {}).get("formats") or [])
|
||||
expected_ids = {"plot_spec", "provenance"}
|
||||
expected_ids.update(
|
||||
artifact_id
|
||||
for artifact_id, (_, _, output_format) in OUTPUT_ARTIFACTS.items()
|
||||
if output_format in requested_formats
|
||||
item.get("key")
|
||||
for item in request.get("outputs", [])
|
||||
if isinstance(item, dict) and isinstance(item.get("key"), str)
|
||||
)
|
||||
if len(manifest) != len(expected_ids):
|
||||
raise SoftwareJobError("job artifact manifest is incomplete")
|
||||
|
|
@ -717,7 +815,7 @@ def update_job_state(node_id: UUID, payload: dict) -> None:
|
|||
raise SoftwareJobError("job state cannot advance from its current status")
|
||||
job.status = (
|
||||
"dispatched"
|
||||
if stage in {"accepted", "waiting_input", "ready_to_run"}
|
||||
if stage in {"accepted", "downloading_inputs", "ready_to_run"}
|
||||
else "running"
|
||||
)
|
||||
job.stage = stage
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ from sqlalchemy import select
|
|||
from core.storage.engine import session_scope
|
||||
from core.storage.models import SoftwareNode, SoftwareNodeEnrollment
|
||||
|
||||
SUPPORTED_CAPABILITIES = frozenset({"origin.plot@v1"})
|
||||
SUPPORTED_CAPABILITIES = frozenset({"origin.plot@v2"})
|
||||
MAX_ENROLLMENT_FAILURES = 5
|
||||
|
||||
|
||||
|
|
@ -44,7 +44,7 @@ def create_enrollment(
|
|||
capabilities: list[str] | None = None,
|
||||
ttl_seconds: int = 600,
|
||||
) -> dict:
|
||||
allowed = list(dict.fromkeys(capabilities or ["origin.plot@v1"]))
|
||||
allowed = list(dict.fromkeys(capabilities or ["origin.plot@v2"]))
|
||||
if not allowed or any(item not in SUPPORTED_CAPABILITIES for item in allowed):
|
||||
raise SoftwareNodeError("unsupported capability")
|
||||
if not 60 <= ttl_seconds <= 3600:
|
||||
|
|
|
|||
|
|
@ -435,39 +435,46 @@ C# COM 只作未覆盖能力或旧版本备用，并与实际 Origin 版本匹
|
|||
|
||||
### 12.2 首期 capability
|
||||
|
||||
只开放 `origin.plot@v1`，接受 CSV、XLSX 或规范化 JSON，不接受任意 Python、LabTalk、模板文件或宏。
|
||||
只开放 `origin.plot@v2`，接受 1–16 个 CSV、XLSX 或规范化 JSON 输入，不接受任意 Python、LabTalk、模板文件或宏。
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"input": {"input_id": "5af18255-c859-43f3-a424-220bc68b7db2", "sheet": "Sheet1"},
|
||||
"plot": {
|
||||
"type": "line_scatter",
|
||||
"x": "Temperature",
|
||||
"y": ["Strength_7d", "Strength_28d"],
|
||||
"template": "publication_double_column",
|
||||
"title": "温度对抗压强度的影响",
|
||||
"x_axis": {"title": "温度", "unit": "°C", "scale": "linear"},
|
||||
"y_axis": {"title": "抗压强度", "unit": "MPa", "scale": "linear"},
|
||||
"legend": {"enabled": true, "position": "top_right"},
|
||||
"error_bars": null
|
||||
"schema_version": 2,
|
||||
"inputs": [
|
||||
{"key": "seven_day", "artifact_id": "5af18255-c859-43f3-a424-220bc68b7db2", "selector": {"sheet": "Sheet1"}},
|
||||
{"key": "twenty_eight_day", "artifact_id": "6bf29366-d960-54f4-b535-331cd79c8ec3"}
|
||||
],
|
||||
"operation": {
|
||||
"plot": {
|
||||
"type": "line_scatter",
|
||||
"series": [
|
||||
{"input": "seven_day", "x": "Temperature", "y": "Strength", "label": "7 d"},
|
||||
{"input": "twenty_eight_day", "x": "Temperature", "y": "Strength", "label": "28 d"}
|
||||
],
|
||||
"template": "publication_double_column",
|
||||
"title": "温度对抗压强度的影响",
|
||||
"x_axis": {"title": "温度", "unit": "°C", "scale": "linear"},
|
||||
"y_axis": {"title": "抗压强度", "unit": "MPa", "scale": "linear"},
|
||||
"legend": {"enabled": true, "position": "top_right"},
|
||||
"error_bars": null
|
||||
}
|
||||
},
|
||||
"output": {
|
||||
"formats": ["opju", "png", "svg", "pdf"],
|
||||
"dpi": 600,
|
||||
"capture_screenshots": true,
|
||||
"record_video": false
|
||||
}
|
||||
"outputs": [
|
||||
{"key": "project", "type": "project", "format": "opju"},
|
||||
{"key": "figure_png", "type": "figure", "format": "png", "options": {"dpi": 600}},
|
||||
{"key": "figure_svg", "type": "figure", "format": "svg"},
|
||||
{"key": "figure_pdf", "type": "figure", "format": "pdf"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
当前图形类型：line、scatter、line_scatter。后续增加 grouped_bar、box、histogram、heatmap、误差棒组合、三元图、等高线、三维曲面、XRD 堆叠图、热分析联图和多面板布局。
|
||||
|
||||
上述 JSON 同时是提交工具与云端账本、Windows Node 之间的 artifact 输入契约。
|
||||
Agent 优先复用已有 artifact UUID；普通 task 文件先调用
|
||||
`register_artifact(path)` 获得 UUID，再以 `input_id`、`plot`、`output` 调用
|
||||
`software_job_submit`。登记输入不会把文件发布为聊天交付物。旧的完整 `request`
|
||||
参数仅作为执行层兼容入口保留，不暴露在模型可见 schema 中。
|
||||
Agent 优先复用已有 artifact UUID；普通 task 文件先逐个调用
|
||||
`register_artifact(path)` 获得 UUID，再以 `inputs`、`operation`、`outputs` 调用
|
||||
`software_job_submit`。登记输入不会把文件发布为聊天交付物。v2 是当前唯一协议，
|
||||
不读取 v1 请求或旧本地任务格式。
|
||||
|
||||
### 12.3 模板与产物
|
||||
|
||||
|
|
@ -476,7 +483,7 @@ Agent 优先复用已有 artifact UUID；普通 task 文件先调用
|
|||
```text
|
||||
D:\ZcbotNode\jobs\<job_id>\
|
||||
├── request\request.json
|
||||
├── input\
|
||||
├── input\<key>\<filename>
|
||||
├── work\
|
||||
├── logs\events.jsonl
|
||||
├── screenshots\
|
||||
|
|
@ -607,7 +614,7 @@ Windows 基线：固定版本并受控更新；禁止休眠和自动锁屏；固
|
|||
## 18. 首期验收标准
|
||||
|
||||
1. Windows 不开放业务入站端口，Node 能通过 mTLS WSS 注册、重连和心跳。
|
||||
2. 云端能发现 `origin.plot@v1`、软件版本、健康和 slot。
|
||||
2. 云端能发现 `origin.plot@v2`、软件版本、健康和 slot。
|
||||
3. 同一幂等键不会创建两个任务。
|
||||
4. 双连接时只有最新 epoch 能接受任务。
|
||||
5. zcbot 能用 CSV 生成指定模板图并获得 `job_id`。
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
> **当前有效的第一阶段开发依据。**本文件取代 `windows-node-mvp.md` 中的 MVP 通信与注册方案。
|
||||
> 长期演进边界见 `windows-node-design.md`。
|
||||
> 首批能力：`origin.plot@v1`。
|
||||
> 首批能力：`origin.plot@v2`。
|
||||
|
||||
## 1. 适用边界
|
||||
|
||||
|
|
@ -82,7 +82,7 @@ Content-Type: application/json
|
|||
"install_id": "019...",
|
||||
"node_version": "0.1.0",
|
||||
"os_version": "Windows 11 Enterprise 24H2",
|
||||
"capabilities": ["origin.plot@v1"]
|
||||
"capabilities": ["origin.plot@v2"]
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -200,12 +200,12 @@ Node 断线且本地任务可能仍在执行时标记 `disconnected`，不得自
|
|||
|
||||
## 7. Origin 任务闭环
|
||||
|
||||
当前实现进度：云端任务账本、幂等提交、短期 offer、Node 本地原子保存与 accept/reject 已落地。输入以任务绑定的 artifact UUID 下载，Node 流式校验大小和 SHA-256 后原子保存。固定 Worker 使用管理员安装的隔离 Python 运行时与随程序发布的 `worker.py` 驱动 Origin，生成 OPJU、PNG、SVG、PDF、plot spec、provenance 和原子 `terminal.json`；运行不绑定单次 WebSocket，断线后继续执行。同一进程按 job 去重，Node 重启后不重复启动已留启动标记但无可信终态的任务。成功产物逐项流式上传到云端隐藏暂存区，云端复核任务身份、固定文件名、大小和 SHA-256 后，一次性发布到 `<working_dir>/origin/<job_id>/` 并登记平台 artifact UUID；Node 以 `upload-complete.json` 恢复中断上传。
|
||||
当前实现进度：云端任务账本、幂等提交、短期 offer、Node 本地原子保存与 accept/reject 已落地。请求以 `inputs[]` 绑定多个 artifact、以 `outputs[]` 声明多个交付物，Node 逐项流式校验大小和 SHA-256 后将输入保存到 `input/<key>/<filename>`。固定 Worker 使用管理员安装的隔离 Python 运行时与随程序发布的 `worker.py` 驱动 Origin，按 `operation.plot.series[]` 从不同输入选择系列，并按输出声明生成 OPJU、PNG、SVG、PDF；plot spec、provenance 和原子 `terminal.json` 是系统强制元数据。运行不绑定单次 WebSocket，断线后继续执行。同一进程按 job 去重，Node 重启后不重复启动已留启动标记但无可信终态的任务。成功产物逐项流式上传到云端隐藏暂存区，云端复核任务身份、固定文件名、大小和 SHA-256 后，一次性发布到 `<working_dir>/origin/<job_id>/` 并登记平台 artifact UUID；Node 以 `upload-complete.json` 恢复中断上传。
|
||||
|
||||
```text
|
||||
用户上传 CSV/XLSX
|
||||
用户上传一个或多个 CSV/XLSX/JSON
|
||||
→ zcbot 生成受控 plot spec
|
||||
→ Node 接收 origin.plot@v1
|
||||
→ Node 接收 origin.plot@v2
|
||||
→ 先持久化 job，再启动 Origin Worker
|
||||
→ originpro 生成 OPJU/PNG/SVG/PDF
|
||||
→ terminal.json 原子记录终态
|
||||
|
|
|
|||
|
|
@ -6,7 +6,6 @@ import tempfile
|
|||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
WORKER_PATH = (
|
||||
Path(__file__).resolve().parents[1] / "windows-node" / "origin-worker" / "worker.py"
|
||||
)
|
||||
|
|
@ -41,6 +40,39 @@ class OriginWorkerUnitTests(unittest.TestCase):
|
|||
self.assertEqual(worker._axis_title({"title": "Stress", "unit": "MPa"}, "Y"), "Stress (MPa)")
|
||||
self.assertEqual(worker._axis_title(None, "Time"), "Time")
|
||||
|
||||
def test_keyed_input_directory_requires_exactly_one_file(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
keyed = root / "input" / "sample"
|
||||
keyed.mkdir(parents=True)
|
||||
path = keyed / "data.csv"
|
||||
path.write_text("x,y\n1,2\n", encoding="utf-8")
|
||||
self.assertEqual(worker._input_file(root, "sample"), path)
|
||||
(keyed / "extra.csv").write_text("x,y\n3,4\n", encoding="utf-8")
|
||||
with self.assertRaisesRegex(ValueError, "INPUT_FILE_COUNT_INVALID:sample"):
|
||||
worker._input_file(root, "sample")
|
||||
|
||||
def test_series_are_resolved_against_each_input(self) -> None:
|
||||
input_data = {
|
||||
"first": (["time", "strength"], [[1, 10]]),
|
||||
"second": (["temperature", "value"], [[20, 30]]),
|
||||
}
|
||||
series = [
|
||||
{"input": "first", "x": "time", "y": "strength", "label": "7 d"},
|
||||
{"input": "second", "x": "temperature", "y": "value", "label": "28 d"},
|
||||
]
|
||||
resolved, labels = worker._resolve_series(input_data, series)
|
||||
self.assertEqual(resolved, [("first", 0, 1, "7 d"), ("second", 0, 1, "28 d")])
|
||||
self.assertEqual(labels, {("first", 1): "7 d", ("second", 1): "28 d"})
|
||||
|
||||
def test_shared_y_column_rejects_conflicting_labels(self) -> None:
|
||||
input_data = {"sample": (["x1", "x2", "y"], [[1, 2, 3]])}
|
||||
with self.assertRaisesRegex(ValueError, "SERIES_LABEL_CONFLICT"):
|
||||
worker._resolve_series(input_data, [
|
||||
{"input": "sample", "x": "x1", "y": "y", "label": "First"},
|
||||
{"input": "sample", "x": "x2", "y": "y", "label": "Second"},
|
||||
])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
|
|||
|
|
@ -22,12 +22,12 @@ class SoftwareJobToolTests(unittest.TestCase):
|
|||
nodes = [
|
||||
{
|
||||
"status": "online",
|
||||
"capabilities": ["origin.plot@v1"],
|
||||
"capabilities": ["origin.plot@v2"],
|
||||
"runtime": {"available_slots": 1},
|
||||
},
|
||||
{
|
||||
"status": "offline",
|
||||
"capabilities": ["origin.plot@v1"],
|
||||
"capabilities": ["origin.plot@v2"],
|
||||
"runtime": {"available_slots": 1},
|
||||
},
|
||||
]
|
||||
|
|
@ -43,36 +43,53 @@ class SoftwareJobToolTests(unittest.TestCase):
|
|||
tool = SoftwareJobSubmitTool(self.user_id, self.task_id)
|
||||
with patch("tools.software_jobs.create_job", return_value=(created, True)) as create:
|
||||
result = json.loads(tool.execute(
|
||||
"origin.plot@v1",
|
||||
input_id=str(artifact_id),
|
||||
plot={"type": "scatter", "x": "x", "y": ["y"]},
|
||||
output={"formats": ["opju", "png"], "dpi": 300},
|
||||
"origin.plot@v2",
|
||||
inputs=[{"key": "sample", "artifact_id": str(artifact_id)}],
|
||||
operation={"plot": {
|
||||
"type": "scatter",
|
||||
"series": [{"input": "sample", "x": "x", "y": "y"}],
|
||||
}},
|
||||
outputs=[
|
||||
{"key": "project", "type": "project", "format": "opju"},
|
||||
{"key": "figure_png", "type": "figure", "format": "png", "options": {"dpi": 300}},
|
||||
],
|
||||
))
|
||||
self.assertTrue(result["created"])
|
||||
self.assertEqual(create.call_args.args[:2], (self.user_id, self.task_id))
|
||||
self.assertEqual(create.call_args.kwargs["capability"], "origin.plot@v1")
|
||||
self.assertEqual(create.call_args.kwargs["capability"], "origin.plot@v2")
|
||||
self.assertEqual(
|
||||
create.call_args.kwargs["request"],
|
||||
{
|
||||
"schema_version": 1,
|
||||
"input": {"input_id": str(artifact_id)},
|
||||
"plot": {"type": "scatter", "x": "x", "y": ["y"]},
|
||||
"output": {"formats": ["opju", "png"], "dpi": 300},
|
||||
"schema_version": 2,
|
||||
"inputs": [{"key": "sample", "artifact_id": str(artifact_id)}],
|
||||
"operation": {"plot": {
|
||||
"type": "scatter",
|
||||
"series": [{"input": "sample", "x": "x", "y": "y"}],
|
||||
}},
|
||||
"outputs": [
|
||||
{"key": "project", "type": "project", "format": "opju"},
|
||||
{"key": "figure_png", "type": "figure", "format": "png", "options": {"dpi": 300}},
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
def test_submit_requires_artifact_uuid(self):
|
||||
result = SoftwareJobSubmitTool(self.user_id, self.task_id).execute(
|
||||
"origin.plot@v1",
|
||||
input_id="data/input.csv",
|
||||
plot={"type": "scatter", "x": "x", "y": ["y"]},
|
||||
output={"formats": ["png"]},
|
||||
"origin.plot@v2",
|
||||
inputs=[{"key": "sample", "artifact_id": "data/input.csv"}],
|
||||
operation={"plot": {
|
||||
"type": "scatter",
|
||||
"series": [{"input": "sample", "x": "x", "y": "y"}],
|
||||
}},
|
||||
outputs=[{"key": "figure_png", "type": "figure", "format": "png"}],
|
||||
)
|
||||
self.assertIn("call register_artifact first", result)
|
||||
|
||||
def test_submit_schema_requires_artifact_input(self):
|
||||
required = SoftwareJobSubmitTool.parameters["required"]
|
||||
self.assertIn("input_id", required)
|
||||
self.assertIn("inputs", required)
|
||||
self.assertIn("operation", required)
|
||||
self.assertIn("outputs", required)
|
||||
self.assertNotIn("input_path", SoftwareJobSubmitTool.parameters["properties"])
|
||||
self.assertNotIn("request", SoftwareJobSubmitTool.parameters["properties"])
|
||||
|
||||
|
|
|
|||
|
|
@ -233,7 +233,7 @@ class SoftwareJobProtocolTests(unittest.TestCase):
|
|||
session = session_scope.return_value.__enter__.return_value
|
||||
job = type("Job", (), {})()
|
||||
job.job_id = uuid4(); job.user_id = uuid4(); job.task_id = uuid4()
|
||||
job.capability = "origin.plot@v1"; job.request_digest = "a" * 64
|
||||
job.capability = "origin.plot@v2"; job.request_digest = "a" * 64
|
||||
job.node_id = None; job.lease_id = None; job.status = "queued"; job.stage = ""
|
||||
job.progress = 0; job.metrics = {}; job.error = {}; job.artifact_manifest = []
|
||||
job.created_at = None; job.started_at = None; job.terminal_at = None
|
||||
|
|
@ -248,9 +248,9 @@ class SoftwareJobProtocolTests(unittest.TestCase):
|
|||
session = session_scope.return_value.__enter__.return_value
|
||||
job = type("Job", (), {})()
|
||||
job.job_id = uuid4(); job.user_id = uuid4(); job.task_id = uuid4()
|
||||
job.capability = "origin.plot@v1"; job.request_digest = "b" * 64
|
||||
job.capability = "origin.plot@v2"; job.request_digest = "b" * 64
|
||||
job.node_id = uuid4(); job.lease_id = uuid4(); job.status = "running"
|
||||
job.stage = "origin_running"; job.progress = 10; job.metrics = {}; job.error = {}
|
||||
job.stage = "software_running"; job.progress = 10; job.metrics = {}; job.error = {}
|
||||
job.artifact_manifest = []; job.created_at = None; job.started_at = None; job.terminal_at = None
|
||||
session.execute.return_value.scalar_one_or_none.return_value = job
|
||||
result, node_message = request_job_cancel(job.user_id, job.job_id)
|
||||
|
|
@ -262,58 +262,88 @@ class SoftwareJobProtocolTests(unittest.TestCase):
|
|||
def test_job_list_is_enriched_for_job_center(self, session_scope) -> None:
|
||||
session = session_scope.return_value.__enter__.return_value
|
||||
job = type("Job", (), {})()
|
||||
job.job_id = uuid4(); job.task_id = uuid4(); job.capability = "origin.plot@v1"
|
||||
job.job_id = uuid4(); job.task_id = uuid4(); job.capability = "origin.plot@v2"
|
||||
job.request_digest = "c" * 64; job.node_id = uuid4(); job.status = "running"
|
||||
job.stage = "origin_running"; job.progress = 20; job.metrics = {}; job.error = {}
|
||||
job.artifact_manifest = []; job.input_manifest = {"filename": "input.xlsx"}
|
||||
job.request = {"plot": {"title": "Test"}, "output": {"formats": ["png"]}}
|
||||
job.stage = "software_running"; job.progress = 20; job.metrics = {}; job.error = {}
|
||||
job.artifact_manifest = []; job.input_manifest = [{"key": "sample", "filename": "input.xlsx"}]
|
||||
job.request = {"operation": {"plot": {"title": "Test"}}, "outputs": [
|
||||
{"key": "figure_png", "type": "figure", "format": "png"}
|
||||
]}
|
||||
job.created_at = None; job.started_at = None; job.terminal_at = None
|
||||
session.execute.return_value.all.return_value = [(job, "材料仿真", "LAB-01")]
|
||||
results = list_jobs(uuid4(), limit=10)
|
||||
self.assertEqual(results[0]["task_name"], "材料仿真")
|
||||
self.assertEqual(results[0]["node_name"], "LAB-01")
|
||||
self.assertEqual(results[0]["request_summary"]["display_name"], "Origin 科研绘图")
|
||||
self.assertEqual(results[0]["request_summary"]["formats"], ["png"])
|
||||
self.assertEqual(results[0]["output_dir"], f"origin/{job.job_id}")
|
||||
|
||||
def test_origin_request_is_canonical_and_rejects_extra_fields(self) -> None:
|
||||
request = {
|
||||
"schema_version": 1,
|
||||
"input": {"input_id": str(uuid4()), "sheet": "Sheet1"},
|
||||
"plot": {"type": "line", "x": "x", "y": ["y"]},
|
||||
"output": {"formats": ["png", "opju"]},
|
||||
"schema_version": 2,
|
||||
"inputs": [
|
||||
{"key": "first", "artifact_id": str(uuid4()), "selector": {"sheet": "Sheet1"}},
|
||||
{"key": "second", "artifact_id": str(uuid4())},
|
||||
],
|
||||
"operation": {"plot": {
|
||||
"type": "line",
|
||||
"series": [
|
||||
{"input": "first", "x": "x", "y": "y"},
|
||||
{"input": "second", "x": "time", "y": "value", "label": "Second"},
|
||||
],
|
||||
}},
|
||||
"outputs": [
|
||||
{"key": "figure_png", "type": "figure", "format": "png", "options": {"dpi": 600}},
|
||||
{"key": "project", "type": "project", "format": "opju"},
|
||||
],
|
||||
}
|
||||
normalized, digest = _canonical_request(request)
|
||||
normalized, digest = _canonical_request("origin.plot@v2", request)
|
||||
self.assertEqual(normalized, request)
|
||||
self.assertEqual(len(digest), 64)
|
||||
with self.assertRaisesRegex(Exception, "invalid origin plot request fields"):
|
||||
_canonical_request({**request, "script": "anything"})
|
||||
_canonical_request("origin.plot@v2", {**request, "script": "anything"})
|
||||
with self.assertRaisesRegex(Exception, "unsupported origin plot fields"):
|
||||
_canonical_request({**request, "plot": {**request["plot"], "script": "anything"}})
|
||||
with self.assertRaisesRegex(Exception, "input.input_id must be an artifact UUID"):
|
||||
_canonical_request({**request, "input": {"input_id": "C:\\data.csv"}})
|
||||
_canonical_request("origin.plot@v2", {**request, "operation": {"plot": {
|
||||
**request["operation"]["plot"], "script": "anything"
|
||||
}}})
|
||||
with self.assertRaisesRegex(Exception, "artifact UUID"):
|
||||
_canonical_request("origin.plot@v2", {**request, "inputs": [{"key": "first", "artifact_id": "C:\\data.csv"}]})
|
||||
|
||||
def test_origin_request_rejects_unimplemented_plot_semantics(self) -> None:
|
||||
request = {
|
||||
"schema_version": 1,
|
||||
"input": {"input_id": str(uuid4())},
|
||||
"plot": {"type": "scatter", "x": "time", "y": ["a", "b"]},
|
||||
"output": {"formats": ["png"], "dpi": 300},
|
||||
"schema_version": 2,
|
||||
"inputs": [{"key": "sample", "artifact_id": str(uuid4())}],
|
||||
"operation": {"plot": {"type": "scatter", "series": [
|
||||
{"input": "sample", "x": "time", "y": "a"},
|
||||
{"input": "sample", "x": "time", "y": "b"},
|
||||
]}},
|
||||
"outputs": [{"key": "figure_png", "type": "figure", "format": "png", "options": {"dpi": 300}}],
|
||||
}
|
||||
for plot, message in (
|
||||
({**request["plot"], "template": "custom"}, "unsupported origin plot template"),
|
||||
({**request["plot"], "x_axis": {"scale": "log10"}}, "invalid x_axis"),
|
||||
({**request["plot"], "legend": {"enabled": False}}, "invalid plot.legend"),
|
||||
({**request["plot"], "y": ["a", "a"]}, "plot.y must contain"),
|
||||
base_plot = request["operation"]["plot"]
|
||||
for case_plot, message in (
|
||||
({**base_plot, "template": "custom"}, "unsupported origin plot template"),
|
||||
({**base_plot, "x_axis": {"scale": "log10"}}, "invalid x_axis"),
|
||||
({**base_plot, "legend": {"enabled": False}}, "invalid plot.legend"),
|
||||
({**base_plot, "series": [base_plot["series"][0], base_plot["series"][0]]}, "plot series must be unique"),
|
||||
):
|
||||
with self.subTest(message=message), self.assertRaisesRegex(Exception, message):
|
||||
_canonical_request({**request, "plot": plot})
|
||||
with self.assertRaisesRegex(Exception, "video recording is not supported"):
|
||||
_canonical_request("origin.plot@v2", {**request, "operation": {"plot": case_plot}})
|
||||
with self.assertRaisesRegex(Exception, "unknown input"):
|
||||
_canonical_request("origin.plot@v2", {**request, "operation": {"plot": {
|
||||
**base_plot,
|
||||
"series": [{"input": "missing", "x": "time", "y": "a"}],
|
||||
}}})
|
||||
with self.assertRaisesRegex(Exception, "key, type, and format do not match"):
|
||||
_canonical_request(
|
||||
{**request, "output": {"formats": ["png"], "record_video": True}}
|
||||
"origin.plot@v2",
|
||||
{**request, "outputs": [{"key": "project", "type": "figure", "format": "png"}]}
|
||||
)
|
||||
|
||||
def test_output_manifest_matches_exact_requested_formats(self) -> None:
|
||||
request = {"output": {"formats": ["opju", "png"]}}
|
||||
def test_output_manifest_matches_exact_requested_outputs(self) -> None:
|
||||
request = {"outputs": [
|
||||
{"key": "project", "type": "project", "format": "opju"},
|
||||
{"key": "figure_png", "type": "figure", "format": "png"},
|
||||
]}
|
||||
manifest = [
|
||||
{"artifact_id": "project", "filename": "project.opju", "media_type": "application/x-origin-project", "size_bytes": 10, "sha256": "a" * 64},
|
||||
{"artifact_id": "figure_png", "filename": "figure.png", "media_type": "image/png", "size_bytes": 20, "sha256": "b" * 64},
|
||||
|
|
@ -375,6 +405,8 @@ class SoftwareJobProtocolTests(unittest.TestCase):
|
|||
).read_text(encoding="utf-8")
|
||||
self.assertIn("digest = sha256()", source)
|
||||
self.assertIn('digest.hexdigest() != item["sha256"]', source)
|
||||
self.assertIn('context["request"].get("outputs", [])', source)
|
||||
self.assertIn("artifact_id not in requested_ids", source)
|
||||
|
||||
@patch("core.software_jobs.session_scope")
|
||||
def test_job_state_restores_disconnected_job(self, session_scope) -> None:
|
||||
|
|
@ -393,12 +425,12 @@ class SoftwareJobProtocolTests(unittest.TestCase):
|
|||
"job_id": str(uuid4()),
|
||||
"lease_id": str(lease_id),
|
||||
"request_digest": digest,
|
||||
"stage": "waiting_input",
|
||||
"stage": "downloading_inputs",
|
||||
"progress": 0,
|
||||
"metrics": {},
|
||||
})
|
||||
self.assertEqual(job.status, "dispatched")
|
||||
self.assertEqual(job.stage, "waiting_input")
|
||||
self.assertEqual(job.stage, "downloading_inputs")
|
||||
|
||||
@patch("core.software_jobs.session_scope")
|
||||
def test_ready_to_run_is_not_reported_as_running(self, session_scope) -> None:
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ class StaticVendorTests(unittest.TestCase):
|
|||
self.assertIn('id="node-enrollment-modal" class="modal"', html)
|
||||
self.assertIn("生成 Windows Node 注册码", html)
|
||||
self.assertIn('"/v1/admin/software-node-enrollments"', admin_js)
|
||||
self.assertIn('capabilities: ["origin.plot@v1"]', admin_js)
|
||||
self.assertIn('capabilities: ["origin.plot@v2"]', admin_js)
|
||||
self.assertIn('origin.health === "ready"', admin_js)
|
||||
self.assertIn("ttl_seconds: 600", admin_js)
|
||||
self.assertIn("navigator.clipboard.writeText(value)", admin_js)
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
|||
'SetRequestHeader("X-Node-Id"',
|
||||
"DataProtectionScope.LocalMachine",
|
||||
"SetAccessRuleProtection(isProtected: true",
|
||||
'"origin.plot@v1"',
|
||||
'"origin.plot@v2"',
|
||||
"NotifyIcon",
|
||||
"ConfigurationForm",
|
||||
"TrayIconFactory.Create",
|
||||
|
|
@ -71,16 +71,42 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
|||
self.assertIn("注册并连接", form)
|
||||
self.assertIn("ContentWidth = 760", form)
|
||||
self.assertIn("Origin 绘图", form)
|
||||
self.assertIn('CreateCapabilityRow("Origin 绘图", "origin.plot@v1")', form)
|
||||
self.assertIn('CreateCapabilityRow("Origin 绘图", "origin.plot@v2")', form)
|
||||
self.assertIn('CreateButton("立即重连", 112, primary: true)', form)
|
||||
self.assertIn("ReconnectRequested?.Invoke()", form)
|
||||
self.assertIn("registrationCard.Visible = !registered", form)
|
||||
self.assertIn("reconnect.Visible = registered", form)
|
||||
self.assertIn("resetIdentity.Visible = registered", form)
|
||||
self.assertIn('CreateSectionTitle("本机任务")', form)
|
||||
self.assertIn("DataGridView", form)
|
||||
self.assertIn("jobInbox.ReadJobSnapshots()", form)
|
||||
self.assertIn('"software_running" => "软件执行中"', form)
|
||||
self.assertNotIn("HttpClient", form)
|
||||
|
||||
tray = (PROJECT / "TrayApplicationContext.cs").read_text(encoding="utf-8")
|
||||
self.assertIn("form.ReconnectRequested += RestartConnection", tray)
|
||||
|
||||
def test_local_job_monitor_is_persisted_and_software_neutral(self) -> None:
|
||||
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
|
||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
||||
models = (PROJECT / "JobMonitorModels.cs").read_text(encoding="utf-8")
|
||||
self.assertIn('"state.json"', inbox)
|
||||
self.assertIn("ReadJobSnapshots", inbox)
|
||||
self.assertIn("AtomicWrite(path, content, overwrite: true)", inbox)
|
||||
for stage in (
|
||||
"accepted",
|
||||
"downloading_inputs",
|
||||
"ready_to_run",
|
||||
"software_running",
|
||||
"uploading_outputs",
|
||||
"succeeded",
|
||||
"failed",
|
||||
"cancelled",
|
||||
):
|
||||
self.assertIn(f'"{stage}"', inbox + connection)
|
||||
self.assertIn("JobDisplaySnapshot", models)
|
||||
self.assertNotIn("origin_running", inbox + connection + models)
|
||||
|
||||
def test_unified_installer_registers_login_startup(self) -> None:
|
||||
launcher = (ROOT / "install-windows-node.bat").read_text(encoding="utf-8")
|
||||
self.assertNotIn("powershell", launcher.lower())
|
||||
|
|
@ -141,7 +167,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
|||
self.assertIn('AutomationProgId = @"Origin.ApplicationSI\\CLSID"', probe)
|
||||
self.assertIn("RegistryHive.LocalMachine", probe)
|
||||
self.assertIn("RegistryHive.CurrentUser", probe)
|
||||
self.assertIn('new("OriginPro", version, "0.2.0", health, detail)', probe)
|
||||
self.assertIn('new("OriginPro", version, "0.3.0", health, detail)', probe)
|
||||
self.assertIn(
|
||||
"&& !jobInbox.HasPendingOriginJobs",
|
||||
connection,
|
||||
|
|
@ -164,9 +190,12 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
|||
def test_job_offer_is_persisted_before_acceptance(self) -> None:
|
||||
inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8")
|
||||
connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8")
|
||||
self.assertIn('capabilityValue.GetString() != "origin.plot@v1"', inbox)
|
||||
self.assertIn('capabilityValue.GetString() != "origin.plot@v2"', inbox)
|
||||
self.assertIn('root.TryGetProperty("input_transfers"', inbox)
|
||||
self.assertIn('"input", key, filename', inbox)
|
||||
self.assertIn("PlotTypes.Contains", inbox)
|
||||
self.assertIn("OutputFormats.Contains", inbox)
|
||||
self.assertIn("IsValidOutputs", inbox)
|
||||
self.assertIn('("figure", "png") => "figure_png"', inbox)
|
||||
self.assertIn("FileOptions.WriteThrough", inbox)
|
||||
self.assertIn("stream.Flush(flushToDisk: true)", inbox)
|
||||
new_record = inbox.split("var record =", 1)[1].split("private static JsonElement?", 1)[0]
|
||||
|
|
@ -183,9 +212,11 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
|||
self.assertIn("StartJobPipeline(socket, acceptedJob)", connection)
|
||||
self.assertIn("inputDownloader.DownloadAsync(job, CancellationToken.None)", connection)
|
||||
self.assertIn('stage = "uploading_outputs"', connection)
|
||||
self.assertIn('stage = "software_running"', connection)
|
||||
self.assertNotIn('stage = "origin_running"', connection)
|
||||
self.assertIn("&& !job.UploadComplete", connection)
|
||||
self.assertIn("StartJobPipeline(socket, job)", connection)
|
||||
self.assertIn('stage = "waiting_input"', connection)
|
||||
self.assertIn('stage = "downloading_inputs"', connection)
|
||||
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', inbox)
|
||||
self.assertIn("AtomicWrite(requestPath, updated, overwrite: true)", inbox)
|
||||
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ class SoftwareCapabilityListTool(_SoftwareJobTool):
|
|||
nodes = list_nodes()
|
||||
items = [{
|
||||
"capability": item,
|
||||
"display_name": "Origin 科研绘图" if item == "origin.plot@v1" else item,
|
||||
"display_name": "Origin 科研绘图" if item == "origin.plot@v2" else item,
|
||||
"available_nodes": sum(
|
||||
1
|
||||
for node in nodes
|
||||
|
|
@ -47,8 +47,8 @@ class SoftwareCapabilityListTool(_SoftwareJobTool):
|
|||
class SoftwareJobSubmitTool(_SoftwareJobTool):
|
||||
name = "software_job_submit"
|
||||
description = (
|
||||
"Submit an Origin plot job using a registered CSV, XLSX, or JSON artifact. "
|
||||
"Call register_artifact first when the input is only a workspace file. "
|
||||
"Submit an Origin plot job using one or more registered CSV, XLSX, or JSON artifacts. "
|
||||
"Call register_artifact first for each workspace file. "
|
||||
"Return immediately with job_id; do not poll continuously or wait for completion."
|
||||
)
|
||||
_axis_schema = {
|
||||
|
|
@ -64,19 +64,22 @@ class SoftwareJobSubmitTool(_SoftwareJobTool):
|
|||
"type": "object",
|
||||
"properties": {
|
||||
"type": {"type": "string", "enum": ["line", "scatter", "line_scatter"]},
|
||||
"x": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 128,
|
||||
"description": "Exact X column name in the input file.",
|
||||
},
|
||||
"y": {
|
||||
"series": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 16,
|
||||
"uniqueItems": True,
|
||||
"items": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"description": "One to sixteen unique Y column names.",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"input": {"type": "string", "pattern": "^[a-z][a-z0-9_]{0,31}$"},
|
||||
"x": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"y": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"label": {"type": "string", "minLength": 1, "maxLength": 200},
|
||||
},
|
||||
"required": ["input", "x", "y"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
"description": "Series with explicit input key and X/Y columns.",
|
||||
},
|
||||
"template": {"type": "string", "enum": ["publication_double_column"]},
|
||||
"title": {"type": "string", "maxLength": 500},
|
||||
|
|
@ -92,84 +95,112 @@ class SoftwareJobSubmitTool(_SoftwareJobTool):
|
|||
},
|
||||
"error_bars": {"type": "null"},
|
||||
},
|
||||
"required": ["type", "x", "y"],
|
||||
"required": ["type", "series"],
|
||||
"additionalProperties": False,
|
||||
}
|
||||
_input_schema = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"key": {"type": "string", "pattern": "^[a-z][a-z0-9_]{0,31}$"},
|
||||
"artifact_id": {
|
||||
"type": "string",
|
||||
"description": "Artifact UUID returned by register_artifact.",
|
||||
},
|
||||
"selector": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"sheet": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
},
|
||||
"required": ["sheet"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
},
|
||||
"required": ["key", "artifact_id"],
|
||||
"additionalProperties": False,
|
||||
}
|
||||
_output_schema = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"formats": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"uniqueItems": True,
|
||||
"items": {"type": "string", "enum": ["opju", "png", "svg", "pdf"]},
|
||||
"key": {
|
||||
"type": "string",
|
||||
"enum": ["project", "figure_png", "figure_svg", "figure_pdf"],
|
||||
},
|
||||
"type": {"type": "string", "enum": ["project", "figure"]},
|
||||
"format": {"type": "string", "enum": ["opju", "png", "svg", "pdf"]},
|
||||
"options": {
|
||||
"type": "object",
|
||||
"properties": {"dpi": {"type": "integer", "minimum": 72, "maximum": 1200}},
|
||||
"required": ["dpi"],
|
||||
"additionalProperties": False,
|
||||
"description": "Only valid for the PNG figure output.",
|
||||
},
|
||||
"dpi": {"type": "integer", "minimum": 72, "maximum": 1200},
|
||||
"capture_screenshots": {"type": "boolean"},
|
||||
"record_video": {"type": "boolean", "enum": [False]},
|
||||
},
|
||||
"required": ["formats"],
|
||||
"required": ["key", "type", "format"],
|
||||
"additionalProperties": False,
|
||||
}
|
||||
parameters = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"capability": {"type": "string", "enum": sorted(SUPPORTED_CAPABILITIES)},
|
||||
"input_id": {
|
||||
"type": "string",
|
||||
"description": "Artifact UUID returned by register_artifact or another artifact-producing flow.",
|
||||
"inputs": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 16,
|
||||
"items": _input_schema,
|
||||
},
|
||||
"sheet": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 128,
|
||||
"description": "Optional XLSX worksheet name.",
|
||||
"operation": {
|
||||
"type": "object",
|
||||
"properties": {"plot": _plot_schema},
|
||||
"required": ["plot"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
"outputs": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 16,
|
||||
"uniqueItems": True,
|
||||
"items": _output_schema,
|
||||
"description": (
|
||||
"Requested deliverables. Use project/project/opju, "
|
||||
"figure_png/figure/png, figure_svg/figure/svg, or figure_pdf/figure/pdf."
|
||||
),
|
||||
},
|
||||
"plot": _plot_schema,
|
||||
"output": _output_schema,
|
||||
"idempotency_key": {
|
||||
"type": "string",
|
||||
"description": "Stable unique key for this exact submission; omit to generate one.",
|
||||
},
|
||||
},
|
||||
"required": ["capability", "input_id", "plot", "output"],
|
||||
"required": ["capability", "inputs", "operation", "outputs"],
|
||||
"additionalProperties": False,
|
||||
}
|
||||
|
||||
def execute(
|
||||
self,
|
||||
capability: str,
|
||||
input_id: str = "",
|
||||
plot: dict | None = None,
|
||||
output: dict | None = None,
|
||||
sheet: str = "",
|
||||
request: dict | None = None,
|
||||
inputs: list[dict] | None = None,
|
||||
operation: dict | None = None,
|
||||
outputs: list[dict] | None = None,
|
||||
idempotency_key: str = "",
|
||||
) -> str:
|
||||
try:
|
||||
if isinstance(input_id, dict) and request is None:
|
||||
request = input_id
|
||||
input_id = ""
|
||||
if request is not None:
|
||||
if input_id or plot is not None or output is not None or sheet:
|
||||
return "[Error] request cannot be combined with input_id, sheet, plot, or output"
|
||||
normalized_request = request
|
||||
else:
|
||||
if not input_id or plot is None or output is None:
|
||||
return "[Error] input_id, plot, and output are required for an Origin job"
|
||||
if not inputs or operation is None or not outputs:
|
||||
return "[Error] inputs, operation, and outputs are required for a software job"
|
||||
canonical_inputs = []
|
||||
for item in inputs:
|
||||
if not isinstance(item, dict):
|
||||
return "[Error] each input must be an object"
|
||||
try:
|
||||
canonical_input_id = str(UUID(str(input_id)))
|
||||
artifact_id = str(UUID(str(item.get("artifact_id") or "")))
|
||||
except ValueError:
|
||||
return "[Error] input_id must be an artifact UUID; call register_artifact first"
|
||||
input_spec = {"input_id": canonical_input_id}
|
||||
if sheet:
|
||||
input_spec["sheet"] = sheet
|
||||
normalized_request = {
|
||||
"schema_version": 1,
|
||||
"input": input_spec,
|
||||
"plot": plot,
|
||||
"output": output,
|
||||
}
|
||||
return "[Error] every artifact_id must be a UUID; call register_artifact first"
|
||||
canonical = {**item, "artifact_id": artifact_id}
|
||||
canonical_inputs.append(canonical)
|
||||
normalized_request = {
|
||||
"schema_version": 2,
|
||||
"inputs": canonical_inputs,
|
||||
"operation": operation,
|
||||
"outputs": outputs,
|
||||
}
|
||||
job, created = create_job(
|
||||
self.user_id,
|
||||
self.task_id,
|
||||
|
|
|
|||
|
|
@ -281,9 +281,10 @@ def register_software_node_routes(app, *, require_user, require_admin) -> None:
|
|||
except SoftwareNodeError as exc:
|
||||
raise HTTPException(400, str(exc)) from exc
|
||||
|
||||
@app.get("/v1/software-jobs/{job_id}/input", tags=["software-nodes"])
|
||||
@app.get("/v1/software-jobs/{job_id}/inputs/{input_key}", tags=["software-nodes"])
|
||||
def download_software_job_input(
|
||||
job_id: UUID,
|
||||
input_key: str,
|
||||
authorization: str | None = Header(default=None),
|
||||
x_node_id: str = Header(default=""),
|
||||
):
|
||||
|
|
@ -292,7 +293,7 @@ def register_software_node_routes(app, *, require_user, require_admin) -> None:
|
|||
authenticate_node(node_id, _bearer(authorization))
|
||||
except (ValueError, SoftwareNodeError) as exc:
|
||||
raise HTTPException(401, "invalid node credentials") from exc
|
||||
item = get_job_input(node_id, job_id)
|
||||
item = get_job_input(node_id, job_id, input_key)
|
||||
if item is None:
|
||||
raise HTTPException(404, "software job input not found")
|
||||
target = safe_join(load_user_root(item["user_id"]), item["current_path"])
|
||||
|
|
@ -340,9 +341,13 @@ def register_software_node_routes(app, *, require_user, require_admin) -> None:
|
|||
metadata = OUTPUT_ARTIFACTS.get(artifact_id)
|
||||
if metadata is None:
|
||||
raise HTTPException(400, "unsupported output artifact identity")
|
||||
filename, _, output_format = metadata
|
||||
requested_formats = set(context["request"].get("output", {}).get("formats") or [])
|
||||
if output_format is not None and output_format not in requested_formats:
|
||||
filename, _, _ = metadata
|
||||
requested_ids = {
|
||||
item.get("key")
|
||||
for item in context["request"].get("outputs", [])
|
||||
if isinstance(item, dict)
|
||||
} | SOFTWARE_JOB_METADATA_IDS
|
||||
if artifact_id not in requested_ids:
|
||||
raise HTTPException(400, "output artifact was not requested")
|
||||
if not 1 <= x_content_length <= MAX_OUTPUT_ARTIFACT_BYTES:
|
||||
raise HTTPException(400, "output artifact size is invalid")
|
||||
|
|
|
|||
|
|
@ -118,7 +118,7 @@ class ExternalSystemCredentialsRequest(BaseModel):
|
|||
|
||||
class SoftwareEnrollmentCreateRequest(BaseModel):
|
||||
expected_name: str = ""
|
||||
capabilities: list[str] = Field(default_factory=lambda: ["origin.plot@v1"])
|
||||
capabilities: list[str] = Field(default_factory=lambda: ["origin.plot@v2"])
|
||||
ttl_seconds: int = 600
|
||||
|
||||
|
||||
|
|
@ -137,5 +137,5 @@ class SoftwareNodeDisableRequest(BaseModel):
|
|||
|
||||
class SoftwareJobCreateRequest(BaseModel):
|
||||
idempotency_key: str
|
||||
capability: str = "origin.plot@v1"
|
||||
capability: str = "origin.plot@v2"
|
||||
request: dict = Field(default_factory=dict)
|
||||
|
|
|
|||
|
|
@ -312,7 +312,7 @@ async function createNodeEnrollment(e) {
|
|||
try {
|
||||
const result = await apiSend("POST", "/v1/admin/software-node-enrollments", {
|
||||
expected_name: $("node-expected-name").value.trim(),
|
||||
capabilities: ["origin.plot@v1"],
|
||||
capabilities: ["origin.plot@v2"],
|
||||
ttl_seconds: 600,
|
||||
});
|
||||
$("node-enrollment-code").textContent = result.enrollment_code || "";
|
||||
|
|
|
|||
|
|
@ -25,8 +25,8 @@ const statusLabel = {
|
|||
succeeded: "已完成", failed: "失败", cancelled: "已取消",
|
||||
};
|
||||
const stageLabel = {
|
||||
accepted: "节点已接收", waiting_input: "正在下载输入文件", ready_to_run: "准备软件环境",
|
||||
origin_running: "Origin 正在生成图表", uploading_outputs: "正在上传结果",
|
||||
accepted: "节点已接收", downloading_inputs: "正在下载输入文件", ready_to_run: "准备软件环境",
|
||||
software_running: "专业软件正在执行", uploading_outputs: "正在上传结果",
|
||||
cancel_requested: "停止请求已发送", terminal: "任务已结束",
|
||||
};
|
||||
|
||||
|
|
@ -165,7 +165,8 @@ function onListScroll(event) {
|
|||
|
||||
function jobCard(job) {
|
||||
const summary = job.request_summary || {};
|
||||
const input = job.input || {};
|
||||
const inputs = Array.isArray(job.input) ? job.input : [];
|
||||
const inputNames = inputs.map((item) => item.filename).filter(Boolean).join("、");
|
||||
const active = ACTIVE.has(job.status);
|
||||
const progress = Math.max(0, Math.min(100, Number(job.progress || 0)));
|
||||
const detail = stageLabel[job.stage] || statusLabel[job.status] || "正在处理";
|
||||
|
|
@ -177,7 +178,7 @@ function jobCard(job) {
|
|||
<span class="sj-status">${statusIcon(job.status)}${escapeHtml(statusLabel[job.status] || job.status)}</span></div>
|
||||
<div class="sj-sub">${icons.activity}<span>${escapeHtml(detail)}${error ? ` · ${escapeHtml(error)}` : ""}</span></div>
|
||||
${active ? `<div class="sj-progress"><i style="width:${progress}%"></i></div>` : ""}
|
||||
<div class="sj-meta">${icons.clock}${escapeHtml(job.task_name || "未命名对话")}${input.filename ? ` · ${escapeHtml(input.filename)}` : ""}${openedText ? ` · 开启于 ${escapeHtml(openedText)}` : ""}</div>
|
||||
<div class="sj-meta">${icons.clock}${escapeHtml(job.task_name || "未命名对话")}${inputNames ? ` · ${escapeHtml(inputNames)}` : ""}${openedText ? ` · 开启于 ${escapeHtml(openedText)}` : ""}</div>
|
||||
<div class="sj-actions">
|
||||
<button class="small" data-job-action="open" data-task-id="${escapeHtml(job.task_id)}">${icons.open}打开对话</button>
|
||||
${job.status === "succeeded" ? `<button class="small primary" data-job-action="analyze" data-task-id="${escapeHtml(job.task_id)}">${icons.analyze}分析结果</button>` : ""}
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
内网 MVP 的 Windows 执行节点，目标运行环境为 Windows 11 Enterprise + .NET 10 SDK 10.0.303。仓库根目录 `global.json` 固定 SDK patch；客户端只使用 .NET Windows Desktop Framework，不依赖第三方 NuGet 包。
|
||||
|
||||
当前实现托盘状态角标、小型配置窗口、注册、DPAPI/ACL 配置保存、WebSocket `hello`/心跳和退避重连，并只读探测 Origin/OriginPro 安装版本、COM 自动化组件与桌面会话状态。Node 可以接收受控的 `origin.plot@v1` offer，在本机任务目录原子保存请求后回报 accept/reject；随后以 Node 身份流式下载任务绑定的 CSV/XLSX/JSON，校验大小与 SHA-256 后原子保存。固定 Origin Worker 独立于单次 WebSocket 执行，断线不终止已启动绘图。成功产物按 manifest 逐项流式上传并由云端复核大小与 SHA-256，全部完成后原子发布到任务工作目录的 `origin/<job_id>/`；中断后按本地 `upload-complete.json` 幂等续传。
|
||||
当前实现托盘状态角标、配置与本机任务窗口、注册、DPAPI/ACL 配置保存、WebSocket `hello`/心跳和退避重连，并只读探测 Origin/OriginPro 安装版本、COM 自动化组件与桌面会话状态。本机任务列表只读取已派发到该 Node 的持久化目录，展示标题、全部输入、通用执行阶段、进度、时间、Job ID 和错误，不查询云端未派发队列。Node 可以接收受控的 `origin.plot@v2` offer，在本机任务目录原子保存请求后回报 accept/reject；随后以 Node 身份逐个流式下载任务绑定的 CSV/XLSX/JSON 到 `input/<key>/<filename>`，校验大小与 SHA-256 后原子保存。固定 Origin Worker 独立于单次 WebSocket 执行，断线不终止已启动绘图。成功产物按 manifest 逐项流式上传并由云端复核大小与 SHA-256，全部完成后原子发布到任务工作目录的 `origin/<job_id>/`；中断后按本地 `upload-complete.json` 幂等续传。
|
||||
|
||||
在仓库根目录执行一条命令生成可分发 ZIP：
|
||||
|
||||
|
|
@ -31,7 +31,7 @@ windows-node/Zcbot.WindowsNode/bin/Debug/net10.0-windows/Zcbot.WindowsNode.exe e
|
|||
windows-node/Zcbot.WindowsNode/bin/Debug/net10.0-windows/Zcbot.WindowsNode.exe
|
||||
```
|
||||
|
||||
直接双击 EXE 默认启动托盘 UI；红点表示未注册或身份失效，黄点表示正在连接，绿点表示在线。双击托盘图标打开配置窗口。无界面运行使用：
|
||||
直接双击 EXE 默认启动托盘 UI；红点表示未注册或身份失效，黄点表示正在连接，绿点表示在线。双击托盘图标打开窗口，可在“本机任务”中查看最近 50 条已接收任务及执行详情。无界面运行使用：
|
||||
|
||||
```powershell
|
||||
Zcbot.WindowsNode.exe run --headless
|
||||
|
|
|
|||
|
|
@ -25,6 +25,11 @@ internal sealed class ConfigurationForm : Form
|
|||
private readonly Label detail = CreateBodyLabel();
|
||||
private readonly Label identity = CreateBodyLabel();
|
||||
private readonly Label capabilitySummary = CreateBodyLabel();
|
||||
private readonly Label jobSummary = CreateBodyLabel();
|
||||
private readonly Label jobDetail = CreateBodyLabel();
|
||||
private readonly DataGridView jobGrid = CreateJobGrid();
|
||||
private readonly JobInboxStore jobInbox = new(NodePaths.ForCurrentMachine().JobsDirectory);
|
||||
private readonly System.Windows.Forms.Timer jobRefreshTimer = new() { Interval = 1000 };
|
||||
private readonly TableLayoutPanel registrationCard;
|
||||
private bool changingStartup;
|
||||
|
||||
|
|
@ -63,7 +68,7 @@ internal sealed class ConfigurationForm : Form
|
|||
AutoSizeMode = AutoSizeMode.GrowAndShrink,
|
||||
Dock = DockStyle.Top,
|
||||
ColumnCount = 1,
|
||||
RowCount = 5,
|
||||
RowCount = 6,
|
||||
BackColor = BackColor,
|
||||
};
|
||||
page.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 100));
|
||||
|
|
@ -92,7 +97,7 @@ internal sealed class ConfigurationForm : Form
|
|||
statusCard.Controls.Add(detail);
|
||||
statusCard.Controls.Add(identity);
|
||||
statusCard.Controls.Add(CreateDivider());
|
||||
statusCard.Controls.Add(CreateCapabilityRow("Origin 绘图", "origin.plot@v1"));
|
||||
statusCard.Controls.Add(CreateCapabilityRow("Origin 绘图", "origin.plot@v2"));
|
||||
statusCard.Controls.Add(capabilitySummary);
|
||||
var resetActions = CreateActions();
|
||||
resetActions.Controls.Add(reconnect);
|
||||
|
|
@ -100,6 +105,15 @@ internal sealed class ConfigurationForm : Form
|
|||
statusCard.Controls.Add(resetActions);
|
||||
page.Controls.Add(statusCard);
|
||||
|
||||
var jobsCard = CreateCard();
|
||||
jobsCard.Controls.Add(CreateSectionTitle("本机任务"));
|
||||
jobsCard.Controls.Add(CreateHint(
|
||||
"仅显示已经派发到本机的任务；状态来自本地持久化记录，断线或重启后仍可查看。"));
|
||||
jobsCard.Controls.Add(jobSummary);
|
||||
jobsCard.Controls.Add(jobGrid);
|
||||
jobsCard.Controls.Add(jobDetail);
|
||||
page.Controls.Add(jobsCard);
|
||||
|
||||
registrationCard = CreateCard();
|
||||
registrationCard.Controls.Add(CreateSectionTitle("首次注册"));
|
||||
registrationCard.Controls.Add(CreateHint(
|
||||
|
|
@ -137,6 +151,11 @@ internal sealed class ConfigurationForm : Form
|
|||
}
|
||||
};
|
||||
startAtLogin.Checked = StartupRegistration.IsEnabled;
|
||||
jobGrid.SelectionChanged += (_, _) => ShowSelectedJob();
|
||||
jobRefreshTimer.Tick += (_, _) => RefreshJobs();
|
||||
jobRefreshTimer.Start();
|
||||
Disposed += (_, _) => jobRefreshTimer.Dispose();
|
||||
RefreshJobs();
|
||||
ApplyStatus(NodeStatus.Create(NodeState.NotRegistered, "尚未注册"), null);
|
||||
}
|
||||
|
||||
|
|
@ -188,6 +207,106 @@ internal sealed class ConfigurationForm : Form
|
|||
return $"{state} · {version}\n{origin.Detail}";
|
||||
}
|
||||
|
||||
private void RefreshJobs()
|
||||
{
|
||||
Guid? selectedId = jobGrid.SelectedRows.Count > 0
|
||||
&& jobGrid.SelectedRows[0].Tag is JobDisplaySnapshot selected
|
||||
? selected.JobId
|
||||
: null;
|
||||
IReadOnlyList<JobDisplaySnapshot> snapshots;
|
||||
try
|
||||
{
|
||||
snapshots = jobInbox.ReadJobSnapshots();
|
||||
}
|
||||
catch (Exception exception) when (
|
||||
exception is IOException or UnauthorizedAccessException)
|
||||
{
|
||||
jobSummary.Text = $"无法读取本机任务：{exception.Message}";
|
||||
return;
|
||||
}
|
||||
jobGrid.Rows.Clear();
|
||||
DataGridViewRow? rowToSelect = null;
|
||||
foreach (var snapshot in snapshots)
|
||||
{
|
||||
var index = jobGrid.Rows.Add(
|
||||
FormatJobStage(snapshot.Stage),
|
||||
snapshot.Title,
|
||||
snapshot.InputFilename,
|
||||
FormatProgress(snapshot),
|
||||
snapshot.AcceptedAt.LocalDateTime.ToString("MM-dd HH:mm"),
|
||||
snapshot.JobId.ToString("N")[..8]);
|
||||
var row = jobGrid.Rows[index];
|
||||
row.Tag = snapshot;
|
||||
row.DefaultCellStyle.ForeColor = snapshot.Stage switch
|
||||
{
|
||||
"failed" => Color.Firebrick,
|
||||
"cancelled" => Color.DimGray,
|
||||
"succeeded" => Color.ForestGreen,
|
||||
_ => Color.FromArgb(30, 41, 59),
|
||||
};
|
||||
if (snapshot.JobId == selectedId)
|
||||
{
|
||||
rowToSelect = row;
|
||||
}
|
||||
}
|
||||
var active = snapshots.Count(item => item.IsActive);
|
||||
jobSummary.Text = snapshots.Count == 0
|
||||
? "暂无本机任务"
|
||||
: $"活动任务 {active} 个 · 最近记录 {snapshots.Count} 条";
|
||||
if (rowToSelect is not null)
|
||||
{
|
||||
rowToSelect.Selected = true;
|
||||
jobGrid.CurrentCell = rowToSelect.Cells[0];
|
||||
}
|
||||
else if (jobGrid.Rows.Count > 0)
|
||||
{
|
||||
jobGrid.Rows[0].Selected = true;
|
||||
jobGrid.CurrentCell = jobGrid.Rows[0].Cells[0];
|
||||
}
|
||||
else
|
||||
{
|
||||
jobDetail.Text = "选择任务后可查看执行阶段、更新时间和完整 Job ID。";
|
||||
}
|
||||
ShowSelectedJob();
|
||||
}
|
||||
|
||||
private void ShowSelectedJob()
|
||||
{
|
||||
if (jobGrid.SelectedRows.Count == 0
|
||||
|| jobGrid.SelectedRows[0].Tag is not JobDisplaySnapshot snapshot)
|
||||
{
|
||||
return;
|
||||
}
|
||||
jobDetail.Text = string.Join("\n", [
|
||||
$"{FormatJobStage(snapshot.Stage)} · {snapshot.Detail}",
|
||||
$"能力：{snapshot.Capability}",
|
||||
$"Job ID：{snapshot.JobId}",
|
||||
$"更新时间：{snapshot.UpdatedAt.LocalDateTime:yyyy-MM-dd HH:mm:ss}",
|
||||
]);
|
||||
}
|
||||
|
||||
private static string FormatProgress(JobDisplaySnapshot snapshot) =>
|
||||
snapshot.Stage == "software_running"
|
||||
? $"已执行 {FormatElapsed(DateTimeOffset.UtcNow - snapshot.UpdatedAt)}"
|
||||
: $"{snapshot.Progress}%";
|
||||
|
||||
private static string FormatElapsed(TimeSpan elapsed) => elapsed.TotalHours >= 1
|
||||
? $"{(int)elapsed.TotalHours}:{elapsed.Minutes:00}:{elapsed.Seconds:00}"
|
||||
: $"{elapsed.Minutes:00}:{elapsed.Seconds:00}";
|
||||
|
||||
private static string FormatJobStage(string stage) => stage switch
|
||||
{
|
||||
"accepted" => "等待处理",
|
||||
"downloading_inputs" => "下载输入",
|
||||
"ready_to_run" => "准备执行",
|
||||
"software_running" => "软件执行中",
|
||||
"uploading_outputs" => "上传结果",
|
||||
"succeeded" => "成功",
|
||||
"failed" => "失败",
|
||||
"cancelled" => "已取消",
|
||||
_ => stage,
|
||||
};
|
||||
|
||||
private void ResetIdentity()
|
||||
{
|
||||
var answer = MessageBox.Show(
|
||||
|
|
@ -289,6 +408,65 @@ internal sealed class ConfigurationForm : Form
|
|||
Margin = new Padding(0, 2, 0, 4),
|
||||
};
|
||||
|
||||
private static DataGridView CreateJobGrid()
|
||||
{
|
||||
var grid = new DataGridView
|
||||
{
|
||||
Height = 238,
|
||||
Dock = DockStyle.Top,
|
||||
Margin = new Padding(0, 8, 0, 8),
|
||||
BackgroundColor = Color.White,
|
||||
BorderStyle = BorderStyle.FixedSingle,
|
||||
AllowUserToAddRows = false,
|
||||
AllowUserToDeleteRows = false,
|
||||
AllowUserToResizeRows = false,
|
||||
AutoSizeRowsMode = DataGridViewAutoSizeRowsMode.None,
|
||||
ColumnHeadersHeight = 34,
|
||||
EnableHeadersVisualStyles = false,
|
||||
MultiSelect = false,
|
||||
ReadOnly = true,
|
||||
RowHeadersVisible = false,
|
||||
RowTemplate = { Height = 34 },
|
||||
SelectionMode = DataGridViewSelectionMode.FullRowSelect,
|
||||
};
|
||||
grid.ColumnHeadersDefaultCellStyle.BackColor = Color.FromArgb(241, 245, 249);
|
||||
grid.ColumnHeadersDefaultCellStyle.ForeColor = Color.FromArgb(51, 65, 85);
|
||||
grid.DefaultCellStyle.SelectionBackColor = Color.FromArgb(219, 234, 254);
|
||||
grid.DefaultCellStyle.SelectionForeColor = Color.FromArgb(30, 64, 175);
|
||||
grid.Columns.Add(new DataGridViewTextBoxColumn
|
||||
{
|
||||
HeaderText = "状态",
|
||||
Width = 105,
|
||||
});
|
||||
grid.Columns.Add(new DataGridViewTextBoxColumn
|
||||
{
|
||||
HeaderText = "任务",
|
||||
AutoSizeMode = DataGridViewAutoSizeColumnMode.Fill,
|
||||
MinimumWidth = 170,
|
||||
});
|
||||
grid.Columns.Add(new DataGridViewTextBoxColumn
|
||||
{
|
||||
HeaderText = "输入",
|
||||
Width = 125,
|
||||
});
|
||||
grid.Columns.Add(new DataGridViewTextBoxColumn
|
||||
{
|
||||
HeaderText = "进度",
|
||||
Width = 95,
|
||||
});
|
||||
grid.Columns.Add(new DataGridViewTextBoxColumn
|
||||
{
|
||||
HeaderText = "接收时间",
|
||||
Width = 100,
|
||||
});
|
||||
grid.Columns.Add(new DataGridViewTextBoxColumn
|
||||
{
|
||||
HeaderText = "Job ID",
|
||||
Width = 76,
|
||||
});
|
||||
return grid;
|
||||
}
|
||||
|
||||
private static Panel CreateDivider() => new()
|
||||
{
|
||||
Height = 1,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ namespace Zcbot.WindowsNode;
|
|||
|
||||
internal static class EnrollmentClient
|
||||
{
|
||||
private static readonly string[] Capabilities = ["origin.plot@v1"];
|
||||
private static readonly string[] Capabilities = ["origin.plot@v2"];
|
||||
|
||||
internal static async Task EnrollAsync(
|
||||
EnrollOptions options, NodeConfigStore store, CancellationToken cancellationToken)
|
||||
|
|
|
|||
|
|
@ -8,7 +8,17 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
private static readonly JsonSerializerOptions JsonOptions = new() { WriteIndented = true };
|
||||
private static readonly HashSet<string> PlotTypes =
|
||||
["line", "scatter", "line_scatter"];
|
||||
private static readonly HashSet<string> OutputFormats = ["opju", "png", "svg", "pdf"];
|
||||
private static readonly HashSet<string> JobStages =
|
||||
[
|
||||
"accepted",
|
||||
"downloading_inputs",
|
||||
"ready_to_run",
|
||||
"software_running",
|
||||
"uploading_outputs",
|
||||
"succeeded",
|
||||
"failed",
|
||||
"cancelled",
|
||||
];
|
||||
|
||||
// Origin 执行槽只由尚无终态的任务占用。成功但上传确认尚未落盘的任务会由
|
||||
// 心跳恢复管线继续重传；上传不使用 Origin，不能反向阻塞新的绘图任务。
|
||||
|
|
@ -41,8 +51,8 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
jobId,
|
||||
leaseId,
|
||||
requestDigest,
|
||||
root.TryGetProperty("input_transfer", out var transfer)
|
||||
? transfer.Clone() : null,
|
||||
root.TryGetProperty("input_transfers", out var transfers)
|
||||
? transfers.Clone() : null,
|
||||
ReadTerminal(Path.Combine(jobDirectory, "terminal.json")),
|
||||
File.Exists(Path.Combine(jobDirectory, "upload-complete.json"))));
|
||||
}
|
||||
|
|
@ -54,6 +64,127 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
return jobs;
|
||||
}
|
||||
|
||||
internal IReadOnlyList<JobDisplaySnapshot> ReadJobSnapshots(int limit = 50)
|
||||
{
|
||||
if (!Directory.Exists(jobsDirectory))
|
||||
{
|
||||
return [];
|
||||
}
|
||||
var snapshots = new List<JobDisplaySnapshot>();
|
||||
foreach (var requestPath in Directory.EnumerateFiles(
|
||||
jobsDirectory, "request.json", SearchOption.AllDirectories))
|
||||
{
|
||||
try
|
||||
{
|
||||
using var requestDocument = JsonDocument.Parse(File.ReadAllBytes(requestPath));
|
||||
var root = requestDocument.RootElement;
|
||||
if (!TryReadGuid(root, "job_id", out var jobId))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
var jobDirectory = Directory.GetParent(
|
||||
Directory.GetParent(requestPath)!.FullName)!.FullName;
|
||||
var acceptedAt = ReadDate(root, "accepted_at")
|
||||
?? new DateTimeOffset(File.GetCreationTimeUtc(requestPath));
|
||||
var capability = ReadString(root, "capability", "unknown");
|
||||
var title = "未命名任务";
|
||||
if (root.TryGetProperty("request", out var request)
|
||||
&& request.TryGetProperty("operation", out var operation)
|
||||
&& operation.TryGetProperty("plot", out var plot))
|
||||
{
|
||||
title = ReadString(plot, "title", title);
|
||||
}
|
||||
var inputFilename = root.TryGetProperty("input_transfers", out var transfers)
|
||||
&& transfers.ValueKind == JsonValueKind.Array
|
||||
? string.Join(", ", transfers.EnumerateArray()
|
||||
.Select(item => ReadString(item, "filename", "-")))
|
||||
: "-";
|
||||
var state = ReadState(Path.Combine(jobDirectory, "state.json"));
|
||||
var terminal = ReadTerminal(Path.Combine(jobDirectory, "terminal.json"));
|
||||
var uploadPath = Path.Combine(jobDirectory, "upload-complete.json");
|
||||
var uploadComplete = File.Exists(uploadPath);
|
||||
var stage = state?.Stage ?? "accepted";
|
||||
var progress = state?.Progress ?? 0;
|
||||
var detail = state?.Detail ?? "任务已由本机接收";
|
||||
var updatedAt = state is not null && state.UpdatedAt != DateTimeOffset.MinValue
|
||||
? state.UpdatedAt
|
||||
: acceptedAt;
|
||||
if (terminal is JsonElement terminalValue)
|
||||
{
|
||||
var terminalStatus = ReadString(terminalValue, "status", "failed");
|
||||
if (terminalStatus == "succeeded" && !uploadComplete)
|
||||
{
|
||||
stage = "uploading_outputs";
|
||||
progress = Math.Max(progress, 90);
|
||||
detail = state?.Stage == "uploading_outputs"
|
||||
? state.Detail
|
||||
: "软件执行完成，等待上传结果";
|
||||
}
|
||||
else
|
||||
{
|
||||
stage = terminalStatus;
|
||||
progress = terminalStatus == "succeeded" ? 100 : progress;
|
||||
detail = TerminalDetail(terminalValue, terminalStatus, detail);
|
||||
}
|
||||
updatedAt = LatestWrite(updatedAt, Path.Combine(jobDirectory, "terminal.json"));
|
||||
}
|
||||
if (uploadComplete)
|
||||
{
|
||||
stage = "succeeded";
|
||||
progress = 100;
|
||||
detail = "结果已上传并由云端确认";
|
||||
updatedAt = LatestWrite(updatedAt, uploadPath);
|
||||
}
|
||||
snapshots.Add(new JobDisplaySnapshot(
|
||||
jobId,
|
||||
capability,
|
||||
title,
|
||||
inputFilename,
|
||||
stage,
|
||||
Math.Clamp(progress, 0, 100),
|
||||
detail,
|
||||
acceptedAt,
|
||||
updatedAt,
|
||||
uploadComplete));
|
||||
}
|
||||
catch (Exception exception) when (
|
||||
exception is JsonException or IOException or UnauthorizedAccessException)
|
||||
{
|
||||
}
|
||||
}
|
||||
return snapshots
|
||||
.OrderByDescending(item => item.IsActive)
|
||||
.ThenByDescending(item => item.UpdatedAt)
|
||||
.Take(Math.Max(1, limit))
|
||||
.ToArray();
|
||||
}
|
||||
|
||||
internal void WriteState(RecoverableJob job, string stage, int progress, string detail)
|
||||
{
|
||||
if (!JobStages.Contains(stage))
|
||||
{
|
||||
throw new InvalidDataException("Unsupported local job stage.");
|
||||
}
|
||||
var path = Path.Combine(
|
||||
jobsDirectory, job.JobId.ToString("D"), "state.json");
|
||||
var content = JsonSerializer.SerializeToUtf8Bytes(new
|
||||
{
|
||||
stage,
|
||||
progress = Math.Clamp(progress, 0, 100),
|
||||
detail = detail[..Math.Min(detail.Length, 500)],
|
||||
updated_at = DateTimeOffset.UtcNow,
|
||||
}, JsonOptions);
|
||||
try
|
||||
{
|
||||
AtomicWrite(path, content, overwrite: true);
|
||||
}
|
||||
catch (Exception exception) when (
|
||||
exception is IOException or UnauthorizedAccessException)
|
||||
{
|
||||
Console.Error.WriteLine($"[WARN] Local job state update failed: {exception.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
internal JobOfferResult Accept(JsonElement payload)
|
||||
{
|
||||
if (!TryReadGuid(payload, "job_id", out var jobId)
|
||||
|
|
@ -62,11 +193,11 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
|| digestValue.ValueKind != JsonValueKind.String
|
||||
|| digestValue.GetString() is not { Length: 64 } requestDigest
|
||||
|| !payload.TryGetProperty("capability", out var capabilityValue)
|
||||
|| capabilityValue.GetString() != "origin.plot@v1"
|
||||
|| capabilityValue.GetString() != "origin.plot@v2"
|
||||
|| !payload.TryGetProperty("request", out var request)
|
||||
|| request.ValueKind != JsonValueKind.Object
|
||||
|| !payload.TryGetProperty("input_transfer", out var inputTransfer)
|
||||
|| !IsValidInputTransfer(inputTransfer))
|
||||
|| !payload.TryGetProperty("input_transfers", out var inputTransfers)
|
||||
|| !IsValidInputTransfers(inputTransfers, jobId))
|
||||
{
|
||||
return JobOfferResult.Reject("invalid_offer");
|
||||
}
|
||||
|
|
@ -74,7 +205,10 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
{
|
||||
return JobOfferResult.Reject("unsupported_request");
|
||||
}
|
||||
|
||||
if (!InputsMatchTransfers(request, inputTransfers))
|
||||
{
|
||||
return JobOfferResult.Reject("invalid_offer");
|
||||
}
|
||||
var directory = Path.Combine(jobsDirectory, jobId.ToString("D"));
|
||||
var requestDirectory = Path.Combine(directory, "request");
|
||||
var requestPath = Path.Combine(requestDirectory, "request.json");
|
||||
|
|
@ -99,13 +233,14 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
job_id = jobId,
|
||||
lease_id = leaseId,
|
||||
request_digest = requestDigest,
|
||||
capability = "origin.plot@v1",
|
||||
capability = "origin.plot@v2",
|
||||
accepted_at = DateTimeOffset.UtcNow,
|
||||
request,
|
||||
input_transfer = inputTransfer,
|
||||
input_transfers = inputTransfers,
|
||||
}, JsonOptions);
|
||||
AtomicWrite(requestPath, updated, overwrite: true);
|
||||
}
|
||||
EnsureAcceptedState(jobId);
|
||||
return JobOfferResult.Accept(jobId, leaseId, requestDigest);
|
||||
}
|
||||
catch (JsonException)
|
||||
|
|
@ -119,14 +254,15 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
job_id = jobId,
|
||||
lease_id = leaseId,
|
||||
request_digest = requestDigest,
|
||||
capability = "origin.plot@v1",
|
||||
capability = "origin.plot@v2",
|
||||
accepted_at = DateTimeOffset.UtcNow,
|
||||
request,
|
||||
input_transfer = inputTransfer,
|
||||
input_transfers = inputTransfers,
|
||||
}, JsonOptions);
|
||||
try
|
||||
{
|
||||
AtomicWrite(requestPath, record, overwrite: false);
|
||||
EnsureAcceptedState(jobId);
|
||||
return JobOfferResult.Accept(jobId, leaseId, requestDigest);
|
||||
}
|
||||
catch (IOException)
|
||||
|
|
@ -145,6 +281,101 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
return document.RootElement.Clone();
|
||||
}
|
||||
|
||||
private void EnsureAcceptedState(Guid jobId)
|
||||
{
|
||||
var path = Path.Combine(jobsDirectory, jobId.ToString("D"), "state.json");
|
||||
if (File.Exists(path))
|
||||
{
|
||||
return;
|
||||
}
|
||||
var content = JsonSerializer.SerializeToUtf8Bytes(new
|
||||
{
|
||||
stage = "accepted",
|
||||
progress = 0,
|
||||
detail = "任务已由本机接收",
|
||||
updated_at = DateTimeOffset.UtcNow,
|
||||
}, JsonOptions);
|
||||
try
|
||||
{
|
||||
AtomicWrite(path, content, overwrite: false);
|
||||
}
|
||||
catch (IOException) when (File.Exists(path))
|
||||
{
|
||||
}
|
||||
catch (Exception exception) when (
|
||||
exception is IOException or UnauthorizedAccessException)
|
||||
{
|
||||
Console.Error.WriteLine($"[WARN] Initial local job state failed: {exception.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
private static LocalJobState? ReadState(string path)
|
||||
{
|
||||
if (!File.Exists(path))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
try
|
||||
{
|
||||
using var document = JsonDocument.Parse(File.ReadAllBytes(path));
|
||||
var root = document.RootElement;
|
||||
var stage = ReadString(root, "stage", "accepted");
|
||||
var progress = root.TryGetProperty("progress", out var progressValue)
|
||||
&& progressValue.TryGetInt32(out var parsedProgress)
|
||||
? parsedProgress
|
||||
: 0;
|
||||
return new LocalJobState(
|
||||
stage,
|
||||
progress,
|
||||
ReadString(root, "detail", ""),
|
||||
ReadDate(root, "updated_at") ?? DateTimeOffset.MinValue);
|
||||
}
|
||||
catch (Exception exception) when (
|
||||
exception is JsonException or IOException or UnauthorizedAccessException)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private static string TerminalDetail(JsonElement terminal, string status, string fallback)
|
||||
{
|
||||
if (terminal.TryGetProperty("error", out var error))
|
||||
{
|
||||
var detail = ReadString(error, "detail", "");
|
||||
if (!string.IsNullOrWhiteSpace(detail))
|
||||
{
|
||||
return detail;
|
||||
}
|
||||
}
|
||||
return status switch
|
||||
{
|
||||
"succeeded" => "软件任务执行成功",
|
||||
"cancelled" => "任务已取消",
|
||||
"failed" => "任务执行失败",
|
||||
_ => fallback,
|
||||
};
|
||||
}
|
||||
|
||||
private static DateTimeOffset LatestWrite(DateTimeOffset current, string path)
|
||||
{
|
||||
var writtenAt = new DateTimeOffset(File.GetLastWriteTimeUtc(path));
|
||||
return writtenAt > current ? writtenAt : current;
|
||||
}
|
||||
|
||||
private static string ReadString(JsonElement value, string name, string fallback) =>
|
||||
value.TryGetProperty(name, out var property)
|
||||
&& property.ValueKind == JsonValueKind.String
|
||||
&& !string.IsNullOrWhiteSpace(property.GetString())
|
||||
? property.GetString()!
|
||||
: fallback;
|
||||
|
||||
private static DateTimeOffset? ReadDate(JsonElement value, string name) =>
|
||||
value.TryGetProperty(name, out var property)
|
||||
&& property.ValueKind == JsonValueKind.String
|
||||
&& DateTimeOffset.TryParse(property.GetString(), out var parsed)
|
||||
? parsed
|
||||
: null;
|
||||
|
||||
private static void AtomicWrite(string path, byte[] content, bool overwrite)
|
||||
{
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
|
||||
|
|
@ -166,77 +397,184 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
}
|
||||
}
|
||||
|
||||
private static bool IsValidRequest(JsonElement request) =>
|
||||
HasOnlyProperties(request, "schema_version", "input", "plot", "output")
|
||||
&& request.TryGetProperty("schema_version", out var schemaVersion)
|
||||
&& schemaVersion.TryGetInt32(out var version)
|
||||
&& version == 1
|
||||
&& request.TryGetProperty("input", out var input)
|
||||
&& input.ValueKind == JsonValueKind.Object
|
||||
&& HasOnlyProperties(input, "input_id", "sheet")
|
||||
&& input.TryGetProperty("input_id", out var inputId)
|
||||
&& inputId.ValueKind == JsonValueKind.String
|
||||
&& !string.IsNullOrWhiteSpace(inputId.GetString())
|
||||
&& request.TryGetProperty("plot", out var plot)
|
||||
&& plot.ValueKind == JsonValueKind.Object
|
||||
&& HasOnlyProperties(
|
||||
plot, "type", "x", "y", "template", "title", "x_axis", "y_axis", "legend", "error_bars")
|
||||
&& plot.TryGetProperty("type", out var plotType)
|
||||
&& plotType.ValueKind == JsonValueKind.String
|
||||
&& PlotTypes.Contains(plotType.GetString() ?? "")
|
||||
&& (!plot.TryGetProperty("title", out var title)
|
||||
|| title.ValueKind == JsonValueKind.String && title.GetString()!.Length <= 500)
|
||||
&& plot.TryGetProperty("x", out var x)
|
||||
&& IsColumnName(x)
|
||||
&& plot.TryGetProperty("y", out var y)
|
||||
&& IsValidYColumns(y)
|
||||
&& (!plot.TryGetProperty("template", out var template)
|
||||
|| template.GetString() == "publication_double_column")
|
||||
&& IsValidAxis(plot, "x_axis")
|
||||
&& IsValidAxis(plot, "y_axis")
|
||||
&& IsValidLegend(plot)
|
||||
&& !plot.TryGetProperty("error_bars", out _)
|
||||
&& request.TryGetProperty("output", out var output)
|
||||
&& output.ValueKind == JsonValueKind.Object
|
||||
&& HasOnlyProperties(output, "formats", "dpi", "capture_screenshots", "record_video")
|
||||
&& output.TryGetProperty("formats", out var formats)
|
||||
&& formats.ValueKind == JsonValueKind.Array
|
||||
&& formats.GetArrayLength() > 0
|
||||
&& IsValidFormats(formats)
|
||||
&& (!output.TryGetProperty("dpi", out var dpi)
|
||||
|| dpi.TryGetInt32(out var dpiValue) && dpiValue is >= 72 and <= 1200)
|
||||
&& IsOptionalBoolean(output, "capture_screenshots")
|
||||
&& IsOptionalBoolean(output, "record_video")
|
||||
&& (!output.TryGetProperty("record_video", out var recordVideo)
|
||||
|| recordVideo.ValueKind == JsonValueKind.False);
|
||||
private static bool IsValidRequest(JsonElement request)
|
||||
{
|
||||
if (!HasOnlyProperties(request, "schema_version", "inputs", "operation", "outputs")
|
||||
|| !request.TryGetProperty("schema_version", out var schemaVersion)
|
||||
|| !schemaVersion.TryGetInt32(out var version)
|
||||
|| version != 2
|
||||
|| !request.TryGetProperty("inputs", out var inputs)
|
||||
|| !IsValidInputBindings(inputs)
|
||||
|| !request.TryGetProperty("operation", out var operation)
|
||||
|| operation.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(operation, "plot")
|
||||
|| !operation.TryGetProperty("plot", out var plot)
|
||||
|| !IsValidPlot(plot, inputs)
|
||||
|| !request.TryGetProperty("outputs", out var outputs))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
return IsValidOutputs(outputs);
|
||||
}
|
||||
|
||||
private static bool IsValidOutputs(JsonElement outputs)
|
||||
{
|
||||
if (outputs.ValueKind != JsonValueKind.Array || outputs.GetArrayLength() is < 1 or > 16)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
var keys = new HashSet<string>(StringComparer.Ordinal);
|
||||
var identities = new HashSet<string>(StringComparer.Ordinal);
|
||||
foreach (var output in outputs.EnumerateArray())
|
||||
{
|
||||
if (output.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(output, "key", "type", "format", "options")
|
||||
|| !output.TryGetProperty("key", out var keyValue)
|
||||
|| keyValue.GetString() is not { } key
|
||||
|| !output.TryGetProperty("type", out var typeValue)
|
||||
|| typeValue.GetString() is not { } outputType
|
||||
|| !output.TryGetProperty("format", out var formatValue)
|
||||
|| formatValue.GetString() is not { } format)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
var expectedKey = (outputType, format) switch
|
||||
{
|
||||
("project", "opju") => "project",
|
||||
("figure", "png") => "figure_png",
|
||||
("figure", "svg") => "figure_svg",
|
||||
("figure", "pdf") => "figure_pdf",
|
||||
_ => "",
|
||||
};
|
||||
if (key != expectedKey || !keys.Add(key) || !identities.Add($"{outputType}\0{format}"))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if (format == "png")
|
||||
{
|
||||
if (output.TryGetProperty("options", out var options)
|
||||
&& (options.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(options, "dpi")
|
||||
|| !options.TryGetProperty("dpi", out var dpi)
|
||||
|| !dpi.TryGetInt32(out var dpiValue)
|
||||
|| dpiValue is < 72 or > 1200))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
else if (output.TryGetProperty("options", out _))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private static bool IsValidInputBindings(JsonElement inputs)
|
||||
{
|
||||
if (inputs.ValueKind != JsonValueKind.Array || inputs.GetArrayLength() is < 1 or > 16)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
var keys = new HashSet<string>(StringComparer.Ordinal);
|
||||
foreach (var input in inputs.EnumerateArray())
|
||||
{
|
||||
if (input.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(input, "key", "artifact_id", "selector")
|
||||
|| !input.TryGetProperty("key", out var keyValue)
|
||||
|| keyValue.GetString() is not { } key
|
||||
|| !IsInputKey(key)
|
||||
|| !keys.Add(key)
|
||||
|| !input.TryGetProperty("artifact_id", out var artifactId)
|
||||
|| !Guid.TryParse(artifactId.GetString(), out _)
|
||||
|| input.TryGetProperty("selector", out var selector)
|
||||
&& !IsValidSelector(selector))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private static bool IsValidPlot(JsonElement plot, JsonElement inputs)
|
||||
{
|
||||
if (plot.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(
|
||||
plot, "type", "series", "template", "title", "x_axis", "y_axis", "legend", "error_bars")
|
||||
|| !plot.TryGetProperty("type", out var plotType)
|
||||
|| !PlotTypes.Contains(plotType.GetString() ?? "")
|
||||
|| plot.TryGetProperty("title", out var title)
|
||||
&& (title.ValueKind != JsonValueKind.String || title.GetString()!.Length > 500)
|
||||
|| !plot.TryGetProperty("series", out var series)
|
||||
|| series.ValueKind != JsonValueKind.Array
|
||||
|| series.GetArrayLength() is < 1 or > 16
|
||||
|| plot.TryGetProperty("template", out var template)
|
||||
&& template.GetString() != "publication_double_column"
|
||||
|| !IsValidAxis(plot, "x_axis")
|
||||
|| !IsValidAxis(plot, "y_axis")
|
||||
|| !IsValidLegend(plot)
|
||||
|| plot.TryGetProperty("error_bars", out _))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
var inputKeys = inputs.EnumerateArray()
|
||||
.Select(item => item.GetProperty("key").GetString()!)
|
||||
.ToHashSet(StringComparer.Ordinal);
|
||||
var identities = new HashSet<string>(StringComparer.Ordinal);
|
||||
var usedInputs = new HashSet<string>(StringComparer.Ordinal);
|
||||
var labels = new Dictionary<string, string>(StringComparer.Ordinal);
|
||||
foreach (var item in series.EnumerateArray())
|
||||
{
|
||||
if (item.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(item, "input", "x", "y", "label")
|
||||
|| !item.TryGetProperty("input", out var input)
|
||||
|| input.GetString() is not { } inputKey
|
||||
|| !inputKeys.Contains(inputKey)
|
||||
|| !item.TryGetProperty("x", out var x)
|
||||
|| !IsColumnName(x)
|
||||
|| !item.TryGetProperty("y", out var y)
|
||||
|| !IsColumnName(y)
|
||||
|| item.TryGetProperty("label", out var label)
|
||||
&& (label.ValueKind != JsonValueKind.String
|
||||
|| label.GetString()!.Length is < 1 or > 200)
|
||||
|| !identities.Add($"{inputKey}\0{x.GetString()}\0{y.GetString()}"))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
usedInputs.Add(inputKey);
|
||||
var labelKey = $"{inputKey}\0{y.GetString()}";
|
||||
var effectiveLabel = item.TryGetProperty("label", out var seriesLabel)
|
||||
? seriesLabel.GetString()!
|
||||
: y.GetString()!;
|
||||
if (labels.TryGetValue(labelKey, out var existingLabel)
|
||||
&& existingLabel != effectiveLabel)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
labels[labelKey] = effectiveLabel;
|
||||
}
|
||||
return usedInputs.SetEquals(inputKeys);
|
||||
}
|
||||
|
||||
private static bool IsValidSelector(JsonElement selector) =>
|
||||
selector.ValueKind == JsonValueKind.Object
|
||||
&& HasOnlyProperties(selector, "sheet")
|
||||
&& selector.TryGetProperty("sheet", out var sheet)
|
||||
&& sheet.ValueKind == JsonValueKind.String
|
||||
&& sheet.GetString()!.Length is >= 1 and <= 128;
|
||||
|
||||
private static bool IsInputKey(string key) =>
|
||||
key.Length is >= 1 and <= 32
|
||||
&& key[0] is >= 'a' and <= 'z'
|
||||
&& key.All(character =>
|
||||
character is >= 'a' and <= 'z'
|
||||
|| character is >= '0' and <= '9'
|
||||
|| character == '_');
|
||||
|
||||
private static bool IsColumnName(JsonElement value) =>
|
||||
value.ValueKind == JsonValueKind.String
|
||||
&& value.GetString() is { Length: >= 1 and <= 128 };
|
||||
|
||||
private static bool IsValidFormats(JsonElement formats)
|
||||
{
|
||||
var values = formats.EnumerateArray().ToArray();
|
||||
return values.All(item =>
|
||||
item.ValueKind == JsonValueKind.String
|
||||
&& OutputFormats.Contains(item.GetString() ?? ""))
|
||||
&& values.Select(item => item.GetString()).Distinct(StringComparer.Ordinal).Count()
|
||||
== values.Length;
|
||||
}
|
||||
|
||||
private static bool IsValidYColumns(JsonElement value)
|
||||
{
|
||||
if (IsColumnName(value)) return true;
|
||||
if (value.ValueKind != JsonValueKind.Array
|
||||
|| value.GetArrayLength() is < 1 or > 16)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
var names = value.EnumerateArray().Select(item => item.GetString()).ToArray();
|
||||
return value.EnumerateArray().All(IsColumnName)
|
||||
&& names.Distinct(StringComparer.Ordinal).Count() == names.Length;
|
||||
}
|
||||
|
||||
private static bool IsValidAxis(JsonElement plot, string name)
|
||||
{
|
||||
if (!plot.TryGetProperty(name, out var axis)) return true;
|
||||
|
|
@ -258,36 +596,94 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
|| position.GetString() == "top_right");
|
||||
}
|
||||
|
||||
private static bool IsOptionalBoolean(JsonElement value, string name) =>
|
||||
!value.TryGetProperty(name, out var property)
|
||||
|| property.ValueKind is JsonValueKind.True or JsonValueKind.False;
|
||||
|
||||
private static bool IsValidInputTransfer(JsonElement transfer) =>
|
||||
transfer.ValueKind == JsonValueKind.Object
|
||||
&& HasOnlyProperties(transfer, "artifact_id", "filename", "size_bytes", "sha256", "download_path")
|
||||
&& transfer.TryGetProperty("artifact_id", out var artifactId)
|
||||
&& Guid.TryParse(artifactId.GetString(), out _)
|
||||
&& transfer.TryGetProperty("filename", out var filename)
|
||||
&& filename.ValueKind == JsonValueKind.String
|
||||
&& Path.GetFileName(filename.GetString()) == filename.GetString()
|
||||
&& transfer.TryGetProperty("size_bytes", out var size)
|
||||
&& size.TryGetInt64(out var sizeBytes)
|
||||
&& sizeBytes is >= 0 and <= 104_857_600
|
||||
&& transfer.TryGetProperty("sha256", out var sha)
|
||||
&& sha.GetString() is { Length: 64 }
|
||||
&& transfer.TryGetProperty("download_path", out var downloadPath)
|
||||
&& downloadPath.GetString()?.StartsWith("/v1/software-jobs/", StringComparison.Ordinal) == true
|
||||
&& downloadPath.GetString()?.EndsWith("/input", StringComparison.Ordinal) == true;
|
||||
|
||||
internal string InputPath(RecoverableJob job)
|
||||
private static bool IsValidInputTransfers(JsonElement transfers, Guid jobId)
|
||||
{
|
||||
if (job.InputTransfer is not JsonElement transfer
|
||||
|| !IsValidInputTransfer(transfer))
|
||||
if (transfers.ValueKind != JsonValueKind.Array
|
||||
|| transfers.GetArrayLength() is < 1 or > 16)
|
||||
{
|
||||
throw new InvalidDataException("Stored input transfer is invalid.");
|
||||
return false;
|
||||
}
|
||||
var keys = new HashSet<string>(StringComparer.Ordinal);
|
||||
long total = 0;
|
||||
foreach (var transfer in transfers.EnumerateArray())
|
||||
{
|
||||
if (transfer.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(
|
||||
transfer, "key", "artifact_id", "filename", "size_bytes", "sha256",
|
||||
"selector", "download_path")
|
||||
|| !transfer.TryGetProperty("key", out var keyValue)
|
||||
|| keyValue.GetString() is not { } key
|
||||
|| !IsInputKey(key)
|
||||
|| !keys.Add(key)
|
||||
|| !transfer.TryGetProperty("artifact_id", out var artifactId)
|
||||
|| !Guid.TryParse(artifactId.GetString(), out _)
|
||||
|| !transfer.TryGetProperty("filename", out var filename)
|
||||
|| filename.ValueKind != JsonValueKind.String
|
||||
|| Path.GetFileName(filename.GetString()) != filename.GetString()
|
||||
|| !transfer.TryGetProperty("size_bytes", out var size)
|
||||
|| !size.TryGetInt64(out var sizeBytes)
|
||||
|| sizeBytes is < 0 or > 104_857_600
|
||||
|| !transfer.TryGetProperty("sha256", out var sha)
|
||||
|| sha.GetString() is not { Length: 64 }
|
||||
|| transfer.TryGetProperty("selector", out var selector)
|
||||
&& !IsValidSelector(selector)
|
||||
|| !transfer.TryGetProperty("download_path", out var downloadPath)
|
||||
|| downloadPath.GetString()
|
||||
!= $"/v1/software-jobs/{jobId:D}/inputs/{key}")
|
||||
{
|
||||
return false;
|
||||
}
|
||||
total += sizeBytes;
|
||||
}
|
||||
return total <= 536_870_912;
|
||||
}
|
||||
|
||||
private static bool InputsMatchTransfers(JsonElement request, JsonElement transfers)
|
||||
{
|
||||
var bindings = request.GetProperty("inputs").EnumerateArray().ToDictionary(
|
||||
item => item.GetProperty("key").GetString()!, StringComparer.Ordinal);
|
||||
if (bindings.Count != transfers.GetArrayLength())
|
||||
{
|
||||
return false;
|
||||
}
|
||||
foreach (var transfer in transfers.EnumerateArray())
|
||||
{
|
||||
var key = transfer.GetProperty("key").GetString()!;
|
||||
if (!bindings.TryGetValue(key, out var binding)
|
||||
|| binding.GetProperty("artifact_id").GetString()
|
||||
!= transfer.GetProperty("artifact_id").GetString())
|
||||
{
|
||||
return false;
|
||||
}
|
||||
var hasBindingSelector = binding.TryGetProperty("selector", out var bindingSelector);
|
||||
var hasTransferSelector = transfer.TryGetProperty("selector", out var transferSelector);
|
||||
if (hasBindingSelector != hasTransferSelector
|
||||
|| hasBindingSelector
|
||||
&& bindingSelector.GetProperty("sheet").GetString()
|
||||
!= transferSelector.GetProperty("sheet").GetString())
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
internal IReadOnlyList<JsonElement> InputTransfers(RecoverableJob job)
|
||||
{
|
||||
if (job.InputTransfers is not JsonElement transfers
|
||||
|| !IsValidInputTransfers(transfers, job.JobId))
|
||||
{
|
||||
throw new InvalidDataException("Stored input transfers are invalid.");
|
||||
}
|
||||
return transfers.EnumerateArray().Select(item => item.Clone()).ToArray();
|
||||
}
|
||||
|
||||
internal string InputPath(RecoverableJob job, JsonElement transfer)
|
||||
{
|
||||
var key = transfer.GetProperty("key").GetString()!;
|
||||
var filename = transfer.GetProperty("filename").GetString()!;
|
||||
return Path.Combine(jobsDirectory, job.JobId.ToString("D"), "input", filename);
|
||||
return Path.Combine(
|
||||
jobsDirectory, job.JobId.ToString("D"), "input", key, filename);
|
||||
}
|
||||
|
||||
internal void WriteTerminal(
|
||||
|
|
@ -330,7 +726,7 @@ internal sealed record RecoverableJob(
|
|||
Guid JobId,
|
||||
Guid LeaseId,
|
||||
string RequestDigest,
|
||||
JsonElement? InputTransfer,
|
||||
JsonElement? InputTransfers,
|
||||
JsonElement? Terminal,
|
||||
bool UploadComplete);
|
||||
|
||||
|
|
|
|||
|
|
@ -8,20 +8,32 @@ internal sealed class JobInputDownloader(NodeConfig config, JobInboxStore inbox)
|
|||
{
|
||||
internal async Task DownloadAsync(RecoverableJob job, CancellationToken cancellationToken)
|
||||
{
|
||||
if (job.InputTransfer is not JsonElement transfer)
|
||||
var transfers = inbox.InputTransfers(job);
|
||||
for (var index = 0; index < transfers.Count; index++)
|
||||
{
|
||||
throw new InvalidDataException("Job input transfer is missing.");
|
||||
inbox.WriteState(
|
||||
job,
|
||||
"downloading_inputs",
|
||||
Math.Max(0, index * 5 / transfers.Count),
|
||||
$"正在下载输入 {index + 1}/{transfers.Count}");
|
||||
await DownloadOneAsync(job, transfers[index], cancellationToken);
|
||||
}
|
||||
}
|
||||
|
||||
private async Task DownloadOneAsync(
|
||||
RecoverableJob job,
|
||||
JsonElement transfer,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var downloadPath = transfer.GetProperty("download_path").GetString()!;
|
||||
if (!downloadPath.StartsWith("/v1/software-jobs/", StringComparison.Ordinal)
|
||||
|| !downloadPath.EndsWith("/input", StringComparison.Ordinal)
|
||||
|| !Uri.TryCreate(downloadPath, UriKind.Relative, out var relativeUri))
|
||||
{
|
||||
throw new InvalidDataException("Job input download path is invalid.");
|
||||
}
|
||||
var expectedSize = transfer.GetProperty("size_bytes").GetInt64();
|
||||
var expectedSha256 = transfer.GetProperty("sha256").GetString()!;
|
||||
var destination = inbox.InputPath(job);
|
||||
var destination = inbox.InputPath(job, transfer);
|
||||
if (File.Exists(destination))
|
||||
{
|
||||
await VerifyExistingAsync(destination, expectedSize, expectedSha256, cancellationToken);
|
||||
|
|
|
|||
|
|
@ -0,0 +1,22 @@
|
|||
namespace Zcbot.WindowsNode;
|
||||
|
||||
internal sealed record LocalJobState(
|
||||
string Stage,
|
||||
int Progress,
|
||||
string Detail,
|
||||
DateTimeOffset UpdatedAt);
|
||||
|
||||
internal sealed record JobDisplaySnapshot(
|
||||
Guid JobId,
|
||||
string Capability,
|
||||
string Title,
|
||||
string InputFilename,
|
||||
string Stage,
|
||||
int Progress,
|
||||
string Detail,
|
||||
DateTimeOffset AcceptedAt,
|
||||
DateTimeOffset UpdatedAt,
|
||||
bool UploadComplete)
|
||||
{
|
||||
internal bool IsActive => Stage is not ("succeeded" or "failed" or "cancelled");
|
||||
}
|
||||
|
|
@ -57,7 +57,7 @@ internal sealed class NodeConfigStore(NodePaths paths)
|
|||
|| string.IsNullOrWhiteSpace(stored.NodeName)
|
||||
|| string.IsNullOrWhiteSpace(token)
|
||||
|| stored.Capabilities.Count == 0
|
||||
|| stored.Capabilities.Any(item => item != "origin.plot@v1"))
|
||||
|| stored.Capabilities.Any(item => item != "origin.plot@v2"))
|
||||
{
|
||||
throw new NodeConfigurationException("Node configuration contains an invalid identity or capability.");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -163,7 +163,7 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
job_id = job.JobId,
|
||||
lease_id = job.LeaseId,
|
||||
request_digest = job.RequestDigest,
|
||||
stage = "waiting_input",
|
||||
stage = "downloading_inputs",
|
||||
progress = 0,
|
||||
metrics = new { },
|
||||
}, cancellationToken);
|
||||
|
|
@ -242,7 +242,7 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
job_id = offerResult.JobId,
|
||||
lease_id = offerResult.LeaseId,
|
||||
request_digest = offerResult.RequestDigest,
|
||||
stage = "waiting_input",
|
||||
stage = "downloading_inputs",
|
||||
progress = 0,
|
||||
metrics = new { },
|
||||
}, cancellationToken);
|
||||
|
|
@ -322,6 +322,7 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
{
|
||||
if (job.Terminal is null)
|
||||
{
|
||||
jobInbox.WriteState(job, "downloading_inputs", 0, "正在下载并校验输入文件");
|
||||
await inputDownloader.DownloadAsync(job, CancellationToken.None);
|
||||
var afterDownload = jobInbox.ReadRecoverableJobs()
|
||||
.Single(item => item.JobId == job.JobId);
|
||||
|
|
@ -330,6 +331,7 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
await TrySendAsync(socket, "job_terminal", cancelledTerminal);
|
||||
return;
|
||||
}
|
||||
jobInbox.WriteState(job, "ready_to_run", 5, "输入文件已就绪，准备启动软件");
|
||||
await TrySendAsync(socket, "job_state", new
|
||||
{
|
||||
job_id = job.JobId,
|
||||
|
|
@ -337,14 +339,20 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
request_digest = job.RequestDigest,
|
||||
stage = "ready_to_run",
|
||||
progress = 5,
|
||||
metrics = new { input_bytes = job.InputTransfer?.GetProperty("size_bytes").GetInt64() },
|
||||
metrics = new
|
||||
{
|
||||
input_count = jobInbox.InputTransfers(job).Count,
|
||||
input_bytes = jobInbox.InputTransfers(job)
|
||||
.Sum(item => item.GetProperty("size_bytes").GetInt64()),
|
||||
},
|
||||
});
|
||||
jobInbox.WriteState(job, "software_running", 10, "Origin 正在生成图形");
|
||||
await TrySendAsync(socket, "job_state", new
|
||||
{
|
||||
job_id = job.JobId,
|
||||
lease_id = job.LeaseId,
|
||||
request_digest = job.RequestDigest,
|
||||
stage = "origin_running",
|
||||
stage = "software_running",
|
||||
progress = 10,
|
||||
metrics = new { },
|
||||
});
|
||||
|
|
@ -358,9 +366,17 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
}
|
||||
if (terminal.GetProperty("status").GetString() != "succeeded")
|
||||
{
|
||||
var terminalStatus = terminal.GetProperty("status").GetString() ?? "failed";
|
||||
var terminalDetail = terminal.TryGetProperty("error", out var error)
|
||||
&& error.TryGetProperty("detail", out var errorDetail)
|
||||
? errorDetail.GetString() ?? "任务执行失败"
|
||||
: "任务执行失败";
|
||||
jobInbox.WriteState(
|
||||
refreshed, terminalStatus, terminalStatus == "cancelled" ? 0 : 10, terminalDetail);
|
||||
await TrySendAsync(socket, "job_terminal", terminal);
|
||||
return;
|
||||
}
|
||||
jobInbox.WriteState(refreshed, "uploading_outputs", 90, "软件执行完成，正在上传结果");
|
||||
await TrySendAsync(socket, "job_state", new
|
||||
{
|
||||
job_id = job.JobId,
|
||||
|
|
@ -371,6 +387,7 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
metrics = new { },
|
||||
});
|
||||
await outputUploader.UploadAsync(refreshed);
|
||||
jobInbox.WriteState(refreshed, "succeeded", 100, "结果已上传并由云端确认");
|
||||
}
|
||||
catch (Exception exception) when (
|
||||
exception is HttpRequestException
|
||||
|
|
@ -384,6 +401,11 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
if (current.Terminal is JsonElement terminal
|
||||
&& terminal.GetProperty("status").GetString() == "succeeded")
|
||||
{
|
||||
jobInbox.WriteState(
|
||||
current,
|
||||
"uploading_outputs",
|
||||
90,
|
||||
$"结果上传暂缓：{exception.Message}");
|
||||
Console.Error.WriteLine($"[WARN] Output upload deferred: {exception.Message}");
|
||||
return;
|
||||
}
|
||||
|
|
@ -392,6 +414,7 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
|
|||
"failed",
|
||||
"INPUT_DOWNLOAD_FAILED",
|
||||
exception.Message[..Math.Min(exception.Message.Length, 500)]);
|
||||
jobInbox.WriteState(job, "failed", 0, exception.Message);
|
||||
var failedTerminal = jobInbox.ReadRecoverableJobs()
|
||||
.Single(item => item.JobId == job.JobId).Terminal;
|
||||
if (failedTerminal is JsonElement payload)
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ internal static class OriginRuntimeProbe
|
|||
}
|
||||
|
||||
private static OriginRuntimeStatus Status(string? version, string health, string detail) =>
|
||||
new("OriginPro", version, "0.2.0", health, detail);
|
||||
new("OriginPro", version, "0.3.0", health, detail);
|
||||
|
||||
private static string? FindInstalledVersion()
|
||||
{
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
"""Fixed Origin adapter for origin.plot@v1.
|
||||
"""Fixed Origin adapter for origin.plot@v2.
|
||||
|
||||
This process accepts exactly one argument: a Node-created job directory. It never
|
||||
installs packages, evaluates user code, downloads data, or resolves paths from the
|
||||
|
|
@ -104,6 +104,35 @@ def _axis_title(axis: Any, fallback: str) -> str:
|
|||
return f"{title} ({unit})" if unit else title
|
||||
|
||||
|
||||
def _input_file(job_dir: Path, key: str) -> Path:
|
||||
directory = job_dir / "input" / key
|
||||
files = [path for path in directory.iterdir() if path.is_file() and not path.name.startswith(".")]
|
||||
if len(files) != 1:
|
||||
raise ValueError(f"INPUT_FILE_COUNT_INVALID:{key}")
|
||||
return files[0]
|
||||
|
||||
|
||||
def _resolve_series(
|
||||
input_data: dict[str, tuple[list[str], list[list[Any]]]],
|
||||
series_specs: list[dict[str, Any]],
|
||||
) -> tuple[list[tuple[str, int, int, str | None]], dict[tuple[str, int], str]]:
|
||||
resolved: list[tuple[str, int, int, str | None]] = []
|
||||
labels: dict[tuple[str, int], str] = {}
|
||||
for series in series_specs:
|
||||
input_key = series["input"]
|
||||
headers, _ = input_data[input_key]
|
||||
x_index = _column_index(headers, series["x"], "x")
|
||||
y_index = _column_index(headers, series["y"], "y")
|
||||
label = series.get("label")
|
||||
label_key = (input_key, y_index)
|
||||
effective_label = label or series["y"]
|
||||
if label_key in labels and labels[label_key] != effective_label:
|
||||
raise ValueError("SERIES_LABEL_CONFLICT")
|
||||
labels[label_key] = effective_label
|
||||
resolved.append((input_key, x_index, y_index, label))
|
||||
return resolved, labels
|
||||
|
||||
|
||||
def _validate_artifact(path: Path, extension: str) -> None:
|
||||
if not path.is_file() or path.stat().st_size == 0:
|
||||
raise RuntimeError(f"{extension.upper()}_EXPORT_EMPTY")
|
||||
|
|
@ -122,21 +151,21 @@ def run(job_dir: Path) -> list[dict[str, Any]]:
|
|||
job_dir = job_dir.resolve(strict=True)
|
||||
request_record = json.loads((job_dir / "request" / "request.json").read_text(encoding="utf-8"))
|
||||
request = request_record["request"]
|
||||
input_files = [path for path in (job_dir / "input").iterdir() if path.is_file() and not path.name.startswith(".")]
|
||||
if len(input_files) != 1:
|
||||
raise ValueError("INPUT_FILE_COUNT_INVALID")
|
||||
headers, rows = _read_rows(input_files[0], request["input"].get("sheet"))
|
||||
plot_spec = request["plot"]
|
||||
input_specs = request["inputs"]
|
||||
input_files = {item["key"]: _input_file(job_dir, item["key"]) for item in input_specs}
|
||||
input_data = {
|
||||
item["key"]: _read_rows(
|
||||
input_files[item["key"]],
|
||||
(item.get("selector") or {}).get("sheet"),
|
||||
)
|
||||
for item in input_specs
|
||||
}
|
||||
plot_spec = request["operation"]["plot"]
|
||||
plot_type = plot_spec["type"]
|
||||
if plot_type not in PLOT_TYPES:
|
||||
raise ValueError("PLOT_TYPE_NOT_IMPLEMENTED")
|
||||
x_index = _column_index(headers, plot_spec.get("x"), "x")
|
||||
y_names = plot_spec.get("y")
|
||||
if isinstance(y_names, str):
|
||||
y_names = [y_names]
|
||||
if not isinstance(y_names, list) or not y_names:
|
||||
raise ValueError("Y_COLUMNS_REQUIRED")
|
||||
y_indexes = [_column_index(headers, name, "y") for name in y_names]
|
||||
series_specs = plot_spec["series"]
|
||||
resolved_series, labels = _resolve_series(input_data, series_specs)
|
||||
|
||||
import originpro as op
|
||||
|
||||
|
|
@ -145,22 +174,37 @@ def run(job_dir: Path) -> list[dict[str, Any]]:
|
|||
op.set_show(False)
|
||||
try:
|
||||
op.new()
|
||||
worksheet = op.new_sheet("w", lname="Data")
|
||||
for index, header in enumerate(headers):
|
||||
worksheet.from_list(index, [row[index] if index < len(row) else None for row in rows], lname=header)
|
||||
worksheets: dict[str, Any] = {}
|
||||
for input_spec in input_specs:
|
||||
input_key = input_spec["key"]
|
||||
headers, rows = input_data[input_key]
|
||||
worksheet = op.new_sheet("w", lname=input_key)
|
||||
worksheets[input_key] = worksheet
|
||||
for index, header in enumerate(headers):
|
||||
column_label = labels.get((input_key, index), header)
|
||||
worksheet.from_list(
|
||||
index,
|
||||
[row[index] if index < len(row) else None for row in rows],
|
||||
lname=column_label,
|
||||
)
|
||||
graph = op.new_graph(template={"line": "line", "scatter": "scatter", "line_scatter": "linesymb"}[plot_type])
|
||||
layer = graph[0]
|
||||
for y_index in y_indexes:
|
||||
layer.add_plot(worksheet, coly=y_index, colx=x_index, type=PLOT_TYPES[plot_type])
|
||||
for input_key, x_index, y_index, _ in resolved_series:
|
||||
layer.add_plot(
|
||||
worksheets[input_key], coly=y_index, colx=x_index, type=PLOT_TYPES[plot_type]
|
||||
)
|
||||
layer.rescale()
|
||||
layer.axis("x").title = _axis_title(plot_spec.get("x_axis"), str(plot_spec.get("x") or "X"))
|
||||
layer.axis("x").title = _axis_title(
|
||||
plot_spec.get("x_axis"), str(series_specs[0].get("x") or "X")
|
||||
)
|
||||
layer.axis("y").title = _axis_title(plot_spec.get("y_axis"), "Y")
|
||||
if plot_spec.get("title"):
|
||||
title = layer.add_label(str(plot_spec["title"]))
|
||||
title.set_int("fsize", 18)
|
||||
title.set_int("left", 2200)
|
||||
title.set_int("top", 120)
|
||||
formats = request["output"]["formats"]
|
||||
requested_outputs = request["outputs"]
|
||||
formats = [item["format"] for item in requested_outputs]
|
||||
if any(item not in FORMATS for item in formats):
|
||||
raise ValueError("OUTPUT_FORMAT_UNSUPPORTED")
|
||||
artifacts: list[dict[str, Any]] = []
|
||||
|
|
@ -170,7 +214,8 @@ def run(job_dir: Path) -> list[dict[str, Any]]:
|
|||
_validate_artifact(project, "opju")
|
||||
artifacts.append(_manifest(project, "application/x-origin-project"))
|
||||
media = {"png": "image/png", "svg": "image/svg+xml", "pdf": "application/pdf"}
|
||||
dpi = request["output"].get("dpi", 300)
|
||||
png_output = next((item for item in requested_outputs if item["format"] == "png"), None)
|
||||
dpi = (png_output.get("options") or {}).get("dpi", 300) if png_output else 300
|
||||
if not isinstance(dpi, int) or isinstance(dpi, bool) or not 72 <= dpi <= 1200:
|
||||
raise ValueError("OUTPUT_DPI_INVALID")
|
||||
pixel_width = round(dpi * 160 / 25.4)
|
||||
|
|
@ -192,12 +237,20 @@ def run(job_dir: Path) -> list[dict[str, Any]]:
|
|||
except PackageNotFoundError:
|
||||
originpro_version = "embedded"
|
||||
provenance = {
|
||||
"adapter_version": "0.2.0",
|
||||
"adapter_version": "0.3.0",
|
||||
"originpro_version": originpro_version,
|
||||
"request_digest": request_record["request_digest"],
|
||||
"input_sha256": _file_sha256(input_files[0]),
|
||||
"requested_dpi": dpi,
|
||||
"png_pixel_width": pixel_width,
|
||||
"inputs": [
|
||||
{
|
||||
"key": item["key"],
|
||||
"filename": input_files[item["key"]].name,
|
||||
"sha256": _file_sha256(input_files[item["key"]]),
|
||||
}
|
||||
for item in input_specs
|
||||
],
|
||||
"outputs": requested_outputs,
|
||||
"requested_dpi": dpi if png_output else None,
|
||||
"png_pixel_width": pixel_width if png_output else None,
|
||||
}
|
||||
plot_spec_path = output / "plot-spec.json"
|
||||
provenance_path = output / "provenance.json"
|
||||
|
|
|
|||
Loading…
Reference in New Issue