feat(origin): expand supported plot types
This commit is contained in:
parent
ef5428c3b7
commit
b77db25440
|
|
@ -15,6 +15,7 @@
|
|||
- Windows Node 窗口新增本机任务列表，可查看任务标题、输入文件、执行阶段、进度、时间、Job ID 和失败详情；任务状态会在本机持久保存，断线或重启后仍可查看。
|
||||
- Windows Node 会自动重试被 Origin 或系统程序短暂占用的输出文件；云端已确认的结果可在重启后直接恢复本机完成状态，并为每个任务保留不含凭据的上传诊断日志。
|
||||
- Origin 绘图任务支持同时绑定多个 CSV、XLSX 或 JSON 输入、从不同输入中选择数据系列，并显式选择需要生成的工程文件和多种图形格式。
|
||||
- Origin 绘图新增柱形图、条形图、分组柱形图、Y 误差棒、等高线、三维曲面、三元图和规则网格热图；这些图形继续使用同一个任务入口，可统一生成工程文件和多种图片格式。
|
||||
|
||||
## 0.65.2 — 2026-08-13
|
||||
|
||||
|
|
|
|||
|
|
@ -468,7 +468,7 @@ Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/software-nodes/c
|
|||
|
||||
第三阶段补齐多输入下载与恢复状态协议：请求使用通用 `inputs[]` 绑定 1–16 个 artifact，并由 `operation.plot.series[]` 以输入 key 引用各自的 X/Y 列；单文件不超过 100 MiB、总量不超过 512 MiB。已有 artifact 可直接提交；普通 task 文件先逐个调用 `register_artifact(path)` 登记稳定身份。Node 以自身 Bearer 身份访问每个任务绑定的只读下载端点，流式写入本 job 的 `input/<key>/<filename>`，同时校验大小与 SHA-256；不暴露工作区路径。Node 会原子读取/补报 `terminal.json`，断线后云端把活动任务标记 `disconnected` 并保留 Node/lease，重连按 job、lease、digest 恢复下载或幂等补报终态，不自动重派。Node UI 的“本机任务”只读取已经 accept 到本机的任务目录，不查询云端未派发 Job；每个任务以原子 `state.json` 持久化 `accepted/downloading_inputs/ready_to_run/software_running/uploading_outputs/succeeded/failed/cancelled` 通用阶段，窗口再与 request、terminal、upload-complete 合并成可恢复视图。
|
||||
|
||||
第四阶段落地固定 Origin Worker：Node 仅从管理员安装的固定 Python 运行时启动随程序发布的 `worker.py`，参数只有本机 job 目录；请求不能指定脚本、解释器或文件路径。Worker 使用 `originpro` 生成 OPJU、PNG、SVG、PDF、plot spec 和 provenance，校验产物签名并原子写入终态；当前受控图形仅含 line、scatter、line_scatter 和双栏出版布局。进程内 pipeline 按 job 去重，并脱离单次 WebSocket 的取消令牌运行；连接中断只延迟状态/终态上报。Node 进程若在 Worker 启动后重启，则保守失败而不重复驱动 Origin，避免无法证明的双执行。
|
||||
第四阶段落地固定 Origin Worker：Node 仅从管理员安装的固定 Python 运行时启动随程序发布的 `worker.py`，参数只有本机 job 目录；请求不能指定脚本、解释器或文件路径。Worker 使用 `originpro` 生成 OPJU、PNG、SVG、PDF、plot spec 和 provenance，校验产物签名并原子写入终态。`origin.plot@v2` 保持单一外层契约，`series[]` 以 `x/y/z/y_error` 统一表达数据角色，再按 `plot.type` 判别必需角色；当前覆盖折线、散点、线点、柱/条形、分组柱形、Y 误差棒、等高线、三维曲面、三元图和规则网格热图，旧 XY 请求原样兼容。进程内 pipeline 按 job 去重，并脱离单次 WebSocket 的取消令牌运行；连接中断只延迟状态/终态上报。Node 进程若在 Worker 启动后重启，则保守失败而不重复驱动 Origin，避免无法证明的双执行。
|
||||
|
||||
第五阶段完成输出上传与发布：Node 只按固定 manifest ID 逐项流式 PUT，并携带 Node、lease、request digest 与内容摘要；云端重新绑定任务身份，不信 Node 提供的路径或媒体类型。文件先进入用户根下隐藏暂存区，固定文件名、单文件/总大小和 SHA-256 全部验证后，把 plot spec、provenance 整理进 `.meta/`，再将完整目录原子移动到 `<working_dir>/origin/<job_id>/`。PNG/SVG/PDF/OPJU 等正式输出登记平台 artifact UUID 和 `software_job_id`，`.meta/` 只落真实文件；成功状态返回 task-relative `output_dir`，Agent 以该目录为起点按需搜索。重复 PUT、complete 和重连均按摘要幂等；部分上传不可见，只有完整集合才能发布。Origin 执行槽与上传确认是两个正交状态：本地已有终态且固定 Worker 已退出时即释放软件执行槽，成功但尚无 `upload-complete.json` 的任务继续后台补传；若云端已经是 succeeded，重复 PUT/complete 必须按数据库持久化 manifest 校验并直接确认，不得按新版本目录规则重新发布旧 Job。
|
||||
|
||||
|
|
|
|||
|
|
@ -22,6 +22,8 @@
|
|||
|
||||
### 2026-08-14
|
||||
|
||||
- **08-14 / Unreleased / Origin 统一图型契约扩展**:`origin.plot@v2` 保持单一入口，`series[]` 增加 `z/y_error` 数据角色并由 `plot.type` 判别校验，在兼容既有 XY 请求的同时新增 column、bar、grouped_column、y_error、contour、surface_3d、ternary、heatmap；规则热图在 Worker 内拒绝缺格、重复坐标、非等间距和非有限数值。云端、Agent 工具、Node 二次校验和固定 Worker 已同步，专项 61 项 unittest、Python 编译与 .NET build 通过，未连接或写入生产 DB。
|
||||
|
||||
- **08-14 / Unreleased / Windows Node 输出上传收敛与诊断**:恢复中的成功任务优先向云端重放完成确认，已发布结果不再重新打开可能被 Origin 占用的 OPJU；文件共享冲突按 0.5/1/2/5 秒有界退避，本地完成标记使用可跨心跳复用的固定 pending 文件，只重试重命名以避开系统程序对每个新文件的重复扫描，并按 Job 写入带阶段、产物、重试次数和 HRESULT 的 1 MiB 轮转诊断日志。同步修正本机更新时间早于接收时间的展示边界；相关 45 项专项 unittest、.NET build 与 diff 检查通过，未写入生产 DB。
|
||||
|
||||
- **08-14 / Unreleased / Windows Node 本机任务监控 + Origin 多输入多输出 v2**:配置窗口新增只读本机任务列表与详情，每秒从 `%ProgramData%/Zcbot/WindowsNode/jobs` 合并 request/state/terminal/upload-complete，展示标题、全部输入、阶段、执行耗时、时间、Job ID 与错误，重启后可恢复且不查询云端未派发 Job。按用户明确授权直接切换到 `origin.plot@v2`：通用 `inputs[] + operation + outputs[]` 支持 1–16 个输入、跨输入系列和多个显式产物，Node 下载到 keyed 目录，Origin Worker 建立多工作表后合并绘图；本地阶段统一为 `downloading_inputs/software_running`，不兼容 v1 请求或旧本地任务。相关专项 71 项 unittest、Python/JavaScript 语法、Ruff 致命规则、.NET build 与 diff 检查通过；完整测试 593 项中非数据库测试通过，3 个数据库集成模块因显式测试库未迁移、缺少 `users` 表而未通过，未连接或写入生产 DB。
|
||||
|
|
|
|||
|
|
@ -17,8 +17,12 @@ from core.storage.models import Artifact, SoftwareJob, SoftwareNode, Task
|
|||
|
||||
OFFER_SECONDS = 60
|
||||
ALLOWED_PLOT_TYPES = frozenset(
|
||||
{"line", "scatter", "line_scatter"}
|
||||
{
|
||||
"line", "scatter", "line_scatter", "column", "bar", "grouped_column",
|
||||
"y_error", "contour", "surface_3d", "ternary", "heatmap",
|
||||
}
|
||||
)
|
||||
XYZ_PLOT_TYPES = frozenset({"contour", "surface_3d", "ternary", "heatmap"})
|
||||
ALLOWED_INPUT_SUFFIXES = frozenset({".csv", ".xlsx", ".json"})
|
||||
MAX_INPUT_BYTES = 100 * 1024 * 1024
|
||||
MAX_INPUTS = 16
|
||||
|
|
@ -102,7 +106,10 @@ def _canonical_origin_plot_request(request: dict) -> tuple[dict, str]:
|
|||
raise SoftwareJobError("input keys must be unique")
|
||||
if not _has_only(
|
||||
plot,
|
||||
{"type", "series", "template", "title", "x_axis", "y_axis", "legend", "error_bars"},
|
||||
{
|
||||
"type", "series", "template", "title", "x_axis", "y_axis", "z_axis",
|
||||
"legend", "error_bars",
|
||||
},
|
||||
):
|
||||
raise SoftwareJobError("unsupported origin plot fields")
|
||||
if plot.get("type") not in ALLOWED_PLOT_TYPES:
|
||||
|
|
@ -116,40 +123,54 @@ def _canonical_origin_plot_request(request: dict) -> tuple[dict, str]:
|
|||
series = plot.get("series")
|
||||
if not isinstance(series, list) or not 1 <= len(series) <= 16:
|
||||
raise SoftwareJobError("plot.series must contain 1 to 16 series")
|
||||
identities: list[tuple[str, str, str]] = []
|
||||
plot_type = plot["type"]
|
||||
if plot_type == "grouped_column" and len(series) < 2:
|
||||
raise SoftwareJobError("grouped_column requires at least two series")
|
||||
if plot_type in XYZ_PLOT_TYPES and len(series) != 1:
|
||||
raise SoftwareJobError(f"{plot_type} requires exactly one XYZ series")
|
||||
required_roles = (
|
||||
{"x", "y", "z"} if plot_type in XYZ_PLOT_TYPES
|
||||
else {"x", "y", "y_error"} if plot_type == "y_error"
|
||||
else {"x", "y"}
|
||||
)
|
||||
identities: list[tuple[str, ...]] = []
|
||||
used_input_keys: set[str] = set()
|
||||
series_labels: dict[tuple[str, str], str] = {}
|
||||
for item in series:
|
||||
if not isinstance(item, dict) or not _has_only(item, {"input", "x", "y", "label"}):
|
||||
if not isinstance(item, dict) or not _has_only(
|
||||
item, {"input", "x", "y", "z", "y_error", "label"}
|
||||
):
|
||||
raise SoftwareJobError("invalid plot series fields")
|
||||
if not {"input", "x", "y"}.issubset(item):
|
||||
raise SoftwareJobError("plot series requires input, x, and y")
|
||||
if not {"input", *required_roles}.issubset(item):
|
||||
roles = ", ".join(sorted(required_roles))
|
||||
raise SoftwareJobError(f"{plot_type} series requires input, {roles}")
|
||||
if {"x", "y", "z", "y_error"}.intersection(item) - required_roles:
|
||||
raise SoftwareJobError(f"{plot_type} series contains unsupported data roles")
|
||||
input_key = item.get("input")
|
||||
x_column = item.get("x")
|
||||
y_column = item.get("y")
|
||||
if input_key not in input_keys:
|
||||
raise SoftwareJobError("plot series references an unknown input")
|
||||
if any(
|
||||
not isinstance(value, str) or not 1 <= len(value) <= 128
|
||||
for value in (x_column, y_column)
|
||||
for value in (item.get(role) for role in required_roles)
|
||||
):
|
||||
raise SoftwareJobError("plot series x and y must be column names")
|
||||
raise SoftwareJobError("plot series data roles must be column names")
|
||||
if "label" in item and (
|
||||
not isinstance(item["label"], str) or not 1 <= len(item["label"]) <= 200
|
||||
):
|
||||
raise SoftwareJobError("plot series label must be a string")
|
||||
y_column = item["y"]
|
||||
label_key = (input_key, y_column)
|
||||
effective_label = item.get("label", y_column)
|
||||
if label_key in series_labels and series_labels[label_key] != effective_label:
|
||||
raise SoftwareJobError("series sharing an input Y column must use one label")
|
||||
series_labels[label_key] = effective_label
|
||||
used_input_keys.add(input_key)
|
||||
identities.append((input_key, x_column, y_column))
|
||||
identities.append((input_key, *(item[role] for role in sorted(required_roles))))
|
||||
if len(identities) != len(set(identities)):
|
||||
raise SoftwareJobError("plot series must be unique")
|
||||
if used_input_keys != set(input_keys):
|
||||
raise SoftwareJobError("every input must be referenced by a plot series")
|
||||
for axis_name in ("x_axis", "y_axis"):
|
||||
for axis_name in ("x_axis", "y_axis", "z_axis"):
|
||||
axis = plot.get(axis_name)
|
||||
if axis is not None and (
|
||||
not isinstance(axis, dict)
|
||||
|
|
|
|||
|
|
@ -468,7 +468,7 @@ C# COM 只作未覆盖能力或旧版本备用，并与实际 Origin 版本匹
|
|||
}
|
||||
```
|
||||
|
||||
当前图形类型：line、scatter、line_scatter。后续增加 grouped_bar、box、histogram、heatmap、误差棒组合、三元图、等高线、三维曲面、XRD 堆叠图、热分析联图和多面板布局。
|
||||
当前仍只开放一个 `origin.plot@v2` 契约。`series[]` 统一使用数据角色字段：常规二维图为 `x/y`，误差棒为 `x/y/y_error`，XYZ 图为 `x/y/z`；服务端和 Node 均按 `type` 做判别校验。图形类型包括 line、scatter、line_scatter、column、bar、grouped_column、y_error、contour、surface_3d、ternary 和 heatmap，其中 heatmap 要求完整、等间距且无重复坐标的规则 XYZ 网格。后续可在同一角色模型内增加 box、histogram、XRD 堆叠图、热分析联图和多面板布局，不拆 capability；只有新增不同执行语义（如拟合、统计分析）时才增加 operation。
|
||||
|
||||
上述 JSON 同时是提交工具与云端账本、Windows Node 之间的 artifact 输入契约。
|
||||
Agent 优先复用已有 artifact UUID；普通 task 文件先逐个调用
|
||||
|
|
@ -589,7 +589,7 @@ Windows 基线：固定版本并受控更新；禁止休眠和自动锁屏；固
|
|||
- enrollment、节点表、WSS Connection Manager；
|
||||
- Node 注册、证书、心跳、能力上报；
|
||||
- job 表、幂等提交、offer/accept 和基本租约；
|
||||
- Origin line/scatter/line_scatter；
|
||||
- Origin 二维 XY、柱/条形、Y 误差棒、XYZ 等高线/三维曲面/三元图和规则网格热图；
|
||||
- 输入下载、分块上传、SHA-256；
|
||||
- zcbot 查询、取消、导入和发布；
|
||||
- 最终截图与稳定错误码。
|
||||
|
|
|
|||
|
|
@ -200,7 +200,7 @@ Node 断线且本地任务可能仍在执行时标记 `disconnected`，不得自
|
|||
|
||||
## 7. Origin 任务闭环
|
||||
|
||||
当前实现进度：云端任务账本、幂等提交、短期 offer、Node 本地原子保存与 accept/reject 已落地。请求以 `inputs[]` 绑定多个 artifact、以 `outputs[]` 声明多个交付物，Node 逐项流式校验大小和 SHA-256 后将输入保存到 `input/<key>/<filename>`。固定 Worker 使用管理员安装的隔离 Python 运行时与随程序发布的 `worker.py` 驱动 Origin，按 `operation.plot.series[]` 从不同输入选择系列，并按输出声明生成 OPJU、PNG、SVG、PDF；plot spec、provenance 和原子 `terminal.json` 是系统强制元数据。运行不绑定单次 WebSocket，断线后继续执行。同一进程按 job 去重，Node 重启后不重复启动已留启动标记但无可信终态的任务。成功产物逐项流式上传到云端隐藏暂存区，云端复核任务身份、固定文件名、大小和 SHA-256 后，一次性发布到 `<working_dir>/origin/<job_id>/` 并登记平台 artifact UUID；Node 以 `upload-complete.json` 恢复中断上传。
|
||||
当前实现进度：云端任务账本、幂等提交、短期 offer、Node 本地原子保存与 accept/reject 已落地。请求以 `inputs[]` 绑定多个 artifact、以 `outputs[]` 声明多个交付物，Node 逐项流式校验大小和 SHA-256 后将输入保存到 `input/<key>/<filename>`。固定 Worker 使用管理员安装的隔离 Python 运行时与随程序发布的 `worker.py` 驱动 Origin；`operation.plot.series[]` 以 `x/y/z/y_error` 统一描述二维、误差棒和 XYZ 数据角色，按 `plot.type` 判别校验并生成 OPJU、PNG、SVG、PDF。当前覆盖折线、散点、线点、柱/条形、分组柱形、Y 误差棒、等高线、三维曲面、三元图和规则网格热图；plot spec、provenance 和原子 `terminal.json` 是系统强制元数据。运行不绑定单次 WebSocket，断线后继续执行。同一进程按 job 去重，Node 重启后不重复启动已留启动标记但无可信终态的任务。成功产物逐项流式上传到云端隐藏暂存区，云端复核任务身份、固定文件名、大小和 SHA-256 后，一次性发布到 `<working_dir>/origin/<job_id>/` 并登记平台 artifact UUID；Node 以 `upload-complete.json` 恢复中断上传。
|
||||
|
||||
```text
|
||||
用户上传一个或多个 CSV/XLSX/JSON
|
||||
|
|
|
|||
|
|
@ -62,7 +62,10 @@ class OriginWorkerUnitTests(unittest.TestCase):
|
|||
{"input": "second", "x": "temperature", "y": "value", "label": "28 d"},
|
||||
]
|
||||
resolved, labels = worker._resolve_series(input_data, series)
|
||||
self.assertEqual(resolved, [("first", 0, 1, "7 d"), ("second", 0, 1, "28 d")])
|
||||
self.assertEqual(resolved, [
|
||||
{"input": "first", "label": "7 d", "x": 0, "y": 1},
|
||||
{"input": "second", "label": "28 d", "x": 0, "y": 1},
|
||||
])
|
||||
self.assertEqual(labels, {("first", 1): "7 d", ("second", 1): "28 d"})
|
||||
|
||||
def test_shared_y_column_rejects_conflicting_labels(self) -> None:
|
||||
|
|
@ -73,6 +76,47 @@ class OriginWorkerUnitTests(unittest.TestCase):
|
|||
{"input": "sample", "x": "x2", "y": "y", "label": "Second"},
|
||||
])
|
||||
|
||||
def test_series_support_xyz_and_y_error_roles(self) -> None:
|
||||
resolved, labels = worker._resolve_series(
|
||||
{"sample": (["x", "y", "z", "sd"], [[0, 1, 2, 0.1]])},
|
||||
[{
|
||||
"input": "sample", "x": "x", "y": "y", "z": "z",
|
||||
"y_error": "sd", "label": "测量值",
|
||||
}],
|
||||
)
|
||||
self.assertEqual(resolved, [{
|
||||
"input": "sample", "label": "测量值", "x": 0, "y": 1,
|
||||
"z": 2, "y_error": 3,
|
||||
}])
|
||||
self.assertEqual(labels, {("sample", 1): "测量值"})
|
||||
|
||||
def test_heatmap_matrix_accepts_complete_unordered_grid(self) -> None:
|
||||
matrix, xy_map = worker._heatmap_matrix(
|
||||
[[1, 20, 4], [0, 10, 1], [1, 10, 2], [0, 20, 3]],
|
||||
{"x": 0, "y": 1, "z": 2},
|
||||
)
|
||||
self.assertEqual(matrix, [[1.0, 2.0], [3.0, 4.0]])
|
||||
self.assertEqual(xy_map, (0.0, 1.0, 10.0, 20.0))
|
||||
|
||||
def test_heatmap_matrix_rejects_invalid_grid(self) -> None:
|
||||
with self.assertRaisesRegex(ValueError, "HEATMAP_GRID_INCOMPLETE"):
|
||||
worker._heatmap_matrix(
|
||||
[[0, 10, 1], [1, 10, 2], [0, 20, 3]],
|
||||
{"x": 0, "y": 1, "z": 2},
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "HEATMAP_COORDINATES_DUPLICATED"):
|
||||
worker._heatmap_matrix(
|
||||
[[0, 10, 1], [0, 10, 2]], {"x": 0, "y": 1, "z": 2}
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "HEATMAP_GRID_NOT_REGULAR"):
|
||||
worker._heatmap_matrix(
|
||||
[
|
||||
[0, 10, 1], [1, 10, 2], [3, 10, 3],
|
||||
[0, 20, 4], [1, 20, 5], [3, 20, 6],
|
||||
],
|
||||
{"x": 0, "y": 1, "z": 2},
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ from sqlalchemy import create_mock_engine
|
|||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
from core.software_jobs import (
|
||||
SoftwareJobError,
|
||||
_canonical_request,
|
||||
_published_output_is_valid,
|
||||
abandon_offer,
|
||||
|
|
@ -339,6 +340,50 @@ class SoftwareJobProtocolTests(unittest.TestCase):
|
|||
{**request, "outputs": [{"key": "project", "type": "figure", "format": "png"}]}
|
||||
)
|
||||
|
||||
def test_origin_request_supports_unified_series_roles(self) -> None:
|
||||
artifact_id = str(uuid4())
|
||||
outputs = [{"key": "figure_png", "type": "figure", "format": "png"}]
|
||||
cases = {
|
||||
"column": [{"input": "sample", "x": "age", "y": "strength"}],
|
||||
"bar": [{"input": "sample", "x": "age", "y": "strength"}],
|
||||
"grouped_column": [
|
||||
{"input": "sample", "x": "age", "y": "strength"},
|
||||
{"input": "sample", "x": "age", "y": "modulus"},
|
||||
],
|
||||
"y_error": [{"input": "sample", "x": "age", "y": "strength", "y_error": "sd"}],
|
||||
"contour": [{"input": "sample", "x": "x", "y": "y", "z": "value"}],
|
||||
"surface_3d": [{"input": "sample", "x": "x", "y": "y", "z": "value"}],
|
||||
"ternary": [{"input": "sample", "x": "a", "y": "b", "z": "c"}],
|
||||
"heatmap": [{"input": "sample", "x": "x", "y": "y", "z": "value"}],
|
||||
}
|
||||
for plot_type, series in cases.items():
|
||||
request = {
|
||||
"schema_version": 2,
|
||||
"inputs": [{"key": "sample", "artifact_id": artifact_id}],
|
||||
"operation": {"plot": {"type": plot_type, "series": series}},
|
||||
"outputs": outputs,
|
||||
}
|
||||
with self.subTest(plot_type=plot_type):
|
||||
normalized, digest = _canonical_request("origin.plot@v2", request)
|
||||
self.assertEqual(normalized, request)
|
||||
self.assertEqual(len(digest), 64)
|
||||
|
||||
def test_origin_request_rejects_roles_that_do_not_match_plot_type(self) -> None:
|
||||
base = {
|
||||
"schema_version": 2,
|
||||
"inputs": [{"key": "sample", "artifact_id": str(uuid4())}],
|
||||
"outputs": [{"key": "figure_png", "type": "figure", "format": "png"}],
|
||||
}
|
||||
invalid_plots = (
|
||||
{"type": "line", "series": [{"input": "sample", "x": "x", "y": "y", "z": "z"}]},
|
||||
{"type": "y_error", "series": [{"input": "sample", "x": "x", "y": "y"}]},
|
||||
{"type": "contour", "series": [{"input": "sample", "x": "x", "y": "y"}]},
|
||||
{"type": "grouped_column", "series": [{"input": "sample", "x": "x", "y": "y"}]},
|
||||
)
|
||||
for plot in invalid_plots:
|
||||
with self.subTest(plot_type=plot["type"]), self.assertRaises(SoftwareJobError):
|
||||
_canonical_request("origin.plot@v2", {**base, "operation": {"plot": plot}})
|
||||
|
||||
def test_output_manifest_matches_exact_requested_outputs(self) -> None:
|
||||
request = {"outputs": [
|
||||
{"key": "project", "type": "project", "format": "opju"},
|
||||
|
|
|
|||
|
|
@ -167,7 +167,7 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
|||
self.assertIn('AutomationProgId = @"Origin.ApplicationSI\\CLSID"', probe)
|
||||
self.assertIn("RegistryHive.LocalMachine", probe)
|
||||
self.assertIn("RegistryHive.CurrentUser", probe)
|
||||
self.assertIn('new("OriginPro", version, "0.3.0", health, detail)', probe)
|
||||
self.assertIn('new("OriginPro", version, "0.4.0", health, detail)', probe)
|
||||
self.assertIn(
|
||||
"&& !jobInbox.HasPendingOriginJobs",
|
||||
connection,
|
||||
|
|
|
|||
|
|
@ -47,7 +47,8 @@ class SoftwareCapabilityListTool(_SoftwareJobTool):
|
|||
class SoftwareJobSubmitTool(_SoftwareJobTool):
|
||||
name = "software_job_submit"
|
||||
description = (
|
||||
"Submit an Origin plot job using one or more registered CSV, XLSX, or JSON artifacts. "
|
||||
"Submit an Origin 2D, error-bar, contour, 3D, ternary, or heatmap plot job using "
|
||||
"one or more registered CSV, XLSX, or JSON artifacts. "
|
||||
"Call register_artifact first for each workspace file. "
|
||||
"Return immediately with job_id; do not poll continuously or wait for completion."
|
||||
)
|
||||
|
|
@ -63,7 +64,10 @@ class SoftwareJobSubmitTool(_SoftwareJobTool):
|
|||
_plot_schema = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"type": {"type": "string", "enum": ["line", "scatter", "line_scatter"]},
|
||||
"type": {"type": "string", "enum": [
|
||||
"line", "scatter", "line_scatter", "column", "bar", "grouped_column",
|
||||
"y_error", "contour", "surface_3d", "ternary", "heatmap",
|
||||
]},
|
||||
"series": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
|
|
@ -74,17 +78,23 @@ class SoftwareJobSubmitTool(_SoftwareJobTool):
|
|||
"input": {"type": "string", "pattern": "^[a-z][a-z0-9_]{0,31}$"},
|
||||
"x": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"y": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"z": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"y_error": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"label": {"type": "string", "minLength": 1, "maxLength": 200},
|
||||
},
|
||||
"required": ["input", "x", "y"],
|
||||
"required": ["input"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
"description": "Series with explicit input key and X/Y columns.",
|
||||
"description": (
|
||||
"Typed data roles. XY plots require x/y; y_error requires x/y/y_error; "
|
||||
"contour, surface_3d, ternary, and heatmap require x/y/z."
|
||||
),
|
||||
},
|
||||
"template": {"type": "string", "enum": ["publication_double_column"]},
|
||||
"title": {"type": "string", "maxLength": 500},
|
||||
"x_axis": _axis_schema,
|
||||
"y_axis": _axis_schema,
|
||||
"z_axis": _axis_schema,
|
||||
"legend": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ windows-node\package-windows-node.bat
|
|||
|
||||
产物位于 `windows-node\dist\zcbot-windows-node-<version>-win-x64.zip`，旁边同时生成 SHA-256 文件。默认不包含 .NET Runtime，目标机器需预装 .NET 10 Desktop Runtime；需要免安装 .NET 的离线包时执行 `package-windows-node.bat --self-contained`。打包脚本是纯 BAT，使用 .NET SDK 以及 Windows 自带的 `tar.exe`、`certutil.exe`。
|
||||
|
||||
固定 Origin Worker 已支持 `line`、`scatter` 和 `line_scatter`，生成 OPJU、PNG、SVG、PDF、plot spec、provenance 与原子 `terminal.json`。运行时独立于 zcbot 服务端 Python。发布目录包含统一安装入口；使用实际运行 Node 的专用 Windows 账号直接双击：
|
||||
固定 Origin Worker 通过统一的 `origin.plot@v2` / `series[]` 数据角色模型支持 `line`、`scatter`、`line_scatter`、`column`、`bar`、`grouped_column`、`y_error`、`contour`、`surface_3d`、`ternary` 和 `heatmap`，生成 OPJU、PNG、SVG、PDF、plot spec、provenance 与原子 `terminal.json`。热图输入必须是完整、等间距且坐标不重复的规则 XYZ 网格。运行时独立于 zcbot 服务端 Python。发布目录包含统一安装入口；使用实际运行 Node 的专用 Windows 账号直接双击：
|
||||
|
||||
```text
|
||||
install-windows-node.bat
|
||||
|
|
|
|||
|
|
@ -7,7 +7,12 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
{
|
||||
private static readonly JsonSerializerOptions JsonOptions = new() { WriteIndented = true };
|
||||
private static readonly HashSet<string> PlotTypes =
|
||||
["line", "scatter", "line_scatter"];
|
||||
[
|
||||
"line", "scatter", "line_scatter", "column", "bar", "grouped_column",
|
||||
"y_error", "contour", "surface_3d", "ternary", "heatmap",
|
||||
];
|
||||
private static readonly HashSet<string> XyzPlotTypes =
|
||||
["contour", "surface_3d", "ternary", "heatmap"];
|
||||
private static readonly HashSet<string> JobStages =
|
||||
[
|
||||
"accepted",
|
||||
|
|
@ -502,9 +507,11 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
{
|
||||
if (plot.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(
|
||||
plot, "type", "series", "template", "title", "x_axis", "y_axis", "legend", "error_bars")
|
||||
plot, "type", "series", "template", "title", "x_axis", "y_axis", "z_axis",
|
||||
"legend", "error_bars")
|
||||
|| !plot.TryGetProperty("type", out var plotType)
|
||||
|| !PlotTypes.Contains(plotType.GetString() ?? "")
|
||||
|| plotType.GetString() is not { } plotTypeName
|
||||
|| !PlotTypes.Contains(plotTypeName)
|
||||
|| plot.TryGetProperty("title", out var title)
|
||||
&& (title.ValueKind != JsonValueKind.String || title.GetString()!.Length > 500)
|
||||
|| !plot.TryGetProperty("series", out var series)
|
||||
|
|
@ -514,11 +521,23 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
&& template.GetString() != "publication_double_column"
|
||||
|| !IsValidAxis(plot, "x_axis")
|
||||
|| !IsValidAxis(plot, "y_axis")
|
||||
|| !IsValidAxis(plot, "z_axis")
|
||||
|| !IsValidLegend(plot)
|
||||
|| plot.TryGetProperty("error_bars", out _))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if ((plotTypeName == "grouped_column" && series.GetArrayLength() < 2)
|
||||
|| (XyzPlotTypes.Contains(plotTypeName) && series.GetArrayLength() != 1))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
var requiredRoles = XyzPlotTypes.Contains(plotTypeName)
|
||||
? new[] { "x", "y", "z" }
|
||||
: plotTypeName == "y_error"
|
||||
? new[] { "x", "y", "y_error" }
|
||||
: new[] { "x", "y" };
|
||||
var allowedRoles = requiredRoles.ToHashSet(StringComparer.Ordinal);
|
||||
var inputKeys = inputs.EnumerateArray()
|
||||
.Select(item => item.GetProperty("key").GetString()!)
|
||||
.ToHashSet(StringComparer.Ordinal);
|
||||
|
|
@ -528,26 +547,29 @@ internal sealed class JobInboxStore(string jobsDirectory)
|
|||
foreach (var item in series.EnumerateArray())
|
||||
{
|
||||
if (item.ValueKind != JsonValueKind.Object
|
||||
|| !HasOnlyProperties(item, "input", "x", "y", "label")
|
||||
|| !HasOnlyProperties(item, "input", "x", "y", "z", "y_error", "label")
|
||||
|| !item.TryGetProperty("input", out var input)
|
||||
|| input.GetString() is not { } inputKey
|
||||
|| !inputKeys.Contains(inputKey)
|
||||
|| !item.TryGetProperty("x", out var x)
|
||||
|| !IsColumnName(x)
|
||||
|| !item.TryGetProperty("y", out var y)
|
||||
|| !IsColumnName(y)
|
||||
|| requiredRoles.Any(role =>
|
||||
!item.TryGetProperty(role, out var column) || !IsColumnName(column))
|
||||
|| new[] { "x", "y", "z", "y_error" }.Any(role =>
|
||||
!allowedRoles.Contains(role) && item.TryGetProperty(role, out _))
|
||||
|| item.TryGetProperty("label", out var label)
|
||||
&& (label.ValueKind != JsonValueKind.String
|
||||
|| label.GetString()!.Length is < 1 or > 200)
|
||||
|| !identities.Add($"{inputKey}\0{x.GetString()}\0{y.GetString()}"))
|
||||
|| !identities.Add(string.Join(
|
||||
"\0", new[] { inputKey }.Concat(requiredRoles.Select(
|
||||
role => item.GetProperty(role).GetString()!)))))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
usedInputs.Add(inputKey);
|
||||
var labelKey = $"{inputKey}\0{y.GetString()}";
|
||||
var yValue = item.GetProperty("y").GetString()!;
|
||||
var labelKey = $"{inputKey}\0{yValue}";
|
||||
var effectiveLabel = item.TryGetProperty("label", out var seriesLabel)
|
||||
? seriesLabel.GetString()!
|
||||
: y.GetString()!;
|
||||
: yValue;
|
||||
if (labels.TryGetValue(labelKey, out var existingLabel)
|
||||
&& existingLabel != effectiveLabel)
|
||||
{
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ internal static class OriginRuntimeProbe
|
|||
}
|
||||
|
||||
private static OriginRuntimeStatus Status(string? version, string health, string detail) =>
|
||||
new("OriginPro", version, "0.3.0", health, detail);
|
||||
new("OriginPro", version, "0.4.0", health, detail);
|
||||
|
||||
private static string? FindInstalledVersion()
|
||||
{
|
||||
|
|
|
|||
|
|
@ -56,7 +56,7 @@ if errorlevel 1 (
|
|||
echo [ERR] Failed to install the Origin worker dependencies.
|
||||
goto :failed
|
||||
)
|
||||
"%RUNTIME_PYTHON%" -c "import originpro, openpyxl; print('[OK] Origin worker Python packages are available.')"
|
||||
"%RUNTIME_PYTHON%" -c "import originpro, openpyxl, numpy; print('[OK] Origin worker Python packages are available.')"
|
||||
if errorlevel 1 (
|
||||
echo [ERR] Origin runtime import verification failed.
|
||||
goto :failed
|
||||
|
|
|
|||
|
|
@ -1,2 +1,3 @@
|
|||
originpro==1.1.15
|
||||
openpyxl==3.1.5
|
||||
numpy==2.2.6
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ from __future__ import annotations
|
|||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
|
|
@ -16,7 +17,19 @@ from importlib.metadata import PackageNotFoundError, version
|
|||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
PLOT_TYPES = {"line": "l", "scatter": "s", "line_scatter": "y"}
|
||||
PLOT_CONFIG = {
|
||||
"line": ("line", "l"),
|
||||
"scatter": ("scatter", "s"),
|
||||
"line_scatter": ("linesymb", "y"),
|
||||
"column": ("column", "c"),
|
||||
"bar": ("bar", 215),
|
||||
"grouped_column": ("column", "c"),
|
||||
"y_error": ("ERRBAR", "y"),
|
||||
"contour": ("TriContour", 243),
|
||||
"surface_3d": ("glCMAP", 103),
|
||||
"ternary": ("ternary", 245),
|
||||
}
|
||||
XYZ_PLOT_TYPES = {"contour", "surface_3d", "ternary", "heatmap"}
|
||||
FORMATS = {"opju", "png", "svg", "pdf"}
|
||||
|
||||
|
||||
|
|
@ -115,24 +128,76 @@ def _input_file(job_dir: Path, key: str) -> Path:
|
|||
def _resolve_series(
|
||||
input_data: dict[str, tuple[list[str], list[list[Any]]]],
|
||||
series_specs: list[dict[str, Any]],
|
||||
) -> tuple[list[tuple[str, int, int, str | None]], dict[tuple[str, int], str]]:
|
||||
resolved: list[tuple[str, int, int, str | None]] = []
|
||||
) -> tuple[list[dict[str, Any]], dict[tuple[str, int], str]]:
|
||||
resolved: list[dict[str, Any]] = []
|
||||
labels: dict[tuple[str, int], str] = {}
|
||||
for series in series_specs:
|
||||
input_key = series["input"]
|
||||
headers, _ = input_data[input_key]
|
||||
x_index = _column_index(headers, series["x"], "x")
|
||||
y_index = _column_index(headers, series["y"], "y")
|
||||
role_indexes = {
|
||||
role: _column_index(headers, series[role], role)
|
||||
for role in ("x", "y", "z", "y_error") if role in series
|
||||
}
|
||||
y_index = role_indexes["y"]
|
||||
label = series.get("label")
|
||||
label_key = (input_key, y_index)
|
||||
effective_label = label or series["y"]
|
||||
if label_key in labels and labels[label_key] != effective_label:
|
||||
raise ValueError("SERIES_LABEL_CONFLICT")
|
||||
labels[label_key] = effective_label
|
||||
resolved.append((input_key, x_index, y_index, label))
|
||||
resolved.append({"input": input_key, "label": label, **role_indexes})
|
||||
return resolved, labels
|
||||
|
||||
|
||||
def _number(value: Any, role: str) -> float:
|
||||
if isinstance(value, bool):
|
||||
raise ValueError(f"{role.upper()}_VALUE_NOT_NUMERIC") # noqa: TRY004
|
||||
try:
|
||||
number = float(value)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"{role.upper()}_VALUE_NOT_NUMERIC") from exc
|
||||
if not math.isfinite(number):
|
||||
raise ValueError(f"{role.upper()}_VALUE_NOT_FINITE")
|
||||
return number
|
||||
|
||||
|
||||
def _is_evenly_spaced(values: list[float]) -> bool:
|
||||
if len(values) <= 2:
|
||||
return True
|
||||
step = values[1] - values[0]
|
||||
tolerance = max(abs(step) * 1e-9, 1e-12)
|
||||
return all(
|
||||
math.isclose(current - previous, step, rel_tol=1e-9, abs_tol=tolerance)
|
||||
for previous, current in zip(values[1:-1], values[2:], strict=True)
|
||||
)
|
||||
|
||||
|
||||
def _heatmap_matrix(
|
||||
rows: list[list[Any]], resolved: dict[str, Any]
|
||||
) -> tuple[list[list[float]], tuple[float, float, float, float]]:
|
||||
points: dict[tuple[float, float], float] = {}
|
||||
for row in rows:
|
||||
try:
|
||||
x = _number(row[resolved["x"]], "x")
|
||||
y = _number(row[resolved["y"]], "y")
|
||||
z = _number(row[resolved["z"]], "z")
|
||||
except IndexError as exc:
|
||||
raise ValueError("HEATMAP_ROW_INCOMPLETE") from exc
|
||||
if (x, y) in points:
|
||||
raise ValueError("HEATMAP_COORDINATES_DUPLICATED")
|
||||
points[(x, y)] = z
|
||||
x_values = sorted({item[0] for item in points})
|
||||
y_values = sorted({item[1] for item in points})
|
||||
if len(x_values) < 2 or len(y_values) < 2:
|
||||
raise ValueError("HEATMAP_GRID_TOO_SMALL")
|
||||
if len(points) != len(x_values) * len(y_values):
|
||||
raise ValueError("HEATMAP_GRID_INCOMPLETE")
|
||||
if not _is_evenly_spaced(x_values) or not _is_evenly_spaced(y_values):
|
||||
raise ValueError("HEATMAP_GRID_NOT_REGULAR")
|
||||
matrix = [[points[(x, y)] for x in x_values] for y in y_values]
|
||||
return matrix, (x_values[0], x_values[-1], y_values[0], y_values[-1])
|
||||
|
||||
|
||||
def _validate_artifact(path: Path, extension: str) -> None:
|
||||
if not path.is_file() or path.stat().st_size == 0:
|
||||
raise RuntimeError(f"{extension.upper()}_EXPORT_EMPTY")
|
||||
|
|
@ -162,7 +227,7 @@ def run(job_dir: Path) -> list[dict[str, Any]]:
|
|||
}
|
||||
plot_spec = request["operation"]["plot"]
|
||||
plot_type = plot_spec["type"]
|
||||
if plot_type not in PLOT_TYPES:
|
||||
if plot_type not in PLOT_CONFIG and plot_type != "heatmap":
|
||||
raise ValueError("PLOT_TYPE_NOT_IMPLEMENTED")
|
||||
series_specs = plot_spec["series"]
|
||||
resolved_series, labels = _resolve_series(input_data, series_specs)
|
||||
|
|
@ -187,17 +252,47 @@ def run(job_dir: Path) -> list[dict[str, Any]]:
|
|||
[row[index] if index < len(row) else None for row in rows],
|
||||
lname=column_label,
|
||||
)
|
||||
graph = op.new_graph(template={"line": "line", "scatter": "scatter", "line_scatter": "linesymb"}[plot_type])
|
||||
layer = graph[0]
|
||||
for input_key, x_index, y_index, _ in resolved_series:
|
||||
layer.add_plot(
|
||||
worksheets[input_key], coly=y_index, colx=x_index, type=PLOT_TYPES[plot_type]
|
||||
if plot_type == "heatmap":
|
||||
import numpy as np
|
||||
|
||||
resolved = resolved_series[0]
|
||||
_, rows = input_data[resolved["input"]]
|
||||
matrix, xy_map = _heatmap_matrix(rows, resolved)
|
||||
matrix_sheet = op.new_sheet("m")
|
||||
matrix_sheet.from_np(np.array(matrix, dtype=float))
|
||||
matrix_sheet.xymap = xy_map
|
||||
matrix_sheet.set_label(
|
||||
0, _axis_title(plot_spec.get("z_axis"), str(series_specs[0]["z"]))
|
||||
)
|
||||
graph = op.new_graph(template="heatmap")
|
||||
layer = graph[0]
|
||||
layer.add_mplot(matrix_sheet, 0, type=105)
|
||||
else:
|
||||
template, origin_plot_type = PLOT_CONFIG[plot_type]
|
||||
graph = op.new_graph(template=template)
|
||||
layer = graph[0]
|
||||
for resolved in resolved_series:
|
||||
arguments = {
|
||||
"coly": resolved["y"],
|
||||
"colx": resolved["x"],
|
||||
"type": origin_plot_type,
|
||||
}
|
||||
if plot_type in XYZ_PLOT_TYPES:
|
||||
arguments["colz"] = resolved["z"]
|
||||
if plot_type == "y_error":
|
||||
arguments["colyerr"] = resolved["y_error"]
|
||||
layer.add_plot(worksheets[resolved["input"]], **arguments)
|
||||
if plot_type == "grouped_column":
|
||||
layer.group()
|
||||
layer.rescale()
|
||||
layer.axis("x").title = _axis_title(
|
||||
plot_spec.get("x_axis"), str(series_specs[0].get("x") or "X")
|
||||
)
|
||||
layer.axis("y").title = _axis_title(plot_spec.get("y_axis"), "Y")
|
||||
if plot_type == "surface_3d":
|
||||
layer.axis("z").title = _axis_title(
|
||||
plot_spec.get("z_axis"), str(series_specs[0].get("z") or "Z")
|
||||
)
|
||||
if plot_spec.get("title"):
|
||||
title = layer.add_label(str(plot_spec["title"]))
|
||||
title.set_int("fsize", 18)
|
||||
|
|
@ -237,7 +332,7 @@ def run(job_dir: Path) -> list[dict[str, Any]]:
|
|||
except PackageNotFoundError:
|
||||
originpro_version = "embedded"
|
||||
provenance = {
|
||||
"adapter_version": "0.3.0",
|
||||
"adapter_version": "0.4.0",
|
||||
"originpro_version": originpro_version,
|
||||
"request_digest": request_record["request_digest"],
|
||||
"inputs": [
|
||||
|
|
|
|||
Loading…
Reference in New Issue