diff --git a/CHANGELOG.md b/CHANGELOG.md index 808465e..d10e291 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ ## Unreleased +- Windows Node 的专业软件适配器现在可作为独立目录更新;后续扩展 Origin 绘图参数时,可只替换适配器并重启本机 Node,无需重装或替换 Node 程序,服务端契约更新也无需重启 zcbot。 + - Origin 绘图新增出版排版控制:可指定毫米画布、坐标范围与对数尺度、刻度角度和字号、标题/图例字号、网格,以及每条曲线的颜色、线型、点型和透明度;旧绘图请求继续沿用默认样式。 - 新增专业软件 Job 中心:Agent 可将当前对话目录内的数据登记为稳定输入,提交、查询和停止 Windows Node 上的受控软件任务;文件栏底部固定展示任务状态,点击可从右侧打开按开启时间倒序、滚动加载的跨对话任务列表,并可收到完成通知或回到原对话分析结果。 - 专业软件任务完成后,输出文件会立即显示在当前对话的文件面板;任务中心收起时也不会再遮挡发送按钮。 diff --git a/DESIGN.md b/DESIGN.md index 6455e9f..138c7f3 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -462,9 +462,9 @@ scheduled_jobs(§8.5) channel_bindings(§8.7,判别列+JSONB) 云端控制面使用独立的 `software_node_enrollments` 与 `software_nodes`,不复用用户外部系统连接。管理员创建的一次性注册码具有 128 bit 随机熵,数据库只保存 SHA-256 摘要;节点注册在行锁事务中校验有效期、预期名称和允许能力,成功后原子消费。每个节点获得独立高熵 Token,数据库只保存 bcrypt 强哈希,明文仅在注册响应出现一次。 -专业软件采用“共享能力契约 + Node adapter”边界。仓库根目录 `software-contracts/*.json` 是语言无关的声明式事实源,描述 capability、请求 JSON Schema、输入额度、输出 manifest、feature 与最低 adapter 版本;Core 启动时自动发现契约,只负责身份、账本、调度、传输、摘要与最终发布,不包含 Origin 或其他软件的操作分支。Windows Node 是可信宿主,负责持久化 job 目录、下载/上传、恢复、取消和 adapter 注册;adapter 才负责探测具体软件、二次语义校验并执行。adapter 可以是 Node 内置 .NET 实现,也可以由固定 runner 启动任意语言的受信进程;进程只接收 job 目录并通过 `state.json`、`terminal.json` 与固定输出目录交接,不把 Python、COM 或某个 SDK 写入通用协议。当前 Origin adapter 的执行体恰好是固定 Python Worker,这是实现选择而非平台契约。 +专业软件采用“共享能力契约 + 本机 adapter”边界。仓库根目录 `software-contracts/*.json` 是语言无关的声明式事实源,描述 capability、请求 JSON Schema、输入额度、输出 manifest、feature 与最低 adapter 版本;Core 按文件签名热加载契约,只负责身份、账本、调度、传输、摘要与最终发布,不包含 Origin 或其他软件的操作分支。热加载先完整校验新快照再原子切换,写入中或非法版本继续使用上一份有效快照。Windows Node 是稳定的可信宿主,只负责持久化 job 目录、下载/上传、恢复、取消,以及从程序目录 `adapters/*/adapter.json` 发现 adapter;Host 对请求只做通用 JSON Schema 校验,具体软件探测和二次语义校验属于 Worker。manifest 声明 capability、实际 adapter 版本、运行类型、入口和契约文件,版本不编译进 EXE。Worker 可以是受管 Python 脚本或独立 EXE,进程只接收 job 目录,并以 `--probe` 返回运行状态,通过 `state.json`、`terminal.json` 与固定输出目录交接;Python、COM 或某个 SDK 都不属于通用协议。当前 Origin adapter 使用 Python 只是实现选择。 -扩展现有 capability 的 feature 时修改共享契约、对应 adapter/Worker 和测试,不修改 Core 调度与 Job 生命周期;增加新专业软件时新增契约,并在目标 Node 安装包的单一 adapter registry 注册实现。Cloud 会自动获得校验、工具 schema、输出发布和能力发现;Node 的注册能力、配置校验与界面展示也从 registry/契约派生。当前 Node 仍按整机单执行槽保守串行,未来只有真实并行软件需求出现时,才把 slot 账本升级为 per-capability 租约,而不改变 Job 协议。 +扩展现有 capability 的 feature 时修改共享契约、对应 Worker 和测试,不修改 Core 调度、Job 生命周期或 Node EXE;增加新专业软件时新增契约与一个 adapter 目录。Cloud 会在下一次契约查询时获得新校验和工具 schema;Node 在启动时从本地目录获得能力、版本和入口。当前选择明确的运维边界:zcbot 契约可热更新,不要求重启;本机 adapter 更新时先退出托盘 Node,整体替换 adapter 目录后重新启动,不重装、不替换 Node EXE,也不建设自动更新平台。当前 Node 仍按整机单执行槽保守串行,未来只有真实并行软件需求出现时,才把 slot 账本升级为 per-capability 租约,而不改变 Job 协议。 Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/software-nodes/connect` WebSocket。进程内 Connection Manager 保证同一节点单活,新连接关闭旧连接;`hello`/`heartbeat` 更新版本、容量、软件健康与最后在线时间。管理员禁用节点时先持久化禁用态,再关闭现有连接;断线收尾不得覆盖禁用态。当前单活只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或将 Node API 固定路由到单一控制面实例。 diff --git a/PROGRESS.md b/PROGRESS.md index e4837de..c9781f6 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -22,6 +22,8 @@ ### 2026-08-14 +- **08-14 / Unreleased / Adapter 目录化更新与契约热加载**:Windows Node 改为从 EXE 同级 `adapters/*/adapter.json` 发现能力,实际版本、Python/EXE 运行类型、入口和契约路径均由 manifest 声明;Host 使用通用 JSON Schema 校验并删除 Origin 专用 C# 校验,软件探测与轴范围、系列角色等语义规则下沉 Worker。Core 契约注册表按文件变化原子热加载,非法中间版本保留上一有效快照,工具 schema、注册默认值和调度查询均动态读取。新增无需编译 Node 的 Origin adapter 独立打包入口;专项 40 项 unittest、Python 编译、Ruff 致命规则、diff 检查、.NET build 与两类实际打包通过,全量 611 项仅 3 个既有数据库集成模块因显式测试库缺少 `users` 表未通过(另跳过 4 项),未连接或写入生产 DB。 + - **08-14 / Unreleased / Origin 出版级排版参数**:`origin.plot@v2` 新增可选画布、轴范围/尺度/步长/刻度排版/网格、标题与图例排版,以及逐系列颜色、线型、点型和透明度;PNG 宽度按物理画布与 DPI 计算,旧请求默认行为不变。共享契约、Node 二次校验、固定 Worker 与运行文档已同步,专项 68 项 unittest、Python 编译、Ruff 致命规则、diff 检查与 .NET build 通过;全量 608 项仅 3 个既有数据库集成模块因显式测试库缺少 `users` 表未通过,未连接或写入生产 DB。 - **08-14 / Unreleased / 专业软件契约与语言无关 adapter 边界**:将 capability 请求 schema、输入额度、输出 manifest、feature/adapter 版本要求和兼容运行时集中到 `software-contracts/*.json`;Core 自动发现契约并通用完成校验、工具 schema、调度、上传发布,不再包含 Origin 分支。Windows Node 以单一 adapter registry 派生注册能力、配置校验、界面和运行时上报,通用 Host 只处理 job 目录协议;adapter 可由 .NET 内置或固定进程以任意语言实现,Origin 的 Python Worker 仅是当前实现。调度同时按 feature/adapter 版本选节点,并跳过队首暂不可执行 Job。专项 82 项 unittest、Python 编译、Ruff 致命规则、diff 检查与 .NET build 通过;全量 605 项仅 3 个数据库集成模块因显式测试库未迁移、缺少 `users` 表而未通过,未连接或写入生产 DB。 diff --git a/RUN.md b/RUN.md index 709c73e..2bcaf62 100644 --- a/RUN.md +++ b/RUN.md @@ -1098,11 +1098,15 @@ cd /d D:\ZcbotNode install-windows-node.bat ``` -若 Python 未加入 PATH,可把绝对路径作为第一个参数,例如 `install-windows-node.bat "C:\Python312\python.exe"`。默认解释器为 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\Scripts\python.exe`。如需使用其他受管解释器,设置机器级 `ZCBOT_ORIGIN_PYTHON` 为绝对 `python.exe` 路径后重启 Node。`node.json`、可恢复任务和 runtime 集中保存在 `%ProgramData%\Zcbot\WindowsNode\`,不会因替换程序目录而丢失。运行时固定依赖见发布目录的 `origin-worker/requirements.txt`;任务请求无权选择解释器、脚本或路径。当前 Worker 支持 CSV/XLSX/JSON 输入,`line`、`scatter`、`line_scatter` 与 OPJU/PNG/SVG/PDF 输出。成功产物由 Node 流式上传,全部校验通过后发布到任务工作目录 `origin//`,plot spec 与 provenance 位于其 `.meta/`;上传中断会在重连时幂等续传。 +若 Python 未加入 PATH,可把绝对路径作为第一个参数,例如 `install-windows-node.bat "C:\Python312\python.exe"`。默认解释器为 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\Scripts\python.exe`。如需使用其他受管解释器,优先设置机器级 `ZCBOT_ADAPTER_ORIGIN_PYTHON`;旧名 `ZCBOT_ORIGIN_PYTHON` 暂时兼容。`node.json`、可恢复任务和 runtime 集中保存在 `%ProgramData%\Zcbot\WindowsNode\`,不会因替换程序目录而丢失。运行时固定依赖见发布目录的 `adapters/origin.plot@v2/requirements.txt`;任务请求无权选择解释器、脚本或路径。当前 Worker 支持 CSV/XLSX/JSON 输入及 OPJU/PNG/SVG/PDF 输出。成功产物由 Node 流式上传,全部校验通过后发布到任务工作目录 `origin//`,plot spec 与 provenance 位于其 `.meta/`;上传中断会在重连时幂等续传。 Web 用户登录后,文件栏 Job 中心会聚合本人最近任务。活动任务约 4 秒刷新一次,空闲时降为约 30 秒;停止已派发任务是协作取消,状态先显示“正在停止”,Node 在线时立即接收,断线后在下次连接或心跳时重放。Agent 可调用 `software_capability_list`、`register_artifact`、`software_job_submit`、`software_job_status` 和 `software_job_cancel`。Origin 输入必须是 artifact:已有 UUID 可直接提交,普通 task 文件先逐个用相对路径登记;登记不会发布聊天交付卡片。提交工具接收 `inputs`、`operation`、`outputs`,支持 1–16 个输入、跨输入系列和多个显式输出;`plot.canvas` 可指定毫米画布,轴可指定范围、步长、尺度、刻度角度/字号、标题字号和网格,`legend` 可控制显隐、位置和字号,`series[].style` 可控制颜色、线宽/线型、点型/点大小和透明度。所有排版字段可选,旧请求保持默认样式。任务只创建固定 v2 schema 的持久任务,不会阻塞当前对话等待完成。成功状态提供 `output_dir`,Agent可在该目录内搜索并分析;正式输出的 artifact 带 `software_job_id`,供结果卡和产物详情展示来源。Node 输出上传的逐任务诊断日志位于 `%ProgramData%\Zcbot\WindowsNode\jobs\\logs\node-output-upload.log`;日志包含上传阶段、产物文件名、重试次数和 Windows `HRESULT`,单文件达到 1 MiB 后轮转一份 `.1`,不记录 Node Token 或认证请求头。 -专业软件契约位于 `software-contracts/*.json`,Core 会在启动时自动发现。新增 feature 时更新对应契约与 Node adapter;新增软件时新增契约,并只在 Windows Node 的 `NodeAdapterRegistry.CreateDefault` 注册已安装实现。通用协议不要求 adapter 使用 Python:实现可为 .NET 内置代码或由固定 runner 启动的任意语言受信进程;Origin 安装器创建 Python 3.12 runtime 只服务当前 Origin adapter。 +专业软件契约位于 `software-contracts/*.json`。zcbot 会在文件变化后校验并热加载完整契约快照,正常修改契约不需要重启服务;非法或尚未写完的文件不会替换上一份有效快照。Node Host 从 EXE 同级的 `adapters/*/adapter.json` 发现本机能力,通用校验只读取 manifest 指向的 JSON Schema;Worker 可以是受管 Python 脚本或独立 EXE。 + +两个 JSON 的职责不同:`adapter.json` 只描述本机如何启动(实际版本、runtime、入口和契约文件名),`origin.plot.v2.json` 描述云端与本机共同遵守的业务请求/输出契约。前者不能替代后者;独立打包脚本只是把根目录的同一份业务契约复制进 adapter 交付目录,不维护第二份源码。 + +只更新 Origin adapter 时,不需要编译、重装或替换 Node EXE:运行 `windows-node\package-origin-adapter.bat` 可直接把 manifest、Worker、依赖清单和契约打成 `dist\origin.plot@v2-adapter.zip`。在 Node 机器上先从托盘退出 zcbot Windows Node,备份并整体替换 EXE 同级的 `adapters\origin.plot@v2\`,再从开始菜单或登录任务启动 Node。不要在任务执行中覆盖目录。若 `requirements.txt` 发生变化,需重新运行统一安装器更新受管 Python runtime;仅 `adapter.json`、契约或 `worker.py` 变化时无需。Node 启动后会用 Worker 的 `--probe` 核对 manifest 与 Worker 版本,版本不一致时该能力保持不可用。 注册配置写入 `%ProgramData%\Zcbot\WindowsNode\node.json`;Token 使用 DPAPI `LocalMachine` 加密,ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号执行统一安装器、注册并运行 Node。当前 MVP 以该账号的登录后计划任务启动,不安装 Windows Service。 diff --git a/core/software_contracts.py b/core/software_contracts.py index 3d45efc..d3d0a8e 100644 --- a/core/software_contracts.py +++ b/core/software_contracts.py @@ -3,7 +3,9 @@ from __future__ import annotations import json +import logging import re +import threading from dataclasses import dataclass from hashlib import sha256 from pathlib import Path, PurePosixPath @@ -13,6 +15,7 @@ from jsonschema import Draft202012Validator, FormatChecker CONTRACT_ROOT = Path(__file__).resolve().parents[1] / "software-contracts" +logger = logging.getLogger(__name__) class SoftwareContractError(ValueError): @@ -212,19 +215,80 @@ def _load_contract(path: Path) -> CapabilityContract: ) -_loaded_contracts = [_load_contract(path) for path in sorted(CONTRACT_ROOT.glob("*.json"))] -CONTRACTS = {contract.capability: contract for contract in _loaded_contracts} -if not CONTRACTS or len(CONTRACTS) != len(_loaded_contracts): - raise RuntimeError("software contracts are missing or contain duplicate capabilities") -SUPPORTED_CAPABILITIES = frozenset(CONTRACTS) -DEFAULT_CAPABILITIES = tuple( - item.capability for item in CONTRACTS.values() if item.default_enrollment -) +class _ContractRegistry: + """Atomically reload contracts after a file change; retain the last valid snapshot.""" + + def __init__(self, root: Path) -> None: + self.root = root + self._lock = threading.RLock() + self._signature: tuple[tuple[str, int, int], ...] = () + self._failed_signature: tuple[tuple[str, int, int], ...] | None = None + self._contracts: dict[str, CapabilityContract] = {} + self._reload(initial=True) + + def _files(self) -> list[Path]: + return sorted(self.root.glob("*.json")) + + def _current_signature(self) -> tuple[tuple[str, int, int], ...]: + return tuple( + (path.name, path.stat().st_mtime_ns, path.stat().st_size) + for path in self._files() + ) + + def _reload(self, *, initial: bool = False) -> None: + with self._lock: + signature = self._current_signature() + if not initial and signature in {self._signature, self._failed_signature}: + return + try: + loaded = [_load_contract(path) for path in self._files()] + contracts = {item.capability: item for item in loaded} + if not contracts or len(contracts) != len(loaded): + raise RuntimeError( + "software contracts are missing or contain duplicate capabilities" + ) + except Exception: + if initial: + raise + self._failed_signature = signature + logger.exception("Ignoring invalid software contract update") + return + self._contracts = contracts + self._signature = signature + self._failed_signature = None + + def contracts(self) -> dict[str, CapabilityContract]: + self._reload() + with self._lock: + return dict(self._contracts) + + +_registry = _ContractRegistry(CONTRACT_ROOT) + + +def get_contracts() -> dict[str, CapabilityContract]: + return _registry.contracts() + + +def supported_capabilities() -> frozenset[str]: + return frozenset(get_contracts()) + + +def default_capabilities() -> tuple[str, ...]: + return tuple( + item.capability for item in get_contracts().values() if item.default_enrollment + ) + + +# Import compatibility only. Runtime paths use the query functions above so edits hot reload. +CONTRACTS = get_contracts() +SUPPORTED_CAPABILITIES = supported_capabilities() +DEFAULT_CAPABILITIES = default_capabilities() def get_contract(capability: str) -> CapabilityContract: try: - return CONTRACTS[capability] + return get_contracts()[capability] except KeyError as exc: raise SoftwareContractError("unsupported capability") from exc diff --git a/core/software_nodes.py b/core/software_nodes.py index f24cccf..2b8aea5 100644 --- a/core/software_nodes.py +++ b/core/software_nodes.py @@ -10,7 +10,7 @@ from uuid import UUID, uuid4 import bcrypt from sqlalchemy import select -from core.software_contracts import DEFAULT_CAPABILITIES, SUPPORTED_CAPABILITIES +from core.software_contracts import default_capabilities, supported_capabilities from core.storage.engine import session_scope from core.storage.models import SoftwareNode, SoftwareNodeEnrollment @@ -44,8 +44,8 @@ def create_enrollment( capabilities: list[str] | None = None, ttl_seconds: int = 600, ) -> dict: - allowed = list(dict.fromkeys(capabilities or DEFAULT_CAPABILITIES)) - if not allowed or any(item not in SUPPORTED_CAPABILITIES for item in allowed): + allowed = list(dict.fromkeys(capabilities or default_capabilities())) + if not allowed or any(item not in supported_capabilities() for item in allowed): raise SoftwareNodeError("unsupported capability") if not 60 <= ttl_seconds <= 3600: raise SoftwareNodeError("ttl_seconds must be between 60 and 3600") diff --git a/tests/test_origin_worker.py b/tests/test_origin_worker.py index e8e8dd7..bbf1a46 100644 --- a/tests/test_origin_worker.py +++ b/tests/test_origin_worker.py @@ -9,6 +9,10 @@ from pathlib import Path WORKER_PATH = ( Path(__file__).resolve().parents[1] / "windows-node" / "origin-worker" / "worker.py" ) +ADAPTER_MANIFEST_PATH = ( + Path(__file__).resolve().parents[1] + / "windows-node" / "adapters" / "origin.plot@v2" / "adapter.json" +) SPEC = importlib.util.spec_from_file_location("zcbot_origin_worker", WORKER_PATH) assert SPEC and SPEC.loader worker = importlib.util.module_from_spec(SPEC) @@ -16,6 +20,21 @@ SPEC.loader.exec_module(worker) class OriginWorkerUnitTests(unittest.TestCase): + @staticmethod + def _request() -> dict: + return { + "inputs": [{"key": "sample"}], + "operation": {"plot": { + "type": "line", + "series": [{"input": "sample", "x": "x", "y": "y"}], + }}, + "outputs": [{"key": "figure_png", "format": "png"}], + } + + def test_worker_version_matches_adapter_manifest(self) -> None: + manifest = json.loads(ADAPTER_MANIFEST_PATH.read_text(encoding="utf-8")) + self.assertEqual(worker.ADAPTER_VERSION, manifest["adapter_version"]) + def test_csv_and_json_inputs_are_read_without_origin(self) -> None: with tempfile.TemporaryDirectory() as directory: root = Path(directory) @@ -147,6 +166,21 @@ class OriginWorkerUnitTests(unittest.TestCase): {"input": "sample", "x": "x2", "y": "y", "label": "Second"}, ]) + def test_worker_owns_cross_field_semantic_validation(self) -> None: + request = self._request() + worker._validate_semantics(request) + + request["operation"]["plot"]["x_axis"] = { + "scale": "log10", "minimum": 0, "maximum": 100 + } + with self.assertRaisesRegex(ValueError, "X_AXIS_LOG_LIMIT_INVALID"): + worker._validate_semantics(request) + + request = self._request() + request["inputs"].append({"key": "unused"}) + with self.assertRaisesRegex(ValueError, "INPUT_BINDINGS_MUST_BE_USED_EXACTLY"): + worker._validate_semantics(request) + def test_series_support_xyz_and_y_error_roles(self) -> None: resolved, labels = worker._resolve_series( {"sample": (["x", "y", "z", "sd"], [[0, 1, 2, 0.1]])}, diff --git a/tests/test_software_contracts.py b/tests/test_software_contracts.py index 7c0223e..b6725b7 100644 --- a/tests/test_software_contracts.py +++ b/tests/test_software_contracts.py @@ -1,9 +1,15 @@ from __future__ import annotations +import json +import tempfile import unittest +from pathlib import Path +from shutil import copy2 from core.software_contracts import ( DEFAULT_CAPABILITIES, + CONTRACT_ROOT, + _ContractRegistry, get_contract, node_supports_request, version_at_least, @@ -64,6 +70,26 @@ class SoftwareContractTests(unittest.TestCase): self.assertTrue(version_at_least("0.10.0", "0.4.0")) self.assertFalse(version_at_least("0.3.9", "0.4.0")) + def test_registry_hot_reloads_and_retains_last_valid_snapshot(self) -> None: + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + target = root / "origin.plot.v2.json" + copy2(CONTRACT_ROOT / target.name, target) + registry = _ContractRegistry(root) + original = registry.contracts()["origin.plot@v2"] + + value = json.loads(target.read_text(encoding="utf-8")) + value["display_name"] = "OriginPro hot reload" + target.write_text(json.dumps(value), encoding="utf-8") + refreshed = registry.contracts()["origin.plot@v2"] + self.assertEqual(refreshed.display_name, "OriginPro hot reload") + + target.write_text("{invalid", encoding="utf-8") + with self.assertLogs("core.software_contracts", level="ERROR"): + retained = registry.contracts()["origin.plot@v2"] + self.assertEqual(retained.display_name, "OriginPro hot reload") + self.assertNotEqual(original.display_name, retained.display_name) + if __name__ == "__main__": unittest.main() diff --git a/tests/test_windows_node_source.py b/tests/test_windows_node_source.py index 9444595..ce24578 100644 --- a/tests/test_windows_node_source.py +++ b/tests/test_windows_node_source.py @@ -9,13 +9,14 @@ PROJECT = ROOT / "Zcbot.WindowsNode" class WindowsNodeSourceTests(unittest.TestCase): - def test_project_targets_net10_windows_forms_without_third_party_packages(self) -> None: + def test_project_targets_net10_windows_forms_with_json_schema_validator(self) -> None: tree = ET.parse(PROJECT / "Zcbot.WindowsNode.csproj") root = tree.getroot() self.assertEqual(root.findtext("./PropertyGroup/TargetFramework"), "net10.0-windows") self.assertEqual(root.findtext("./PropertyGroup/UseWindowsForms"), "true") self.assertEqual(root.findtext("./PropertyGroup/OutputType"), "WinExe") - self.assertEqual(root.findall("./ItemGroup/PackageReference"), []) + packages = root.findall("./ItemGroup/PackageReference") + self.assertEqual([item.attrib["Include"] for item in packages], ["JsonSchema.Net"]) def test_node_protocol_and_secret_storage_markers_are_present(self) -> None: source = "\n".join(path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs")) @@ -26,7 +27,7 @@ class WindowsNodeSourceTests(unittest.TestCase): 'SetRequestHeader("X-Node-Id"', "DataProtectionScope.LocalMachine", "SetAccessRuleProtection(isProtected: true", - '"origin.plot.v2.json"', + '"adapter.json"', "NotifyIcon", "ConfigurationForm", "TrayIconFactory.Create", @@ -38,7 +39,7 @@ class WindowsNodeSourceTests(unittest.TestCase): source = "\n".join( path.read_text(encoding="utf-8") for path in PROJECT.glob("*.cs") - if path.name != "OriginWorkerRunner.cs" + if path.name != "AdapterProcessRunner.cs" ) for forbidden in ("Process.Start", "cmd.exe", "powershell.exe", "LabTalk"): self.assertNotIn(forbidden, source) @@ -135,7 +136,8 @@ class WindowsNodeSourceTests(unittest.TestCase): project = (PROJECT / "Zcbot.WindowsNode.csproj").read_text(encoding="utf-8") self.assertIn("..\\install-windows-node.bat", project) self.assertIn("..\\origin-worker\\requirements.txt", project) - self.assertIn("..\\..\\software-contracts\\*.json", project) + self.assertIn("..\\adapters\\origin.plot@v2\\adapter.json", project) + self.assertIn("adapters\\origin.plot@v2\\origin.plot.v2.json", project) self.assertFalse((ROOT / "install-windows-node.ps1").exists()) self.assertFalse((ROOT / "install-origin-runtime.ps1").exists()) self.assertFalse((ROOT / "install-startup.ps1").exists()) @@ -147,7 +149,7 @@ class WindowsNodeSourceTests(unittest.TestCase): self.assertIn('if /I "%~1"=="--self-contained"', script) self.assertIn("dotnet publish", script) self.assertIn('"install-windows-node.bat"', script) - self.assertIn('"software-contracts\\origin.plot.v2.json"', script) + self.assertIn('"adapters\\origin.plot@v2\\origin.plot.v2.json"', script) self.assertIn("tar.exe -a -c -f", script) self.assertIn("certutil.exe -hashfile", script) self.assertFalse((ROOT / "package-windows-node.ps1").exists()) @@ -163,13 +165,13 @@ class WindowsNodeSourceTests(unittest.TestCase): self.assertIn("节点身份已被服务端拒绝", connection) self.assertNotIn("Node credentials were rejected", connection) - def test_origin_runtime_probe_is_read_only_and_reported(self) -> None: - probe = (PROJECT / "OriginRuntimeProbe.cs").read_text(encoding="utf-8") + def test_adapter_runtime_probe_is_worker_owned_and_reported(self) -> None: + runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8") + worker = (ROOT / "origin-worker" / "worker.py").read_text(encoding="utf-8") connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8") - self.assertIn('AutomationProgId = @"Origin.ApplicationSI\\CLSID"', probe) - self.assertIn("RegistryHive.LocalMachine", probe) - self.assertIn("RegistryHive.CurrentUser", probe) - self.assertIn("OriginPlotNodeAdapter.CurrentAdapterVersion", probe) + self.assertIn('r"Origin.ApplicationSI\\CLSID"', worker) + self.assertIn('["--probe"]', runner) + self.assertIn('root.GetProperty("adapter_version")', runner) self.assertIn( "&& !jobInbox.HasPendingJobs", connection, @@ -179,8 +181,7 @@ class WindowsNodeSourceTests(unittest.TestCase): "ReadRecoverableJobs().Any(item => item.Terminal is null)", (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8"), ) - self.assertNotIn("CreateInstance", probe) - self.assertNotIn("Process.Start", probe) + self.assertNotIn("CreateInstance", worker) for marker in ( "software_version = origin.SoftwareVersion", "adapter_version = origin.AdapterVersion", @@ -193,11 +194,11 @@ class WindowsNodeSourceTests(unittest.TestCase): inbox = (PROJECT / "JobInboxStore.cs").read_text(encoding="utf-8") connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8") self.assertIn("adapters.Find(capability)", inbox) + self.assertIn("adapter.ValidateRequest(request)", inbox) + self.assertNotIn("IsValidOriginRequest", inbox) + self.assertNotIn("IsValidPlot", inbox) self.assertIn('root.TryGetProperty("input_transfers"', inbox) self.assertIn('"input", key, filename', inbox) - self.assertIn("supportedPlotTypes.Contains", inbox) - self.assertIn("IsValidOutputs", inbox) - self.assertIn('("figure", "png") => "figure_png"', inbox) self.assertIn("FileOptions.WriteThrough", inbox) self.assertIn("stream.Flush(flushToDisk: true)", inbox) new_record = inbox.split("var record =", 1)[1].split("private static JsonElement?", 1)[0] @@ -232,26 +233,26 @@ class WindowsNodeSourceTests(unittest.TestCase): self.assertIn("File.Move(temporaryPath, destination, overwrite: false)", downloader) self.assertNotIn("Process.Start", downloader) - def test_origin_worker_launch_is_fixed_and_terminal_driven(self) -> None: - runner = (PROJECT / "OriginWorkerRunner.cs").read_text(encoding="utf-8") + def test_adapter_worker_launch_is_manifest_driven_and_terminal_driven(self) -> None: + runner = (PROJECT / "AdapterProcessRunner.cs").read_text(encoding="utf-8") connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8") project = (PROJECT / "Zcbot.WindowsNode.csproj").read_text(encoding="utf-8") worker = (ROOT / "origin-worker" / "worker.py").read_text(encoding="utf-8") self.assertIn('Environment.GetEnvironmentVariable("ZCBOT_ORIGIN_PYTHON")', runner) self.assertIn( - 'Path.Combine(paths.RootDirectory, "runtimes", "origin", "Scripts", "python.exe")', + 'Path.Combine(paths.RootDirectory, "runtimes", runtimeId, "Scripts", "python.exe")', runner, ) self.assertIn("UseShellExecute = false", runner) - self.assertIn("startInfo.ArgumentList.Add(workerScript)", runner) - self.assertIn("startInfo.ArgumentList.Add(jobDirectory)", runner) + self.assertIn("command.PrefixArguments.Concat(arguments)", runner) + self.assertIn("descriptor.EntrypointPath", runner) self.assertIn('Path.Combine(jobDirectory, "terminal.json")', runner) self.assertIn('"NODE_RESTARTED_DURING_JOB"', runner) self.assertIn("CancellationTokenSource.CreateLinkedTokenSource", runner) self.assertIn("process.Kill(entireProcessTree: true)", runner) self.assertIn('type.GetString() == "job_cancel"', connection) self.assertIn('"cancelled", "USER_CANCELLED"', connection) - self.assertIn("origin-worker\\worker.py", project) + self.assertIn("adapters\\origin.plot@v2\\worker.py", project) self.assertIn("if op.oext:", worker) self.assertIn("op.exit()", worker) self.assertIn("op.new_graph", worker) @@ -262,6 +263,14 @@ class WindowsNodeSourceTests(unittest.TestCase): for forbidden in ("subprocess", "eval(", "exec(", "os.system"): self.assertNotIn(forbidden, worker) + def test_origin_adapter_can_be_packaged_without_building_node(self) -> None: + script = (ROOT / "package-origin-adapter.bat").read_text(encoding="utf-8") + connection = (PROJECT / "NodeConnectionLoop.cs").read_text(encoding="utf-8") + self.assertIn('"adapters\\origin.plot@v2\\adapter.json"', script) + self.assertIn('"origin-worker\\worker.py"', script) + self.assertIn('"..\\software-contracts\\origin.plot.v2.json"', script) + self.assertNotIn("dotnet", script.lower()) + uploader = (PROJECT / "JobOutputUploader.cs").read_text(encoding="utf-8") self.assertIn('new AuthenticationHeaderValue("Bearer", config.NodeToken)', uploader) self.assertIn('DefaultRequestHeaders.Add("X-Node-Id"', uploader) diff --git a/tools/software_jobs.py b/tools/software_jobs.py index c18b7b4..0f16050 100644 --- a/tools/software_jobs.py +++ b/tools/software_jobs.py @@ -5,10 +5,10 @@ import json from uuid import UUID, uuid4 from core.software_contracts import ( - CONTRACTS, - SUPPORTED_CAPABILITIES, + get_contracts, get_contract, node_available_slots, + supported_capabilities, ) from core.software_jobs import ( SoftwareJobError, @@ -25,7 +25,7 @@ from .base import Tool def _contract_property_schema(name: str) -> dict: schemas = [ item.submission_schema()["properties"][name] - for item in CONTRACTS.values() + for item in get_contracts().values() ] unique = {json.dumps(item, ensure_ascii=False, sort_keys=True): item for item in schemas} values = list(unique.values()) @@ -56,7 +56,7 @@ class SoftwareCapabilityListTool(_SoftwareJobTool): and item in (node.get("capabilities") or []) and node_available_slots(item, node.get("runtime") or {}) > 0 ), - } for item in sorted(SUPPORTED_CAPABILITIES)] + } for item in sorted(supported_capabilities())] return json.dumps({"capabilities": items}, ensure_ascii=False) @@ -67,21 +67,36 @@ class SoftwareJobSubmitTool(_SoftwareJobTool): "capability contract. Call register_artifact first for workspace files. Return " "immediately with job_id; do not poll continuously or wait for completion." ) - parameters = { - "type": "object", - "properties": { - "capability": {"type": "string", "enum": sorted(SUPPORTED_CAPABILITIES)}, - "inputs": _contract_property_schema("inputs"), - "operation": _contract_property_schema("operation"), - "outputs": _contract_property_schema("outputs"), - "idempotency_key": { - "type": "string", - "description": "Stable unique key for this exact submission; omit to generate one.", + @staticmethod + def _parameters() -> dict: + return { + "type": "object", + "properties": { + "capability": { + "type": "string", "enum": sorted(supported_capabilities()) + }, + "inputs": _contract_property_schema("inputs"), + "operation": _contract_property_schema("operation"), + "outputs": _contract_property_schema("outputs"), + "idempotency_key": { + "type": "string", + "description": ( + "Stable unique key for this exact submission; omit to generate one." + ), + }, }, - }, - "required": ["capability", "inputs", "operation", "outputs"], - "additionalProperties": False, - } + "required": ["capability", "inputs", "operation", "outputs"], + "additionalProperties": False, + } + + # Compatibility for callers inspecting the class; Tool.schema below is always fresh. + parameters = _parameters() + + @property + def schema(self) -> dict: + value = super().schema + value["function"]["parameters"] = self._parameters() + return value def execute( self, diff --git a/web/routers/software_nodes.py b/web/routers/software_nodes.py index 860c33f..9efb122 100644 --- a/web/routers/software_nodes.py +++ b/web/routers/software_nodes.py @@ -22,11 +22,7 @@ from fastapi.responses import FileResponse from core.artifact_lifecycle import register_published_artifacts from core.paths import from_db_path -from core.software_contracts import ( - DEFAULT_CAPABILITIES, - SoftwareContractError, - get_contract, -) +from core.software_contracts import SoftwareContractError, default_capabilities, get_contract from core.software_jobs import ( MAX_OUTPUT_ARTIFACT_BYTES, MAX_OUTPUT_TOTAL_BYTES, @@ -218,7 +214,7 @@ def _organize_staged_metadata( def _publish_software_job_outputs(job_id: UUID, context: dict, manifest: list[dict]) -> list[dict]: - capability = context.get("capability", DEFAULT_CAPABILITIES[0]) + capability = context.get("capability", default_capabilities()[0]) contract = get_contract(capability) root = load_user_root(context["user_id"]) working_dir = _task_working_dir(root, context["working_dir"]) diff --git a/web/schemas.py b/web/schemas.py index 2f206cb..fbc8a4b 100644 --- a/web/schemas.py +++ b/web/schemas.py @@ -6,7 +6,7 @@ from uuid import UUID from pydantic import BaseModel, Field -from core.software_contracts import DEFAULT_CAPABILITIES +from core.software_contracts import default_capabilities class TaskCreateRequest(BaseModel): @@ -120,7 +120,7 @@ class ExternalSystemCredentialsRequest(BaseModel): class SoftwareEnrollmentCreateRequest(BaseModel): expected_name: str = "" - capabilities: list[str] = Field(default_factory=lambda: list(DEFAULT_CAPABILITIES)) + capabilities: list[str] = Field(default_factory=lambda: list(default_capabilities())) ttl_seconds: int = 600 @@ -139,5 +139,5 @@ class SoftwareNodeDisableRequest(BaseModel): class SoftwareJobCreateRequest(BaseModel): idempotency_key: str - capability: str = DEFAULT_CAPABILITIES[0] + capability: str = Field(default_factory=lambda: default_capabilities()[0]) request: dict = Field(default_factory=dict) diff --git a/windows-node/README.md b/windows-node/README.md index 968739b..92bf77e 100644 --- a/windows-node/README.md +++ b/windows-node/README.md @@ -2,7 +2,7 @@ 内网 MVP 的 Windows 执行节点,目标运行环境为 Windows 11 Enterprise + .NET 10 SDK 10.0.303。仓库根目录 `global.json` 固定 SDK patch;客户端只使用 .NET Windows Desktop Framework,不依赖第三方 NuGet 包。 -当前实现托盘状态角标、配置与本机任务窗口、注册、DPAPI/ACL 配置保存、WebSocket `hello`/心跳和退避重连。Node Host 以语言无关的 job 目录协议负责持久化、下载、恢复、取消和上传,具体软件由 `NodeAdapterRegistry` 中的 adapter 负责探测、校验和执行;adapter 可为 .NET 内置实现,也可由固定 runner 启动任意语言的受信进程。注册能力、配置校验和界面从 registry 与随包发布的 `software-contracts/*.json` 派生。当前安装包注册 `origin.plot@v2`,其 adapter 使用 Python Worker 驱动 Origin/OriginPro,但 Python 不是 Node 通用协议的一部分。本机任务列表只读取已派发到该 Node 的持久化目录,不查询云端未派发队列;成功产物按 manifest 上传并由云端复核,全部完成后原子发布,中断后按本地 `upload-complete.json` 幂等续传。 +当前实现托盘状态角标、配置与本机任务窗口、注册、DPAPI/ACL 配置保存、WebSocket `hello`/心跳和退避重连。Node Host 以语言无关的 job 目录协议负责持久化、下载、恢复、取消和上传,并从 EXE 同级 `adapters/*/adapter.json` 发现能力;Host 只按 manifest 指向的 JSON Schema 通用校验请求,软件探测、语义校验和执行都由 Worker 完成。Worker 可为受管 Python 脚本或独立 EXE。当前安装包带 `origin.plot@v2` Python Worker,但 Python 不是 Node 通用协议的一部分。本机任务列表只读取已派发到该 Node 的持久化目录;成功产物按 manifest 上传并由云端复核,全部完成后原子发布,中断后按本地 `upload-complete.json` 幂等续传。 在仓库根目录执行一条命令生成可分发 ZIP: @@ -14,6 +14,8 @@ windows-node\package-windows-node.bat 固定 Origin Worker 通过统一的 `origin.plot@v2` / `series[]` 数据角色模型支持 `line`、`scatter`、`line_scatter`、`column`、`bar`、`grouped_column`、`y_error`、`contour`、`surface_3d`、`ternary` 和 `heatmap`,并支持毫米画布、轴范围/尺度/刻度排版、标题、图例及逐系列样式,生成 OPJU、PNG、SVG、PDF、plot spec、provenance 与原子 `terminal.json`。热图输入必须是完整、等间距且坐标不重复的规则 XYZ 网格。运行时独立于 zcbot 服务端 Python。发布目录包含统一安装入口;使用实际运行 Node 的专用 Windows 账号直接双击: +日常只更新 adapter 时运行 `package-origin-adapter.bat`,无需编译 Node。退出托盘 Node 后,整体替换安装目录中的 `adapters\origin.plot@v2\` 再启动即可;不要在任务执行期间覆盖文件。只有依赖清单变化才需要重新运行统一安装器。 + ```text install-windows-node.bat ``` diff --git a/windows-node/Zcbot.WindowsNode/AdapterProcessRunner.cs b/windows-node/Zcbot.WindowsNode/AdapterProcessRunner.cs new file mode 100644 index 0000000..65d58dc --- /dev/null +++ b/windows-node/Zcbot.WindowsNode/AdapterProcessRunner.cs @@ -0,0 +1,254 @@ +using System.Collections.Concurrent; +using System.ComponentModel; +using System.Diagnostics; +using System.Text; +using System.Text.Json; + +namespace Zcbot.WindowsNode; + +internal sealed class AdapterProcessRunner(AdapterDescriptor descriptor, JobInboxStore inbox) +{ + private static readonly TimeSpan WorkerTimeout = TimeSpan.FromMinutes(30); + private readonly ConcurrentDictionary active = new(); + private readonly ConcurrentDictionary cancellations = new(); + + internal bool HasActiveJobs => !active.IsEmpty; + + internal Task RunAsync(RecoverableJob job) => + active.GetOrAdd(job.JobId, _ => RunOnceAsync(job, CancellationFor(job.JobId).Token)); + + internal void Cancel(Guid jobId) => CancellationFor(jobId).Cancel(); + + internal AdapterRuntimeStatus Probe() + { + try + { + var command = ResolveCommand(); + using var process = Start(command, descriptor.DirectoryPath, ["--probe"]); + var stdout = process.StandardOutput.ReadToEndAsync(); + var stderr = process.StandardError.ReadToEndAsync(); + using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + try + { + process.WaitForExitAsync(timeout.Token).GetAwaiter().GetResult(); + } + catch (OperationCanceledException) + { + process.Kill(entireProcessTree: true); + return Unavailable("Adapter probe timed out."); + } + var output = stdout.GetAwaiter().GetResult(); + var error = stderr.GetAwaiter().GetResult(); + if (process.ExitCode != 0) + { + return Unavailable( + string.IsNullOrWhiteSpace(error) ? "Adapter probe failed." : error.Trim()); + } + using var document = JsonDocument.Parse(output); + var root = document.RootElement; + var reportedVersion = root.GetProperty("adapter_version").GetString(); + if (!descriptor.Manifest.AdapterVersion.Equals(reportedVersion, StringComparison.Ordinal)) + { + return Unavailable("Adapter manifest and worker versions do not match."); + } + return new AdapterRuntimeStatus( + root.GetProperty("software").GetString() ?? descriptor.Contract.DisplayName, + root.TryGetProperty("software_version", out var softwareVersion) + ? softwareVersion.GetString() + : null, + descriptor.Manifest.AdapterVersion, + root.GetProperty("health").GetString() ?? "unavailable", + root.GetProperty("detail").GetString() ?? "Adapter probe returned no detail."); + } + catch (Exception exception) when ( + exception is IOException + or JsonException + or UnauthorizedAccessException + or InvalidOperationException + or Win32Exception) + { + return Unavailable(exception.Message); + } + } + + private AdapterRuntimeStatus Unavailable(string detail) => + new( + descriptor.Contract.DisplayName, + null, + descriptor.Manifest.AdapterVersion, + "unavailable", + detail[..Math.Min(500, detail.Length)]); + + private CancellationTokenSource CancellationFor(Guid jobId) => + cancellations.GetOrAdd(jobId, _ => new CancellationTokenSource()); + + private async Task RunOnceAsync(RecoverableJob job, CancellationToken cancellationToken) + { + try + { + var paths = NodePaths.ForCurrentMachine(); + var jobDirectory = Path.Combine(paths.JobsDirectory, job.JobId.ToString("D")); + var terminalPath = Path.Combine(jobDirectory, "terminal.json"); + if (File.Exists(terminalPath)) return; + + var markerPath = Path.Combine(jobDirectory, "worker-started.json"); + if (File.Exists(markerPath)) + { + inbox.WriteTerminal( + job, + "failed", + ErrorCode("NODE_RESTARTED_DURING_JOB"), + "The node restarted after adapter execution began and cannot prove the prior worker state."); + return; + } + + var command = ResolveCommand(); + WriteMarker(markerPath, command); + using var process = Start(command, jobDirectory, [jobDirectory]); + var stdout = process.StandardOutput.ReadToEndAsync(); + var stderr = process.StandardError.ReadToEndAsync(); + using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); + timeout.CancelAfter(WorkerTimeout); + try + { + await process.WaitForExitAsync(timeout.Token); + } + catch (OperationCanceledException) + { + process.Kill(entireProcessTree: true); + if (cancellationToken.IsCancellationRequested) + { + inbox.WriteTerminal(job, "cancelled", "USER_CANCELLED", "Cancelled by user."); + } + else + { + inbox.WriteTerminal( + job, "failed", ErrorCode("WORKER_TIMEOUT"), + "Adapter worker exceeded 30 minutes."); + } + return; + } + var output = await stdout; + var error = await stderr; + WriteDiagnostic(jobDirectory, output, error, process.ExitCode); + if (!File.Exists(terminalPath)) + { + inbox.WriteTerminal( + job, + "failed", + ErrorCode("WORKER_NO_TERMINAL"), + $"Adapter worker exited with code {process.ExitCode} without terminal.json."); + } + } + catch (Exception exception) when ( + exception is IOException + or JsonException + or UnauthorizedAccessException + or InvalidOperationException + or Win32Exception) + { + inbox.WriteTerminal( + job, + "failed", + ErrorCode("WORKER_START_FAILED"), + exception.Message[..Math.Min(500, exception.Message.Length)]); + } + finally + { + active.TryRemove(job.JobId, out _); + if (cancellations.TryRemove(job.JobId, out var cancellation)) cancellation.Dispose(); + } + } + + private string ErrorCode(string suffix) => + descriptor.Manifest.Capability.StartsWith("origin.", StringComparison.Ordinal) + ? suffix == "NODE_RESTARTED_DURING_JOB" ? suffix : $"ORIGIN_{suffix}" + : suffix == "NODE_RESTARTED_DURING_JOB" ? suffix : $"ADAPTER_{suffix}"; + + private ProcessCommand ResolveCommand() + { + if (descriptor.Manifest.Runtime == "executable") + { + return new ProcessCommand(descriptor.EntrypointPath, []); + } + var runtimeId = descriptor.Manifest.RuntimeId!; + var environmentName = "ZCBOT_ADAPTER_" + + runtimeId.ToUpperInvariant().Replace('-', '_') + "_PYTHON"; + var configured = Environment.GetEnvironmentVariable(environmentName); + if (string.IsNullOrWhiteSpace(configured) && runtimeId == "origin") + { + configured = Environment.GetEnvironmentVariable("ZCBOT_ORIGIN_PYTHON"); + } + var paths = NodePaths.ForCurrentMachine(); + var candidate = string.IsNullOrWhiteSpace(configured) + ? Path.Combine(paths.RootDirectory, "runtimes", runtimeId, "Scripts", "python.exe") + : configured; + if (!Path.IsPathFullyQualified(candidate)) + { + throw new InvalidOperationException("Managed Python interpreter path is not absolute."); + } + var interpreter = Path.GetFullPath(candidate); + if (!File.Exists(interpreter) + || !Path.GetFileName(interpreter).Equals("python.exe", StringComparison.OrdinalIgnoreCase)) + { + throw new InvalidOperationException("Managed Python interpreter is unavailable."); + } + return new ProcessCommand(interpreter, [descriptor.EntrypointPath]); + } + + private static Process Start( + ProcessCommand command, string workingDirectory, IReadOnlyList arguments) + { + var startInfo = new ProcessStartInfo + { + FileName = command.Filename, + WorkingDirectory = workingDirectory, + UseShellExecute = false, + CreateNoWindow = true, + RedirectStandardOutput = true, + RedirectStandardError = true, + StandardOutputEncoding = Encoding.UTF8, + StandardErrorEncoding = Encoding.UTF8, + }; + foreach (var argument in command.PrefixArguments.Concat(arguments)) + { + startInfo.ArgumentList.Add(argument); + } + return Process.Start(startInfo) + ?? throw new InvalidOperationException("Adapter worker did not start."); + } + + private void WriteMarker(string path, ProcessCommand command) + { + var value = JsonSerializer.SerializeToUtf8Bytes(new + { + started_at = DateTimeOffset.UtcNow, + node_pid = Environment.ProcessId, + capability = descriptor.Manifest.Capability, + adapter_version = descriptor.Manifest.AdapterVersion, + runtime = descriptor.Manifest.Runtime, + executable = command.Filename, + entrypoint = descriptor.EntrypointPath, + }); + using var stream = new FileStream( + path, FileMode.CreateNew, FileAccess.Write, FileShare.None, + bufferSize: 4096, FileOptions.WriteThrough); + stream.Write(value); + stream.Flush(flushToDisk: true); + } + + private static void WriteDiagnostic(string jobDirectory, string output, string error, int exitCode) + { + var logs = Path.Combine(jobDirectory, "logs"); + Directory.CreateDirectory(logs); + var value = JsonSerializer.Serialize(new + { + exit_code = exitCode, + stdout = output[..Math.Min(output.Length, 16 * 1024)], + stderr = error[..Math.Min(error.Length, 16 * 1024)], + }); + File.WriteAllText(Path.Combine(logs, "worker-process.json"), value, Encoding.UTF8); + } + + private sealed record ProcessCommand(string Filename, IReadOnlyList PrefixArguments); +} diff --git a/windows-node/Zcbot.WindowsNode/AdapterRuntimeStatus.cs b/windows-node/Zcbot.WindowsNode/AdapterRuntimeStatus.cs new file mode 100644 index 0000000..2f345a0 --- /dev/null +++ b/windows-node/Zcbot.WindowsNode/AdapterRuntimeStatus.cs @@ -0,0 +1,8 @@ +namespace Zcbot.WindowsNode; + +internal sealed record AdapterRuntimeStatus( + string Software, + string? SoftwareVersion, + string AdapterVersion, + string Health, + string Detail); diff --git a/windows-node/Zcbot.WindowsNode/JobInboxStore.cs b/windows-node/Zcbot.WindowsNode/JobInboxStore.cs index bda7693..90a8aaf 100644 --- a/windows-node/Zcbot.WindowsNode/JobInboxStore.cs +++ b/windows-node/Zcbot.WindowsNode/JobInboxStore.cs @@ -403,187 +403,6 @@ internal sealed class JobInboxStore(string jobsDirectory) } } - internal static bool IsValidOriginRequest( - JsonElement request, IReadOnlyList supportedPlotTypes) - { - if (!HasOnlyProperties(request, "schema_version", "inputs", "operation", "outputs") - || !request.TryGetProperty("schema_version", out var schemaVersion) - || !schemaVersion.TryGetInt32(out var version) - || version != 2 - || !request.TryGetProperty("inputs", out var inputs) - || !IsValidInputBindings(inputs) - || !request.TryGetProperty("operation", out var operation) - || operation.ValueKind != JsonValueKind.Object - || !HasOnlyProperties(operation, "plot") - || !operation.TryGetProperty("plot", out var plot) - || !IsValidPlot(plot, inputs, supportedPlotTypes) - || !request.TryGetProperty("outputs", out var outputs)) - { - return false; - } - return IsValidOutputs(outputs); - } - - private static bool IsValidOutputs(JsonElement outputs) - { - if (outputs.ValueKind != JsonValueKind.Array || outputs.GetArrayLength() is < 1 or > 16) - { - return false; - } - var keys = new HashSet(StringComparer.Ordinal); - var identities = new HashSet(StringComparer.Ordinal); - foreach (var output in outputs.EnumerateArray()) - { - if (output.ValueKind != JsonValueKind.Object - || !HasOnlyProperties(output, "key", "type", "format", "options") - || !output.TryGetProperty("key", out var keyValue) - || keyValue.GetString() is not { } key - || !output.TryGetProperty("type", out var typeValue) - || typeValue.GetString() is not { } outputType - || !output.TryGetProperty("format", out var formatValue) - || formatValue.GetString() is not { } format) - { - return false; - } - var expectedKey = (outputType, format) switch - { - ("project", "opju") => "project", - ("figure", "png") => "figure_png", - ("figure", "svg") => "figure_svg", - ("figure", "pdf") => "figure_pdf", - _ => "", - }; - if (key != expectedKey || !keys.Add(key) || !identities.Add($"{outputType}\0{format}")) - { - return false; - } - if (format == "png") - { - if (output.TryGetProperty("options", out var options) - && (options.ValueKind != JsonValueKind.Object - || !HasOnlyProperties(options, "dpi") - || !options.TryGetProperty("dpi", out var dpi) - || !dpi.TryGetInt32(out var dpiValue) - || dpiValue is < 72 or > 1200)) - { - return false; - } - } - else if (output.TryGetProperty("options", out _)) - { - return false; - } - } - return true; - } - - private static bool IsValidInputBindings(JsonElement inputs) - { - if (inputs.ValueKind != JsonValueKind.Array || inputs.GetArrayLength() is < 1 or > 16) - { - return false; - } - var keys = new HashSet(StringComparer.Ordinal); - foreach (var input in inputs.EnumerateArray()) - { - if (input.ValueKind != JsonValueKind.Object - || !HasOnlyProperties(input, "key", "artifact_id", "selector") - || !input.TryGetProperty("key", out var keyValue) - || keyValue.GetString() is not { } key - || !IsInputKey(key) - || !keys.Add(key) - || !input.TryGetProperty("artifact_id", out var artifactId) - || !Guid.TryParse(artifactId.GetString(), out _) - || input.TryGetProperty("selector", out var selector) - && !IsValidSelector(selector)) - { - return false; - } - } - return true; - } - - private static bool IsValidPlot( - JsonElement plot, JsonElement inputs, IReadOnlyList supportedPlotTypes) - { - if (plot.ValueKind != JsonValueKind.Object - || !HasOnlyProperties( - plot, "type", "series", "template", "title", "x_axis", "y_axis", "z_axis", - "legend", "error_bars", "title_style", "canvas") - || !plot.TryGetProperty("type", out var plotType) - || plotType.GetString() is not { } plotTypeName - || !supportedPlotTypes.Contains(plotTypeName, StringComparer.Ordinal) - || plot.TryGetProperty("title", out var title) - && (title.ValueKind != JsonValueKind.String || title.GetString()!.Length > 500) - || !plot.TryGetProperty("series", out var series) - || series.ValueKind != JsonValueKind.Array - || series.GetArrayLength() is < 1 or > 16 - || plot.TryGetProperty("template", out var template) - && template.GetString() != "publication_double_column" - || !IsValidTextStyle(plot, "title_style") - || !IsValidCanvas(plot) - || !IsValidAxis(plot, "x_axis") - || !IsValidAxis(plot, "y_axis") - || !IsValidAxis(plot, "z_axis") - || !IsValidLegend(plot) - || plot.TryGetProperty("error_bars", out _)) - { - return false; - } - if ((plotTypeName == "grouped_column" && series.GetArrayLength() < 2) - || (XyzPlotTypes.Contains(plotTypeName) && series.GetArrayLength() != 1)) - { - return false; - } - var requiredRoles = XyzPlotTypes.Contains(plotTypeName) - ? new[] { "x", "y", "z" } - : plotTypeName == "y_error" - ? new[] { "x", "y", "y_error" } - : new[] { "x", "y" }; - var allowedRoles = requiredRoles.ToHashSet(StringComparer.Ordinal); - var inputKeys = inputs.EnumerateArray() - .Select(item => item.GetProperty("key").GetString()!) - .ToHashSet(StringComparer.Ordinal); - var identities = new HashSet(StringComparer.Ordinal); - var usedInputs = new HashSet(StringComparer.Ordinal); - var labels = new Dictionary(StringComparer.Ordinal); - foreach (var item in series.EnumerateArray()) - { - if (item.ValueKind != JsonValueKind.Object - || !HasOnlyProperties(item, "input", "x", "y", "z", "y_error", "label", "style") - || !item.TryGetProperty("input", out var input) - || input.GetString() is not { } inputKey - || !inputKeys.Contains(inputKey) - || requiredRoles.Any(role => - !item.TryGetProperty(role, out var column) || !IsColumnName(column)) - || new[] { "x", "y", "z", "y_error" }.Any(role => - !allowedRoles.Contains(role) && item.TryGetProperty(role, out _)) - || item.TryGetProperty("label", out var label) - && (label.ValueKind != JsonValueKind.String - || label.GetString()!.Length is < 1 or > 200) - || !IsValidSeriesStyle(item) - || !identities.Add(string.Join( - "\0", new[] { inputKey }.Concat(requiredRoles.Select( - role => item.GetProperty(role).GetString()!))))) - { - return false; - } - usedInputs.Add(inputKey); - var yValue = item.GetProperty("y").GetString()!; - var labelKey = $"{inputKey}\0{yValue}"; - var effectiveLabel = item.TryGetProperty("label", out var seriesLabel) - ? seriesLabel.GetString()! - : yValue; - if (labels.TryGetValue(labelKey, out var existingLabel) - && existingLabel != effectiveLabel) - { - return false; - } - labels[labelKey] = effectiveLabel; - } - return usedInputs.SetEquals(inputKeys); - } - private static bool IsValidSelector(JsonElement selector) => selector.ValueKind == JsonValueKind.Object && HasOnlyProperties(selector, "sheet") @@ -599,108 +418,6 @@ internal sealed class JobInboxStore(string jobsDirectory) || character is >= '0' and <= '9' || character == '_'); - private static bool IsColumnName(JsonElement value) => - value.ValueKind == JsonValueKind.String - && value.GetString() is { Length: >= 1 and <= 128 }; - - private static bool IsValidAxis(JsonElement plot, string name) - { - if (!plot.TryGetProperty(name, out var axis)) return true; - if (axis.ValueKind != JsonValueKind.Object - || !HasOnlyProperties( - axis, "title", "unit", "scale", "minimum", "maximum", "major_step", - "tick_label_angle", "tick_label_font_size", "title_font_size", "grid")) - { - return false; - } - if (axis.TryGetProperty("title", out var titleValue) - && titleValue.ValueKind != JsonValueKind.String - || axis.TryGetProperty("unit", out var unitValue) - && unitValue.ValueKind != JsonValueKind.String - || axis.TryGetProperty("scale", out var scale) - && scale.GetString() is not ("linear" or "log10" or "ln" or "log2") - || axis.TryGetProperty("grid", out var grid) - && grid.GetString() is not ("none" or "major" or "major_minor") - || !IsOptionalNumber(axis, "minimum") - || !IsOptionalNumber(axis, "maximum") - || !IsOptionalNumber(axis, "major_step", 0, double.PositiveInfinity, false) - || !IsOptionalNumber(axis, "tick_label_angle", -180, 180) - || !IsOptionalNumber(axis, "tick_label_font_size", 6, 72) - || !IsOptionalNumber(axis, "title_font_size", 6, 72)) - { - return false; - } - return !axis.TryGetProperty("minimum", out var minimum) - || !axis.TryGetProperty("maximum", out var maximum) - || minimum.GetDouble() < maximum.GetDouble(); - } - - private static bool IsValidLegend(JsonElement plot) - { - if (!plot.TryGetProperty("legend", out var legend)) return true; - return legend.ValueKind == JsonValueKind.Object - && HasOnlyProperties(legend, "enabled", "position", "font_size") - && (!legend.TryGetProperty("enabled", out var enabled) - || enabled.ValueKind is JsonValueKind.True or JsonValueKind.False) - && (!legend.TryGetProperty("position", out var position) - || position.GetString() is "top_left" or "top_right" or "bottom_left" or "bottom_right") - && IsOptionalNumber(legend, "font_size", 6, 72); - } - - private static bool IsValidTextStyle(JsonElement parent, string name) => - !parent.TryGetProperty(name, out var style) - || style.ValueKind == JsonValueKind.Object - && HasOnlyProperties(style, "font_size") - && IsOptionalNumber(style, "font_size", 6, 72); - - private static bool IsValidCanvas(JsonElement plot) => - !plot.TryGetProperty("canvas", out var canvas) - || canvas.ValueKind == JsonValueKind.Object - && HasOnlyProperties(canvas, "width_mm", "height_mm") - && canvas.TryGetProperty("width_mm", out _) - && canvas.TryGetProperty("height_mm", out _) - && IsOptionalNumber(canvas, "width_mm", 40, 1000) - && IsOptionalNumber(canvas, "height_mm", 40, 1000); - - private static bool IsValidSeriesStyle(JsonElement series) - { - if (!series.TryGetProperty("style", out var style)) return true; - return style.ValueKind == JsonValueKind.Object - && HasOnlyProperties( - style, "color", "line_width", "line_style", "symbol", "symbol_size", - "transparency") - && (!style.TryGetProperty("color", out var color) - || color.ValueKind == JsonValueKind.String && IsHexColor(color.GetString()!)) - && (!style.TryGetProperty("line_style", out var lineStyle) - || lineStyle.GetString() is "solid" or "dash" or "dot" or "dash_dot" or "dash_dot_dot") - && (!style.TryGetProperty("symbol", out var symbol) - || symbol.GetString() is "circle" or "square" or "triangle_up" or "diamond" or "cross" or "plus") - && IsOptionalNumber(style, "line_width", 0.1, 20) - && IsOptionalNumber(style, "symbol_size", 1, 100) - && (!style.TryGetProperty("transparency", out var transparency) - || transparency.TryGetInt32(out var value) && value is >= 0 and <= 100); - } - - private static bool IsOptionalNumber( - JsonElement parent, string name, double minimum = double.NegativeInfinity, - double maximum = double.PositiveInfinity, bool inclusiveMinimum = true) - { - if (!parent.TryGetProperty(name, out var value)) return true; - if (value.ValueKind != JsonValueKind.Number || !value.TryGetDouble(out var number) - || double.IsNaN(number) || double.IsInfinity(number)) - { - return false; - } - return (inclusiveMinimum ? number >= minimum : number > minimum) && number <= maximum; - } - - private static bool IsHexColor(string value) => - value.Length == 7 && value[0] == '#' - && value.Skip(1).All(character => - character is >= '0' and <= '9' - || character is >= 'a' and <= 'f' - || character is >= 'A' and <= 'F'); - private static bool IsValidInputTransfers(JsonElement transfers, Guid jobId) { if (transfers.ValueKind != JsonValueKind.Array diff --git a/windows-node/Zcbot.WindowsNode/NodeAdapters.cs b/windows-node/Zcbot.WindowsNode/NodeAdapters.cs index 08ac36a..814f096 100644 --- a/windows-node/Zcbot.WindowsNode/NodeAdapters.cs +++ b/windows-node/Zcbot.WindowsNode/NodeAdapters.cs @@ -1,4 +1,6 @@ +using Json.Schema; using System.Text.Json; +using System.Text.RegularExpressions; namespace Zcbot.WindowsNode; @@ -16,54 +18,193 @@ internal interface INodeAdapter void Cancel(Guid jobId); } -internal sealed class OriginPlotNodeAdapter(JobInboxStore inbox) : INodeAdapter -{ - internal const string CurrentAdapterVersion = "0.5.0"; - internal const string ContractFilename = "origin.plot.v2.json"; - private static readonly NodeAdapterContract Contract = - NodeAdapterContract.Load(ContractFilename, CurrentAdapterVersion); - private readonly OriginWorkerRunner runner = new(inbox); - - internal static string CapabilityName => Contract.Capability; - - public string Capability => Contract.Capability; - public string DisplayName => Contract.DisplayName; - public string AdapterVersion => CurrentAdapterVersion; - public IReadOnlyList Features => Contract.Features; - public bool HasActiveJobs => runner.HasActiveJobs; - public string RunningDetail => "Origin 正在生成图形"; - public AdapterRuntimeStatus DetectRuntime() => OriginRuntimeProbe.Detect(); - public bool ValidateRequest(JsonElement request) => - JobInboxStore.IsValidOriginRequest(request, Features); - public Task RunAsync(RecoverableJob job) => runner.RunAsync(job); - public void Cancel(Guid jobId) => runner.Cancel(jobId); -} +internal sealed record AdapterManifest( + string Capability, + string AdapterVersion, + string Runtime, + string? RuntimeId, + string Entrypoint, + string Contract, + string RunningDetail); internal sealed record NodeAdapterContract( - string Capability, string DisplayName, IReadOnlyList Features) + string Capability, + string DisplayName, + IReadOnlyList Features, + JsonSchema RequestSchema); + +internal sealed record AdapterDescriptor( + string DirectoryPath, + string EntrypointPath, + AdapterManifest Manifest, + NodeAdapterContract Contract) { - internal static NodeAdapterContract Load(string filename, string adapterVersion) + internal static AdapterDescriptor Load(string directory) { - var path = Path.GetFullPath( - Path.Combine(AppContext.BaseDirectory, "software-contracts", filename)); - using var document = JsonDocument.Parse(File.ReadAllBytes(path)); - var root = document.RootElement; - var capability = root.GetProperty("capability").GetString() - ?? throw new InvalidDataException("Adapter contract capability is missing."); - var displayName = root.GetProperty("display_name").GetString() - ?? throw new InvalidDataException("Adapter contract display name is missing."); - var current = ParseVersion(adapterVersion); - var features = root.GetProperty("features").EnumerateObject() - .Where(item => ParseVersion(item.Value.GetString() ?? "0.0.0") <= current) + var root = Path.GetFullPath(directory); + var manifestPath = ResolveFile(root, "adapter.json"); + using var manifestDocument = JsonDocument.Parse(File.ReadAllBytes(manifestPath)); + var manifestRoot = manifestDocument.RootElement; + RequireOnlyProperties( + manifestRoot, "capability", "adapter_version", "runtime", "runtime_id", + "entrypoint", "contract", "running_detail"); + var manifest = new AdapterManifest( + RequiredString(manifestRoot, "capability"), + RequiredVersion(manifestRoot, "adapter_version"), + RequiredString(manifestRoot, "runtime"), + OptionalString(manifestRoot, "runtime_id"), + RequiredString(manifestRoot, "entrypoint"), + RequiredString(manifestRoot, "contract"), + RequiredString(manifestRoot, "running_detail")); + if (!Regex.IsMatch(manifest.Capability, "^[a-z][a-z0-9_.-]+@v[1-9][0-9]*$")) + { + throw new InvalidDataException("Adapter capability is invalid."); + } + if (manifest.Runtime is not ("python" or "executable")) + { + throw new InvalidDataException("Adapter runtime must be python or executable."); + } + if (manifest.Runtime == "python" + && (manifest.RuntimeId is null + || !Regex.IsMatch(manifest.RuntimeId, "^[a-z][a-z0-9_-]{0,31}$"))) + { + throw new InvalidDataException("Python adapter runtime_id is invalid."); + } + if (manifest.Runtime == "executable" && manifest.RuntimeId is not null) + { + throw new InvalidDataException("Executable adapter must not declare runtime_id."); + } + + var contractPath = ResolveFile(root, manifest.Contract); + using var contractDocument = JsonDocument.Parse(File.ReadAllBytes(contractPath)); + var contractRoot = contractDocument.RootElement; + var capability = RequiredString(contractRoot, "capability"); + if (!capability.Equals(manifest.Capability, StringComparison.Ordinal)) + { + throw new InvalidDataException("Adapter manifest capability does not match its contract."); + } + var displayName = RequiredString(contractRoot, "display_name"); + var currentVersion = ParseVersion(manifest.AdapterVersion); + var features = contractRoot.GetProperty("features").EnumerateObject() + .Where(item => ParseVersion(item.Value.GetString() ?? "0.0.0") <= currentVersion) .Select(item => item.Name) .ToArray(); - return new NodeAdapterContract(capability, displayName, features); + var schema = JsonSchema.Build(contractRoot.GetProperty("request_schema")); + var entrypointPath = ResolveFile(root, manifest.Entrypoint); + if ((manifest.Runtime == "python" + && !Path.GetExtension(entrypointPath).Equals(".py", StringComparison.OrdinalIgnoreCase)) + || (manifest.Runtime == "executable" + && !Path.GetExtension(entrypointPath).Equals(".exe", StringComparison.OrdinalIgnoreCase))) + { + throw new InvalidDataException("Adapter entrypoint extension does not match its runtime."); + } + return new AdapterDescriptor( + root, + entrypointPath, + manifest, + new NodeAdapterContract(capability, displayName, features, schema)); } + private static string ResolveFile(string root, string relativePath) + { + if (string.IsNullOrWhiteSpace(relativePath) || Path.IsPathFullyQualified(relativePath)) + { + throw new InvalidDataException("Adapter file path must be relative."); + } + var resolved = Path.GetFullPath(Path.Combine(root, relativePath)); + if (!resolved.StartsWith(root + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase) + || !File.Exists(resolved)) + { + throw new InvalidDataException("Adapter file is missing or outside its directory."); + } + return resolved; + } + + private static void RequireOnlyProperties(JsonElement value, params string[] names) + { + if (value.ValueKind != JsonValueKind.Object) + { + throw new InvalidDataException("Adapter manifest must be an object."); + } + var allowed = names.ToHashSet(StringComparer.Ordinal); + if (value.EnumerateObject().Any(item => !allowed.Contains(item.Name))) + { + throw new InvalidDataException("Adapter manifest contains unknown properties."); + } + } + + private static string RequiredString(JsonElement value, string name) => + value.TryGetProperty(name, out var property) + && property.ValueKind == JsonValueKind.String + && !string.IsNullOrWhiteSpace(property.GetString()) + ? property.GetString()! + : throw new InvalidDataException($"Adapter property {name} is missing."); + + private static string RequiredVersion(JsonElement value, string name) + { + var version = RequiredString(value, name); + if (!Regex.IsMatch(version, "^[0-9]+\\.[0-9]+\\.[0-9]+$") + || !Version.TryParse(version, out _)) + { + throw new InvalidDataException($"Adapter property {name} is not a semantic version."); + } + return version; + } + + private static string? OptionalString(JsonElement value, string name) => + !value.TryGetProperty(name, out var property) || property.ValueKind == JsonValueKind.Null + ? null + : property.ValueKind == JsonValueKind.String + ? property.GetString() + : throw new InvalidDataException($"Adapter property {name} must be a string."); + private static Version ParseVersion(string value) => Version.TryParse(value, out var version) ? version : new Version(0, 0, 0); } +internal sealed class ProcessNodeAdapter : INodeAdapter +{ + private static readonly EvaluationOptions SchemaOptions = new() + { + OutputFormat = OutputFormat.Flag, + RequireFormatValidation = true, + }; + private readonly AdapterProcessRunner runner; + private AdapterRuntimeStatus? cachedRuntime; + private DateTimeOffset runtimeCheckedAt; + + internal ProcessNodeAdapter(AdapterDescriptor descriptor, JobInboxStore inbox) + { + Descriptor = descriptor; + runner = new AdapterProcessRunner(descriptor, inbox); + } + + internal AdapterDescriptor Descriptor { get; } + public string Capability => Descriptor.Contract.Capability; + public string DisplayName => Descriptor.Contract.DisplayName; + public string AdapterVersion => Descriptor.Manifest.AdapterVersion; + public IReadOnlyList Features => Descriptor.Contract.Features; + public bool HasActiveJobs => runner.HasActiveJobs; + public string RunningDetail => Descriptor.Manifest.RunningDetail; + + public AdapterRuntimeStatus DetectRuntime() + { + if (cachedRuntime is null + || DateTimeOffset.UtcNow - runtimeCheckedAt > TimeSpan.FromSeconds(30)) + { + cachedRuntime = runner.Probe(); + runtimeCheckedAt = DateTimeOffset.UtcNow; + } + return cachedRuntime; + } + + public bool ValidateRequest(JsonElement request) => + Descriptor.Contract.RequestSchema.Evaluate(request, SchemaOptions).IsValid; + + public Task RunAsync(RecoverableJob job) => runner.RunAsync(job); + public void Cancel(Guid jobId) => runner.Cancel(jobId); +} + internal sealed class NodeAdapterRegistry { private readonly IReadOnlyDictionary adapters; @@ -73,21 +214,30 @@ internal sealed class NodeAdapterRegistry adapters = values.ToDictionary(item => item.Capability, StringComparer.Ordinal); } + internal static string AdapterRoot => + Path.GetFullPath(Path.Combine(AppContext.BaseDirectory, "adapters")); + internal static NodeAdapterRegistry CreateDefault(JobInboxStore inbox) => - new([new OriginPlotNodeAdapter(inbox)]); + new(DiscoverDescriptors().Select(item => new ProcessNodeAdapter(item, inbox))); - internal static IReadOnlyList InstalledContracts { get; } = - [ - NodeAdapterContract.Load( - OriginPlotNodeAdapter.ContractFilename, - OriginPlotNodeAdapter.CurrentAdapterVersion), - ]; + internal static IReadOnlyList InstalledContracts => + DiscoverDescriptors().Select(item => item.Contract).ToArray(); - internal static IReadOnlyList InstalledCapabilities { get; } = + internal static IReadOnlyList InstalledCapabilities => InstalledContracts.Select(item => item.Capability).ToArray(); internal IReadOnlyCollection All => adapters.Values.ToArray(); internal INodeAdapter? Find(string capability) => adapters.TryGetValue(capability, out var adapter) ? adapter : null; + + private static IReadOnlyList DiscoverDescriptors() + { + if (!Directory.Exists(AdapterRoot)) return []; + return Directory.EnumerateDirectories(AdapterRoot) + .Where(path => File.Exists(Path.Combine(path, "adapter.json"))) + .OrderBy(path => path, StringComparer.OrdinalIgnoreCase) + .Select(AdapterDescriptor.Load) + .ToArray(); + } } diff --git a/windows-node/Zcbot.WindowsNode/NodeConnectionLoop.cs b/windows-node/Zcbot.WindowsNode/NodeConnectionLoop.cs index b19182f..84036e3 100644 --- a/windows-node/Zcbot.WindowsNode/NodeConnectionLoop.cs +++ b/windows-node/Zcbot.WindowsNode/NodeConnectionLoop.cs @@ -483,7 +483,6 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action? private object RuntimePayload() { var root = Path.GetPathRoot(Environment.SystemDirectory) ?? "C:\\"; - var origin = OriginRuntimeProbe.Detect(); var capabilityRuntime = adapters.All .Where(item => config.Capabilities.Contains(item.Capability, StringComparer.Ordinal)) .ToDictionary( @@ -503,7 +502,11 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action? detail = runtime.Detail, }; }); - var originAdapter = adapters.Find(OriginPlotNodeAdapter.CapabilityName); + // Keep the legacy top-level Origin payload during its deprecation window. + var originAdapter = adapters.Find("origin.plot@v2"); + var origin = originAdapter?.DetectRuntime() + ?? new AdapterRuntimeStatus( + "OriginPro", null, "0.0.0", "unavailable", "Origin adapter is not installed."); return new { install_id = config.InstallId, diff --git a/windows-node/Zcbot.WindowsNode/OriginRuntimeProbe.cs b/windows-node/Zcbot.WindowsNode/OriginRuntimeProbe.cs deleted file mode 100644 index 535f0c1..0000000 --- a/windows-node/Zcbot.WindowsNode/OriginRuntimeProbe.cs +++ /dev/null @@ -1,102 +0,0 @@ -using Microsoft.Win32; -using System.Security; - -namespace Zcbot.WindowsNode; - -internal sealed record AdapterRuntimeStatus( - string Software, - string? SoftwareVersion, - string AdapterVersion, - string Health, - string Detail); - -internal static class OriginRuntimeProbe -{ - private static readonly Lazy Current = new(DetectCore); - private const string AutomationProgId = @"Origin.ApplicationSI\CLSID"; - - internal static AdapterRuntimeStatus Detect() => Current.Value; - - private static AdapterRuntimeStatus DetectCore() - { - try - { - var version = FindInstalledVersion(); - using var automationKey = Registry.ClassesRoot.OpenSubKey(AutomationProgId); - var automationRegistered = automationKey is not null; - if (version is null && !automationRegistered) - { - return Status(null, "unavailable", "未检测到 Origin/OriginPro 安装"); - } - if (!automationRegistered) - { - return Status(version, "unavailable", "已检测到 Origin,但 COM 自动化组件未注册"); - } - if (!Environment.UserInteractive) - { - return Status(version, "unavailable", "Origin 需要交互式 Windows 桌面会话"); - } - var interpreter = OriginWorkerRuntime.ResolveInterpreter(); - if (interpreter is null) - { - return Status( - version, - "unavailable", - "Origin 可用,但固定 Python 运行时缺失;请配置 ZCBOT_ORIGIN_PYTHON"); - } - return Status(version, "ready", $"Origin COM 与固定 Python 运行时可用({interpreter})"); - } - catch (Exception exception) when ( - exception is SecurityException or UnauthorizedAccessException or IOException) - { - return Status(null, "unavailable", $"Origin 运行时探测失败:{exception.Message}"); - } - } - - private static AdapterRuntimeStatus Status(string? version, string health, string detail) => - new("OriginPro", version, OriginPlotNodeAdapter.CurrentAdapterVersion, health, detail); - - private static string? FindInstalledVersion() - { - var candidates = new List(); - foreach (var hive in new[] { RegistryHive.LocalMachine, RegistryHive.CurrentUser }) - { - foreach (var view in new[] { RegistryView.Registry64, RegistryView.Registry32 }) - { - using var baseKey = RegistryKey.OpenBaseKey(hive, view); - using var uninstall = baseKey.OpenSubKey( - @"SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall"); - if (uninstall is null) - { - continue; - } - foreach (var keyName in uninstall.GetSubKeyNames()) - { - using var product = uninstall.OpenSubKey(keyName); - var name = product?.GetValue("DisplayName") as string; - var publisher = product?.GetValue("Publisher") as string; - if (!IsOriginProduct(name, publisher)) - { - continue; - } - var version = product?.GetValue("DisplayVersion") as string; - if (!string.IsNullOrWhiteSpace(version)) - { - candidates.Add(version.Trim()); - } - } - } - } - return candidates.OrderByDescending(ParseVersion).ThenByDescending(x => x).FirstOrDefault(); - } - - private static bool IsOriginProduct(string? name, string? publisher) => - !string.IsNullOrWhiteSpace(name) - && (name.Equals("Origin", StringComparison.OrdinalIgnoreCase) - || name.StartsWith("Origin ", StringComparison.OrdinalIgnoreCase) - || name.StartsWith("OriginPro", StringComparison.OrdinalIgnoreCase)) - && (publisher?.Contains("OriginLab", StringComparison.OrdinalIgnoreCase) ?? false); - - private static Version ParseVersion(string value) => - Version.TryParse(value, out var version) ? version : new Version(0, 0); -} diff --git a/windows-node/Zcbot.WindowsNode/OriginWorkerRunner.cs b/windows-node/Zcbot.WindowsNode/OriginWorkerRunner.cs deleted file mode 100644 index 8162579..0000000 --- a/windows-node/Zcbot.WindowsNode/OriginWorkerRunner.cs +++ /dev/null @@ -1,166 +0,0 @@ -using System.Collections.Concurrent; -using System.Diagnostics; -using System.Text; -using System.Text.Json; - -namespace Zcbot.WindowsNode; - -internal sealed class OriginWorkerRunner(JobInboxStore inbox) -{ - private static readonly TimeSpan WorkerTimeout = TimeSpan.FromMinutes(30); - private readonly ConcurrentDictionary active = new(); - private readonly ConcurrentDictionary cancellations = new(); - - internal bool HasActiveJobs => !active.IsEmpty; - - internal Task RunAsync(RecoverableJob job) => - active.GetOrAdd(job.JobId, _ => RunOnceAsync(job, CancellationFor(job.JobId).Token)); - - internal void Cancel(Guid jobId) - { - CancellationFor(jobId).Cancel(); - } - - private CancellationTokenSource CancellationFor(Guid jobId) => - cancellations.GetOrAdd(jobId, _ => new CancellationTokenSource()); - - private async Task RunOnceAsync(RecoverableJob job, CancellationToken cancellationToken) - { - try - { - var paths = NodePaths.ForCurrentMachine(); - var jobDirectory = Path.Combine(paths.JobsDirectory, job.JobId.ToString("D")); - var terminalPath = Path.Combine(jobDirectory, "terminal.json"); - if (File.Exists(terminalPath)) return; - - var markerPath = Path.Combine(jobDirectory, "worker-started.json"); - if (File.Exists(markerPath)) - { - inbox.WriteTerminal( - job, - "failed", - "NODE_RESTARTED_DURING_JOB", - "The node restarted after Origin execution began and cannot prove the prior worker state."); - return; - } - - var interpreter = OriginWorkerRuntime.ResolveInterpreter() - ?? throw new InvalidOperationException("The fixed Origin Python interpreter is unavailable."); - var workerScript = Path.GetFullPath( - Path.Combine(AppContext.BaseDirectory, "origin-worker", "worker.py")); - if (!File.Exists(workerScript)) - { - throw new FileNotFoundException("The fixed Origin worker script is missing.", workerScript); - } - WriteMarker(markerPath, interpreter, workerScript); - - var startInfo = new ProcessStartInfo - { - FileName = interpreter, - WorkingDirectory = jobDirectory, - UseShellExecute = false, - CreateNoWindow = true, - RedirectStandardOutput = true, - RedirectStandardError = true, - StandardOutputEncoding = Encoding.UTF8, - StandardErrorEncoding = Encoding.UTF8, - }; - startInfo.ArgumentList.Add(workerScript); - startInfo.ArgumentList.Add(jobDirectory); - using var process = Process.Start(startInfo) - ?? throw new InvalidOperationException("The fixed Origin worker did not start."); - var stdout = process.StandardOutput.ReadToEndAsync(); - var stderr = process.StandardError.ReadToEndAsync(); - using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); - timeout.CancelAfter(WorkerTimeout); - try - { - await process.WaitForExitAsync(timeout.Token); - } - catch (OperationCanceledException) - { - process.Kill(entireProcessTree: true); - if (cancellationToken.IsCancellationRequested) - { - inbox.WriteTerminal(job, "cancelled", "USER_CANCELLED", "Cancelled by user."); - } - else - { - inbox.WriteTerminal(job, "failed", "ORIGIN_WORKER_TIMEOUT", "Origin worker exceeded 30 minutes."); - } - return; - } - var output = await stdout; - var error = await stderr; - WriteDiagnostic(jobDirectory, output, error, process.ExitCode); - if (!File.Exists(terminalPath)) - { - inbox.WriteTerminal( - job, - "failed", - "ORIGIN_WORKER_NO_TERMINAL", - $"Origin worker exited with code {process.ExitCode} without terminal.json."); - } - } - catch (Exception exception) when ( - exception is IOException - or JsonException - or UnauthorizedAccessException - or InvalidOperationException) - { - inbox.WriteTerminal(job, "failed", "ORIGIN_WORKER_START_FAILED", exception.Message[..Math.Min(500, exception.Message.Length)]); - } - finally - { - active.TryRemove(job.JobId, out _); - if (cancellations.TryRemove(job.JobId, out var cancellation)) cancellation.Dispose(); - } - } - - private static void WriteMarker(string path, string interpreter, string workerScript) - { - var value = JsonSerializer.SerializeToUtf8Bytes(new - { - started_at = DateTimeOffset.UtcNow, - node_pid = Environment.ProcessId, - interpreter, - worker_script = workerScript, - }); - using var stream = new FileStream( - path, FileMode.CreateNew, FileAccess.Write, FileShare.None, - bufferSize: 4096, FileOptions.WriteThrough); - stream.Write(value); - stream.Flush(flushToDisk: true); - } - - private static void WriteDiagnostic(string jobDirectory, string output, string error, int exitCode) - { - var logs = Path.Combine(jobDirectory, "logs"); - Directory.CreateDirectory(logs); - var value = JsonSerializer.Serialize(new - { - exit_code = exitCode, - stdout = output[..Math.Min(output.Length, 16 * 1024)], - stderr = error[..Math.Min(error.Length, 16 * 1024)], - }); - File.WriteAllText(Path.Combine(logs, "worker-process.json"), value, Encoding.UTF8); - } -} - -internal static class OriginWorkerRuntime -{ - internal static string? ResolveInterpreter() - { - var paths = NodePaths.ForCurrentMachine(); - var configured = Environment.GetEnvironmentVariable("ZCBOT_ORIGIN_PYTHON"); - var candidate = string.IsNullOrWhiteSpace(configured) - ? Path.Combine(paths.RootDirectory, "runtimes", "origin", "Scripts", "python.exe") - : configured; - if (!Path.IsPathFullyQualified(candidate)) return null; - var resolved = Path.GetFullPath(candidate); - return File.Exists(resolved) - && Path.GetFileName(resolved).Equals("python.exe", StringComparison.OrdinalIgnoreCase) - ? resolved - : null; - } -} diff --git a/windows-node/Zcbot.WindowsNode/Zcbot.WindowsNode.csproj b/windows-node/Zcbot.WindowsNode/Zcbot.WindowsNode.csproj index 61a1d91..e589cfe 100644 --- a/windows-node/Zcbot.WindowsNode/Zcbot.WindowsNode.csproj +++ b/windows-node/Zcbot.WindowsNode/Zcbot.WindowsNode.csproj @@ -9,21 +9,29 @@ true Zcbot.WindowsNode Zcbot.WindowsNode - 0.1.0 + 0.2.0 - - software-contracts\%(Filename)%(Extension) + + + + + adapters\origin.plot@v2\adapter.json + PreserveNewest + PreserveNewest + + + adapters\origin.plot@v2\origin.plot.v2.json PreserveNewest PreserveNewest - origin-worker\worker.py + adapters\origin.plot@v2\worker.py PreserveNewest PreserveNewest - origin-worker\requirements.txt + adapters\origin.plot@v2\requirements.txt PreserveNewest PreserveNewest diff --git a/windows-node/Zcbot.WindowsNode/packages.lock.json b/windows-node/Zcbot.WindowsNode/packages.lock.json index 83dc7a3..8b68987 100644 --- a/windows-node/Zcbot.WindowsNode/packages.lock.json +++ b/windows-node/Zcbot.WindowsNode/packages.lock.json @@ -1,7 +1,36 @@ { "version": 1, "dependencies": { - "net10.0-windows7.0": {}, + "net10.0-windows7.0": { + "JsonSchema.Net": { + "type": "Direct", + "requested": "[9.4.0, )", + "resolved": "9.4.0", + "contentHash": "muE4nPuzbD9x5XA1mkXJNqX4mhz47oF3EY8qFLY1pyUY7lhXFKq65t/dSKIHeNSLBV17zwZuq24MD1em90rMNg==", + "dependencies": { + "JsonPointer.Net": "7.0.2" + } + }, + "Humanizer.Core": { + "type": "Transitive", + "resolved": "3.0.10", + "contentHash": "yZIhtw8sYuvsONzQbZxWpR60tMWYHXoo0DL6nyOqSFiU5POjBTSEyWFpTQtJEZuy+oqiYTXKXY/Mjx7KnqIQFw==" + }, + "Json.More.Net": { + "type": "Transitive", + "resolved": "3.0.1", + "contentHash": "fRctF2J2SILYG6wqP21drmeEODmCVkVQ/b3MndDu2fT1swfySyUgq7ePCk+aENGlDcIm05fyfjh9vcuqDEfv3w==" + }, + "JsonPointer.Net": { + "type": "Transitive", + "resolved": "7.0.2", + "contentHash": "oClYHv2ooeRrtPZyC9sb/Za/ie5pGhjTbNHlAyFg4XOCyU+606FdoZKS7UdaWvtsPlrF+U2w0Ja3TRvn5+spyA==", + "dependencies": { + "Humanizer.Core": "3.0.10", + "Json.More.Net": "3.0.1" + } + } + }, "net10.0-windows7.0/win-x64": {} } -} +} \ No newline at end of file diff --git a/windows-node/adapters/origin.plot@v2/adapter.json b/windows-node/adapters/origin.plot@v2/adapter.json new file mode 100644 index 0000000..707829e --- /dev/null +++ b/windows-node/adapters/origin.plot@v2/adapter.json @@ -0,0 +1,9 @@ +{ + "capability": "origin.plot@v2", + "adapter_version": "0.5.0", + "runtime": "python", + "runtime_id": "origin", + "entrypoint": "worker.py", + "contract": "origin.plot.v2.json", + "running_detail": "Origin 正在生成图形" +} diff --git a/windows-node/install-windows-node.bat b/windows-node/install-windows-node.bat index 9390dd0..f3b64fe 100644 --- a/windows-node/install-windows-node.bat +++ b/windows-node/install-windows-node.bat @@ -3,7 +3,7 @@ setlocal EnableExtensions cd /d "%~dp0" set "NODE_EXE=%~dp0Zcbot.WindowsNode.exe" -set "REQUIREMENTS=%~dp0origin-worker\requirements.txt" +set "REQUIREMENTS=%~dp0adapters\origin.plot@v2\requirements.txt" set "RUNTIME_DIR=%ProgramData%\Zcbot\WindowsNode\runtimes\origin" set "RUNTIME_PYTHON=%RUNTIME_DIR%\Scripts\python.exe" set "PYTHON_EXE=" diff --git a/windows-node/origin-worker/worker.py b/windows-node/origin-worker/worker.py index 443dfe3..928ce11 100644 --- a/windows-node/origin-worker/worker.py +++ b/windows-node/origin-worker/worker.py @@ -52,6 +52,67 @@ LEGEND_POSITIONS = { "bottom_left": (700, 7200), "bottom_right": (6800, 7200), } +ADAPTER_VERSION = "0.5.0" + + +def _probe() -> int: + health = "ready" + detail = "Origin COM 与托管 Python 运行时可用" + software_version = None + try: + if sys.platform != "win32": + raise RuntimeError("Origin adapter requires Windows") + import winreg + import originpro + + with winreg.OpenKey(winreg.HKEY_CLASSES_ROOT, r"Origin.ApplicationSI\CLSID"): + pass + originpro_version = version("originpro") + detail = f"Origin COM 与托管 Python 运行时可用(originpro {originpro_version})" + del originpro + except (FileNotFoundError, ImportError, OSError, PackageNotFoundError, RuntimeError) as exc: + health = "unavailable" + detail = str(exc) + print(json.dumps({ + "adapter_version": ADAPTER_VERSION, + "software": "OriginPro", + "software_version": software_version, + "health": health, + "detail": detail, + }, ensure_ascii=False)) + return 0 + + +def _validate_semantics(request: dict[str, Any]) -> None: + input_keys = {item["key"] for item in request["inputs"]} + plot = request["operation"]["plot"] + plot_type = plot["type"] + series = plot["series"] + used_inputs = {item["input"] for item in series} + if used_inputs != input_keys: + raise ValueError("INPUT_BINDINGS_MUST_BE_USED_EXACTLY") + if plot_type == "grouped_column" and len(series) < 2: + raise ValueError("GROUPED_COLUMN_REQUIRES_MULTIPLE_SERIES") + if plot_type in XYZ_PLOT_TYPES and len(series) != 1: + raise ValueError("XYZ_PLOT_REQUIRES_ONE_SERIES") + required_roles = ( + ("x", "y", "z") if plot_type in XYZ_PLOT_TYPES + else ("x", "y", "y_error") if plot_type == "y_error" + else ("x", "y") + ) + for item in series: + if any(role not in item for role in required_roles): + raise ValueError("SERIES_REQUIRED_ROLE_MISSING") + for axis_name in ("x_axis", "y_axis", "z_axis"): + axis = plot.get(axis_name) or {} + minimum = axis.get("minimum") + maximum = axis.get("maximum") + if minimum is not None and maximum is not None and minimum >= maximum: + raise ValueError(f"{axis_name.upper()}_LIMITS_INVALID") + if axis.get("scale") in {"log10", "ln", "log2"} and ( + minimum is not None and minimum <= 0 or maximum is not None and maximum <= 0 + ): + raise ValueError(f"{axis_name.upper()}_LOG_LIMIT_INVALID") def _atomic_json(path: Path, value: Any) -> None: @@ -302,6 +363,7 @@ def run(job_dir: Path) -> list[dict[str, Any]]: job_dir = job_dir.resolve(strict=True) request_record = json.loads((job_dir / "request" / "request.json").read_text(encoding="utf-8")) request = request_record["request"] + _validate_semantics(request) input_specs = request["inputs"] input_files = {item["key"]: _input_file(job_dir, item["key"]) for item in input_specs} input_data = { @@ -425,7 +487,7 @@ def run(job_dir: Path) -> list[dict[str, Any]]: except PackageNotFoundError: originpro_version = "embedded" provenance = { - "adapter_version": "0.5.0", + "adapter_version": ADAPTER_VERSION, "originpro_version": originpro_version, "request_digest": request_record["request_digest"], "inputs": [ @@ -454,6 +516,8 @@ def run(job_dir: Path) -> list[dict[str, Any]]: def main() -> int: + if sys.argv[1:] == ["--probe"]: + return _probe() if len(sys.argv) != 2: print("[ERR] Usage: worker.py ", file=sys.stderr) return 2 diff --git a/windows-node/package-origin-adapter.bat b/windows-node/package-origin-adapter.bat new file mode 100644 index 0000000..7926c53 --- /dev/null +++ b/windows-node/package-origin-adapter.bat @@ -0,0 +1,46 @@ +@echo off +setlocal EnableExtensions EnableDelayedExpansion +cd /d "%~dp0" + +set "OUTPUT_ROOT=%~f1" +if "%~1"=="" set "OUTPUT_ROOT=%~dp0dist" +set "ADAPTER_DIR_NAME=origin.plot@v2" +set "ARCHIVE_NAME=origin.plot@v2-adapter.zip" +set "PACKAGE_DIR=!OUTPUT_ROOT!\!ADAPTER_DIR_NAME!" +set "ARCHIVE_PATH=!OUTPUT_ROOT!\!ARCHIVE_NAME!" +set "HASH_PATH=!ARCHIVE_PATH!.sha256" + +where.exe tar.exe >nul 2>&1 +if errorlevel 1 goto :missing_tool +where.exe certutil.exe >nul 2>&1 +if errorlevel 1 goto :missing_tool +if not exist "!OUTPUT_ROOT!" mkdir "!OUTPUT_ROOT!" +if exist "!PACKAGE_DIR!" rmdir /s /q "!PACKAGE_DIR!" +if exist "!ARCHIVE_PATH!" del /f /q "!ARCHIVE_PATH!" +if exist "!HASH_PATH!" del /f /q "!HASH_PATH!" +mkdir "!PACKAGE_DIR!" + +copy /y "adapters\origin.plot@v2\adapter.json" "!PACKAGE_DIR!\adapter.json" >nul +copy /y "origin-worker\worker.py" "!PACKAGE_DIR!\worker.py" >nul +copy /y "origin-worker\requirements.txt" "!PACKAGE_DIR!\requirements.txt" >nul +copy /y "..\software-contracts\origin.plot.v2.json" "!PACKAGE_DIR!\origin.plot.v2.json" >nul +if errorlevel 1 goto :failed + +tar.exe -a -c -f "!ARCHIVE_PATH!" -C "!OUTPUT_ROOT!" "!ADAPTER_DIR_NAME!" +if errorlevel 1 goto :failed +set "SHA256=" +for /f "tokens=*" %%H in ('certutil.exe -hashfile "!ARCHIVE_PATH!" SHA256 ^| findstr.exe /R /C:"^[0-9A-Fa-f][0-9A-Fa-f ]*[0-9A-Fa-f]$"') do set "SHA256=%%H" +set "SHA256=!SHA256: =!" +if not defined SHA256 goto :failed +>"!HASH_PATH!" echo !SHA256! !ARCHIVE_NAME! +echo [OK] Adapter package: !ARCHIVE_PATH! +echo [OK] SHA256: !SHA256! +exit /b 0 + +:missing_tool +echo [ERR] Windows tar.exe and certutil.exe are required. +exit /b 1 + +:failed +echo [ERR] Origin adapter packaging failed. +exit /b 1 diff --git a/windows-node/package-windows-node.bat b/windows-node/package-windows-node.bat index 893b3dc..f38db66 100644 --- a/windows-node/package-windows-node.bat +++ b/windows-node/package-windows-node.bat @@ -63,9 +63,10 @@ if errorlevel 1 ( for %%F in ( "Zcbot.WindowsNode.exe" "install-windows-node.bat" - "origin-worker\worker.py" - "origin-worker\requirements.txt" - "software-contracts\origin.plot.v2.json" + "adapters\origin.plot@v2\adapter.json" + "adapters\origin.plot@v2\worker.py" + "adapters\origin.plot@v2\requirements.txt" + "adapters\origin.plot@v2\origin.plot.v2.json" ) do ( if not exist "!PUBLISH_DIR!\%%~F" ( echo [ERR] Published package is incomplete: %%~F