feat(origin): 新增受控科研分析能力
This commit is contained in:
parent
631a635eab
commit
3068da491f
|
|
@ -8,6 +8,8 @@
|
|||
|
||||
## Unreleased
|
||||
|
||||
- Origin 新增受控科研分析，可进行数据体检、归一化、平滑、求导、积分和线性拟合，并交付结果表、诊断信息及可继续检查的 Origin 工程。
|
||||
|
||||
- 平台托管的文献、内部材料库和 Materials Project 来源现在使用来源明确的工具，并可在单个来源不可用时继续使用其他来源；访问凭据不进入任务沙箱。
|
||||
|
||||
## 0.68.0 — 2026-08-24
|
||||
|
|
|
|||
|
|
@ -483,6 +483,8 @@ Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/software-nodes/c
|
|||
|
||||
第四阶段落地固定 Origin Worker：Node 仅从管理员安装的固定 Python 运行时启动随程序发布的 `worker.py`，参数只有本机 job 目录；请求不能指定脚本、解释器或文件路径。Worker 使用 `originpro` 生成 OPJU、PNG、SVG、PDF、plot spec 和 provenance，校验产物签名并原子写入终态。`origin.plot@v2` 保持单一外层契约，旧单图 `series[]` 以 `x/y/z/y_error` 统一表达数据角色，再按 `plot.type` 判别必需角色；当前覆盖折线、散点、线点、柱/条形、分组柱形、Y 误差棒、等高线、三维曲面、三元图和规则网格热图，并以可选的 `canvas`、轴排版、图例、标题和 series style 统一表达出版级尺寸与样式。新增复合图不继续枚举 `dual_axis_line` 等组合类型，而使用 `multi_panel` feature：`panels[]` 显式描述 1–4 个二维 panel，每条系列用 `kind` 选择基础二维图型、用 `y_axis` 绑定左右轴，并可提供对称 `x_error/y_error`；`layout` 只允许 1×1、1×2、2×1、2×2 四种受控布局。Worker 从固定基础二维图创建受控原生图层网格及 right-Y 图层，不依赖带隐式主题分组的 panel 模板，也不接受请求传入模板名；Origin 不支持同一 3D 图窗多图层，因此 contour、surface、ternary 和 heatmap 不混入本阶段 panel。旧 XY 请求原样兼容。进程内 pipeline 按 job 去重，并脱离单次 WebSocket 的取消令牌运行；连接中断只延迟状态/终态上报。Node 进程若在 Worker 启动后重启，则保守失败而不重复驱动 Origin，避免无法证明的双执行。
|
||||
|
||||
Origin 科研分析以独立 `origin.analysis@v1` capability 承载，不把数据处理、拟合和统计继续塞入 `origin.plot@v2`。首批白名单 operation 为 `data_check/normalize/smooth/differentiate/integrate/linear_fit`：合同绑定单个 CSV/XLSX/JSON artifact 和显式 X/Y 列，固定 Worker 使用版本化数值算法生成结果表与机器可读诊断，再把原始数据、结果和诊断写入可编辑 OPJU；线性拟合额外调用 Origin 内置 `LinearFit` 生成原生报告。标准化数值结果以独立 oracle 为发布依据，Origin 报告用于工程内继续检查和编辑，避免把正确性绑定到本机语言相关的层级报告表解析。输出固定包含 OPJU、CSV、diagnostics、analysis spec 与 provenance，可选 XLSX；分析 capability 为无状态一次性 Job，结果 CSV/XLSX 可登记为后续 plot Job 输入，不与绘图混在同一请求或 Workspace 事务中。`data_check` 命中请求指定的问题时返回 `diagnostics.passed=false` 而非让 Job 失败，使 Agent 能先体检再决定是否占用后续分析资源。请求仍不能提供 Python、LabTalk、函数表达式、模板或路径；两份 Origin adapter 共享同一受管 `origin` runtime，并由整机单执行槽保守串行。
|
||||
|
||||
Recipe 是专业软件的声明式目标状态，不是任意脚本或逐次鼠标命令。`origin.plot@v2` 以新增 `plot.type=recipe + recipe_version=1` 复用已验证的 `layout/panels/series/axis/legend` 组合结构；旧 `multi_panel` 与 Recipe 进入同一个受控执行器，单图和特殊统计图继续保留兼容入口。模型可以组合契约允许的原子图形能力，但不能提供 Python、LabTalk、解释器、模板名或文件路径。其他软件可采用同一通用 Job 外壳和各自的声明式 Recipe，不建设跨软件万能 DSL；API/SDK executor 优先，未来的 UI Automation 或 Computer Use 仅作为 adapter 内部执行后端，不改变云端 Recipe。
|
||||
|
||||
Blender 入口为 `blender.scene.author@v3`，唯一场景类型是 `recipe`；它是 Blender 专属的声明式静态场景目标状态，不是跨软件万能 DSL，也不是 Python、逐步按键或任意节点脚本。Recipe v1 组合带稳定 ID、父子层级和集合归属的 geometry/generator/text/empty 对象：geometry 覆盖 box/cylinder/cone/sphere/torus/plane、二维轮廓拉伸、轴向旋转、路径扫掠和有壁厚直管，受控 modifier 覆盖 boolean/bevel/mirror/array/solidify，并统一声明材质、变换、灯光、世界背景和 1–8 个相机视图；`rotary_kiln` 只是 generator 组件，可与其他对象处于同一场景，不再占据顶层 capability。adapter 1.0.0 把合同校验后的 Recipe 编译为固定 `bpy` 调用，限制请求 256 KiB、200 个声明对象、展开后 1000 个 Blender 对象、100 万 mesh polygon、材质/灯光/视图和修改器数量；Worker 还机械校验 ID 唯一性、引用、父子关系、布尔目标、管壁和请求预览的一致性。输出固定包含 `.blend`、首张 PNG、场景清单和溯源，可再选择最多七张 PNG 与 GLB；预览用 `preview_01`–`preview_08` 稳定槽位，视图业务名称记录在 manifest。
|
||||
|
|
|
|||
|
|
@ -20,6 +20,8 @@
|
|||
---
|
||||
## 已完成关键能力
|
||||
|
||||
- **08-26 / Unreleased / Origin 科研分析 MVP**：新增非默认的 `origin.analysis@v1` / adapter 0.1.0，与稳定的 `origin.plot@v2` 分离；首批提供数据体检、最大值/面积/参考点归一化、固定局部多项式平滑、一/二阶导数、区间/累计梯形积分和带 95% 置信区间的线性拟合，固定输出可编辑 OPJU、CSV、诊断、分析规格与溯源，可选 XLSX，线性拟合工程同时保留 Origin 原生报告。六个合成真值用例在 Origin 2024 / originpro 1.1.15 / numpy 2.2.6 上完成生产 Worker、数值 oracle、OPJU 重开和进程释放门；前三例来自组合执行的逐例通过记录，后三例分别通过，目标机 COM 完整退出约需 2 分钟。Origin/合同/Job/Node 专项 159 项 unittest、Release .NET build、完整 Node publish 与独立 adapter ZIP 校验通过，未连接或写入数据库。
|
||||
|
||||
- **08-26 / Unreleased / platform_sources bounded context**：将 paper_server、内部材料库和 Materials Project 收敛为独立顶层包，以显式可信 Provider 列表统一可用性与生命周期，单来源配置/装配失败不阻断其他来源；`core/tool_registry.py` 只保留一个构建入口，与用户连接 `external_systems` 完全独立。模型工具统一改为来源明确的 `paper_server_*`、`materials_library_*`、`materials_project_*`，保留材料库批量检索、MP 专用 SDK/CIF/entries 语义及 paper_server 安全下载边界；部署环境变量仍是事实源，无数据库迁移、无版本提升。
|
||||
|
||||
- **08-26 / Unreleased / paper_server 宿主侧受控访问**：新增按配置注册的 `paper_server_search` / `paper_server_get` / `paper_server_fetch` typed tools，`publication_type` 直接映射后端 `type` 过滤且保留开放原始类型；API Key 从 `run_python` 和 Docker sandbox 透传中移除，下载增加同源 URL、100 MiB 上限和原子落盘约束。literature / brief 指引、smoke、RUN、Skill 清单及回归测试同步迁移，线上只读 `type=book` 验证通过，不提升版本号。
|
||||
|
|
|
|||
6
RUN.md
6
RUN.md
|
|
@ -1160,6 +1160,12 @@ Web 用户登录后，文件栏 Job 中心会聚合本人最近任务。活动
|
|||
|
||||
Origin adapter 1.1.0 起随包提供固定真机 QA harness，但暂不接入 Node UI。在专用测试节点确认没有活动 Job 后，从 PowerShell 使用 Origin 卡片已安装的受管解释器运行：`<data_root>\runtimes\origin\Scripts\python.exe <node_program>\adapters\origin.plot@v2\acceptance.py --work-root <new_empty_parent>\origin-acceptance-YYYYMMDD`。`--work-root` 必须指向尚不存在的新目录；adapter 1.3.0 默认依次执行 annotations、recipe_2x2、heatmap、surface_3d、stacked、stacked_line、violin、band，也可重复传入 `--case <name>` 只跑指定用例。每个用例先通过生产 Worker 子进程生成 OPJU/PNG/SVG/PDF，再由独立 Origin 会话重开工程，核对图层、系列、工作表/矩阵和数值 oracle，复导三种图件，并等待 Origin 进程退出；必要时可用 `--release-wait <seconds>` 调整释放等待时间。最终报告位于 `<work-root>\acceptance-report.json`。输入均为脚本生成的合成数据，但 PNG/SVG/PDF 仍需人工完成视觉 gate 后才能判定目标机器通过。
|
||||
|
||||
`origin.analysis@v1` 是与绘图分离的无状态科研分析入口，首批 operation 为 `data_check`、`normalize`、`smooth`、`differentiate`、`integrate`、`linear_fit`。请求只绑定一个 CSV/XLSX/JSON artifact；除 `data_check` 可只检查整表外，其余操作必须显式提供 `input/x/y`。数据体检通过 `parameters.fail_on` 指定需要阻断后续流程的问题，命中时 Job 仍成功并在 `diagnostics.json` 返回 `passed=false`；调用方应读取诊断后再决定是否提交分析。归一化的 `max_abs` 使用最大绝对值，`area` 使用有符号梯形积分，`reference` 在数据范围内按 X 线性插值；平滑固定为完整奇数窗口的局部多项式，求导支持一/二阶且接受非等间距 X，积分输出累计值及可选 `from/to` 区间值，线性拟合输出参数、95% t 置信区间、R²、RMSE、拟合值和残差，并在 OPJU 内保留 Origin 原生线性拟合报告。所有数值操作拒绝缺失、非有限、重复或非严格递增的 X/Y 数据，不做隐式排序或删行。
|
||||
|
||||
分析任务固定发布 `analysis.opju`、`result-table.csv` 和 `diagnostics.json`，隐藏保存完整 analysis spec 与 provenance；请求 `outputs=[{"key":"result_workbook","type":"table","format":"xlsx"}]` 时再增加 XLSX。CSV/XLSX 可登记为后续 `origin.plot@v2` Job 输入，但分析与绘图不合并成同一个请求。`origin.analysis@v1` 与 `origin.plot@v2` 共享 Origin 软件位置及 `runtimes\origin`，不会在 UI 生成第二张软件卡；完整 Node 包会同时携带两份 adapter。只更新分析 adapter 时运行 `windows-node\package-origin-analysis-adapter.bat`，退出 Node 后整体替换 `adapters\origin.analysis@v1\` 再启动；该独立包假定目标节点已经安装 Origin 受管 runtime。
|
||||
|
||||
分析 adapter 自带六个固定真机用例。运行：`<data_root>\runtimes\origin\Scripts\python.exe <node_program>\adapters\origin.analysis@v1\acceptance.py --work-root <new_empty_parent>\origin-analysis-acceptance-YYYYMMDD --release-wait 180`。可用 `--case <name>` 逐例执行；每例通过生产 Worker 生成标准产物，再用独立 Origin 会话重开 OPJU，核对必需工作表、结果摘要和数值 oracle，并等待新 Origin 进程完全退出。目标测试机观察到 COM 退出可能接近 2 分钟，因此整套六例需预留较长命令时间；查询进程列表的瞬时失败会固定重试三次。最终以 `acceptance-report.json` 的 `passed=true`、各 case 的 `oracle_validation.passed=true` 和 `origin_processes_released=true` 共同作为通过条件。
|
||||
|
||||
Blender adapter 可独立运行 `windows-node\package-blender-adapter.bat` 打成 `dist\blender.scene.author@v3-adapter.zip`，替换规则与 Origin 相同。升级时先从托盘退出 Node，删除程序目录中的 `adapters\blender.scene.author@v1\`、`adapters\blender.scene.author@v2\`，整体放入 v3 目录后重启；数据根目录中的旧 Workspace 不在清理范围。Blender 的 `requirements.txt` 当前不安装第三方包，`bpy` 不安装到轻量启动 runtime。
|
||||
|
||||
注册配置写入 `<data_root>\node.json`；Token 使用 DPAPI `LocalMachine` 加密，ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号配置、注册并运行 Node。当前 MVP 可由 UI 写入该账号的登录启动项，不安装 Windows Service。
|
||||
|
|
|
|||
|
|
@ -0,0 +1,298 @@
|
|||
{
|
||||
"capability": "origin.analysis@v1",
|
||||
"display_name": "Origin 科研分析",
|
||||
"default_enrollment": false,
|
||||
"output_namespace": "origin_analysis",
|
||||
"input_policy": {
|
||||
"suffixes": [".csv", ".xlsx", ".json"],
|
||||
"max_count": 1,
|
||||
"max_bytes": 104857600,
|
||||
"max_total_bytes": 104857600
|
||||
},
|
||||
"outputs": {
|
||||
"project": {
|
||||
"filename": "analysis.opju",
|
||||
"media_type": "application/x-origin-project",
|
||||
"relative_path": "analysis.opju",
|
||||
"publish": true,
|
||||
"required": true
|
||||
},
|
||||
"result_table": {
|
||||
"filename": "result-table.csv",
|
||||
"media_type": "text/csv",
|
||||
"relative_path": "result-table.csv",
|
||||
"publish": true,
|
||||
"required": true
|
||||
},
|
||||
"result_workbook": {
|
||||
"filename": "result-table.xlsx",
|
||||
"media_type": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
|
||||
"relative_path": "result-table.xlsx",
|
||||
"publish": true,
|
||||
"required": false
|
||||
},
|
||||
"diagnostics": {
|
||||
"filename": "diagnostics.json",
|
||||
"media_type": "application/json",
|
||||
"relative_path": "diagnostics.json",
|
||||
"publish": true,
|
||||
"required": true
|
||||
},
|
||||
"analysis_spec": {
|
||||
"filename": "analysis-spec.json",
|
||||
"media_type": "application/json",
|
||||
"relative_path": ".meta/analysis-spec.json",
|
||||
"publish": false,
|
||||
"required": true
|
||||
},
|
||||
"provenance": {
|
||||
"filename": "provenance.json",
|
||||
"media_type": "application/json",
|
||||
"relative_path": ".meta/provenance.json",
|
||||
"publish": false,
|
||||
"required": true
|
||||
}
|
||||
},
|
||||
"feature_path": ["operation", "analysis", "type"],
|
||||
"features": {
|
||||
"data_check": "0.1.0",
|
||||
"normalize": "0.1.0",
|
||||
"smooth": "0.1.0",
|
||||
"differentiate": "0.1.0",
|
||||
"integrate": "0.1.0",
|
||||
"linear_fit": "0.1.0"
|
||||
},
|
||||
"summary": {
|
||||
"title_path": ["operation", "analysis", "title"]
|
||||
},
|
||||
"legacy_runtime": null,
|
||||
"workspace": null,
|
||||
"request_schema": {
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"type": "object",
|
||||
"x-maxBytes": 65536,
|
||||
"required": ["schema_version", "inputs", "operation", "outputs"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"schema_version": {"const": 1},
|
||||
"inputs": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["key", "artifact_id"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"key": {"type": "string", "pattern": "^[a-z][a-z0-9_]{0,31}$"},
|
||||
"artifact_id": {"type": "string", "format": "uuid"},
|
||||
"selector": {
|
||||
"type": "object",
|
||||
"required": ["sheet"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"sheet": {"type": "string", "minLength": 1, "maxLength": 128}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"operation": {
|
||||
"type": "object",
|
||||
"required": ["analysis"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"analysis": {
|
||||
"type": "object",
|
||||
"required": ["type", "title", "data", "parameters"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"type": {
|
||||
"enum": [
|
||||
"data_check",
|
||||
"normalize",
|
||||
"smooth",
|
||||
"differentiate",
|
||||
"integrate",
|
||||
"linear_fit"
|
||||
]
|
||||
},
|
||||
"title": {"type": "string", "minLength": 1, "maxLength": 200},
|
||||
"data": {"$ref": "#/$defs/data_binding"},
|
||||
"parameters": {"type": "object"}
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"if": {"properties": {"type": {"const": "data_check"}}},
|
||||
"then": {
|
||||
"properties": {
|
||||
"parameters": {"$ref": "#/$defs/data_check_parameters"}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"if": {"properties": {"type": {"const": "normalize"}}},
|
||||
"then": {
|
||||
"properties": {
|
||||
"data": {"$ref": "#/$defs/xy_binding"},
|
||||
"parameters": {"$ref": "#/$defs/normalize_parameters"}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"if": {"properties": {"type": {"const": "smooth"}}},
|
||||
"then": {
|
||||
"properties": {
|
||||
"data": {"$ref": "#/$defs/xy_binding"},
|
||||
"parameters": {"$ref": "#/$defs/smooth_parameters"}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"if": {"properties": {"type": {"const": "differentiate"}}},
|
||||
"then": {
|
||||
"properties": {
|
||||
"data": {"$ref": "#/$defs/xy_binding"},
|
||||
"parameters": {"$ref": "#/$defs/differentiate_parameters"}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"if": {"properties": {"type": {"const": "integrate"}}},
|
||||
"then": {
|
||||
"properties": {
|
||||
"data": {"$ref": "#/$defs/xy_binding"},
|
||||
"parameters": {"$ref": "#/$defs/integrate_parameters"}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"if": {"properties": {"type": {"const": "linear_fit"}}},
|
||||
"then": {
|
||||
"properties": {
|
||||
"data": {"$ref": "#/$defs/xy_binding"},
|
||||
"parameters": {"$ref": "#/$defs/linear_fit_parameters"}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"outputs": {
|
||||
"type": "array",
|
||||
"maxItems": 1,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["key", "type", "format"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"key": {"const": "result_workbook"},
|
||||
"type": {"const": "table"},
|
||||
"format": {"const": "xlsx"}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"data_binding": {
|
||||
"type": "object",
|
||||
"required": ["input"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"input": {"type": "string", "pattern": "^[a-z][a-z0-9_]{0,31}$"},
|
||||
"x": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"y": {"type": "string", "minLength": 1, "maxLength": 128}
|
||||
}
|
||||
},
|
||||
"xy_binding": {
|
||||
"type": "object",
|
||||
"required": ["input", "x", "y"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"input": {"type": "string", "pattern": "^[a-z][a-z0-9_]{0,31}$"},
|
||||
"x": {"type": "string", "minLength": 1, "maxLength": 128},
|
||||
"y": {"type": "string", "minLength": 1, "maxLength": 128}
|
||||
}
|
||||
},
|
||||
"data_check_parameters": {
|
||||
"type": "object",
|
||||
"description": "数据体检始终返回诊断产物；命中 fail_on 时 diagnostics.passed=false，但 Job 本身成功完成，供调用方决定是否继续分析。",
|
||||
"required": ["fail_on"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"fail_on": {
|
||||
"type": "array",
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"enum": [
|
||||
"missing",
|
||||
"non_finite",
|
||||
"duplicate_x",
|
||||
"non_monotonic_x",
|
||||
"uneven_spacing"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"normalize_parameters": {
|
||||
"type": "object",
|
||||
"description": "max_abs 使用最大绝对值；area 使用有符号梯形积分；reference 在数据范围内按 X 线性插值得到归一化除数。",
|
||||
"required": ["method"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"method": {"enum": ["max_abs", "area", "reference"]},
|
||||
"reference_x": {"type": "number"}
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"if": {"properties": {"method": {"const": "reference"}}},
|
||||
"then": {"required": ["reference_x"]},
|
||||
"else": {"not": {"required": ["reference_x"]}}
|
||||
}
|
||||
]
|
||||
},
|
||||
"smooth_parameters": {
|
||||
"type": "object",
|
||||
"description": "固定局部多项式平滑；窗口必须为奇数，边界使用平移后的完整窗口，不补造数据点。",
|
||||
"required": ["method", "window", "polynomial_order"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"method": {"const": "savitzky_golay"},
|
||||
"window": {"type": "integer", "minimum": 5, "maximum": 101},
|
||||
"polynomial_order": {"type": "integer", "minimum": 2, "maximum": 5}
|
||||
}
|
||||
},
|
||||
"differentiate_parameters": {
|
||||
"type": "object",
|
||||
"description": "使用支持非等间距 X 的二阶边界数值梯度，按 order 重复求导。",
|
||||
"required": ["order"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"order": {"type": "integer", "enum": [1, 2]}
|
||||
}
|
||||
},
|
||||
"integrate_parameters": {
|
||||
"type": "object",
|
||||
"description": "使用梯形积分输出从首个 X 开始的累计积分；from/to 省略时报告完整范围积分。",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"from": {"type": "number"},
|
||||
"to": {"type": "number"}
|
||||
}
|
||||
},
|
||||
"linear_fit_parameters": {
|
||||
"type": "object",
|
||||
"description": "固定最小二乘线性拟合并输出 95% t 置信区间、R²、RMSE 和残差；同时在 OPJU 中生成 Origin 原生线性拟合报告。",
|
||||
"required": ["include_intercept", "confidence_level"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"include_intercept": {"type": "boolean"},
|
||||
"confidence_level": {"const": 0.95}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,309 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import numpy as np
|
||||
|
||||
from core.software_contracts import get_contract
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
ADAPTER_ROOT = ROOT / "windows-node" / "adapters" / "origin.analysis@v1"
|
||||
WORKER_PATH = ADAPTER_ROOT / "worker.py"
|
||||
SPEC = importlib.util.spec_from_file_location("zcbot_origin_analysis_worker", WORKER_PATH)
|
||||
assert SPEC and SPEC.loader
|
||||
worker = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(worker)
|
||||
|
||||
ACCEPTANCE_PATH = ADAPTER_ROOT / "acceptance.py"
|
||||
ACCEPTANCE_SPEC = importlib.util.spec_from_file_location(
|
||||
"zcbot_origin_analysis_acceptance", ACCEPTANCE_PATH
|
||||
)
|
||||
assert ACCEPTANCE_SPEC and ACCEPTANCE_SPEC.loader
|
||||
acceptance = importlib.util.module_from_spec(ACCEPTANCE_SPEC)
|
||||
ACCEPTANCE_SPEC.loader.exec_module(acceptance)
|
||||
|
||||
|
||||
def _request(operation: str, parameters: dict, *, data: dict | None = None) -> dict:
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"inputs": [
|
||||
{
|
||||
"key": "sample",
|
||||
"artifact_id": "f4186347-65cc-4f07-9c26-bf11992beef8",
|
||||
}
|
||||
],
|
||||
"operation": {
|
||||
"analysis": {
|
||||
"type": operation,
|
||||
"title": f"Test {operation}",
|
||||
"data": data or {"input": "sample", "x": "x", "y": "y"},
|
||||
"parameters": parameters,
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
}
|
||||
|
||||
|
||||
class OriginAnalysisContractTests(unittest.TestCase):
|
||||
def test_contract_exposes_six_guarded_operations_and_fixed_outputs(self) -> None:
|
||||
contract = get_contract("origin.analysis@v1")
|
||||
self.assertFalse(contract.default_enrollment)
|
||||
self.assertEqual(
|
||||
set(contract.features),
|
||||
{
|
||||
"data_check",
|
||||
"normalize",
|
||||
"smooth",
|
||||
"differentiate",
|
||||
"integrate",
|
||||
"linear_fit",
|
||||
},
|
||||
)
|
||||
request = _request("normalize", {"method": "max_abs"})
|
||||
normalized, digest = contract.normalize_request(request)
|
||||
self.assertEqual(normalized, request)
|
||||
self.assertEqual(len(digest), 64)
|
||||
self.assertEqual(
|
||||
set(contract.expected_outputs(request)),
|
||||
{"project", "result_table", "diagnostics", "analysis_spec", "provenance"},
|
||||
)
|
||||
|
||||
workbook_request = json.loads(json.dumps(request))
|
||||
workbook_request["outputs"] = [
|
||||
{"key": "result_workbook", "type": "table", "format": "xlsx"}
|
||||
]
|
||||
normalized, _ = contract.normalize_request(workbook_request)
|
||||
self.assertEqual(normalized, workbook_request)
|
||||
self.assertIn("result_workbook", contract.expected_outputs(workbook_request))
|
||||
|
||||
def test_contract_rejects_scripts_and_operation_specific_parameter_drift(self) -> None:
|
||||
contract = get_contract("origin.analysis@v1")
|
||||
request = _request("smooth", {
|
||||
"method": "savitzky_golay",
|
||||
"window": 7,
|
||||
"polynomial_order": 3,
|
||||
})
|
||||
contract.normalize_request(request)
|
||||
|
||||
request["operation"]["analysis"]["parameters"]["script"] = "anything"
|
||||
with self.assertRaises(ValueError):
|
||||
contract.normalize_request(request)
|
||||
|
||||
reference = _request("normalize", {"method": "reference"})
|
||||
with self.assertRaises(ValueError):
|
||||
contract.normalize_request(reference)
|
||||
|
||||
def test_worker_and_manifest_versions_match(self) -> None:
|
||||
manifest = json.loads((ADAPTER_ROOT / "adapter.json").read_text(encoding="utf-8"))
|
||||
self.assertEqual(worker.ADAPTER_VERSION, manifest["adapter_version"])
|
||||
|
||||
|
||||
class OriginAnalysisNumericalTests(unittest.TestCase):
|
||||
def test_data_check_profiles_columns_and_reports_x_issues(self) -> None:
|
||||
headers = ["x", "y", "label"]
|
||||
rows = [[1, 2, "a"], [1, None, "b"], [3.5, float("inf"), "c"]]
|
||||
spec = _request(
|
||||
"data_check",
|
||||
{"fail_on": ["missing", "non_finite", "duplicate_x", "uneven_spacing"]},
|
||||
)["operation"]["analysis"]
|
||||
result_headers, result_rows, diagnostics = worker.analyze(headers, rows, spec)
|
||||
self.assertEqual(result_headers[0], "column")
|
||||
self.assertEqual(len(result_rows), 3)
|
||||
self.assertFalse(diagnostics["passed"])
|
||||
self.assertEqual(
|
||||
set(diagnostics["failed_checks"]),
|
||||
{"missing", "non_finite", "duplicate_x"},
|
||||
)
|
||||
|
||||
def test_normalize_supports_max_area_and_interpolated_reference(self) -> None:
|
||||
x = np.array([0.0, 1.0, 2.0])
|
||||
y = np.array([0.0, 2.0, 4.0])
|
||||
_, rows, diagnostics = worker._normalize(x, y, {"method": "max_abs"})
|
||||
self.assertEqual([item[2] for item in rows], [0.0, 0.5, 1.0])
|
||||
self.assertEqual(diagnostics["divisor"], 4.0)
|
||||
|
||||
_, _, diagnostics = worker._normalize(x, y, {"method": "area"})
|
||||
self.assertEqual(diagnostics["divisor"], 4.0)
|
||||
|
||||
_, rows, diagnostics = worker._normalize(
|
||||
x, y, {"method": "reference", "reference_x": 1.5}
|
||||
)
|
||||
self.assertEqual(diagnostics["divisor"], 3.0)
|
||||
self.assertAlmostEqual(rows[-1][2], 4 / 3)
|
||||
|
||||
def test_smooth_reproduces_quadratic_and_rejects_even_window(self) -> None:
|
||||
x = np.arange(7, dtype=float)
|
||||
y = x**2 + 2 * x + 3
|
||||
_, rows, diagnostics = worker._smooth(
|
||||
x,
|
||||
y,
|
||||
{"method": "savitzky_golay", "window": 5, "polynomial_order": 2},
|
||||
)
|
||||
np.testing.assert_allclose([item[2] for item in rows], y, atol=1e-10)
|
||||
self.assertAlmostEqual(diagnostics["rmse"], 0.0, places=10)
|
||||
with self.assertRaisesRegex(ValueError, "SMOOTH_WINDOW_MUST_BE_ODD"):
|
||||
worker._smooth(
|
||||
x,
|
||||
y,
|
||||
{"method": "savitzky_golay", "window": 6, "polynomial_order": 2},
|
||||
)
|
||||
|
||||
def test_differentiate_and_integrate_match_polynomial_oracles(self) -> None:
|
||||
x = np.arange(5, dtype=float)
|
||||
y = x**2
|
||||
_, derivative_rows, _ = worker._differentiate(x, y, {"order": 1})
|
||||
np.testing.assert_allclose([item[2] for item in derivative_rows], 2 * x)
|
||||
|
||||
_, integral_rows, diagnostics = worker._integrate(x, 2 * x, {})
|
||||
np.testing.assert_allclose([item[2] for item in integral_rows], x**2)
|
||||
self.assertEqual(diagnostics["full_integral"], 16.0)
|
||||
_, _, diagnostics = worker._integrate(x, 2 * x, {"from": 1.0, "to": 3.0})
|
||||
self.assertEqual(diagnostics["integral"], 8.0)
|
||||
|
||||
def test_linear_fit_returns_parameters_intervals_and_residuals(self) -> None:
|
||||
x = np.arange(1, 7, dtype=float)
|
||||
y = 2.5 * x - 1.25
|
||||
headers, rows, diagnostics = worker._linear_fit(
|
||||
x,
|
||||
y,
|
||||
{"include_intercept": True, "confidence_level": 0.95},
|
||||
)
|
||||
self.assertEqual(headers, ["x", "y", "fitted_y", "residual"])
|
||||
self.assertAlmostEqual(diagnostics["slope"], 2.5)
|
||||
self.assertAlmostEqual(diagnostics["intercept"], -1.25)
|
||||
self.assertAlmostEqual(diagnostics["r_squared"], 1.0)
|
||||
np.testing.assert_allclose([item[3] for item in rows], 0, atol=1e-12)
|
||||
self.assertLessEqual(
|
||||
diagnostics["slope_confidence_interval"][0], diagnostics["slope"]
|
||||
)
|
||||
self.assertGreaterEqual(
|
||||
diagnostics["slope_confidence_interval"][1], diagnostics["slope"]
|
||||
)
|
||||
|
||||
def test_xy_values_require_finite_strictly_increasing_x(self) -> None:
|
||||
binding = {"input": "sample", "x": "x", "y": "y"}
|
||||
with self.assertRaisesRegex(ValueError, "X_VALUES_DUPLICATED"):
|
||||
worker._xy_values(["x", "y"], [[1, 2], [1, 3]], binding)
|
||||
with self.assertRaisesRegex(ValueError, "Y_VALUE_NOT_FINITE"):
|
||||
worker._xy_values(["x", "y"], [[1, 2], [2, float("nan")]], binding)
|
||||
|
||||
def test_csv_json_and_xlsx_outputs_are_deterministically_readable(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
csv_path = root / "input.csv"
|
||||
csv_path.write_text("x,y\n1,2\n3,4\n", encoding="utf-8")
|
||||
self.assertEqual(
|
||||
worker._read_rows(csv_path, None),
|
||||
(["x", "y"], [["1", "2"], ["3", "4"]]),
|
||||
)
|
||||
json_path = root / "input.json"
|
||||
json_path.write_text(
|
||||
json.dumps([{"x": 1, "y": 2}, {"x": 3, "y": 4}]),
|
||||
encoding="utf-8",
|
||||
)
|
||||
self.assertEqual(
|
||||
worker._read_rows(json_path, None), (["x", "y"], [[1, 2], [3, 4]])
|
||||
)
|
||||
|
||||
result_path = root / "result-table.csv"
|
||||
worker._atomic_csv(result_path, ["x", "y"], [[1, 2], [3, 4]])
|
||||
self.assertTrue(result_path.read_bytes().startswith(b"\xef\xbb\xbf"))
|
||||
workbook_path = root / "result-table.xlsx"
|
||||
worker._atomic_xlsx(workbook_path, ["x", "y"], [[1, 2], [3, 4]])
|
||||
self.assertTrue(workbook_path.is_file())
|
||||
|
||||
def test_run_builds_fixed_manifest_without_origin_side_effects_in_request(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
job_dir = Path(directory)
|
||||
(job_dir / "request").mkdir()
|
||||
(job_dir / "input" / "sample").mkdir(parents=True)
|
||||
(job_dir / "input" / "sample" / "sample.csv").write_text(
|
||||
"x,y\n0,0\n1,2\n2,4\n", encoding="utf-8"
|
||||
)
|
||||
request = _request("normalize", {"method": "max_abs"})
|
||||
record = {
|
||||
"job_id": "job",
|
||||
"lease_id": "lease",
|
||||
"request_digest": "a" * 64,
|
||||
"request": request,
|
||||
}
|
||||
worker._atomic_json(job_dir / "request" / "request.json", record)
|
||||
|
||||
def fake_project(path, *_args):
|
||||
path.write_bytes(b"origin-project" * 16)
|
||||
|
||||
with patch.object(worker, "_write_origin_project", side_effect=fake_project):
|
||||
artifacts = worker.run(job_dir)
|
||||
|
||||
self.assertEqual(
|
||||
{item["artifact_id"] for item in artifacts},
|
||||
{"project", "result_table", "diagnostics", "analysis_spec", "provenance"},
|
||||
)
|
||||
diagnostics = json.loads(
|
||||
(job_dir / "output" / "diagnostics.json").read_text(encoding="utf-8")
|
||||
)
|
||||
self.assertEqual(diagnostics["divisor"], 4.0)
|
||||
provenance = json.loads(
|
||||
(job_dir / "output" / "provenance.json").read_text(encoding="utf-8")
|
||||
)
|
||||
self.assertEqual(provenance["request_digest"], "a" * 64)
|
||||
|
||||
|
||||
class OriginAnalysisAcceptanceTests(unittest.TestCase):
|
||||
def test_fixed_acceptance_cases_are_contract_valid(self) -> None:
|
||||
contract = get_contract("origin.analysis@v1")
|
||||
cases = acceptance._cases()
|
||||
self.assertEqual(
|
||||
list(cases),
|
||||
[
|
||||
"data_check",
|
||||
"normalize",
|
||||
"smooth",
|
||||
"differentiate",
|
||||
"integrate",
|
||||
"linear_fit",
|
||||
],
|
||||
)
|
||||
for name, case in cases.items():
|
||||
normalized, digest = contract.normalize_request(case["request"])
|
||||
self.assertEqual(normalized, case["request"], name)
|
||||
self.assertEqual(len(digest), 64)
|
||||
|
||||
def test_acceptance_oracles_are_independently_recomputed(self) -> None:
|
||||
for name, case in acceptance._cases().items():
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "sample.csv"
|
||||
path.write_text(case["csv"], encoding="utf-8", newline="")
|
||||
headers, rows = worker._read_rows(path, None)
|
||||
result_headers, result_rows, diagnostics = worker.analyze(
|
||||
headers, rows, case["request"]["operation"]["analysis"]
|
||||
)
|
||||
validation = acceptance._validate_oracle(
|
||||
name,
|
||||
diagnostics,
|
||||
result_headers,
|
||||
result_rows,
|
||||
case["oracle"],
|
||||
)
|
||||
self.assertTrue(validation["passed"], name)
|
||||
|
||||
def test_stage_writes_keyed_input_and_stable_request_digest(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
case = acceptance._cases()["linear_fit"]
|
||||
job_dir = acceptance._stage(root, "linear_fit", case)
|
||||
record = json.loads(
|
||||
(job_dir / "request" / "request.json").read_text(encoding="utf-8")
|
||||
)
|
||||
_, digest = acceptance._canonical_request(case["request"])
|
||||
self.assertEqual(record["request_digest"], digest)
|
||||
self.assertTrue((job_dir / "input" / "sample" / "sample.csv").is_file())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -82,6 +82,7 @@ class SoftwareNodeSecurityTests(unittest.TestCase):
|
|||
self.assertEqual(
|
||||
set(enrollment.allowed_capabilities),
|
||||
{
|
||||
"origin.analysis@v1",
|
||||
"origin.plot@v2",
|
||||
"ansys.geometry.inspect@v1",
|
||||
"ansys.mechanical.static_structural@v2",
|
||||
|
|
|
|||
|
|
@ -304,6 +304,8 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
|||
self.assertIn("adapters\\%(RecursiveDir)%(Filename)%(Extension)", project)
|
||||
self.assertIn("..\\..\\software-contracts\\origin.plot.v2.json", project)
|
||||
self.assertIn("adapters\\origin.plot@v2\\origin.plot.v2.json", project)
|
||||
self.assertIn("..\\..\\software-contracts\\origin.analysis.v1.json", project)
|
||||
self.assertIn("adapters\\origin.analysis@v1\\origin.analysis.v1.json", project)
|
||||
self.assertIn(
|
||||
"..\\..\\software-contracts\\ansys.geometry.inspect.v1.json",
|
||||
project,
|
||||
|
|
@ -541,6 +543,24 @@ class WindowsNodeSourceTests(unittest.TestCase):
|
|||
tray,
|
||||
)
|
||||
|
||||
def test_origin_analysis_adapter_can_be_packaged_without_building_node(self) -> None:
|
||||
script = (ROOT / "package-origin-analysis-adapter.bat").read_text(encoding="utf-8")
|
||||
self.assertIn('"adapters\\origin.analysis@v1\\adapter.json"', script)
|
||||
self.assertIn('"adapters\\origin.analysis@v1\\worker.py"', script)
|
||||
self.assertIn('"adapters\\origin.analysis@v1\\acceptance.py"', script)
|
||||
self.assertIn('"..\\software-contracts\\origin.analysis.v1.json"', script)
|
||||
self.assertNotIn("dotnet", script.lower())
|
||||
|
||||
worker = (
|
||||
ROOT / "adapters" / "origin.analysis@v1" / "worker.py"
|
||||
).read_text(encoding="utf-8")
|
||||
for forbidden in ("eval(", "exec(", "os.system"):
|
||||
self.assertNotIn(forbidden, worker)
|
||||
self.assertIn("op.LinearFit()", worker)
|
||||
self.assertIn("op.save", worker)
|
||||
self.assertIn('if op.oext:', worker)
|
||||
self.assertIn("op.exit()", worker)
|
||||
|
||||
def test_blender_adapter_can_be_packaged_without_building_node(self) -> None:
|
||||
script = (ROOT / "package-blender-adapter.bat").read_text(encoding="utf-8")
|
||||
self.assertIn("adapters\\blender.scene.author@v3", script)
|
||||
|
|
|
|||
|
|
@ -25,6 +25,11 @@
|
|||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
|
||||
</Content>
|
||||
<Content Include="..\..\software-contracts\origin.analysis.v1.json">
|
||||
<Link>adapters\origin.analysis@v1\origin.analysis.v1.json</Link>
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
|
||||
</Content>
|
||||
<Content Include="..\..\software-contracts\ansys.geometry.inspect.v1.json">
|
||||
<Link>adapters\ansys.geometry.inspect@v1\ansys.geometry.inspect.v1.json</Link>
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,397 @@
|
|||
"""Run fixed origin.analysis@v1 acceptance cases on a dedicated Windows node."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import locale
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from importlib.metadata import PackageNotFoundError, version
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
|
||||
ADAPTER_DIR = Path(__file__).resolve().parent
|
||||
WORKER_PATH = ADAPTER_DIR / "worker.py"
|
||||
_WORKER_SPEC = importlib.util.spec_from_file_location(
|
||||
"zcbot_origin_analysis_acceptance_worker", WORKER_PATH
|
||||
)
|
||||
if _WORKER_SPEC is None or _WORKER_SPEC.loader is None:
|
||||
raise RuntimeError("ORIGIN_ANALYSIS_WORKER_IMPORT_FAILED")
|
||||
worker = importlib.util.module_from_spec(_WORKER_SPEC)
|
||||
_WORKER_SPEC.loader.exec_module(worker)
|
||||
|
||||
REPORT_SCHEMA_VERSION = 1
|
||||
ORIGIN_PROCESS_PATTERN = r"^origin(?:\d+)?(?:_?\d+)?(?:64)?\.exe$"
|
||||
|
||||
|
||||
def _artifact_id(case_name: str) -> str:
|
||||
return str(uuid5(NAMESPACE_URL, f"zcbot-origin-analysis-acceptance:{case_name}"))
|
||||
|
||||
|
||||
def _request(case_name: str, operation: str, parameters: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"inputs": [{"key": "sample", "artifact_id": _artifact_id(case_name)}],
|
||||
"operation": {
|
||||
"analysis": {
|
||||
"type": operation,
|
||||
"title": f"Origin analysis acceptance: {operation}",
|
||||
"data": {"input": "sample", "x": "x", "y": "y"},
|
||||
"parameters": parameters,
|
||||
}
|
||||
},
|
||||
"outputs": (
|
||||
[{"key": "result_workbook", "type": "table", "format": "xlsx"}]
|
||||
if operation == "linear_fit"
|
||||
else []
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def _cases() -> dict[str, dict[str, Any]]:
|
||||
data_check = _request(
|
||||
"data_check",
|
||||
"data_check",
|
||||
{"fail_on": ["missing", "non_finite", "duplicate_x", "uneven_spacing"]},
|
||||
)
|
||||
return {
|
||||
"data_check": {
|
||||
"csv": "x,y\n0,1\n1,\n1,inf\n3,4\n",
|
||||
"request": data_check,
|
||||
"oracle": {
|
||||
"passed": False,
|
||||
"failed_checks": ["duplicate_x", "missing", "non_finite", "uneven_spacing"],
|
||||
},
|
||||
},
|
||||
"normalize": {
|
||||
"csv": "x,y\n0,0\n1,2\n2,4\n",
|
||||
"request": _request(
|
||||
"normalize", "normalize", {"method": "reference", "reference_x": 1.5}
|
||||
),
|
||||
"oracle": {"divisor": 3.0, "last_value": 4 / 3},
|
||||
},
|
||||
"smooth": {
|
||||
"csv": "x,y\n" + "\n".join(f"{x},{x*x + 2*x + 3}" for x in range(7)) + "\n",
|
||||
"request": _request(
|
||||
"smooth",
|
||||
"smooth",
|
||||
{"method": "savitzky_golay", "window": 5, "polynomial_order": 2},
|
||||
),
|
||||
"oracle": {"rmse": 0.0},
|
||||
},
|
||||
"differentiate": {
|
||||
"csv": "x,y\n" + "\n".join(f"{x},{x*x}" for x in range(5)) + "\n",
|
||||
"request": _request("differentiate", "differentiate", {"order": 1}),
|
||||
"oracle": {"derived": [0.0, 2.0, 4.0, 6.0, 8.0]},
|
||||
},
|
||||
"integrate": {
|
||||
"csv": "x,y\n" + "\n".join(f"{x},{2*x}" for x in range(5)) + "\n",
|
||||
"request": _request("integrate", "integrate", {"from": 1.0, "to": 3.0}),
|
||||
"oracle": {"integral": 8.0, "full_integral": 16.0},
|
||||
},
|
||||
"linear_fit": {
|
||||
"csv": "x,y\n" + "\n".join(f"{x},{2.5*x - 1.25}" for x in range(1, 7)) + "\n",
|
||||
"request": _request(
|
||||
"linear_fit",
|
||||
"linear_fit",
|
||||
{"include_intercept": True, "confidence_level": 0.95},
|
||||
),
|
||||
"oracle": {"slope": 2.5, "intercept": -1.25, "r_squared": 1.0},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _canonical_request(request: dict[str, Any]) -> tuple[str, str]:
|
||||
encoded = json.dumps(request, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
||||
return encoded, hashlib.sha256(encoded.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _stage(root: Path, case_name: str, case: dict[str, Any]) -> Path:
|
||||
job_dir = root / case_name
|
||||
request_dir = job_dir / "request"
|
||||
input_dir = job_dir / "input" / "sample"
|
||||
request_dir.mkdir(parents=True)
|
||||
input_dir.mkdir(parents=True)
|
||||
(input_dir / "sample.csv").write_text(case["csv"], encoding="utf-8", newline="")
|
||||
_, digest = _canonical_request(case["request"])
|
||||
record = {
|
||||
"job_id": str(uuid5(NAMESPACE_URL, f"zcbot-origin-analysis-job:{case_name}")),
|
||||
"lease_id": str(uuid5(NAMESPACE_URL, f"zcbot-origin-analysis-lease:{case_name}")),
|
||||
"request_digest": digest,
|
||||
"request": case["request"],
|
||||
}
|
||||
worker._atomic_json(request_dir / "request.json", record)
|
||||
return job_dir
|
||||
|
||||
|
||||
def _read_result(path: Path) -> tuple[list[str], list[list[float | str | None]]]:
|
||||
with path.open("r", encoding="utf-8-sig", newline="") as handle:
|
||||
rows = list(csv.reader(handle))
|
||||
converted: list[list[float | str | None]] = []
|
||||
for row in rows[1:]:
|
||||
values: list[float | str | None] = []
|
||||
for value in row:
|
||||
if value == "":
|
||||
values.append(None)
|
||||
continue
|
||||
try:
|
||||
values.append(float(value))
|
||||
except ValueError:
|
||||
values.append(value)
|
||||
converted.append(values)
|
||||
return rows[0], converted
|
||||
|
||||
|
||||
def _assert_close(actual: float, expected: float, tolerance: float = 1e-9) -> None:
|
||||
if abs(float(actual) - float(expected)) > tolerance:
|
||||
raise RuntimeError(f"NUMERICAL_ORACLE_MISMATCH:{actual}:{expected}")
|
||||
|
||||
|
||||
def _validate_oracle(
|
||||
case_name: str,
|
||||
diagnostics: dict[str, Any],
|
||||
result_headers: list[str],
|
||||
result_rows: list[list[Any]],
|
||||
oracle: dict[str, Any],
|
||||
) -> dict[str, Any]:
|
||||
if case_name == "data_check":
|
||||
if diagnostics["passed"] is not oracle["passed"]:
|
||||
raise RuntimeError("DATA_CHECK_PASS_STATE_MISMATCH")
|
||||
if diagnostics["failed_checks"] != oracle["failed_checks"]:
|
||||
raise RuntimeError("DATA_CHECK_CODES_MISMATCH")
|
||||
elif case_name == "normalize":
|
||||
_assert_close(diagnostics["divisor"], oracle["divisor"])
|
||||
_assert_close(result_rows[-1][result_headers.index("normalized_y")], oracle["last_value"])
|
||||
elif case_name == "smooth":
|
||||
_assert_close(diagnostics["rmse"], oracle["rmse"], 1e-10)
|
||||
elif case_name == "differentiate":
|
||||
values = [row[result_headers.index("derivative_order_1")] for row in result_rows]
|
||||
for actual, expected in zip(values, oracle["derived"], strict=True):
|
||||
_assert_close(actual, expected)
|
||||
elif case_name == "integrate":
|
||||
_assert_close(diagnostics["integral"], oracle["integral"])
|
||||
_assert_close(diagnostics["full_integral"], oracle["full_integral"])
|
||||
elif case_name == "linear_fit":
|
||||
for name in ("slope", "intercept", "r_squared"):
|
||||
_assert_close(diagnostics[name], oracle[name])
|
||||
return {"passed": True, "oracle": oracle}
|
||||
|
||||
|
||||
def _validate_manifest(job_dir: Path, case: dict[str, Any]) -> dict[str, Any]:
|
||||
terminal = json.loads((job_dir / "terminal.json").read_text(encoding="utf-8"))
|
||||
if terminal.get("status") != "succeeded":
|
||||
raise RuntimeError(f"WORKER_FAILED:{terminal.get('error')}")
|
||||
expected = {"project", "result_table", "diagnostics", "analysis_spec", "provenance"}
|
||||
if case["request"]["outputs"]:
|
||||
expected.add("result_workbook")
|
||||
manifest = terminal.get("artifact_manifest") or []
|
||||
by_id = {item["artifact_id"]: item for item in manifest}
|
||||
if set(by_id) != expected:
|
||||
raise RuntimeError(f"ARTIFACT_MANIFEST_MISMATCH:{sorted(by_id)}")
|
||||
output = job_dir / "output"
|
||||
paths = {artifact_id: output / item["filename"] for artifact_id, item in by_id.items()}
|
||||
for artifact_id, path in paths.items():
|
||||
if not path.is_file() or path.stat().st_size != by_id[artifact_id]["size_bytes"]:
|
||||
raise RuntimeError(f"ARTIFACT_INVALID:{artifact_id}")
|
||||
if worker._file_sha256(path) != by_id[artifact_id]["sha256"]:
|
||||
raise RuntimeError(f"ARTIFACT_DIGEST_MISMATCH:{artifact_id}")
|
||||
diagnostics = json.loads(paths["diagnostics"].read_text(encoding="utf-8"))
|
||||
headers, rows = _read_result(paths["result_table"])
|
||||
oracle = _validate_oracle(case["request"]["operation"]["analysis"]["type"], diagnostics, headers, rows, case["oracle"])
|
||||
return {
|
||||
"artifacts": {
|
||||
key: {"size_bytes": path.stat().st_size, "sha256": worker._file_sha256(path)}
|
||||
for key, path in sorted(paths.items())
|
||||
},
|
||||
"oracle_validation": oracle,
|
||||
}
|
||||
|
||||
|
||||
def _reopen_project(job_dir: Path) -> dict[str, Any]:
|
||||
import originpro as op
|
||||
|
||||
project = job_dir / "output" / "analysis.opju"
|
||||
op.set_show(False)
|
||||
try:
|
||||
if not op.open(str(project), readonly=True):
|
||||
raise RuntimeError("OPJU_REOPEN_FAILED")
|
||||
books = list(op.pages("w"))
|
||||
sheets = [sheet for book in books for sheet in book]
|
||||
names = [
|
||||
str(sheet.lname or getattr(book, "lname", "") or getattr(book, "name", "") or "")
|
||||
for book in books
|
||||
for sheet in book
|
||||
]
|
||||
for required in ("RawData", "Result", "Diagnostics"):
|
||||
if required not in names:
|
||||
raise RuntimeError(f"OPJU_SHEET_MISSING:{required}")
|
||||
return {"worksheet_count": len(sheets), "worksheet_names": names}
|
||||
finally:
|
||||
if op.oext:
|
||||
op.exit()
|
||||
|
||||
|
||||
def _environment_fingerprint() -> dict[str, Any]:
|
||||
try:
|
||||
originpro_version = version("originpro")
|
||||
except PackageNotFoundError:
|
||||
originpro_version = "unknown"
|
||||
return {
|
||||
"adapter_version": worker.ADAPTER_VERSION,
|
||||
"algorithm_version": worker.ALGORITHM_VERSION,
|
||||
"originpro_version": originpro_version,
|
||||
"numpy_version": worker.np.__version__,
|
||||
"python_version": platform.python_version(),
|
||||
"platform": platform.platform(),
|
||||
"locale": locale.getlocale(),
|
||||
"execution_mode": "hidden",
|
||||
"configured_origin_executable": bool(os.environ.get("ZCBOT_ORIGIN_EXE")),
|
||||
}
|
||||
|
||||
|
||||
def _origin_processes() -> dict[int, str]:
|
||||
completed = None
|
||||
for attempt in range(3):
|
||||
completed = subprocess.run(
|
||||
["tasklist.exe", "/fo", "csv", "/nh"],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
if completed.returncode == 0:
|
||||
break
|
||||
if attempt < 2:
|
||||
time.sleep(1)
|
||||
if completed is None or completed.returncode != 0:
|
||||
detail = completed.stderr[-500:] if completed is not None else "not started"
|
||||
raise RuntimeError(f"TASKLIST_FAILED:{detail}")
|
||||
result: dict[int, str] = {}
|
||||
for row in csv.reader(completed.stdout.splitlines()):
|
||||
name = row[0].casefold() if row else ""
|
||||
if len(row) >= 2 and re.fullmatch(ORIGIN_PROCESS_PATTERN, name):
|
||||
result[int(row[1])] = row[0]
|
||||
return result
|
||||
|
||||
|
||||
def _wait_for_origin_release(baseline: dict[int, str], timeout_seconds: int) -> None:
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
while True:
|
||||
remaining = {
|
||||
pid: name for pid, name in _origin_processes().items() if pid not in baseline
|
||||
}
|
||||
if not remaining:
|
||||
return
|
||||
if time.monotonic() >= deadline:
|
||||
raise RuntimeError(f"ORIGIN_PROCESS_REMAINS:{remaining}")
|
||||
time.sleep(2)
|
||||
|
||||
|
||||
def _run_case(
|
||||
root: Path,
|
||||
case_name: str,
|
||||
case: dict[str, Any],
|
||||
baseline: dict[int, str],
|
||||
release_wait: int,
|
||||
) -> dict[str, Any]:
|
||||
job_dir = _stage(root, case_name, case)
|
||||
started = time.monotonic()
|
||||
completed = subprocess.run(
|
||||
[sys.executable, str(WORKER_PATH), str(job_dir)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
check=False,
|
||||
)
|
||||
if completed.returncode != 0:
|
||||
raise RuntimeError(f"WORKER_PROCESS_FAILED:{completed.stderr[-500:]}")
|
||||
validation = _validate_manifest(job_dir, case)
|
||||
reopened = _reopen_project(job_dir)
|
||||
_wait_for_origin_release(baseline, release_wait)
|
||||
_, digest = _canonical_request(case["request"])
|
||||
return {
|
||||
"case": case_name,
|
||||
"elapsed_seconds": round(time.monotonic() - started, 3),
|
||||
"request_digest": digest,
|
||||
"validation": validation,
|
||||
"reopen": reopened,
|
||||
"origin_processes_released": True,
|
||||
}
|
||||
|
||||
|
||||
def run_suite(
|
||||
root: Path,
|
||||
selected: list[str] | None = None,
|
||||
*,
|
||||
release_wait: int = 60,
|
||||
) -> dict[str, Any]:
|
||||
cases = _cases()
|
||||
names = selected or list(cases)
|
||||
unknown = sorted(set(names) - cases.keys())
|
||||
if unknown:
|
||||
raise ValueError(f"UNKNOWN_ACCEPTANCE_CASES:{','.join(unknown)}")
|
||||
root.mkdir(parents=True, exist_ok=False)
|
||||
baseline = _origin_processes()
|
||||
report = {
|
||||
"schema_version": REPORT_SCHEMA_VERSION,
|
||||
"started_at": datetime.now(timezone.utc).isoformat(),
|
||||
"environment": _environment_fingerprint(),
|
||||
"selected_cases": names,
|
||||
"cases": [],
|
||||
"passed": False,
|
||||
}
|
||||
report_path = root / "acceptance-report.json"
|
||||
try:
|
||||
for index, name in enumerate(names, start=1):
|
||||
report["cases"].append(
|
||||
_run_case(root, name, cases[name], baseline, release_wait)
|
||||
)
|
||||
worker._atomic_json(report_path, report)
|
||||
print(f"[OK] Origin analysis acceptance {index}/{len(names)}: {name}")
|
||||
report["passed"] = True
|
||||
return report
|
||||
except Exception as exc:
|
||||
report["failure"] = {
|
||||
"type": type(exc).__name__,
|
||||
"detail": str(exc)[:1000],
|
||||
"completed_cases": len(report["cases"]),
|
||||
}
|
||||
raise
|
||||
finally:
|
||||
report["completed_at"] = datetime.now(timezone.utc).isoformat()
|
||||
worker._atomic_json(report_path, report)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Run fixed Origin analysis acceptance cases.")
|
||||
parser.add_argument("--work-root", type=Path, required=True)
|
||||
parser.add_argument("--release-wait", type=int, default=60)
|
||||
parser.add_argument(
|
||||
"--case", action="append", choices=tuple(_cases()), dest="cases"
|
||||
)
|
||||
args = parser.parse_args()
|
||||
if sys.platform != "win32":
|
||||
raise RuntimeError("Origin analysis acceptance requires Windows")
|
||||
if args.release_wait < 1:
|
||||
raise ValueError("release-wait must be positive")
|
||||
root = args.work_root.resolve()
|
||||
report = run_suite(root, args.cases, release_wait=args.release_wait)
|
||||
print(f"[OK] Origin analysis acceptance passed. Report: {root / 'acceptance-report.json'}")
|
||||
print(f"[INFO] Cases: {len(report['cases'])}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
{
|
||||
"capability": "origin.analysis@v1",
|
||||
"adapter_version": "0.1.0",
|
||||
"runtime": "python",
|
||||
"runtime_id": "origin",
|
||||
"entrypoint": "worker.py",
|
||||
"contract": "origin.analysis.v1.json",
|
||||
"worker_timeout_minutes": 30,
|
||||
"running_detail": "Origin 正在执行科研分析"
|
||||
}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
originpro==1.1.15
|
||||
openpyxl==3.1.5
|
||||
numpy==2.2.6
|
||||
|
|
@ -0,0 +1,792 @@
|
|||
"""Fixed Origin adapter for origin.analysis@v1.
|
||||
|
||||
The worker accepts one Node-created job directory. It performs only versioned,
|
||||
declarative analysis operations, writes the raw data and results into an Origin
|
||||
project, and never evaluates user code or accepts user-controlled paths.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from importlib.metadata import PackageNotFoundError, version
|
||||
from itertools import pairwise
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
ADAPTER_VERSION = "0.1.0"
|
||||
ALGORITHM_VERSION = "origin-analysis-mvp-1"
|
||||
OPERATIONS = {
|
||||
"data_check",
|
||||
"normalize",
|
||||
"smooth",
|
||||
"differentiate",
|
||||
"integrate",
|
||||
"linear_fit",
|
||||
}
|
||||
OUTPUT_MEDIA = {
|
||||
"analysis.opju": ("project", "application/x-origin-project"),
|
||||
"result-table.csv": ("result_table", "text/csv"),
|
||||
"result-table.xlsx": (
|
||||
"result_workbook",
|
||||
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
|
||||
),
|
||||
"diagnostics.json": ("diagnostics", "application/json"),
|
||||
"analysis-spec.json": ("analysis_spec", "application/json"),
|
||||
"provenance.json": ("provenance", "application/json"),
|
||||
}
|
||||
|
||||
|
||||
def _probe() -> int:
|
||||
health = "ready"
|
||||
detail = "Origin COM 与科研分析运行时可用"
|
||||
software_version = None
|
||||
try:
|
||||
if sys.platform != "win32":
|
||||
raise RuntimeError("Origin analysis adapter requires Windows")
|
||||
import winreg
|
||||
|
||||
import originpro
|
||||
|
||||
with winreg.OpenKey(winreg.HKEY_CLASSES_ROOT, r"Origin.ApplicationSI\CLSID"):
|
||||
pass
|
||||
originpro_version = version("originpro")
|
||||
numpy_version = version("numpy")
|
||||
detail = (
|
||||
"Origin COM 与科研分析运行时可用"
|
||||
f"（originpro {originpro_version}，numpy {numpy_version}）"
|
||||
)
|
||||
del originpro
|
||||
except (
|
||||
FileNotFoundError,
|
||||
ImportError,
|
||||
OSError,
|
||||
PackageNotFoundError,
|
||||
RuntimeError,
|
||||
) as exc:
|
||||
health = "unavailable"
|
||||
detail = str(exc)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"adapter_version": ADAPTER_VERSION,
|
||||
"software": "OriginPro",
|
||||
"software_version": software_version,
|
||||
"health": health,
|
||||
"detail": detail,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def _atomic_json(path: Path, value: Any) -> None:
|
||||
temporary = path.with_name(path.name + ".tmp-" + os.urandom(8).hex())
|
||||
try:
|
||||
with temporary.open("w", encoding="utf-8", newline="\n") as handle:
|
||||
json.dump(value, handle, ensure_ascii=False, indent=2, allow_nan=False)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _atomic_csv(path: Path, headers: list[str], rows: list[list[Any]]) -> None:
|
||||
temporary = path.with_name(path.name + ".tmp-" + os.urandom(8).hex())
|
||||
try:
|
||||
with temporary.open("w", encoding="utf-8-sig", newline="") as handle:
|
||||
writer = csv.writer(handle, lineterminator="\n")
|
||||
writer.writerow(headers)
|
||||
writer.writerows(rows)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _atomic_xlsx(path: Path, headers: list[str], rows: list[list[Any]]) -> None:
|
||||
from openpyxl import Workbook
|
||||
|
||||
temporary = path.with_name(path.name + ".tmp-" + os.urandom(8).hex() + ".xlsx")
|
||||
try:
|
||||
workbook = Workbook(write_only=True)
|
||||
worksheet = workbook.create_sheet("Result")
|
||||
worksheet.append(headers)
|
||||
for row in rows:
|
||||
worksheet.append(row)
|
||||
workbook.save(temporary)
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _read_rows(path: Path, sheet: str | None) -> tuple[list[str], list[list[Any]]]:
|
||||
suffix = path.suffix.lower()
|
||||
if suffix == ".csv":
|
||||
with path.open("r", encoding="utf-8-sig", newline="") as handle:
|
||||
rows = list(csv.reader(handle))
|
||||
if len(rows) < 2:
|
||||
raise ValueError("CSV_INPUT_EMPTY")
|
||||
return [str(item) for item in rows[0]], rows[1:]
|
||||
if suffix == ".json":
|
||||
value = json.loads(path.read_text(encoding="utf-8"))
|
||||
if isinstance(value, list) and value and all(isinstance(item, dict) for item in value):
|
||||
headers = list(value[0])
|
||||
if any(set(item) != set(headers) for item in value):
|
||||
raise ValueError("JSON_OBJECT_COLUMNS_MISMATCH")
|
||||
return headers, [[item.get(name) for name in headers] for item in value]
|
||||
if (
|
||||
isinstance(value, dict)
|
||||
and value
|
||||
and all(isinstance(item, list) for item in value.values())
|
||||
):
|
||||
headers = list(value)
|
||||
length = max(len(value[name]) for name in headers)
|
||||
return headers, [
|
||||
[value[name][index] if index < len(value[name]) else None for name in headers]
|
||||
for index in range(length)
|
||||
]
|
||||
raise ValueError("JSON_INPUT_SHAPE_UNSUPPORTED")
|
||||
if suffix == ".xlsx":
|
||||
from openpyxl import load_workbook
|
||||
|
||||
workbook = load_workbook(path, read_only=True, data_only=True)
|
||||
try:
|
||||
worksheet = workbook[sheet] if sheet else workbook.active
|
||||
rows = list(worksheet.iter_rows(values_only=True))
|
||||
finally:
|
||||
workbook.close()
|
||||
if len(rows) < 2:
|
||||
raise ValueError("XLSX_INPUT_EMPTY")
|
||||
return [str(item or "") for item in rows[0]], [list(row) for row in rows[1:]]
|
||||
raise ValueError("INPUT_TYPE_UNSUPPORTED")
|
||||
|
||||
|
||||
def _input_file(job_dir: Path, key: str) -> Path:
|
||||
directory = job_dir / "input" / key
|
||||
files = [
|
||||
path for path in directory.iterdir() if path.is_file() and not path.name.startswith(".")
|
||||
]
|
||||
if len(files) != 1:
|
||||
raise ValueError(f"INPUT_FILE_COUNT_INVALID:{key}")
|
||||
return files[0]
|
||||
|
||||
|
||||
def _file_sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _manifest(path: Path) -> dict[str, Any]:
|
||||
artifact_id, media_type = OUTPUT_MEDIA[path.name]
|
||||
return {
|
||||
"artifact_id": artifact_id,
|
||||
"filename": path.name,
|
||||
"media_type": media_type,
|
||||
"size_bytes": path.stat().st_size,
|
||||
"sha256": _file_sha256(path),
|
||||
}
|
||||
|
||||
|
||||
def _missing(value: Any) -> bool:
|
||||
return value is None or (isinstance(value, str) and not value.strip())
|
||||
|
||||
|
||||
def _finite_number(value: Any, role: str, row_number: int) -> float:
|
||||
if isinstance(value, bool) or _missing(value):
|
||||
raise ValueError(f"{role.upper()}_VALUE_NOT_NUMERIC:row={row_number}")
|
||||
try:
|
||||
result = float(value)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"{role.upper()}_VALUE_NOT_NUMERIC:row={row_number}") from exc
|
||||
if not math.isfinite(result):
|
||||
raise ValueError(f"{role.upper()}_VALUE_NOT_FINITE:row={row_number}")
|
||||
return result
|
||||
|
||||
|
||||
def _column_index(headers: list[str], name: Any, role: str) -> int:
|
||||
if not isinstance(name, str) or name not in headers:
|
||||
raise ValueError(f"{role.upper()}_COLUMN_NOT_FOUND")
|
||||
return headers.index(name)
|
||||
|
||||
|
||||
def _xy_values(
|
||||
headers: list[str], rows: list[list[Any]], binding: dict[str, Any]
|
||||
) -> tuple[np.ndarray, np.ndarray]:
|
||||
x_index = _column_index(headers, binding.get("x"), "x")
|
||||
y_index = _column_index(headers, binding.get("y"), "y")
|
||||
x = np.array(
|
||||
[
|
||||
_finite_number(row[x_index] if x_index < len(row) else None, "x", index)
|
||||
for index, row in enumerate(rows, start=2)
|
||||
],
|
||||
dtype=float,
|
||||
)
|
||||
y = np.array(
|
||||
[
|
||||
_finite_number(row[y_index] if y_index < len(row) else None, "y", index)
|
||||
for index, row in enumerate(rows, start=2)
|
||||
],
|
||||
dtype=float,
|
||||
)
|
||||
if len(x) < 2:
|
||||
raise ValueError("ANALYSIS_REQUIRES_AT_LEAST_TWO_ROWS")
|
||||
differences = np.diff(x)
|
||||
if np.any(differences == 0):
|
||||
raise ValueError("X_VALUES_DUPLICATED")
|
||||
if np.any(differences < 0):
|
||||
raise ValueError("X_VALUES_NOT_STRICTLY_INCREASING")
|
||||
return x, y
|
||||
|
||||
|
||||
def _json_number(value: float) -> float | None:
|
||||
return float(value) if math.isfinite(float(value)) else None
|
||||
|
||||
|
||||
def _column_profile(name: str, values: list[Any]) -> dict[str, Any]:
|
||||
missing = sum(_missing(item) for item in values)
|
||||
numeric: list[float] = []
|
||||
non_numeric = 0
|
||||
non_finite = 0
|
||||
for item in values:
|
||||
if _missing(item):
|
||||
continue
|
||||
if isinstance(item, bool):
|
||||
non_numeric += 1
|
||||
continue
|
||||
try:
|
||||
number = float(item)
|
||||
except (TypeError, ValueError):
|
||||
non_numeric += 1
|
||||
continue
|
||||
if not math.isfinite(number):
|
||||
non_finite += 1
|
||||
continue
|
||||
numeric.append(number)
|
||||
return {
|
||||
"column": name,
|
||||
"row_count": len(values),
|
||||
"non_missing": len(values) - missing,
|
||||
"missing": missing,
|
||||
"numeric": len(numeric),
|
||||
"non_numeric": non_numeric,
|
||||
"non_finite": non_finite,
|
||||
"minimum": min(numeric) if numeric else None,
|
||||
"maximum": max(numeric) if numeric else None,
|
||||
"mean": float(np.mean(numeric)) if numeric else None,
|
||||
}
|
||||
|
||||
|
||||
def _data_check(
|
||||
headers: list[str], rows: list[list[Any]], spec: dict[str, Any]
|
||||
) -> tuple[list[str], list[list[Any]], dict[str, Any]]:
|
||||
profiles = [
|
||||
_column_profile(
|
||||
header,
|
||||
[row[index] if index < len(row) else None for row in rows],
|
||||
)
|
||||
for index, header in enumerate(headers)
|
||||
]
|
||||
issues: list[dict[str, Any]] = []
|
||||
if any(item["missing"] for item in profiles):
|
||||
issues.append({"code": "missing", "detail": "输入包含缺失值。"})
|
||||
if any(item["non_finite"] for item in profiles):
|
||||
issues.append({"code": "non_finite", "detail": "输入包含非有限数值。"})
|
||||
binding = spec["data"]
|
||||
if "x" in binding:
|
||||
x_index = _column_index(headers, binding["x"], "x")
|
||||
x_values: list[float] = []
|
||||
x_complete = True
|
||||
for row_number, row in enumerate(rows, start=2):
|
||||
try:
|
||||
x_values.append(
|
||||
_finite_number(
|
||||
row[x_index] if x_index < len(row) else None,
|
||||
"x",
|
||||
row_number,
|
||||
)
|
||||
)
|
||||
except ValueError:
|
||||
x_complete = False
|
||||
break
|
||||
if x_complete and len(x_values) >= 2:
|
||||
differences = np.diff(np.array(x_values, dtype=float))
|
||||
if np.any(differences == 0):
|
||||
issues.append({"code": "duplicate_x", "detail": "X 列包含重复值。"})
|
||||
if np.any(differences < 0):
|
||||
issues.append(
|
||||
{"code": "non_monotonic_x", "detail": "X 列不是严格递增。"}
|
||||
)
|
||||
positive = differences[differences > 0]
|
||||
if len(positive) >= 2:
|
||||
relative_span = float(np.ptp(positive) / np.mean(positive))
|
||||
if relative_span > 1e-6:
|
||||
issues.append(
|
||||
{
|
||||
"code": "uneven_spacing",
|
||||
"detail": "X 列采样间隔不均匀。",
|
||||
"relative_spacing_span": relative_span,
|
||||
}
|
||||
)
|
||||
fail_on = set(spec["parameters"]["fail_on"])
|
||||
failed_codes = sorted({item["code"] for item in issues} & fail_on)
|
||||
result_headers = [
|
||||
"column",
|
||||
"row_count",
|
||||
"non_missing",
|
||||
"missing",
|
||||
"numeric",
|
||||
"non_numeric",
|
||||
"non_finite",
|
||||
"minimum",
|
||||
"maximum",
|
||||
"mean",
|
||||
]
|
||||
result_rows = [[item.get(header) for header in result_headers] for item in profiles]
|
||||
diagnostics = {
|
||||
"operation": "data_check",
|
||||
"passed": not failed_codes,
|
||||
"failed_checks": failed_codes,
|
||||
"issues": issues,
|
||||
"row_count": len(rows),
|
||||
"column_count": len(headers),
|
||||
}
|
||||
return result_headers, result_rows, diagnostics
|
||||
|
||||
|
||||
def _normalize(
|
||||
x: np.ndarray, y: np.ndarray, parameters: dict[str, Any]
|
||||
) -> tuple[list[str], list[list[Any]], dict[str, Any]]:
|
||||
method = parameters["method"]
|
||||
selected_x = None
|
||||
if method == "max_abs":
|
||||
divisor = float(np.max(np.abs(y)))
|
||||
elif method == "area":
|
||||
divisor = float(np.trapezoid(y, x))
|
||||
elif method == "reference":
|
||||
reference_x = float(parameters["reference_x"])
|
||||
if reference_x < x[0] or reference_x > x[-1]:
|
||||
raise ValueError("REFERENCE_X_OUTSIDE_DATA_RANGE")
|
||||
divisor = float(np.interp(reference_x, x, y))
|
||||
selected_x = reference_x
|
||||
else:
|
||||
raise ValueError("NORMALIZE_METHOD_UNSUPPORTED")
|
||||
scale = max(1.0, float(np.max(np.abs(y))))
|
||||
if abs(divisor) <= np.finfo(float).eps * scale:
|
||||
raise ValueError("NORMALIZE_DIVISOR_IS_ZERO")
|
||||
normalized = y / divisor
|
||||
rows = [[float(a), float(b), float(c)] for a, b, c in zip(x, y, normalized, strict=True)]
|
||||
return ["x", "y", "normalized_y"], rows, {
|
||||
"operation": "normalize",
|
||||
"method": method,
|
||||
"divisor": divisor,
|
||||
"reference_x": selected_x,
|
||||
"row_count": len(rows),
|
||||
}
|
||||
|
||||
|
||||
def _smooth(
|
||||
x: np.ndarray, y: np.ndarray, parameters: dict[str, Any]
|
||||
) -> tuple[list[str], list[list[Any]], dict[str, Any]]:
|
||||
window = int(parameters["window"])
|
||||
polynomial_order = int(parameters["polynomial_order"])
|
||||
if window % 2 == 0:
|
||||
raise ValueError("SMOOTH_WINDOW_MUST_BE_ODD")
|
||||
if polynomial_order >= window:
|
||||
raise ValueError("SMOOTH_POLYNOMIAL_ORDER_INVALID")
|
||||
if window > len(x):
|
||||
raise ValueError("SMOOTH_WINDOW_EXCEEDS_ROW_COUNT")
|
||||
half = window // 2
|
||||
smoothed = np.empty_like(y)
|
||||
for index in range(len(x)):
|
||||
start = min(max(index - half, 0), len(x) - window)
|
||||
stop = start + window
|
||||
local_x = x[start:stop] - x[index]
|
||||
coefficients = np.polynomial.polynomial.polyfit(
|
||||
local_x, y[start:stop], polynomial_order
|
||||
)
|
||||
smoothed[index] = coefficients[0]
|
||||
residual = y - smoothed
|
||||
rows = [
|
||||
[float(a), float(b), float(c), float(d)]
|
||||
for a, b, c, d in zip(x, y, smoothed, residual, strict=True)
|
||||
]
|
||||
return ["x", "y", "smoothed_y", "residual"], rows, {
|
||||
"operation": "smooth",
|
||||
"method": "savitzky_golay",
|
||||
"window": window,
|
||||
"polynomial_order": polynomial_order,
|
||||
"rmse": float(np.sqrt(np.mean(residual**2))),
|
||||
"boundary_handling": "shifted_full_window_local_polynomial",
|
||||
"row_count": len(rows),
|
||||
}
|
||||
|
||||
|
||||
def _differentiate(
|
||||
x: np.ndarray, y: np.ndarray, parameters: dict[str, Any]
|
||||
) -> tuple[list[str], list[list[Any]], dict[str, Any]]:
|
||||
if len(x) < 3:
|
||||
raise ValueError("DIFFERENTIATE_REQUIRES_AT_LEAST_THREE_ROWS")
|
||||
order = int(parameters["order"])
|
||||
derivative = y.copy()
|
||||
for _ in range(order):
|
||||
derivative = np.gradient(derivative, x, edge_order=2)
|
||||
rows = [
|
||||
[float(a), float(b), float(c)]
|
||||
for a, b, c in zip(x, y, derivative, strict=True)
|
||||
]
|
||||
return ["x", "y", f"derivative_order_{order}"], rows, {
|
||||
"operation": "differentiate",
|
||||
"order": order,
|
||||
"method": "numpy_gradient_nonuniform",
|
||||
"edge_order": 2,
|
||||
"row_count": len(rows),
|
||||
}
|
||||
|
||||
|
||||
def _cumulative_trapezoid(x: np.ndarray, y: np.ndarray) -> np.ndarray:
|
||||
cumulative = np.zeros_like(y)
|
||||
cumulative[1:] = np.cumsum(np.diff(x) * (y[:-1] + y[1:]) / 2)
|
||||
return cumulative
|
||||
|
||||
|
||||
def _integrate(
|
||||
x: np.ndarray, y: np.ndarray, parameters: dict[str, Any]
|
||||
) -> tuple[list[str], list[list[Any]], dict[str, Any]]:
|
||||
cumulative = _cumulative_trapezoid(x, y)
|
||||
lower = float(parameters.get("from", x[0]))
|
||||
upper = float(parameters.get("to", x[-1]))
|
||||
if lower >= upper:
|
||||
raise ValueError("INTEGRATE_RANGE_INVALID")
|
||||
if lower < x[0] or upper > x[-1]:
|
||||
raise ValueError("INTEGRATE_RANGE_OUTSIDE_DATA")
|
||||
integral = float(np.interp(upper, x, cumulative) - np.interp(lower, x, cumulative))
|
||||
rows = [
|
||||
[float(a), float(b), float(c)]
|
||||
for a, b, c in zip(x, y, cumulative, strict=True)
|
||||
]
|
||||
return ["x", "y", "cumulative_integral"], rows, {
|
||||
"operation": "integrate",
|
||||
"method": "trapezoid",
|
||||
"from": lower,
|
||||
"to": upper,
|
||||
"integral": integral,
|
||||
"full_integral": float(cumulative[-1]),
|
||||
"row_count": len(rows),
|
||||
}
|
||||
|
||||
|
||||
_T_CRITICAL_95 = (
|
||||
12.706,
|
||||
4.303,
|
||||
3.182,
|
||||
2.776,
|
||||
2.571,
|
||||
2.447,
|
||||
2.365,
|
||||
2.306,
|
||||
2.262,
|
||||
2.228,
|
||||
2.201,
|
||||
2.179,
|
||||
2.160,
|
||||
2.145,
|
||||
2.131,
|
||||
2.120,
|
||||
2.110,
|
||||
2.101,
|
||||
2.093,
|
||||
2.086,
|
||||
2.080,
|
||||
2.074,
|
||||
2.069,
|
||||
2.064,
|
||||
2.060,
|
||||
2.056,
|
||||
2.052,
|
||||
2.048,
|
||||
2.045,
|
||||
2.042,
|
||||
)
|
||||
|
||||
|
||||
def _t_critical_95(degrees_of_freedom: int) -> float:
|
||||
if degrees_of_freedom <= 0:
|
||||
raise ValueError("LINEAR_FIT_DEGREES_OF_FREEDOM_INVALID")
|
||||
if degrees_of_freedom <= len(_T_CRITICAL_95):
|
||||
return _T_CRITICAL_95[degrees_of_freedom - 1]
|
||||
anchors = ((30, 2.042), (40, 2.021), (60, 2.000), (120, 1.980), (10**9, 1.960))
|
||||
for (left_df, left_value), (right_df, right_value) in pairwise(anchors):
|
||||
if degrees_of_freedom <= right_df:
|
||||
ratio = (degrees_of_freedom - left_df) / (right_df - left_df)
|
||||
return left_value + ratio * (right_value - left_value)
|
||||
return 1.960
|
||||
|
||||
|
||||
def _linear_fit(
|
||||
x: np.ndarray, y: np.ndarray, parameters: dict[str, Any]
|
||||
) -> tuple[list[str], list[list[Any]], dict[str, Any]]:
|
||||
include_intercept = bool(parameters["include_intercept"])
|
||||
design = np.column_stack((x, np.ones_like(x))) if include_intercept else x[:, None]
|
||||
coefficients, _, rank, _ = np.linalg.lstsq(design, y, rcond=None)
|
||||
if rank != design.shape[1]:
|
||||
raise ValueError("LINEAR_FIT_DESIGN_RANK_DEFICIENT")
|
||||
fitted = design @ coefficients
|
||||
residual = y - fitted
|
||||
parameter_count = design.shape[1]
|
||||
degrees_of_freedom = len(x) - parameter_count
|
||||
if degrees_of_freedom <= 0:
|
||||
raise ValueError("LINEAR_FIT_INSUFFICIENT_ROWS")
|
||||
sum_squared_error = float(residual @ residual)
|
||||
mean_squared_error = sum_squared_error / degrees_of_freedom
|
||||
covariance = mean_squared_error * np.linalg.inv(design.T @ design)
|
||||
standard_errors = np.sqrt(np.diag(covariance))
|
||||
slope = float(coefficients[0])
|
||||
intercept = float(coefficients[1]) if include_intercept else 0.0
|
||||
slope_se = float(standard_errors[0])
|
||||
intercept_se = float(standard_errors[1]) if include_intercept else 0.0
|
||||
t_critical = _t_critical_95(degrees_of_freedom)
|
||||
slope_ci = [slope - t_critical * slope_se, slope + t_critical * slope_se]
|
||||
intercept_ci = (
|
||||
[intercept - t_critical * intercept_se, intercept + t_critical * intercept_se]
|
||||
if include_intercept
|
||||
else [0.0, 0.0]
|
||||
)
|
||||
total_sum_squares = (
|
||||
float(np.sum((y - np.mean(y)) ** 2))
|
||||
if include_intercept
|
||||
else float(y @ y)
|
||||
)
|
||||
r_squared = 1 - sum_squared_error / total_sum_squares if total_sum_squares else 1.0
|
||||
rows = [
|
||||
[float(a), float(b), float(c), float(d)]
|
||||
for a, b, c, d in zip(x, y, fitted, residual, strict=True)
|
||||
]
|
||||
return ["x", "y", "fitted_y", "residual"], rows, {
|
||||
"operation": "linear_fit",
|
||||
"include_intercept": include_intercept,
|
||||
"confidence_level": 0.95,
|
||||
"slope": slope,
|
||||
"intercept": intercept,
|
||||
"slope_standard_error": slope_se,
|
||||
"intercept_standard_error": intercept_se,
|
||||
"slope_confidence_interval": slope_ci,
|
||||
"intercept_confidence_interval": intercept_ci,
|
||||
"t_critical": t_critical,
|
||||
"degrees_of_freedom": degrees_of_freedom,
|
||||
"r_squared": r_squared,
|
||||
"rmse": math.sqrt(mean_squared_error),
|
||||
"sum_squared_error": sum_squared_error,
|
||||
"row_count": len(rows),
|
||||
}
|
||||
|
||||
|
||||
def analyze(
|
||||
headers: list[str], rows: list[list[Any]], analysis: dict[str, Any]
|
||||
) -> tuple[list[str], list[list[Any]], dict[str, Any]]:
|
||||
operation = analysis["type"]
|
||||
if operation not in OPERATIONS:
|
||||
raise ValueError("ANALYSIS_TYPE_NOT_IMPLEMENTED")
|
||||
if operation == "data_check":
|
||||
return _data_check(headers, rows, analysis)
|
||||
x, y = _xy_values(headers, rows, analysis["data"])
|
||||
parameters = analysis["parameters"]
|
||||
functions = {
|
||||
"normalize": _normalize,
|
||||
"smooth": _smooth,
|
||||
"differentiate": _differentiate,
|
||||
"integrate": _integrate,
|
||||
"linear_fit": _linear_fit,
|
||||
}
|
||||
return functions[operation](x, y, parameters)
|
||||
|
||||
|
||||
def _origin_value(value: Any) -> Any:
|
||||
if value is None or isinstance(value, (str, int, float, bool)):
|
||||
return value
|
||||
return json.dumps(value, ensure_ascii=False, sort_keys=True)
|
||||
|
||||
|
||||
def _write_origin_project(
|
||||
project: Path,
|
||||
headers: list[str],
|
||||
rows: list[list[Any]],
|
||||
result_headers: list[str],
|
||||
result_rows: list[list[Any]],
|
||||
diagnostics: dict[str, Any],
|
||||
analysis: dict[str, Any],
|
||||
) -> None:
|
||||
import originpro as op
|
||||
|
||||
op.set_show(False)
|
||||
try:
|
||||
op.new()
|
||||
raw_sheet = op.new_sheet("w", lname="RawData")
|
||||
for index, header in enumerate(headers):
|
||||
raw_sheet.from_list(
|
||||
index,
|
||||
[row[index] if index < len(row) else None for row in rows],
|
||||
lname=header,
|
||||
)
|
||||
result_sheet = op.new_sheet("w", lname="Result")
|
||||
for index, header in enumerate(result_headers):
|
||||
result_sheet.from_list(
|
||||
index,
|
||||
[row[index] if index < len(row) else None for row in result_rows],
|
||||
lname=header,
|
||||
)
|
||||
diagnostic_sheet = op.new_sheet("w", lname="Diagnostics")
|
||||
diagnostic_items = list(diagnostics.items())
|
||||
diagnostic_sheet.from_list(0, [item[0] for item in diagnostic_items], lname="metric")
|
||||
diagnostic_sheet.from_list(
|
||||
1,
|
||||
[_origin_value(item[1]) for item in diagnostic_items],
|
||||
lname="value",
|
||||
)
|
||||
if analysis["type"] == "linear_fit":
|
||||
binding = analysis["data"]
|
||||
x_index = _column_index(headers, binding["x"], "x")
|
||||
y_index = _column_index(headers, binding["y"], "y")
|
||||
fit = op.LinearFit()
|
||||
fit.set_data(raw_sheet, x_index, y_index)
|
||||
if not analysis["parameters"]["include_intercept"]:
|
||||
fit.fix_intercept(0)
|
||||
fit.report(3)
|
||||
op.save(str(project))
|
||||
finally:
|
||||
if op.oext:
|
||||
op.exit()
|
||||
|
||||
|
||||
def _validate_project(path: Path) -> None:
|
||||
if not path.is_file() or path.stat().st_size < 128:
|
||||
raise RuntimeError("ORIGIN_PROJECT_INVALID")
|
||||
|
||||
|
||||
def run(job_dir: Path) -> list[dict[str, Any]]:
|
||||
job_dir = job_dir.resolve(strict=True)
|
||||
request_record = json.loads(
|
||||
(job_dir / "request" / "request.json").read_text(encoding="utf-8")
|
||||
)
|
||||
request = request_record["request"]
|
||||
input_spec = request["inputs"][0]
|
||||
analysis = request["operation"]["analysis"]
|
||||
if analysis["data"]["input"] != input_spec["key"]:
|
||||
raise ValueError("ANALYSIS_INPUT_KEY_NOT_BOUND")
|
||||
input_path = _input_file(job_dir, input_spec["key"])
|
||||
headers, rows = _read_rows(input_path, (input_spec.get("selector") or {}).get("sheet"))
|
||||
if not headers or len(set(headers)) != len(headers) or any(not item for item in headers):
|
||||
raise ValueError("INPUT_HEADERS_INVALID")
|
||||
result_headers, result_rows, diagnostics = analyze(headers, rows, analysis)
|
||||
|
||||
output = job_dir / "output"
|
||||
output.mkdir(exist_ok=True)
|
||||
result_table = output / "result-table.csv"
|
||||
diagnostics_path = output / "diagnostics.json"
|
||||
analysis_spec_path = output / "analysis-spec.json"
|
||||
provenance_path = output / "provenance.json"
|
||||
project_path = output / "analysis.opju"
|
||||
_atomic_csv(result_table, result_headers, result_rows)
|
||||
_atomic_json(diagnostics_path, diagnostics)
|
||||
_atomic_json(analysis_spec_path, request)
|
||||
_write_origin_project(
|
||||
project_path,
|
||||
headers,
|
||||
rows,
|
||||
result_headers,
|
||||
result_rows,
|
||||
diagnostics,
|
||||
analysis,
|
||||
)
|
||||
_validate_project(project_path)
|
||||
try:
|
||||
originpro_version = version("originpro")
|
||||
except PackageNotFoundError:
|
||||
originpro_version = "embedded"
|
||||
provenance = {
|
||||
"adapter_version": ADAPTER_VERSION,
|
||||
"algorithm_version": ALGORITHM_VERSION,
|
||||
"originpro_version": originpro_version,
|
||||
"numpy_version": np.__version__,
|
||||
"request_digest": request_record["request_digest"],
|
||||
"input": {
|
||||
"key": input_spec["key"],
|
||||
"filename": input_path.name,
|
||||
"sha256": _file_sha256(input_path),
|
||||
"sheet": (input_spec.get("selector") or {}).get("sheet"),
|
||||
},
|
||||
"operation": analysis["type"],
|
||||
"origin_linear_fit_report": analysis["type"] == "linear_fit",
|
||||
}
|
||||
_atomic_json(provenance_path, provenance)
|
||||
|
||||
artifacts = [
|
||||
_manifest(project_path),
|
||||
_manifest(result_table),
|
||||
_manifest(diagnostics_path),
|
||||
_manifest(analysis_spec_path),
|
||||
_manifest(provenance_path),
|
||||
]
|
||||
if any(item["key"] == "result_workbook" for item in request["outputs"]):
|
||||
workbook_path = output / "result-table.xlsx"
|
||||
_atomic_xlsx(workbook_path, result_headers, result_rows)
|
||||
artifacts.append(_manifest(workbook_path))
|
||||
return artifacts
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if sys.argv[1:] == ["--probe"]:
|
||||
return _probe()
|
||||
if len(sys.argv) != 2:
|
||||
print("[ERR] Usage: worker.py <job-directory>", file=sys.stderr)
|
||||
return 2
|
||||
job_dir = Path(sys.argv[1])
|
||||
request_record: dict[str, Any] = {}
|
||||
try:
|
||||
request_record = json.loads(
|
||||
(job_dir / "request" / "request.json").read_text(encoding="utf-8")
|
||||
)
|
||||
artifacts = run(job_dir)
|
||||
terminal = {
|
||||
"job_id": request_record["job_id"],
|
||||
"lease_id": request_record["lease_id"],
|
||||
"request_digest": request_record["request_digest"],
|
||||
"status": "succeeded",
|
||||
"error": {},
|
||||
"artifact_manifest": artifacts,
|
||||
"terminal_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
_atomic_json(job_dir / "artifacts.json", artifacts)
|
||||
_atomic_json(job_dir / "terminal.json", terminal)
|
||||
print("[OK] Origin analysis job completed.")
|
||||
return 0
|
||||
except Exception as exception: # noqa: BLE001 - terminalize every worker failure
|
||||
terminal = {
|
||||
"job_id": request_record.get("job_id", ""),
|
||||
"lease_id": request_record.get("lease_id", ""),
|
||||
"request_digest": request_record.get("request_digest", ""),
|
||||
"status": "failed",
|
||||
"error": {"code": type(exception).__name__, "detail": str(exception)[:500]},
|
||||
"artifact_manifest": [],
|
||||
"terminal_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
_atomic_json(job_dir / "terminal.json", terminal)
|
||||
print(f"[ERR] {type(exception).__name__}: {exception}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
@echo off
|
||||
setlocal EnableExtensions EnableDelayedExpansion
|
||||
cd /d "%~dp0"
|
||||
|
||||
set "OUTPUT_ROOT=%~f1"
|
||||
if "%~1"=="" set "OUTPUT_ROOT=%~dp0dist"
|
||||
set "ADAPTER_DIR_NAME=origin.analysis@v1"
|
||||
set "ARCHIVE_NAME=origin.analysis@v1-adapter.zip"
|
||||
set "PACKAGE_DIR=!OUTPUT_ROOT!\!ADAPTER_DIR_NAME!"
|
||||
set "ARCHIVE_PATH=!OUTPUT_ROOT!\!ARCHIVE_NAME!"
|
||||
set "HASH_PATH=!ARCHIVE_PATH!.sha256"
|
||||
|
||||
where.exe tar.exe >nul 2>&1
|
||||
if errorlevel 1 goto :missing_tool
|
||||
where.exe certutil.exe >nul 2>&1
|
||||
if errorlevel 1 goto :missing_tool
|
||||
if not exist "!OUTPUT_ROOT!" mkdir "!OUTPUT_ROOT!"
|
||||
if exist "!PACKAGE_DIR!" rmdir /s /q "!PACKAGE_DIR!"
|
||||
if exist "!ARCHIVE_PATH!" del /f /q "!ARCHIVE_PATH!"
|
||||
if exist "!HASH_PATH!" del /f /q "!HASH_PATH!"
|
||||
mkdir "!PACKAGE_DIR!"
|
||||
|
||||
copy /y "adapters\origin.analysis@v1\adapter.json" "!PACKAGE_DIR!\adapter.json" >nul
|
||||
copy /y "adapters\origin.analysis@v1\worker.py" "!PACKAGE_DIR!\worker.py" >nul
|
||||
copy /y "adapters\origin.analysis@v1\acceptance.py" "!PACKAGE_DIR!\acceptance.py" >nul
|
||||
copy /y "adapters\origin.analysis@v1\requirements.txt" "!PACKAGE_DIR!\requirements.txt" >nul
|
||||
copy /y "..\software-contracts\origin.analysis.v1.json" "!PACKAGE_DIR!\origin.analysis.v1.json" >nul
|
||||
if errorlevel 1 goto :failed
|
||||
|
||||
tar.exe -a -c -f "!ARCHIVE_PATH!" -C "!OUTPUT_ROOT!" "!ADAPTER_DIR_NAME!"
|
||||
if errorlevel 1 goto :failed
|
||||
set "SHA256="
|
||||
for /f "tokens=*" %%H in ('certutil.exe -hashfile "!ARCHIVE_PATH!" SHA256 ^| findstr.exe /R /C:"^[0-9A-Fa-f][0-9A-Fa-f ]*[0-9A-Fa-f]$"') do set "SHA256=%%H"
|
||||
set "SHA256=!SHA256: =!"
|
||||
if not defined SHA256 goto :failed
|
||||
>"!HASH_PATH!" echo !SHA256! !ARCHIVE_NAME!
|
||||
echo [OK] Adapter package: !ARCHIVE_PATH!
|
||||
echo [OK] SHA256: !SHA256!
|
||||
exit /b 0
|
||||
|
||||
:missing_tool
|
||||
echo [ERR] Windows tar.exe and certutil.exe are required.
|
||||
exit /b 1
|
||||
|
||||
:failed
|
||||
echo [ERR] Origin analysis adapter packaging failed.
|
||||
exit /b 1
|
||||
Loading…
Reference in New Issue