feat(software): add professional software job center

This commit is contained in:
caoqianming 2026-08-13 15:59:31 +08:00
parent ded8cba54c
commit 8aafe3272f
18 changed files with 703 additions and 14 deletions

View File

@ -8,7 +8,7 @@
## Unreleased ## Unreleased
<!-- 在这里整理尚未发布的用户可感知变化;功能稳定后随 release commit 填入版本号和日期。 --> - 新增专业软件 Job 中心Agent 可提交、查询和停止 Windows Node 上的受控软件任务;用户可在右下角跨对话查看进度、收到完成通知,并回到原对话分析结果。
## 0.65.2 — 2026-08-13 ## 0.65.2 — 2026-08-13

View File

@ -472,7 +472,9 @@ Node 通过 `Authorization: Bearer` 与 `X-Node-Id` 建立 `/v1/software-nodes/c
第五阶段完成输出上传与发布Node 只按固定 manifest ID 逐项流式 PUT并携带 Node、lease、request digest 与内容摘要;云端重新绑定任务身份,不信 Node 提供的路径或媒体类型。文件先进入用户根下隐藏暂存区,固定文件名、单文件/总大小和 SHA-256 全部验证后,目录级原子移动到 `<working_dir>/origin/<job_id>/`,再登记平台 artifact UUID 并写成功终态。重复 PUT、complete 和重连均按摘要幂等;部分上传不可见,只有完整集合才能发布。 第五阶段完成输出上传与发布Node 只按固定 manifest ID 逐项流式 PUT并携带 Node、lease、request digest 与内容摘要;云端重新绑定任务身份,不信 Node 提供的路径或媒体类型。文件先进入用户根下隐藏暂存区,固定文件名、单文件/总大小和 SHA-256 全部验证后,目录级原子移动到 `<working_dir>/origin/<job_id>/`,再登记平台 artifact UUID 并写成功终态。重复 PUT、complete 和重连均按摘要幂等;部分上传不可见,只有完整集合才能发布。
后续仍需实现协作取消和 Token 轮换;不得以任意命令或脚本接口临时代替。当前单活与 offer 选择仍只覆盖单 Web 进程,生产启用多实例前必须增加 Redis/PG fencing 或固定路由到单一控制面实例。 第六阶段增加用户级 Job 中心与 Agent typed tools。`software_capability_list` 只暴露固定能力及当前在线空闲节点数,`software_job_submit/status/cancel` 在构造时绑定当前 user/task模型不能跨用户或跨对话指定归属。右下角 Job 中心按用户聚合各对话任务,活动期短轮询、空闲期降频;终态变化通知用户,成功任务可回到原对话发起分析。取消采用协作协议:未派发任务直接终止,已派发任务先进入 `cancelling`,云端通过 WebSocket 发送并在心跳时重放 `job_cancel`Node 杀死固定 Worker 进程树后回报 `cancelled`;终态写入仍由云端账本裁决。
后续仍需实现 Token 轮换;不得以任意命令或脚本接口临时代替。当前 Job 中心采用轮询而非用户事件推送,单活与 offer 选择仍只覆盖单 Web 进程;生产启用多实例前必须增加 Redis/PG fencing 或固定路由到单一控制面实例。
--- ---

View File

@ -2,7 +2,7 @@
> 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9` > 配合 `DESIGN.md`。本文件只记 phase 状态、决策偏差、文件量、下一步。每条 1-2 句:做了啥 + 关键判断;细节查 `git log` / `git diff` / `DESIGN §7.9`
最后更新:2026-08-13(用户消息结构化附件与图片预览,bump 0.65.2) 最后更新:2026-08-13(专业软件 Job 中心开发中,未发版)
--- ---
@ -22,6 +22,8 @@
### 2026-08-13 ### 2026-08-13
- **08-13 / Unreleased / 专业软件 Job 中心**:Agent 新增固定能力发现、提交、状态和取消工具后端提供用户级跨对话任务列表与协作取消Windows Node 可终止固定 Worker 进程树并幂等回报取消终态Web 右下角展示活动/最近任务、进度、完成通知、停止与回到原对话分析入口。相关 Python 78 项、JavaScript 语法、Python 编译、.NET build 与 diff 检查通过;数据库保持在 0031未连库、未执行 migration。
- **08-13 / 0.65.2 / 用户消息结构化附件 + 对话内图片预览**:新增 0031 `messages.attachment_refs`,新客户端将附件作为结构化字段发送,数据库正文仅保留用户自然语言;后端按 task working_dir 校验路径并在内存模型上下文中补附件提示,旧客户端与历史正文标记继续兼容。用户消息即时态和历史态统一展示附件 chip图片额外显示可点击缩略图相关 Python 34 项、全部前端 Node 26 项、Python/JavaScript 语法、Alembic 单 head 与 diff 检查通过,未连接或迁移生产 DB。 - **08-13 / 0.65.2 / 用户消息结构化附件 + 对话内图片预览**:新增 0031 `messages.attachment_refs`,新客户端将附件作为结构化字段发送,数据库正文仅保留用户自然语言;后端按 task working_dir 校验路径并在内存模型上下文中补附件提示,旧客户端与历史正文标记继续兼容。用户消息即时态和历史态统一展示附件 chip图片额外显示可点击缩略图相关 Python 34 项、全部前端 Node 26 项、Python/JavaScript 语法、Alembic 单 head 与 diff 检查通过,未连接或迁移生产 DB。
### 2026-08-12 ### 2026-08-12

6
RUN.md
View File

@ -1054,13 +1054,15 @@ sudo xfs_quota -x -c "limit -p bhard=10g zcbot_<user_uuid>" /opt
### Windows Node 内网 MVP开发中 ### Windows Node 内网 MVP开发中
先执行 `alembic upgrade head` 创建 `software_node_enrollments``software_nodes`。不要在未确认目标数据库时运行迁移;本机 `.env``ZCBOT_DB_URL` 可能是生产隧道。 先执行 `alembic upgrade head` 创建 `software_node_enrollments`、`software_nodes` 和 `software_jobs`。不要在未确认目标数据库时运行迁移;本机 `.env``ZCBOT_DB_URL` 可能是生产隧道。
云端当前提供: 云端当前提供:
- 管理员 `POST /v1/admin/software-node-enrollments` 创建一次性注册码; - 管理员 `POST /v1/admin/software-node-enrollments` 创建一次性注册码;
- Node `POST /v1/software-nodes/enroll` 注册并一次性取得 `node_id`、`node_token` - Node `POST /v1/software-nodes/enroll` 注册并一次性取得 `node_id`、`node_token`
- Node 携带 `Authorization: Bearer <node_token>``X-Node-Id` 连接 `WS /v1/software-nodes/connect` - Node 携带 `Authorization: Bearer <node_token>``X-Node-Id` 连接 `WS /v1/software-nodes/connect`
- 用户或 Agent 通过 `POST /v1/tasks/{task_id}/software-jobs` 提交专业软件任务;
- 用户通过 `GET /v1/software-jobs` 查看本人跨对话任务,可用 `task_id`、`active_only` 和 `limit` 筛选,`POST /v1/software-jobs/{job_id}/cancel` 请求停止;
- 管理员 `GET /v1/admin/software-nodes` 查看节点,`PATCH /v1/admin/software-nodes/{node_id}` 启停节点,`DELETE /v1/admin/software-nodes/{node_id}` 永久删除节点身份。 - 管理员 `GET /v1/admin/software-nodes` 查看节点,`PATCH /v1/admin/software-nodes/{node_id}` 启停节点,`DELETE /v1/admin/software-nodes/{node_id}` 永久删除节点身份。
Node API 只能绑定受控内网地址并由安全组限制来源 IP。当前 HTTP/WS 链路不加密;跨安全域、公网或不可信终端接入前,必须先升级 HTTPS/WSS。多 Web 实例部署时Node API 暂时固定路由到单一实例,直至 Connection Manager 增加跨实例 fencing。 Node API 只能绑定受控内网地址并由安全组限制来源 IP。当前 HTTP/WS 链路不加密;跨安全域、公网或不可信终端接入前,必须先升级 HTTPS/WSS。多 Web 实例部署时Node API 暂时固定路由到单一实例,直至 Connection Manager 增加跨实例 fencing。
@ -1084,6 +1086,8 @@ windows-node\install-origin-runtime.ps1 -BootstrapPython D:\programs\Python312\p
默认解释器为 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\python.exe`。如需放在其他受管目录,设置机器级 `ZCBOT_ORIGIN_PYTHON` 为绝对 `python.exe` 路径后重启 Node。运行时固定依赖见 `windows-node/origin-worker/requirements.txt`;任务请求无权选择解释器、脚本或路径。当前 Worker 支持 CSV/XLSX/JSON 输入,`line`、`scatter`、`line_scatter` 与 OPJU/PNG/SVG/PDF 输出。成功产物由 Node 流式上传,全部校验通过后发布到任务工作目录 `origin/<job_id>/`;上传中断会在重连时幂等续传。 默认解释器为 `%ProgramData%\Zcbot\WindowsNode\runtimes\origin\python.exe`。如需放在其他受管目录,设置机器级 `ZCBOT_ORIGIN_PYTHON` 为绝对 `python.exe` 路径后重启 Node。运行时固定依赖见 `windows-node/origin-worker/requirements.txt`;任务请求无权选择解释器、脚本或路径。当前 Worker 支持 CSV/XLSX/JSON 输入,`line`、`scatter`、`line_scatter` 与 OPJU/PNG/SVG/PDF 输出。成功产物由 Node 流式上传,全部校验通过后发布到任务工作目录 `origin/<job_id>/`;上传中断会在重连时幂等续传。
Web 用户登录后,右下角 Job 中心会聚合本人最近任务。活动任务约 4 秒刷新一次,空闲时降为约 30 秒停止已派发任务是协作取消状态先显示“正在停止”Node 在线时立即接收断线后在下次连接或心跳时重放。Agent 可调用 `software_capability_list`、`software_job_submit`、`software_job_status` 和 `software_job_cancel`;提交工具只创建固定 schema 的持久任务,不会阻塞当前对话等待完成。
注册配置写入 `%ProgramData%\Zcbot\WindowsNode\node.json`Token 使用 DPAPI `LocalMachine` 加密ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号注册并运行 Node。当前 MVP 以该账号的登录后计划任务启动,不安装 Windows Service。 注册配置写入 `%ProgramData%\Zcbot\WindowsNode\node.json`Token 使用 DPAPI `LocalMachine` 加密ACL 仅允许注册账号和 `SYSTEM`。应始终用同一专用 Windows 账号注册并运行 Node。当前 MVP 以该账号的登录后计划任务启动,不安装 Windows Service。
直接双击 EXE 启动托盘 UI红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开配置窗。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless` 直接双击 EXE 启动托盘 UI红点为未注册/身份失效,黄点为连接中,绿点为在线;双击托盘图标打开配置窗。原 CLI 注册入口继续保留,无 UI 模式使用 `Zcbot.WindowsNode.exe run --headless`

View File

@ -8,7 +8,7 @@ from datetime import datetime, timedelta, timezone
from hashlib import sha256 from hashlib import sha256
from uuid import UUID, uuid4 from uuid import UUID, uuid4
from sqlalchemy import select from sqlalchemy import desc, select
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
from core.software_nodes import SUPPORTED_CAPABILITIES from core.software_nodes import SUPPORTED_CAPABILITIES
@ -157,6 +157,121 @@ def _job_dict(row: SoftwareJob) -> dict:
} }
def list_jobs(
user_id: UUID,
*,
task_id: UUID | None = None,
active_only: bool = False,
limit: int = 50,
) -> list[dict]:
"""列出用户的软件任务;用于全局 Job 中心和 Agent 查询。"""
limit = max(1, min(int(limit), 100))
with session_scope() as session:
statement = (
select(SoftwareJob, Task.name, SoftwareNode.name)
.join(Task, Task.task_id == SoftwareJob.task_id)
.outerjoin(SoftwareNode, SoftwareNode.node_id == SoftwareJob.node_id)
.where(SoftwareJob.user_id == user_id)
)
if task_id is not None:
statement = statement.where(SoftwareJob.task_id == task_id)
if active_only:
statement = statement.where(
SoftwareJob.status.in_(
{
"queued",
"offered",
"dispatched",
"running",
"disconnected",
"cancelling",
}
)
)
rows = session.execute(
statement.order_by(
desc(SoftwareJob.created_at), desc(SoftwareJob.job_id)
).limit(limit)
).all()
results: list[dict] = []
for job, task_name, node_name in rows:
item = _job_dict(job)
item.update(
{
"task_name": task_name,
"node_name": node_name,
"input": job.input_manifest,
"request_summary": _request_summary(job),
}
)
results.append(item)
return results
def _request_summary(job: SoftwareJob) -> dict:
plot = job.request.get("plot") or {}
output = job.request.get("output") or {}
return {
"display_name": (
"Origin 科研绘图"
if job.capability == "origin.plot@v1"
else job.capability
),
"title": str(plot.get("title") or ""),
"formats": list(output.get("formats") or []),
}
def request_job_cancel(user_id: UUID, job_id: UUID) -> tuple[dict, dict | None]:
"""持久化取消意图queued 直接终止,已分派任务返回 Node 消息。"""
now = datetime.now(timezone.utc)
with session_scope() as session:
job = session.execute(
select(SoftwareJob).where(
SoftwareJob.job_id == job_id,
SoftwareJob.user_id == user_id,
).with_for_update()
).scalar_one_or_none()
if job is None:
raise SoftwareJobError("job not found")
if job.status in {"succeeded", "failed", "cancelled"}:
return _job_dict(job), None
if job.status == "queued" or job.node_id is None or job.lease_id is None:
job.status = "cancelled"
job.stage = "terminal"
job.error = {"code": "USER_CANCELLED", "detail": "Cancelled before dispatch."}
job.terminal_at = now
return _job_dict(job), None
job.status = "cancelling"
job.stage = "cancel_requested"
payload = {
"job_id": str(job.job_id),
"lease_id": str(job.lease_id),
"request_digest": job.request_digest,
}
return _job_dict(job), {"node_id": job.node_id, "payload": payload}
def pending_node_cancellations(node_id: UUID) -> list[dict]:
"""节点重连或心跳时重放未确认的取消意图。"""
with session_scope() as session:
rows = session.execute(
select(SoftwareJob).where(
SoftwareJob.node_id == node_id,
SoftwareJob.status == "cancelling",
)
).scalars()
return [
{
"job_id": str(job.job_id),
"lease_id": str(job.lease_id),
"request_digest": job.request_digest,
}
for job in rows
if job.lease_id is not None
]
def create_job( def create_job(
user_id: UUID, user_id: UUID,
task_id: UUID, task_id: UUID,
@ -475,7 +590,7 @@ def respond_to_offer(node_id: UUID, *, accepted: bool, payload: dict) -> None:
raise SoftwareJobError("job offer is stale or does not belong to this node") raise SoftwareJobError("job offer is stale or does not belong to this node")
if ( if (
accepted accepted
and job.status in {"dispatched", "running", "succeeded", "failed", "cancelled"} and job.status in {"dispatched", "running", "cancelling", "succeeded", "failed", "cancelled"}
and payload.get("request_digest") == job.request_digest and payload.get("request_digest") == job.request_digest
): ):
return return
@ -518,7 +633,7 @@ def update_job_state(node_id: UUID, payload: dict) -> None:
select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update() select(SoftwareJob).where(SoftwareJob.job_id == job_id).with_for_update()
).scalar_one_or_none() ).scalar_one_or_none()
_assert_job_message(job, node_id, lease_id, digest) _assert_job_message(job, node_id, lease_id, digest)
if job.status in {"succeeded", "failed", "cancelled"}: if job.status in {"succeeded", "failed", "cancelled", "cancelling"}:
return return
if not _can_accept_state(job.status): if not _can_accept_state(job.status):
raise SoftwareJobError("job state cannot advance from its current status") raise SoftwareJobError("job state cannot advance from its current status")
@ -578,7 +693,7 @@ def record_job_terminal(node_id: UUID, payload: dict) -> None:
if job.status != terminal_status: if job.status != terminal_status:
raise SoftwareJobError("job terminal status conflicts with existing terminal") raise SoftwareJobError("job terminal status conflicts with existing terminal")
return return
if job.status not in {"offered", "dispatched", "running", "disconnected"}: if job.status not in {"offered", "dispatched", "running", "disconnected", "cancelling"}:
raise SoftwareJobError("job terminal cannot advance from its current status") raise SoftwareJobError("job terminal cannot advance from its current status")
job.status = terminal_status job.status = terminal_status
job.stage = "terminal" job.stage = "terminal"

View File

@ -47,6 +47,12 @@ from tools.run_python import RunPythonTool
from tools.schedule import ( from tools.schedule import (
ScheduleCancelTool, ScheduleCreateTool, ScheduleListTool, ScheduleUpdateTool, ScheduleCancelTool, ScheduleCreateTool, ScheduleListTool, ScheduleUpdateTool,
) )
from tools.software_jobs import (
SoftwareCapabilityListTool,
SoftwareJobCancelTool,
SoftwareJobStatusTool,
SoftwareJobSubmitTool,
)
from tools.seedance import SeedanceTool from tools.seedance import SeedanceTool
from tools.seedream import SeedreamTool from tools.seedream import SeedreamTool
from tools.send_email import SendEmailTool, smtp_configured from tools.send_email import SendEmailTool, smtp_configured
@ -207,6 +213,14 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
ScheduleCancelTool(ctx.uid, **base), ScheduleCancelTool(ctx.uid, **base),
] ]
def _software_jobs() -> list:
return [
SoftwareCapabilityListTool(ctx.uid, ctx.task_id, **base),
SoftwareJobSubmitTool(ctx.uid, ctx.task_id, **base),
SoftwareJobStatusTool(ctx.uid, ctx.task_id, **base),
SoftwareJobCancelTool(ctx.uid, ctx.task_id, **base),
]
def _send_email() -> list: def _send_email() -> list:
# base_dir 用 working_dir_path(宿主 task 目录):send_email 在宿主进程读附件, # base_dir 用 working_dir_path(宿主 task 目录):send_email 在宿主进程读附件,
# docker 下 agent 给的相对路径相对容器 workdir=task_dir,翻回宿主即 working_dir_path; # docker 下 agent 给的相对路径相对容器 workdir=task_dir,翻回宿主即 working_dir_path;
@ -294,6 +308,7 @@ def build_tools(ctx: ToolContext) -> dict[str, Any]:
("skill_authoring", lambda: True, _skill_authoring), ("skill_authoring", lambda: True, _skill_authoring),
# 定时 run 内不挂 schedule_*(防任务造任务自我繁殖);仅交互对话可建/管 job。 # 定时 run 内不挂 schedule_*(防任务造任务自我繁殖);仅交互对话可建/管 job。
("schedules", lambda: not ctx.scheduled_run, _schedules), ("schedules", lambda: not ctx.scheduled_run, _schedules),
("software_jobs", lambda: not ctx.scheduled_run, _software_jobs),
("send_email", smtp_configured, _send_email), ("send_email", smtp_configured, _send_email),
("wechat_push", wechat_push_available, _wechat_push), ("wechat_push", wechat_push_available, _wechat_push),
("run_python", lambda: ctx.caps.enable_run_python, _run_python), ("run_python", lambda: ctx.caps.enable_run_python, _run_python),

View File

@ -0,0 +1,80 @@
from __future__ import annotations
import json
import unittest
from unittest.mock import patch
from uuid import uuid4
from tools.software_jobs import (
SoftwareCapabilityListTool,
SoftwareJobCancelTool,
SoftwareJobStatusTool,
SoftwareJobSubmitTool,
)
class SoftwareJobToolTests(unittest.TestCase):
def setUp(self):
self.user_id = uuid4()
self.task_id = uuid4()
def test_capability_list_reports_current_capacity(self):
nodes = [
{
"status": "online",
"capabilities": ["origin.plot@v1"],
"runtime": {"available_slots": 1},
},
{
"status": "offline",
"capabilities": ["origin.plot@v1"],
"runtime": {"available_slots": 1},
},
]
with patch("tools.software_jobs.list_nodes", return_value=nodes):
result = json.loads(
SoftwareCapabilityListTool(self.user_id, self.task_id).execute()
)
self.assertEqual(result["capabilities"][0]["available_nodes"], 1)
def test_submit_injects_current_user_and_task(self):
created = {"job_id": str(uuid4()), "status": "queued"}
tool = SoftwareJobSubmitTool(self.user_id, self.task_id)
with patch("tools.software_jobs.create_job", return_value=(created, True)) as create:
result = json.loads(tool.execute("origin.plot@v1", {"input_id": str(uuid4())}))
self.assertTrue(result["created"])
self.assertEqual(create.call_args.args[:2], (self.user_id, self.task_id))
self.assertEqual(create.call_args.kwargs["capability"], "origin.plot@v1")
def test_status_and_cancel_reject_cross_task_job(self):
foreign = {"job_id": str(uuid4()), "task_id": str(uuid4())}
with patch("tools.software_jobs.get_job", return_value=foreign):
status = SoftwareJobStatusTool(self.user_id, self.task_id).execute(
foreign["job_id"]
)
cancel = SoftwareJobCancelTool(self.user_id, self.task_id).execute(
foreign["job_id"]
)
self.assertIn("not found", status)
self.assertIn("not found", cancel)
def test_cancel_uses_user_scoped_service(self):
job_id = uuid4()
current = {"job_id": str(job_id), "task_id": str(self.task_id)}
cancelled = {**current, "status": "cancelled"}
with (
patch("tools.software_jobs.get_job", return_value=current),
patch(
"tools.software_jobs.request_job_cancel",
return_value=(cancelled, None),
) as request_cancel,
):
result = json.loads(
SoftwareJobCancelTool(self.user_id, self.task_id).execute(str(job_id))
)
self.assertEqual(result["status"], "cancelled")
request_cancel.assert_called_once_with(self.user_id, job_id)
if __name__ == "__main__":
unittest.main()

View File

@ -25,6 +25,8 @@ from core.software_jobs import (
respond_to_offer, respond_to_offer,
update_job_state, update_job_state,
validate_output_manifest, validate_output_manifest,
list_jobs,
request_job_cancel,
) )
from web.routers.software_nodes import NodeConnectionManager, _bearer from web.routers.software_nodes import NodeConnectionManager, _bearer
@ -130,6 +132,52 @@ class SoftwareNodeMigrationTests(unittest.TestCase):
class SoftwareJobProtocolTests(unittest.TestCase): class SoftwareJobProtocolTests(unittest.TestCase):
@patch("core.software_jobs.session_scope")
def test_queued_job_cancels_without_node_message(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
job = type("Job", (), {})()
job.job_id = uuid4(); job.user_id = uuid4(); job.task_id = uuid4()
job.capability = "origin.plot@v1"; job.request_digest = "a" * 64
job.node_id = None; job.lease_id = None; job.status = "queued"; job.stage = ""
job.progress = 0; job.metrics = {}; job.error = {}; job.artifact_manifest = []
job.created_at = None; job.started_at = None; job.terminal_at = None
session.execute.return_value.scalar_one_or_none.return_value = job
result, node_message = request_job_cancel(job.user_id, job.job_id)
self.assertEqual(result["status"], "cancelled")
self.assertIsNone(node_message)
self.assertEqual(job.error["code"], "USER_CANCELLED")
@patch("core.software_jobs.session_scope")
def test_running_job_persists_cancel_before_sending(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
job = type("Job", (), {})()
job.job_id = uuid4(); job.user_id = uuid4(); job.task_id = uuid4()
job.capability = "origin.plot@v1"; job.request_digest = "b" * 64
job.node_id = uuid4(); job.lease_id = uuid4(); job.status = "running"
job.stage = "origin_running"; job.progress = 10; job.metrics = {}; job.error = {}
job.artifact_manifest = []; job.created_at = None; job.started_at = None; job.terminal_at = None
session.execute.return_value.scalar_one_or_none.return_value = job
result, node_message = request_job_cancel(job.user_id, job.job_id)
self.assertEqual(result["status"], "cancelling")
self.assertEqual(node_message["node_id"], job.node_id)
self.assertEqual(node_message["payload"]["lease_id"], str(job.lease_id))
@patch("core.software_jobs.session_scope")
def test_job_list_is_enriched_for_job_center(self, session_scope) -> None:
session = session_scope.return_value.__enter__.return_value
job = type("Job", (), {})()
job.job_id = uuid4(); job.task_id = uuid4(); job.capability = "origin.plot@v1"
job.request_digest = "c" * 64; job.node_id = uuid4(); job.status = "running"
job.stage = "origin_running"; job.progress = 20; job.metrics = {}; job.error = {}
job.artifact_manifest = []; job.input_manifest = {"filename": "input.xlsx"}
job.request = {"plot": {"title": "Test"}, "output": {"formats": ["png"]}}
job.created_at = None; job.started_at = None; job.terminal_at = None
session.execute.return_value.all.return_value = [(job, "材料仿真", "LAB-01")]
results = list_jobs(uuid4(), limit=10)
self.assertEqual(results[0]["task_name"], "材料仿真")
self.assertEqual(results[0]["node_name"], "LAB-01")
self.assertEqual(results[0]["request_summary"]["display_name"], "Origin 科研绘图")
def test_origin_request_is_canonical_and_rejects_extra_fields(self) -> None: def test_origin_request_is_canonical_and_rejects_extra_fields(self) -> None:
request = { request = {
"schema_version": 1, "schema_version": 1,

View File

@ -26,6 +26,15 @@ def _frontend_source() -> str:
class StaticVendorTests(unittest.TestCase): class StaticVendorTests(unittest.TestCase):
def test_dev_console_has_software_job_center(self) -> None:
html = DEV_HTML.read_text(encoding="utf-8")
source = (JS_DIR / "software_jobs.js").read_text(encoding="utf-8")
self.assertIn('id="software-job-center"', html)
self.assertIn('id="software-job-toggle"', html)
self.assertIn('"/v1/software-jobs?limit=50"', source)
self.assertIn("/cancel`", source)
self.assertIn("分析结果", source)
def test_admin_can_create_windows_node_enrollment_code(self) -> None: def test_admin_can_create_windows_node_enrollment_code(self) -> None:
html = ADMIN_HTML.read_text(encoding="utf-8") html = ADMIN_HTML.read_text(encoding="utf-8")
admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8") admin_js = (JS_DIR / "admin.js").read_text(encoding="utf-8")

View File

@ -111,6 +111,8 @@ class AuthGateTests(unittest.TestCase):
("POST", "/v1/asr/transcribe"), ("POST", "/v1/asr/transcribe"),
("GET", "/v1/admin/overview"), ("GET", "/v1/admin/overview"),
("GET", "/v1/admin/software-nodes"), ("GET", "/v1/admin/software-nodes"),
("GET", "/v1/software-jobs"),
("POST", "/v1/software-jobs/00000000-0000-0000-0000-000000000000/cancel"),
("DELETE", "/v1/admin/software-nodes/00000000-0000-0000-0000-000000000000"), ("DELETE", "/v1/admin/software-nodes/00000000-0000-0000-0000-000000000000"),
("GET", "/v1/admin/tool-wire-health"), ("GET", "/v1/admin/tool-wire-health"),
("GET", "/v1/admin/external-system-definitions"), ("GET", "/v1/admin/external-system-definitions"),

View File

@ -169,7 +169,10 @@ class WindowsNodeSourceTests(unittest.TestCase):
self.assertIn("startInfo.ArgumentList.Add(jobDirectory)", runner) self.assertIn("startInfo.ArgumentList.Add(jobDirectory)", runner)
self.assertIn('Path.Combine(jobDirectory, "terminal.json")', runner) self.assertIn('Path.Combine(jobDirectory, "terminal.json")', runner)
self.assertIn('"NODE_RESTARTED_DURING_JOB"', runner) self.assertIn('"NODE_RESTARTED_DURING_JOB"', runner)
self.assertNotIn("RunAsync(RecoverableJob job, CancellationToken", runner) self.assertIn("CancellationTokenSource.CreateLinkedTokenSource", runner)
self.assertIn("process.Kill(entireProcessTree: true)", runner)
self.assertIn('type.GetString() == "job_cancel"', connection)
self.assertIn('"cancelled", "USER_CANCELLED"', connection)
self.assertIn("origin-worker\\worker.py", project) self.assertIn("origin-worker\\worker.py", project)
self.assertIn("if op.oext:", worker) self.assertIn("if op.oext:", worker)
self.assertIn("op.exit()", worker) self.assertIn("op.exit()", worker)

122
tools/software_jobs.py Normal file
View File

@ -0,0 +1,122 @@
"""Agent 可调用的专业软件任务工具。"""
from __future__ import annotations
import json
from uuid import UUID, uuid4
from core.software_jobs import (
SoftwareJobError,
create_job,
get_job,
list_jobs,
request_job_cancel,
)
from core.software_nodes import SUPPORTED_CAPABILITIES, list_nodes
from .base import Tool
class _SoftwareJobTool(Tool):
def __init__(self, user_id: UUID, task_id: UUID, **kwargs) -> None:
super().__init__(**kwargs)
self.user_id = user_id
self.task_id = task_id
class SoftwareCapabilityListTool(_SoftwareJobTool):
name = "software_capability_list"
description = "List professional software capabilities available through managed Windows nodes."
parameters = {"type": "object", "properties": {}, "additionalProperties": False}
def execute(self) -> str:
nodes = list_nodes()
items = [{
"capability": item,
"display_name": "Origin 科研绘图" if item == "origin.plot@v1" else item,
"available_nodes": sum(
1
for node in nodes
if node["status"] == "online"
and item in (node.get("capabilities") or [])
and (node.get("runtime") or {}).get("available_slots", 0) > 0
),
} for item in sorted(SUPPORTED_CAPABILITIES)]
return json.dumps({"capabilities": items}, ensure_ascii=False)
class SoftwareJobSubmitTool(_SoftwareJobTool):
name = "software_job_submit"
description = (
"Submit a persistent professional software job for the current task. "
"Return immediately with job_id; do not poll continuously or wait for completion."
)
parameters = {
"type": "object",
"properties": {
"capability": {"type": "string", "enum": sorted(SUPPORTED_CAPABILITIES)},
"request": {"type": "object"},
"idempotency_key": {
"type": "string",
"description": "Stable unique key for this exact submission; omit to generate one.",
},
},
"required": ["capability", "request"],
"additionalProperties": False,
}
def execute(self, capability: str, request: dict, idempotency_key: str = "") -> str:
try:
job, created = create_job(
self.user_id,
self.task_id,
idempotency_key=idempotency_key.strip() or str(uuid4()),
capability=capability,
request=request,
)
return json.dumps({**job, "created": created}, ensure_ascii=False)
except SoftwareJobError as exc:
return f"[Error] {exc}"
class SoftwareJobStatusTool(_SoftwareJobTool):
name = "software_job_status"
description = "Check one software job, or list recent jobs in the current task when job_id is omitted."
parameters = {
"type": "object",
"properties": {"job_id": {"type": "string"}},
"additionalProperties": False,
}
def execute(self, job_id: str = "") -> str:
try:
if job_id.strip():
item = get_job(self.user_id, UUID(job_id.strip()))
if item is None or item["task_id"] != str(self.task_id):
return "[Error] software job not found"
return json.dumps(item, ensure_ascii=False)
return json.dumps(
{"results": list_jobs(self.user_id, task_id=self.task_id, limit=20)},
ensure_ascii=False,
)
except ValueError:
return "[Error] invalid job_id"
class SoftwareJobCancelTool(_SoftwareJobTool):
name = "software_job_cancel"
description = "Request cancellation of a software job in the current task after the user asks to stop it."
parameters = {
"type": "object",
"properties": {"job_id": {"type": "string"}},
"required": ["job_id"],
"additionalProperties": False,
}
def execute(self, job_id: str) -> str:
try:
item = get_job(self.user_id, UUID(job_id.strip()))
if item is None or item["task_id"] != str(self.task_id):
return "[Error] software job not found"
job, _ = request_job_cancel(self.user_id, UUID(job_id.strip()))
return json.dumps(job, ensure_ascii=False)
except (ValueError, SoftwareJobError) as exc:
return f"[Error] {exc}"

View File

@ -33,7 +33,10 @@ from core.software_jobs import (
get_job_output_context, get_job_output_context,
mark_node_jobs_disconnected, mark_node_jobs_disconnected,
offer_next_job, offer_next_job,
pending_node_cancellations,
record_job_terminal, record_job_terminal,
list_jobs,
request_job_cancel,
respond_to_offer, respond_to_offer,
update_job_state, update_job_state,
validate_output_manifest, validate_output_manifest,
@ -375,6 +378,10 @@ def register_software_node_routes(app, *, require_user, require_admin) -> None:
websocket, websocket,
{"type": "connected", "heartbeat_seconds": 15}, {"type": "connected", "heartbeat_seconds": 15},
) )
for cancel in await asyncio.to_thread(pending_node_cancellations, node_id):
await node_connections.send_on(
node_id, websocket, {"type": "job_cancel", "payload": cancel}
)
while True: while True:
message = await websocket.receive_json() message = await websocket.receive_json()
message_type = message.get("type") message_type = message.get("type")
@ -434,6 +441,10 @@ def register_software_node_routes(app, *, require_user, require_admin) -> None:
await node_connections.send_on(node_id, websocket, await node_connections.send_on(node_id, websocket,
{"type": "ack", "message_id": message.get("message_id")} {"type": "ack", "message_id": message.get("message_id")}
) )
for cancel in await asyncio.to_thread(pending_node_cancellations, node_id):
await node_connections.send_on(
node_id, websocket, {"type": "job_cancel", "payload": cancel}
)
offer = await asyncio.to_thread( offer = await asyncio.to_thread(
offer_next_job, await node_connections.node_ids() offer_next_job, await node_connections.node_ids()
) )
@ -446,7 +457,13 @@ def register_software_node_routes(app, *, require_user, require_admin) -> None:
await asyncio.to_thread( await asyncio.to_thread(
abandon_offer, offer["node_id"], offer["payload"] abandon_offer, offer["node_id"], offer["payload"]
) )
except (SoftwareNodeError, WebSocketDisconnect, RuntimeError, ValueError): except (
SoftwareJobError,
SoftwareNodeError,
WebSocketDisconnect,
RuntimeError,
ValueError,
):
pass pass
finally: finally:
if await node_connections.remove(node_id, websocket): if await node_connections.remove(node_id, websocket):
@ -479,6 +496,35 @@ def register_software_node_routes(app, *, require_user, require_admin) -> None:
detail = str(exc) detail = str(exc)
raise HTTPException(404 if detail == "task not found" else 400, detail) from exc raise HTTPException(404 if detail == "task not found" else 400, detail) from exc
@app.get("/v1/software-jobs", tags=["software-jobs"])
def read_software_jobs(
task_id: UUID | None = None,
active_only: bool = False,
limit: int = 50,
user_id: UUID = Depends(require_user), # noqa: B008
):
return {
"results": list_jobs(
user_id, task_id=task_id, active_only=active_only, limit=limit
)
}
@app.post("/v1/software-jobs/{job_id}/cancel", tags=["software-jobs"])
async def cancel_software_job(
job_id: UUID,
user_id: UUID = Depends(require_user), # noqa: B008
):
try:
job, message = await asyncio.to_thread(request_job_cancel, user_id, job_id)
except SoftwareJobError as exc:
raise HTTPException(404, str(exc)) from exc
if message is not None:
await node_connections.send(
message["node_id"],
{"type": "job_cancel", "payload": message["payload"]},
)
return job
@app.get("/v1/software-jobs/{job_id}", tags=["software-jobs"]) @app.get("/v1/software-jobs/{job_id}", tags=["software-jobs"])
def read_software_job( def read_software_job(
job_id: UUID, job_id: UUID,

View File

@ -1069,6 +1069,34 @@
overflow: hidden; text-overflow: ellipsis; white-space: nowrap; margin-top: 2px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; margin-top: 2px;
} }
.proc-toast.fail { border-color: rgba(192,57,43,0.5); } .proc-toast.fail { border-color: rgba(192,57,43,0.5); }
#software-job-center {
position: fixed; right: 16px; bottom: 16px; z-index: 115; display: none;
width: min(380px, calc(100vw - 24px)); font-size: 13px;
}
#software-job-center.show { display: block; }
#software-job-toggle {
margin-left: auto; display: flex; align-items: center; gap: 8px;
border-radius: 999px; padding: 8px 13px; background: #fff;
border: 1px solid var(--border); box-shadow: 0 4px 16px rgba(0,0,0,.16); cursor: pointer;
}
#software-job-center.expanded #software-job-toggle { border-radius: 0 0 10px 10px; }
#software-job-panel {
max-height: min(520px, calc(100vh - 110px)); overflow: auto; background: #fff;
border: 1px solid var(--border); border-bottom: 0; border-radius: 10px 10px 0 0;
box-shadow: 0 -4px 20px rgba(0,0,0,.16); padding: 8px;
}
.sj-card { padding: 10px; border-bottom: 1px solid var(--border); }
.sj-card:last-child { border-bottom: 0; }
.sj-title { display: flex; justify-content: space-between; gap: 10px; }
.sj-title span { color: var(--muted); white-space: nowrap; }
.sj-card.failed .sj-title span { color: var(--danger); }
.sj-card.disconnected .sj-title span, .sj-card.cancelling .sj-title span { color: #a66514; }
.sj-sub, .sj-meta { margin-top: 4px; color: var(--muted); font-size: 11px; overflow-wrap: anywhere; }
.sj-progress { height: 4px; margin-top: 7px; border-radius: 4px; background: var(--panel-muted); overflow: hidden; }
.sj-progress i { display: block; height: 100%; background: var(--accent); transition: width .25s ease; }
.sj-actions { display: flex; justify-content: flex-end; gap: 6px; margin-top: 8px; }
.sj-empty { padding: 22px; text-align: center; color: var(--muted); }
#software-job-center.show ~ #proc-toasts { bottom: 62px; }
/* media tool 摘要 banner(model / size / cost / elapsed,折叠态也可见) */ /* media tool 摘要 banner(model / size / cost / elapsed,折叠态也可见) */
.tool-banner { .tool-banner {
display: inline-flex; flex-wrap: wrap; gap: 6px; display: inline-flex; flex-wrap: wrap; gap: 6px;
@ -2174,6 +2202,13 @@
</div> </div>
</div> </div>
<div id="software-job-center" aria-live="polite">
<div id="software-job-panel" hidden></div>
<button id="software-job-toggle" type="button" title="展开专业软件任务">
<span>专业软件任务</span><strong id="software-job-count">0 条</strong>
</button>
</div>
<script type="module" src="js/main.js"></script> <script type="module" src="js/main.js"></script>
</body> </body>
</html> </html>

View File

@ -19,6 +19,7 @@ import { loadFolderSuggestions } from "./newtask.js";
import { embedInit } from "./embed.js"; import { embedInit } from "./embed.js";
import { loadTaskList, loadModels, loadChannelCards, selectTask, showNewConversationDraft } from "./chat.js"; import { loadTaskList, loadModels, loadChannelCards, selectTask, showNewConversationDraft } from "./chat.js";
import { refreshProcs } from "./procs.js"; import { refreshProcs } from "./procs.js";
import { initSoftwareJobs } from "./software_jobs.js";
// ───── enter app ───── // ───── enter app ─────
export function enterApp(initialTaskId = "") { export function enterApp(initialTaskId = "") {
@ -40,6 +41,7 @@ export function enterApp(initialTaskId = "") {
loadStorage(); // 顶栏存储用量(后台扫描快照,非实时) loadStorage(); // 顶栏存储用量(后台扫描快照,非实时)
loadRole(); // 拉 /v1/me,admin 才显「管理」入口(/static/admin.html) loadRole(); // 拉 /v1/me,admin 才显「管理」入口(/static/admin.html)
refreshProcs(); // 后台进程(bg proc):上个会话遗留的 running proc 恢复轮询/展示 refreshProcs(); // 后台进程(bg proc):上个会话遗留的 running proc 恢复轮询/展示
initSoftwareJobs();
} }
// 顶栏用户名:默认显 name(兜底 user_name → email → uid8),title 悬浮给完整身份。 // 顶栏用户名:默认显 name(兜底 user_name → email → uid8),title 悬浮给完整身份。

View File

@ -0,0 +1,144 @@
// 专业软件任务中心:用户级轮询、跨对话状态、取消与终态通知。
import { api } from "./api.js";
import { state } from "./state.js";
import { $ } from "./dom.js";
import { escapeHtml } from "./format.js";
import { selectTask } from "./chat.js";
import { dialogConfirm, message } from "./dialog.js";
const ACTIVE = new Set(["queued", "offered", "dispatched", "running", "disconnected", "cancelling"]);
const TERMINAL = new Set(["succeeded", "failed", "cancelled"]);
const POLL_ACTIVE_MS = 4000;
const POLL_IDLE_MS = 30000;
let timer = null;
let known = new Map();
let jobs = [];
let expanded = false;
let initialized = false;
const statusLabel = {
queued: "等待计算节点", offered: "正在分配节点", dispatched: "节点已接收",
running: "正在执行", disconnected: "节点连接中断", cancelling: "正在停止",
succeeded: "已完成", failed: "失败", cancelled: "已取消",
};
const stageLabel = {
accepted: "节点已接收", waiting_input: "正在下载输入文件", ready_to_run: "准备软件环境",
origin_running: "Origin 正在生成图表", uploading_outputs: "正在上传结果",
cancel_requested: "停止请求已发送", terminal: "任务已结束",
};
export function initSoftwareJobs() {
if (initialized || !$("software-job-center")) return;
initialized = true;
$("software-job-toggle").onclick = () => { expanded = !expanded; render(); };
document.addEventListener("click", (event) => {
const center = $("software-job-center");
if (expanded && center && !center.contains(event.target)) { expanded = false; render(); }
});
refreshSoftwareJobs();
}
export async function refreshSoftwareJobs() {
if (!state.token) return;
try {
const data = await api("GET", "/v1/software-jobs?limit=50");
const next = data.results || [];
next.forEach((job) => {
const previous = known.get(job.job_id);
if (previous && ACTIVE.has(previous) && TERMINAL.has(job.status)) notifyTerminal(job);
known.set(job.job_id, job.status);
});
jobs = next;
render();
} catch (_) { /* 后台刷新失败静默,下轮恢复 */ }
schedule();
}
function schedule() {
if (timer) clearTimeout(timer);
const delay = jobs.some((job) => ACTIVE.has(job.status)) ? POLL_ACTIVE_MS : POLL_IDLE_MS;
timer = setTimeout(refreshSoftwareJobs, document.hidden ? Math.max(delay, 30000) : delay);
}
document.addEventListener("visibilitychange", () => {
if (!document.hidden && state.token) refreshSoftwareJobs(); else schedule();
});
function render() {
const center = $("software-job-center");
const panel = $("software-job-panel");
const active = jobs.filter((job) => ACTIVE.has(job.status));
const recent = jobs.slice(0, 10);
center.classList.toggle("show", active.length > 0 || recent.length > 0);
center.classList.toggle("expanded", expanded);
$("software-job-count").textContent = active.length ? `${active.length} 运行中` : `${jobs.length}`;
panel.hidden = !expanded;
if (!expanded) return;
panel.innerHTML = recent.length
? recent.map(jobCard).join("")
: '<div class="sj-empty">暂无专业软件任务</div>';
panel.querySelectorAll("[data-job-action]").forEach((button) => {
button.onclick = (event) => handleAction(event, button);
});
}
function jobCard(job) {
const summary = job.request_summary || {};
const input = job.input || {};
const active = ACTIVE.has(job.status);
const progress = Math.max(0, Math.min(100, Number(job.progress || 0)));
const detail = stageLabel[job.stage] || statusLabel[job.status] || "正在处理";
const error = job.error && (job.error.detail || job.error.code);
return `<article class="sj-card ${escapeHtml(job.status)}" data-job-id="${escapeHtml(job.job_id)}">
<div class="sj-title"><strong>${escapeHtml(summary.display_name || job.capability)}</strong>
<span>${escapeHtml(statusLabel[job.status] || job.status)}</span></div>
<div class="sj-sub">${escapeHtml(detail)}${error ? ` · ${escapeHtml(error)}` : ""}</div>
${active ? `<div class="sj-progress"><i style="width:${progress}%"></i></div>` : ""}
<div class="sj-meta">${escapeHtml(job.task_name || "未命名对话")}${input.filename ? ` · ${escapeHtml(input.filename)}` : ""}</div>
<div class="sj-actions">
<button class="small" data-job-action="open" data-task-id="${escapeHtml(job.task_id)}">打开对话</button>
${job.status === "succeeded" ? `<button class="small primary" data-job-action="analyze" data-task-id="${escapeHtml(job.task_id)}">分析结果</button>` : ""}
${active && job.status !== "cancelling" ? '<button class="small danger" data-job-action="cancel">停止</button>' : ""}
</div>
</article>`;
}
async function handleAction(event, button) {
event.stopPropagation();
const card = button.closest("[data-job-id]");
const jobId = card.dataset.jobId;
const action = button.dataset.jobAction;
if (action === "cancel") {
if (!await dialogConfirm({
title: "停止专业软件任务",
message: "确定停止这个任务?已经产生但尚未发布的中间输出可能不会保留。",
okText: "停止", danger: true,
})) return;
button.disabled = true;
try { await api("POST", `/v1/software-jobs/${jobId}/cancel`); }
catch (error) { message(error.message || "停止失败", "error"); }
refreshSoftwareJobs();
return;
}
const taskId = button.dataset.taskId;
if (taskId) await selectTask(taskId);
expanded = false;
render();
if (action === "analyze") {
setTimeout(() => {
const input = $("chat-input");
if (!input) return;
input.value = `请分析专业软件任务 ${jobId} 的结果,结合输出图表和输入数据总结主要结论。`;
input.focus();
input.dispatchEvent(new Event("input", { bubbles: true }));
}, 400);
}
}
function notifyTerminal(job) {
const ok = job.status === "succeeded";
const label = ok ? "已完成" : (job.status === "cancelled" ? "已取消" : "失败");
const summary = job.request_summary || {};
message(`${summary.display_name || "专业软件任务"}${label}`, ok ? "success" : "error", 6000);
expanded = true;
}

View File

@ -251,11 +251,47 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
StartJobPipeline(socket, acceptedJob); StartJobPipeline(socket, acceptedJob);
} }
} }
else if (type.GetString() == "job_cancel"
&& document.RootElement.TryGetProperty("payload", out var cancelPayload)
&& TryCancelJob(cancelPayload, out var cancelledJob))
{
workerRunner.Cancel(cancelledJob.JobId);
jobInbox.WriteTerminal(
cancelledJob, "cancelled", "USER_CANCELLED", "Cancelled by user.");
await SendAsync(socket, "job_terminal", new
{
job_id = cancelledJob.JobId,
lease_id = cancelledJob.LeaseId,
request_digest = cancelledJob.RequestDigest,
status = "cancelled",
error = new { code = "USER_CANCELLED", detail = "Cancelled by user." },
artifact_manifest = Array.Empty<object>(),
}, cancellationToken);
}
} }
message.SetLength(0); message.SetLength(0);
} }
} }
private bool TryCancelJob(JsonElement payload, out RecoverableJob job)
{
job = null!;
if (!payload.TryGetProperty("job_id", out var jobIdValue)
|| !Guid.TryParse(jobIdValue.GetString(), out var jobId)
|| !payload.TryGetProperty("lease_id", out var leaseIdValue)
|| !Guid.TryParse(leaseIdValue.GetString(), out var leaseId)
|| !payload.TryGetProperty("request_digest", out var digestValue))
{
return false;
}
job = jobInbox.ReadRecoverableJobs().SingleOrDefault(item =>
item.JobId == jobId
&& item.LeaseId == leaseId
&& item.RequestDigest == digestValue.GetString()
&& item.Terminal is null)!;
return job is not null;
}
private void StartJobPipeline(ClientWebSocket socket, RecoverableJob job) private void StartJobPipeline(ClientWebSocket socket, RecoverableJob job)
{ {
var completion = new TaskCompletionSource( var completion = new TaskCompletionSource(
@ -287,6 +323,13 @@ internal sealed class NodeConnectionLoop(NodeConfig config, Action<NodeStatus>?
if (job.Terminal is null) if (job.Terminal is null)
{ {
await inputDownloader.DownloadAsync(job, CancellationToken.None); await inputDownloader.DownloadAsync(job, CancellationToken.None);
var afterDownload = jobInbox.ReadRecoverableJobs()
.Single(item => item.JobId == job.JobId);
if (afterDownload.Terminal is JsonElement cancelledTerminal)
{
await TrySendAsync(socket, "job_terminal", cancelledTerminal);
return;
}
await TrySendAsync(socket, "job_state", new await TrySendAsync(socket, "job_state", new
{ {
job_id = job.JobId, job_id = job.JobId,

View File

@ -9,11 +9,20 @@ internal sealed class OriginWorkerRunner(JobInboxStore inbox)
{ {
private static readonly TimeSpan WorkerTimeout = TimeSpan.FromMinutes(30); private static readonly TimeSpan WorkerTimeout = TimeSpan.FromMinutes(30);
private readonly ConcurrentDictionary<Guid, Task> active = new(); private readonly ConcurrentDictionary<Guid, Task> active = new();
private readonly ConcurrentDictionary<Guid, CancellationTokenSource> cancellations = new();
internal Task RunAsync(RecoverableJob job) => internal Task RunAsync(RecoverableJob job) =>
active.GetOrAdd(job.JobId, _ => RunOnceAsync(job)); active.GetOrAdd(job.JobId, _ => RunOnceAsync(job, CancellationFor(job.JobId).Token));
private async Task RunOnceAsync(RecoverableJob job) internal void Cancel(Guid jobId)
{
CancellationFor(jobId).Cancel();
}
private CancellationTokenSource CancellationFor(Guid jobId) =>
cancellations.GetOrAdd(jobId, _ => new CancellationTokenSource());
private async Task RunOnceAsync(RecoverableJob job, CancellationToken cancellationToken)
{ {
try try
{ {
@ -60,7 +69,7 @@ internal sealed class OriginWorkerRunner(JobInboxStore inbox)
?? throw new InvalidOperationException("The fixed Origin worker did not start."); ?? throw new InvalidOperationException("The fixed Origin worker did not start.");
var stdout = process.StandardOutput.ReadToEndAsync(); var stdout = process.StandardOutput.ReadToEndAsync();
var stderr = process.StandardError.ReadToEndAsync(); var stderr = process.StandardError.ReadToEndAsync();
using var timeout = new CancellationTokenSource(); using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeout.CancelAfter(WorkerTimeout); timeout.CancelAfter(WorkerTimeout);
try try
{ {
@ -69,7 +78,14 @@ internal sealed class OriginWorkerRunner(JobInboxStore inbox)
catch (OperationCanceledException) catch (OperationCanceledException)
{ {
process.Kill(entireProcessTree: true); process.Kill(entireProcessTree: true);
inbox.WriteTerminal(job, "failed", "ORIGIN_WORKER_TIMEOUT", "Origin worker exceeded 30 minutes."); if (cancellationToken.IsCancellationRequested)
{
inbox.WriteTerminal(job, "cancelled", "USER_CANCELLED", "Cancelled by user.");
}
else
{
inbox.WriteTerminal(job, "failed", "ORIGIN_WORKER_TIMEOUT", "Origin worker exceeded 30 minutes.");
}
return; return;
} }
var output = await stdout; var output = await stdout;
@ -95,6 +111,7 @@ internal sealed class OriginWorkerRunner(JobInboxStore inbox)
finally finally
{ {
active.TryRemove(job.JobId, out _); active.TryRemove(job.JobId, out _);
if (cancellations.TryRemove(job.JobId, out var cancellation)) cancellation.Dispose();
} }
} }