 caoqianming
|
3ab18f3ed9
|
perf(resm): scihub改用curl-cffi轻量解析, 移除oa_url内联Playwright回退
download_pdf 链两处都不再在 worker 里起浏览器(此前是并发扩容的内存瓶颈):
- Sci-Hub: 新增 d_scihub.download_by_doi_http, 用 curl-cffi 取页面解析内嵌
PDF 地址直接下载并校验魔数, 覆盖各域名 embed#pdf/iframe#pdf/location.href
结构; 缓存存活域名做秒级故障切换; 撞 Cloudflare/DDoS 挑战返回
scihub_need_play 交独立脚本兜底。浏览器版 download_paper_by_doi 保留。
- oa_url: 移除策略3内联 Playwright(还会开有头窗口), curl-cffi 失败即打
oa_url_need_play, 交 scripts/get_pdf_by_playwright.py 兜底。
run_async 保留(该脚本从 tasks 导入)。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-21 15:33:15 +08:00 |
TianyangZhang