mirror of
https://github.com/GopeedLab/gopeed.git
synced 2026-11-14 12:23:43 +00:00
Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Levi
|
e6f134b29d | ||
|
Levi
|
d4ea62ca62 | ||
|
Levi
|
71999c0905 | ||
|
Levi
|
54fc0f7289 | ||
|
Levi
|
917fc64046 | ||
|
Levi
|
4ffb172259 |
No files matched your search
@@ -1,3 +0,0 @@
|
|||||||
/testdata/extensions/
|
|
||||||
/bin/
|
|
||||||
__pycache__/
|
|
||||||
@@ -1,40 +0,0 @@
|
|||||||
# moejs 实验评估
|
|
||||||
|
|
||||||
目标按顺序判断：**先能跑通 Gopeed 的真实扩展，再判断收益是否足够大，最后才考虑正式替换。**
|
|
||||||
|
|
||||||
这是独立 Go module。生产引擎、根目录依赖和默认构建不变。结论、实测数据及限制见 [REPORT.md](REPORT.md)。
|
|
||||||
|
|
||||||
## 复现
|
|
||||||
|
|
||||||
从本目录运行，需要 Git、Python 3 和 Go 1.26.0：
|
|
||||||
|
|
||||||
```sh
|
|
||||||
python3 fetch_sources.py
|
|
||||||
go test -v -timeout 3m
|
|
||||||
# 可选：向 Bilibili / YouTube / Twitter 等公开接口发请求；不需要账户 Cookie。
|
|
||||||
GOPEED_MOEJS_LIVE=1 go test -run TestLiveResolve -v -timeout 4m
|
|
||||||
# 单独执行，期间避免其他构建/测试负载。
|
|
||||||
GOMAXPROCS=1 go test -pgo=off -run '^$' -bench . -benchmem -benchtime=500ms -count=5 -timeout 8m > results/bench.txt
|
|
||||||
python3 summarize.py
|
|
||||||
python3 measure_size.py
|
|
||||||
# macOS/Linux：独立进程的 CPU、峰值 RSS、存活堆和关闭后内存。默认 5 轮。
|
|
||||||
python3 measure_resources.py > results/resources-run.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
`fetch_sources.py` 只取固定提交的原始 `dist/index.js`，校验已记录的 SHA-256，不执行 npm 安装或构建。下载文件和二进制不入库。`results/sources.json` 是来源清单。
|
|
||||||
|
|
||||||
## 实验范围
|
|
||||||
|
|
||||||
- goja 基线调用真实的 `pkg/download/engine.NewEngine`。
|
|
||||||
- moejs 原型使用其 eventloop、Web API，加载 Gopeed 现有的 polyfill 和 XHR JS；保留 mutable builtins，与 goja 的默认语义接近。
|
|
||||||
- 三个完整扩展包执行 `onResolve`，两端逐字比较资源输出和请求路径；B 站包含 SDK/WBI 签名逻辑，YouTube 包含 YouTube.js 的元数据解析。
|
|
||||||
- `replay.js` 用固定的合成 HTTP 响应隔离网络波动；没有把 SDK 函数替换成假函数。YouTube 的 session/config 请求固定返回 503，走库的本地 session fallback，再提供 player JSON。
|
|
||||||
- Blob 注册、FFmpeg 和 WebView 为模拟宿主。FFmpeg 的 mock 会拒绝真正合并；WebView 只接受 UA 查询。`onStart/onError` 只验证 mock 宿主下的回调流程，**不代表下载或 SABR/PoToken 已验证**。
|
|
||||||
- `TestMoeResolveWriteback` 演示显式的 Go request 拷入、JS resource 拷出，并验证 Gopeed `Resource.Validate()`。真实任务的可变对象/方法还需专门移植。
|
|
||||||
- `TestAsyncHTTP` 使用本地 HTTP server，验证异步 JSON 和 AbortController 请求取消。
|
|
||||||
- `BenchmarkBufferedTransport` 在两个引擎中使用同一段 JS 的 buffered Response，检查速度差是否主要来自原生 Response/Stream 实现。
|
|
||||||
- `BenchmarkExtension/*/cold` 包含建引擎、polyfill、编译和加载完整 bundle、一次 resolve 及关闭；`warm` 复用已加载的 runtime。冷启动更接近当前 `doTrigger` 路径。
|
|
||||||
- `measure_resources.py` 为每个样本启动独立 worker，交替引擎顺序，以进程 user+system CPU 时间比较相同工作量；另测保留 1/8 个 runtime 的内存。解释与结果见 [RESOURCES.md](RESOURCES.md)。两个引擎都链接在同一个测量程序中，RSS 是原型 worker 的占用，不是最终 Gopeed 应用占用。
|
|
||||||
- `measure_size.py` 实测独立解释器程序和真实 CLI 的双引擎共存成本；没有伪造“已经完全替换”的安装包大小。CLI 构建可能临时补充本实验 module 的依赖，最后执行 tidy。
|
|
||||||
|
|
||||||
测试只覆盖本报告所列路径。未指定 `GOPEED_MOEJS_LIVE=1` 时，扩展回放不访问外网；固定 bundle 需要事先下载。
|
|
||||||
@@ -1,128 +0,0 @@
|
|||||||
# moejs 替换 goja 的实验评估
|
|
||||||
|
|
||||||
日期：2026-11-14。目标顺序：**真实扩展先跑通 → 判断实际收益 → 再决定是否正式替换**。
|
|
||||||
|
|
||||||
## 判断
|
|
||||||
|
|
||||||
**CPU/内存补测后，建议维持 goja，暂不投入完整替换。**
|
|
||||||
|
|
||||||
新增的 [CPU 与内存测量](RESOURCES.md) 表明：当前冷路径 B 站/YouTube 的实际 CPU 时间只降低约 12% / 15%，驻留内存没有一致改善，连续解析的进程峰值 RSS 还更高。结合迁移成本和包体结果，收益不足以推进替换。
|
|
||||||
|
|
||||||
三个真实扩展的完整 bundle 已在 moejs 中跑通离线解析和公开网站联网解析，结果与 goja 相同。热运行 CPU 收益明显；当前 Gopeed 的事件路径每次新建引擎、编译并加载 bundle，冷启动的收益远小于热运行倍数。压缩体积基本没有显著收益。
|
|
||||||
|
|
||||||
“扩展解析可运行”已验证。“真实下载、FFmpeg 合并、YouTube SABR/PoToken、WebView 和 Go 任务生命周期完整可用”尚未验证，不能把本实验当成这些路径已经兼容。
|
|
||||||
|
|
||||||
## 版本与方法
|
|
||||||
|
|
||||||
- Gopeed 基线：`be0a395447d788419e83acdfd693e48a1a1706bf`，分支基于该版 main。
|
|
||||||
- goja：`v0.0.0-20260106131823-651366fbe6e3`，沿用项目依赖。
|
|
||||||
- moejs：`v0.0.0-20261113160646-b2c7fed9c42d`，固定上游提交；纯 Go，当前主模块的 Go 1.26.0 足够。
|
|
||||||
- Apple M1、8 GiB RAM、macOS 14.6.1、Go 1.26.0；性能测试 `GOMAXPROCS=1`、`-pgo=off`、每项 500 ms、5 次重复，报告中位数。没有套用上游的 PGO 或上游插件跑分。
|
|
||||||
- 最终性能测量单独执行，未与本次构建或联网测试并发。记录在 [toolchain.txt](results/toolchain.txt)、[bench.txt](results/bench.txt)。`ns_min/ns_max` 见 [bench-summary.json](results/bench-summary.json)。这不是多机器统计结论。
|
|
||||||
- goja 使用生产 `engine.NewEngine`；moejs 使用其 eventloop/webapi 加 Gopeed 的 polyfill/XHR。两侧都运行未经改写的发布 bundle。moejs 使用 `MutableIntrinsics: true`，保留 goja 的可写 builtin 语义。
|
|
||||||
- 冷运行包含建引擎、宿主 JS 初始化、编译/加载 bundle、一次解析及关闭；热运行复用已加载 runtime。宿主初始化能力不完全对等：goja 含完整生产注入，moejs 尚未移植 File/VM/FFmpeg 等原生桥。**冷运行数字是两个现有原型的对比，不是完成生产移植后的承诺。**
|
|
||||||
- 回放提供合成 HTTP 响应，不替换 Bilibili SDK、YouTube.js 或 Twitter SDK。YouTube session/config 固定返回 503，验证本地 session fallback 与 player JSON 解析；未覆盖成功 config、真实 player decipher 或媒体流。
|
|
||||||
- 基准中 Gopeed namespace/task/Blob 等由相同 JS mock 提供，结果以 JSON 字符串导出用于比较；不包含完整 Go 对象桥接及持久化的成本，原型的 Resource 拷出另有功能验证，未纳入时延表。
|
|
||||||
- 每次结果包含文件名、URL、labels、请求路径，两端比较相同输出。额外使用相同 JS 的 buffered Response 再测热运行，降低原生 Response/Stream 差异的影响。
|
|
||||||
|
|
||||||
## 真实扩展验证
|
|
||||||
|
|
||||||
| 扩展 | 固定发布包提交 | bundle 大小 | 回放 / 联网解析 | 已验证的范围 |
|
|
||||||
| --- | --- | ---: | --- | --- |
|
|
||||||
| Bilibili | [`b91db1e`](https://github.com/monkeyWie/gopeed-extension-bilibili/tree/b91db1ea52e975d8de9823f01a7af5f9ea83792f) | 920,556 B | 两个引擎均成功，输出相同 | 真实 SDK/WBI 请求签名、3 分 P、画质选择、资源组装 |
|
|
||||||
| YouTube | [`eeb78d1`](https://github.com/monkeyWie/gopeed-extension-youtube/tree/eeb78d168e053e9a2aad1dbaa309fc11f6992f7c) | 3,409,614 B | 两个引擎均成功，输出相同 | 完整 YouTube.js bundle 初始化、公开视频 metadata/title、下载任务描述 |
|
|
||||||
| Twitter | [`a963428`](https://github.com/monkeyWie/gopeed-extension-twitter/tree/a963428bbfdbee9510bf224b00b384e37514ce77) | 66,243 B | 两个引擎均成功，输出相同 | guest token/GraphQL SDK 路径、图片和最高码率视频筛选 |
|
|
||||||
|
|
||||||
联网输入：B 站 `BV1BJ4m1e7g8?p=1-3`、YouTube `dQw4w9WgXcQ`、Twitter `1734145541599383880`。未使用账户 Cookie。B 站实际返回了 480p fallback，两个引擎一致。联网不是固定延迟源，不用单次耗时推导速度倍数。
|
|
||||||
|
|
||||||
`onStart/onError` 在同一个实验 runtime 的模拟 task/blob/webview 宿主下跑通；B 站 CDN 切换和 YouTube 重试注册能返回相同结果。**这不包含真正调用 Blob opener 或读取/合并媒体。** WebView 只模拟 `navigator.userAgent`，FFmpeg mock 遇到真实 merge 会报错，不会伪造下载成功。
|
|
||||||
|
|
||||||
另有两个直接宿主检查：moejs 的显式 Go request 拷入、resource 拷出经过 `base.Resource.Validate()`；本地 HTTP server 的异步 JSON 和取消请求在两侧均可完成。取消后 HTTP server 的请求 context 均结束。但原生产 goja fetch 的取消 catch 路径因 `DOMException` 未定义而返回 `ReferenceError`，moejs 返回传入的取消原因；这个已有宿主差异被单独记录，不能视为错误语义已完全兼容，详见 [http.txt](results/http.txt)。
|
|
||||||
|
|
||||||
原始证据：[compatibility.txt](results/compatibility.txt)、[live.txt](results/live.txt)、[sources.json](results/sources.json)。
|
|
||||||
|
|
||||||
## 性能实测
|
|
||||||
|
|
||||||
### 热解析与冷解析
|
|
||||||
|
|
||||||
| 扩展 | 热 goja / moejs (ms) | 热速度倍数 | 冷 goja / moejs (ms) | 冷耗时下降 |
|
|
||||||
| --- | ---: | ---: | ---: | ---: |
|
|
||||||
| bilibili | 21.188 / 3.389 | 6.25× | 118.861 / 104.827 | 11.8% |
|
|
||||||
| youtube | 3.193 / 0.277 | 11.54× | 283.412 / 239.646 | 15.4% |
|
|
||||||
| twitter | 3.575 / 0.505 | 7.08× | 17.239 / 11.849 | 31.3% |
|
|
||||||
|
|
||||||
### 相同 JS buffered Response 的对照
|
|
||||||
|
|
||||||
| 扩展 | goja (ms) | moejs (ms) | 速度倍数 | 每次分配 goja / moejs (KiB) |
|
|
||||||
| --- | ---: | ---: | ---: | ---: |
|
|
||||||
| bilibili | 22.076 | 3.528 | 6.26× | 8036.0 / 1267.2 |
|
|
||||||
| youtube | 2.895 | 0.290 | 10.00× | 1071.1 / 151.3 |
|
|
||||||
| twitter | 3.450 | 0.491 | 7.03× | 1527.4 / 238.3 |
|
|
||||||
|
|
||||||
更换为相同的 JS Response 后仍有 6–10 倍差距，说明收益并不只来自 moejs 的原生 Response/Stream。但 URL/Headers 等底层实现仍不同，因此它仍是完整运行时对照，不能宣称是严格隔离的解释器指令效率。
|
|
||||||
|
|
||||||
### 空 runtime 加一次 `1+1` 的对照
|
|
||||||
|
|
||||||
| 配置 | 耗时 (µs) | 每次分配 (KiB) |
|
|
||||||
| --- | ---: | ---: |
|
|
||||||
| goja | 2.85 | 5.1 |
|
|
||||||
| moejs-mutable | 48.42 | 103.6 |
|
|
||||||
| moejs-shared | 3.86 | 5.2 |
|
|
||||||
|
|
||||||
moejs 的 mutable runtime 创建在本机反而明显更贵。shared frozen 模式能降低这部分成本，但改变 builtin 的可变性；本报告没有把这项不同合同的优化算入默认迁移收益。上述项也包含编译小脚本，并非只测构造函数。
|
|
||||||
|
|
||||||
|
|
||||||
热运行的明显收益不能直接解释成“下载快了数倍”。本次固定 JSON 较小，没有大 playlist、真实 YouTube player JS、PoToken、SABR、账号登录或 FFmpeg 媒体吞吐的测量。`B/op` 是每次调用累计分配量，**不是常驻内存、峰值 RSS 或每个 runtime 的存活内存**。
|
|
||||||
|
|
||||||
当前 [`doTrigger`](../../pkg/download/extension.go) 在匹配每个事件后调用 `newExtensionEngine` 并执行 bundle；[`engineSession`](../../pkg/download/extension_engine_session.go) 为 Blob/WebView 保活，未提供已编译 bundle 的通用缓存。因此首个事件更接近 cold 项。即使热运行快很多，也不能套用同样倍数预测首次解析。未来可以单独比较“两引擎均缓存编译产物”，避免把缓存改进归因于换引擎。
|
|
||||||
|
|
||||||
## 构建体积
|
|
||||||
|
|
||||||
`CGO_ENABLED=0`、`-trimpath -pgo=off -ldflags='-s -w'`。gzip 使用 Python 默认 level 9、`mtime=0`。gzip 只是相同压缩算法的代理，不是 Flutter installer、APK、IPA 或最终 zip 的测量。
|
|
||||||
|
|
||||||
### 独立解释器程序
|
|
||||||
|
|
||||||
| 目标 | goja (MiB) | moejs (MiB) | 未压缩减少 (MiB) | gzip 减少 (KiB) |
|
|
||||||
| --- | ---: | ---: | ---: | ---: |
|
|
||||||
| darwin/arm64 | 8.86 | 7.74 | 1.12 | 36.0 |
|
|
||||||
| linux/amd64 | 9.14 | 8.06 | 1.07 | 13.0 |
|
|
||||||
| windows/amd64 | 9.36 | 8.30 | 1.07 | 9.9 |
|
|
||||||
| android/arm64 | 9.56 | 8.44 | 1.12 | 37.3 |
|
|
||||||
|
|
||||||
### 真实 Gopeed CLI（`nosqlite`）的双引擎过渡成本
|
|
||||||
|
|
||||||
| 目标 | 原生产 CLI (MiB) | 加入 moejs/webapi (MiB) | 未压缩增加 (MiB) | gzip 增加 (MiB) |
|
|
||||||
| --- | ---: | ---: | ---: | ---: |
|
|
||||||
| darwin/arm64 | 49.63 | 56.06 | 6.43 | 2.50 |
|
|
||||||
| linux/amd64 | 51.25 | 57.92 | 6.68 | 2.69 |
|
|
||||||
| windows/amd64 | 52.42 | 59.23 | 6.81 | 2.73 |
|
|
||||||
|
|
||||||
|
|
||||||
独立解释器程序支持从命令行传入任意 JS，避免 linker 把执行功能全部裁掉；它不含完整宿主能力。真实 CLI 的第二项通过构建 overlay 额外链接 moejs、eventloop、webapi，**仍然保留生产 goja，默认扩展执行仍走 goja**。这测的是双引擎过渡阶段成本，没有伪装成完整替换的成品。
|
|
||||||
|
|
||||||
结论：独立引擎未压缩可以少约 1.1 MiB，但压缩后只省十几到几十 KiB；没有证据表明换引擎能让最终安装包大幅变小。双引擎灰度还会增大包。完全移除 goja 后的 CLI、桌面 c-shared、Android/iOS 库和 Flutter 安装包必须等完整桥接完成后再测。本次 Android 只交叉编译独立解释器，不代表 APK/FFI 已验证。
|
|
||||||
|
|
||||||
原始证据：[size.json](results/size.json)、[size-build.txt](results/size-build.txt)，构建脚本 [measure_size.py](measure_size.py)。
|
|
||||||
|
|
||||||
## 迁移阻塞与工作量
|
|
||||||
|
|
||||||
| 接入点 | 实测或代码事实 | 正式替换需要完成 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Go struct / map 语义 | goja 对 `ctx.res`、`ctx.req.url` 的修改写回 Go；moejs `FromGo` 是 JS 快照，Go 值不变；Go 方法也不会自动映射 | 显式对象/方法包装及写回策略，覆盖 req/task/meta/extra/settings/storage，验证引用与持久化语义 |
|
|
||||||
| 数字 labels | B 站给 labels 写数字；goja 自动转成 `map[string]string`，直接 JSON 拷出会报类型错 | 原型已做 string 规范化；生产要定义完整转换和错误策略 |
|
|
||||||
| 事件循环和取消 | `goja_nodejs` 与现有 `Post`、Promise/事件超时直接绑定；moejs 有不同的 loop/API | 对齐并发归属、超时、Close、pending Promise、回调错误、后台任务生命周期，迁移现有 cancellation 测试 |
|
|
||||||
| fetch/XHR/streams | XHR JS 可复用；本次 fetch 使用 moejs webapi；本地取消请求已验证 | 保留代理、默认 UA、headers、Cookie、stream backpressure、body ownership 和 session cleanup；跑现有 Fetch WPT/流测试于新宿主 |
|
|
||||||
| Blob/FFmpeg/WebView | 生产注入层大量引用 goja `Value`、`Object`、`FunctionCall` | 移植 Blob opener/reader、FFmpeg WASM、WebView 回调与函数源码序列化，验证取消/恢复/重试 |
|
|
||||||
| File/VM/MessageError/logging | 现有 Go 方法/错误和值导出不是 moejs 的直接替代 API | 建原生函数包装，保留 `MessageError` 分类、日志格式、File 行为与子 VM 隔离 |
|
|
||||||
| 可写 builtins | 三个回放也能在 frozen 模式运行，但修改 `Array.prototype` 会失败 | 默认保持 mutable；除非扩展能力合同允许冻结，不能把共享 builtin 的启动/内存优势直接作为默认收益 |
|
|
||||||
|
|
||||||
上游 [Go values in](https://github.com/MoeForge/moejs/blob/b2c7fed9c42dd1ba8ce5ea844850cba6a1a95fb3/docs/guide.md#go-values-in) 说明快照/方法转换规则；[runtime.go](https://github.com/MoeForge/moejs/blob/b2c7fed9c42dd1ba8ce5ea844850cba6a1a95fb3/runtime.go) 说明共享冻结 builtin 和 goroutine 约束；[README](https://github.com/MoeForge/moejs/blob/b2c7fed9c42dd1ba8ce5ea844850cba6a1a95fb3/README.md#status) 仍将项目标为 alpha、API 可能变化。要固定提交，不能把上游语言测试通过数当成 Gopeed 宿主兼容证明。分发需要随带上游 [THIRD_PARTY_NOTICES](https://github.com/MoeForge/moejs/blob/b2c7fed9c42dd1ba8ce5ea844850cba6a1a95fb3/THIRD_PARTY_NOTICES)。
|
|
||||||
|
|
||||||
## 后续替换门槛
|
|
||||||
|
|
||||||
1. 在实验构建中跑完整 Downloader → extension → Go task/resource → Blob/FFmpeg/WebView 路径，实测 B 站与 YouTube 下载输出文件、取消、断点恢复、重试；不再依赖本次模拟宿主。
|
|
||||||
2. 新引擎跑现有扩展、Fetch/Stream、WebView、FFmpeg、事件取消测试；验证无 pending promise、session 泄漏及并发访问错误。
|
|
||||||
3. 在实际每事件新建 runtime 的方式下比较总耗时、CPU、峰值/常驻内存和最终安装包；增加至少一台 Windows、一台 Android/iOS 目标以及真实大型 player/playlist。
|
|
||||||
4. 再决定冷路径与实际解析体验的提升是否足以抵消桥接成本。新增资源测量不支持在当前架构下继续完整移植；未来若有真实的长时间计算/环境复用场景，再重新评估。**最终包体和完整下载性能目前没有“大幅提升”的证据**。
|
|
||||||
|
|
||||||
本 PR 仅交付可复现的调研/原型，不切换默认引擎。生产 engine 原有测试通过，记录在 [production-engine-tests.txt](results/production-engine-tests.txt)。复现命令见 [README.md](README.md)。
|
|
||||||
@@ -1,86 +0,0 @@
|
|||||||
# CPU 与内存补充测量
|
|
||||||
|
|
||||||
日期：2026-11-14。沿用 [主报告](REPORT.md) 的引擎版本、三个固定扩展包、回放输入和 M1 / Go 1.26.0 环境。
|
|
||||||
|
|
||||||
**建议维持 goja，暂不投入完整替换。** 在当前每次事件新建引擎、重新编译加载的方式下，B 站/YouTube 的 CPU 只减少约 12% / 15%，没有一致的内存优势；连续解析时峰值 RSS 更高。Twitter 的 CPU 降幅较大，但每次只省约 5 ms。这些收益不足以抵消完整宿主桥接、兼容性和维护成本。
|
|
||||||
|
|
||||||
## 测量方式
|
|
||||||
|
|
||||||
- 每个样本使用新的独立 OS 进程，两个引擎交替先后顺序，5 轮重复，报告中位数；最终数据共 120 个进程样本。测量时未与本次其他构建或测试并发。
|
|
||||||
- `GOMAXPROCS=1 GOGC=100 GOMEMLIMIT=off`，不使用 PGO。CPU 是 `getrusage(RUSAGE_SELF)` 的 **user + system 时间**，除以完成次数；包含 GC 和编译开销，不使用瞬时 CPU 占用百分比，也不把墙钟耗时直接当 CPU 时间。
|
|
||||||
- `cold`：一次进程内先预热一次全局初始化，再连续完成 20 次“新建环境、克隆 bundle 输入字符串、编译加载、解析、关闭”。接近当前反复触发扩展事件的方式。每次使用独立 bundle 字符串，模拟生产文件读取后的输入所有权。
|
|
||||||
- `warm`：环境已加载，连续解析 200 次。仅用于确认复用环境时的 CPU 上限；当前生产事件路径没有这样复用。
|
|
||||||
- 内存组不预建引擎，从相同测量程序基线开始，分别建立并保留 1 / 8 个完整环境，每个环境执行一次解析后保持存活。8 个环境是同时保留，不是同时并行执行；不模拟实际 Blob/FFmpeg 媒体数据。
|
|
||||||
- 存活堆：`runtime.MemStats.HeapAlloc` 相对建引擎前的增量。连续 GC 并给异步清理调度机会，至少 3 轮、最多 8 轮，直到相邻值差异小于 64 KiB；不稳定则测量失败。保留源字符串并用 `runtime.KeepAlive` 保证环境仍存活，避免编译器提前释放。
|
|
||||||
- RSS：通过 `ps` 记录当前物理驻留快照，通过 `getrusage` 记录进程历史峰值；macOS 的峰值单位为字节，Linux 转换为字节。`ps`、稳定 GC 和内存快照都在计时区间外，`RUSAGE_SELF` 不包括 sampler 子进程 CPU。
|
|
||||||
- 两个引擎链接在相同测量 worker 中，RSS 含公共 Go/测试框架/代码页和 allocator 保留页，**不是完整 Gopeed 应用或最终单引擎产品的内存预测**。存活堆增量用来区分扩展保留对象和进程历史峰值。
|
|
||||||
- 全部输出按 SHA-256 比较一致，没有通过减少文件或请求处理量获得更好的数字。宿主 mock、HTTP fixture、未移植能力及真实下载尚未验证等限制与主报告相同。
|
|
||||||
|
|
||||||
## 冷路径 CPU 与连续解析峰值
|
|
||||||
|
|
||||||
| 扩展 | goja CPU (ms/次) | moejs CPU (ms/次) | CPU 减少 | goja 峰值 RSS (MiB) | moejs 峰值 RSS (MiB) |
|
|
||||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
|
||||||
| bilibili | 118.962 | 104.081 | 12.5% | 154.7 | 172.3 |
|
|
||||||
| youtube | 278.476 | 237.330 | 14.8% | 283.8 | 358.9 |
|
|
||||||
| twitter | 15.555 | 10.594 | 31.9% | 79.7 | 96.8 |
|
|
||||||
|
|
||||||
|
|
||||||
峰值是连续 20 次冷解析期间**整个 worker 的进程峰值**，不是单个 runtime 的堆大小。峰值会受 Go GC/allocator 与异步清理时机影响；不能将它当作关闭后仍存活的对象。原始数据保留每轮值，汇总文件也提供 min/max。
|
|
||||||
|
|
||||||
| 扩展 | goja 累计分配 (MiB/次) | moejs 累计分配 (MiB/次) | 分配减少 |
|
|
||||||
| --- | ---: | ---: | ---: |
|
|
||||||
| bilibili | 73.69 | 60.17 | 18.3% |
|
|
||||||
| youtube | 159.91 | 146.82 | 8.2% |
|
|
||||||
| twitter | 10.96 | 7.73 | 29.5% |
|
|
||||||
|
|
||||||
|
|
||||||
moejs 的累计分配量确实较少，但这没有转化为更低的连续解析峰值 RSS。`B/op`、GC 后存活堆、RSS 是不同指标，不能互相替代。
|
|
||||||
|
|
||||||
## 保留环境时的内存
|
|
||||||
|
|
||||||
| 扩展 | 保留环境数 | goja 存活堆增量 (MiB) | moejs 存活堆增量 (MiB) | goja RSS 快照 (MiB) | moejs RSS 快照 (MiB) |
|
|
||||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
|
||||||
| bilibili | 1 | 15.87 | 15.66 | 114.2 | 111.2 |
|
|
||||||
| bilibili | 8 | 126.72 | 120.60 | 362.0 | 302.6 |
|
|
||||||
| youtube | 1 | 32.48 | 41.22 | 203.7 | 192.2 |
|
|
||||||
| youtube | 8 | 259.64 | 320.54 | 627.8 | 680.1 |
|
|
||||||
| twitter | 1 | 2.53 | 2.63 | 75.6 | 74.7 |
|
|
||||||
| twitter | 8 | 20.06 | 19.48 | 106.9 | 93.5 |
|
|
||||||
|
|
||||||
|
|
||||||
B 站一个环境的存活堆几乎没有变化；YouTube 反而更高。多环境情况下也没有稳定、普遍的显著节省。RSS 快照包含 allocator 页面，因此与存活堆的变化比例不同。
|
|
||||||
|
|
||||||
最初单次 GC 的 YouTube 堆快照约为 32.5 → 42.2 MiB。考虑 moejs 异步清理编译缓存后，已重测驻留组，表中使用稳定 GC 的结果，不能将初始 42.2 MiB 当成最终驻留值。CPU 和冷路径峰值取自第一次进程资源测量；这些计时和峰值发生在强制 GC 之前，不受驻留测量修正影响。
|
|
||||||
|
|
||||||
关闭全部环境并等稳定 GC 后，额外存活 Go 堆回落到 goja 约 0.03–0.05 MiB、moejs 约 0.18–0.97 MiB。这包含一次性进程缓存；本实验没有据此认定泄漏。RSS 不一定同步回落，不能只看关闭后的 RSS 判断对象是否已释放。
|
|
||||||
|
|
||||||
## 热路径 CPU
|
|
||||||
|
|
||||||
| 扩展 | goja CPU (ms/次) | moejs CPU (ms/次) | CPU 减少 |
|
|
||||||
| --- | ---: | ---: | ---: |
|
|
||||||
| bilibili | 19.688 | 3.173 | 83.9% |
|
|
||||||
| youtube | 2.905 | 0.215 | 92.6% |
|
|
||||||
| twitter | 3.265 | 0.453 | 86.1% |
|
|
||||||
|
|
||||||
|
|
||||||
热路径的大幅 CPU 节省再次成立。但它要求持有并复用已加载的环境；对当前每个事件重建环境的实现，应该采用上面的 cold 结果评估迁移。
|
|
||||||
|
|
||||||
## 决策
|
|
||||||
|
|
||||||
按当前扩展和架构，CPU/内存补测没有提供足够的替换依据。尤其 B 站和 YouTube 的 CPU 节省有限，内存没有一致改善，加上主报告里压缩体积几乎不变，应先维持 goja，把本 PR 留作调研记录。
|
|
||||||
|
|
||||||
未来若出现长时间执行计算、反复复用环境的真实扩展，再重新评估；若独立研究编译缓存或环境复用，必须同时在 goja 和 moejs 上比较收益，不能把架构优化全部归给引擎替换。
|
|
||||||
|
|
||||||
## 复现和证据
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cd experiments/moejs
|
|
||||||
python3 fetch_sources.py
|
|
||||||
python3 measure_resources.py > results/resources-run.txt
|
|
||||||
# 只重新汇总已保存的原始样本，不启动新测量
|
|
||||||
python3 measure_resources.py --summarize-only
|
|
||||||
# 必要时只重测驻留组，保留之前的 CPU 样本
|
|
||||||
python3 measure_resources.py --memory-only > results/resources-memory-run.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
原始样本：[resources.json](results/resources.json)；汇总：[resources-summary.json](results/resources-summary.json)。首次资源运行日志：[resources-run.txt](results/resources-run.txt)；稳定 GC 后重测驻留组日志：[resources-memory-run.txt](results/resources-memory-run.txt)。首次日志中的驻留组已被后者替代。当前 worker：[resource_worker_test.go](resource_worker_test.go)；驱动：[measure_resources.py](measure_resources.py)；功能检查：[resources-tests.txt](results/resources-tests.txt)。CPU/warm 组中单次 GC 的快照保留作原始记录，内存结论只使用重新测得的 `retained` 组。
|
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
package evaluation
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/GopeedLab/gopeed/pkg/base"
|
|
||||||
"github.com/MoeForge/moejs"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Demonstrate an explicit copy-in/copy-out resolve bridge, since FromGo does
|
|
||||||
// not provide goja's live Go-struct wrappers. Existing task wrappers still need
|
|
||||||
// native methods and lifecycle ownership before this can ship.
|
|
||||||
func TestMoeResolveWriteback(t *testing.T) {
|
|
||||||
r := setup(t, "moejs", "bilibili").(*Moe)
|
|
||||||
req := &base.Request{URL: "https://www.bilibili.com/video/BV1BJ4m1e7g8?p=2"}
|
|
||||||
done := make(chan error, 1)
|
|
||||||
if !r.loop.RunOnLoop(func(rt *moejs.Runtime) { done <- rt.SetGlobal("hostContext", map[string]any{"req": req}) }) {
|
|
||||||
t.Fatal("loop stopped")
|
|
||||||
}
|
|
||||||
if e := <-done; e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
v, e := r.Eval(`(async()=>{await handlers.onResolve(hostContext);
|
|
||||||
// Goja coerces numeric JS label values into map[string]string. JSON does not.
|
|
||||||
for (const f of hostContext.res.files) {
|
|
||||||
if (f.req.labels) f.req.labels = Object.fromEntries(Object.entries(f.req.labels).map(([k,v])=>[k,String(v)]));
|
|
||||||
}
|
|
||||||
return JSON.stringify(hostContext.res)})()`)
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
var res base.Resource
|
|
||||||
if e = json.Unmarshal([]byte(fmt.Sprint(v)), &res); e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
if e = res.Validate(); e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
if len(res.Files) != 1 || res.Files[0].Name != "BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4" {
|
|
||||||
t.Fatalf("unexpected resource: %+v", res)
|
|
||||||
}
|
|
||||||
t.Logf("Go Resource.Validate passed, name=%s", res.Files[0].Name)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestBufferedReplay(t *testing.T) {
|
|
||||||
for _, name := range []string{"bilibili", "youtube", "twitter"} {
|
|
||||||
t.Run(name, func(t *testing.T) {
|
|
||||||
var want any
|
|
||||||
for _, engine := range []string{"goja", "moejs"} {
|
|
||||||
t.Run(engine, func(t *testing.T) {
|
|
||||||
r := setup(t, engine, name)
|
|
||||||
v, e := r.Eval("Response = BufferedResponse; resolveOnce()")
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
if engine == "goja" {
|
|
||||||
want = v
|
|
||||||
} else if v != want {
|
|
||||||
t.Fatalf("output differs: %v / %v", want, v)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
//go:build !moejs
|
|
||||||
|
|
||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"github.com/dop251/goja"
|
|
||||||
"os"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
source := "1+1"
|
|
||||||
if len(os.Args) > 1 {
|
|
||||||
source = os.Args[1]
|
|
||||||
}
|
|
||||||
rt := goja.New()
|
|
||||||
v, e := rt.RunString(source)
|
|
||||||
if e != nil {
|
|
||||||
panic(e)
|
|
||||||
}
|
|
||||||
fmt.Println(v.Export())
|
|
||||||
}
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
//go:build moejs
|
|
||||||
|
|
||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"github.com/MoeForge/moejs"
|
|
||||||
"os"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
source := "1+1"
|
|
||||||
if len(os.Args) > 1 {
|
|
||||||
source = os.Args[1]
|
|
||||||
}
|
|
||||||
rt := moejs.NewRuntime(moejs.Options{MutableIntrinsics: true})
|
|
||||||
script, e := moejs.CompileScript("size", source)
|
|
||||||
if e != nil {
|
|
||||||
panic(e)
|
|
||||||
}
|
|
||||||
v, e := rt.RunScript(script)
|
|
||||||
if e != nil {
|
|
||||||
panic(e)
|
|
||||||
}
|
|
||||||
out, e := rt.ToGo(v)
|
|
||||||
if e != nil {
|
|
||||||
panic(e)
|
|
||||||
}
|
|
||||||
fmt.Println(out)
|
|
||||||
}
|
|
||||||
@@ -1,139 +0,0 @@
|
|||||||
package evaluation
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"testing"
|
|
||||||
)
|
|
||||||
|
|
||||||
func source(t testing.TB, name string) string {
|
|
||||||
t.Helper()
|
|
||||||
b, e := os.ReadFile("testdata/extensions/" + name + ".js")
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal("run python3 fetch_sources.py first: ", e)
|
|
||||||
}
|
|
||||||
return string(b)
|
|
||||||
}
|
|
||||||
func setup(t testing.TB, engine, name string) Runtime {
|
|
||||||
t.Helper()
|
|
||||||
r, e := New(engine)
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
t.Cleanup(r.Close)
|
|
||||||
b, e := os.ReadFile("replay.js")
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
for _, s := range []string{fmt.Sprintf("var extensionName = %q;", name), string(b), source(t, name)} {
|
|
||||||
if _, e = r.Eval(s); e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return r
|
|
||||||
}
|
|
||||||
func TestExtensionReplay(t *testing.T) {
|
|
||||||
for _, name := range []string{"bilibili", "youtube", "twitter"} {
|
|
||||||
t.Run(name, func(t *testing.T) {
|
|
||||||
var want any
|
|
||||||
for _, engine := range []string{"goja", "moejs"} {
|
|
||||||
t.Run(engine, func(t *testing.T) {
|
|
||||||
r := setup(t, engine, name)
|
|
||||||
v, e := r.Eval("resolveOnce()")
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
t.Log(v)
|
|
||||||
if engine == "goja" {
|
|
||||||
want = v
|
|
||||||
} else if want != nil && v != want {
|
|
||||||
t.Fatalf("output differs\ngoja: %s\nmoejs: %s", want, v)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
func BenchmarkExtension(b *testing.B) {
|
|
||||||
for _, name := range []string{"bilibili", "youtube", "twitter"} {
|
|
||||||
for _, engine := range []string{"goja", "moejs"} {
|
|
||||||
b.Run(name+"/"+engine+"/warm", func(b *testing.B) {
|
|
||||||
r := setup(b, engine, name)
|
|
||||||
if _, e := r.Eval("resolveOnce()"); e != nil {
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
b.ReportAllocs()
|
|
||||||
b.ResetTimer()
|
|
||||||
for i := 0; i < b.N; i++ {
|
|
||||||
if _, e := r.Eval("resolveOnce()"); e != nil {
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
b.Run(name+"/"+engine+"/cold", func(b *testing.B) {
|
|
||||||
src := source(b, name)
|
|
||||||
p, e := os.ReadFile("replay.js")
|
|
||||||
if e != nil {
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
b.ReportAllocs()
|
|
||||||
b.ResetTimer()
|
|
||||||
for i := 0; i < b.N; i++ {
|
|
||||||
r, e := New(engine)
|
|
||||||
if e != nil {
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
for _, s := range []string{fmt.Sprintf("var extensionName=%q;", name), string(p), src, "resolveOnce()"} {
|
|
||||||
if _, e = r.Eval(s); e != nil {
|
|
||||||
r.Close()
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
r.Close()
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestExtensionLifecycle(t *testing.T) {
|
|
||||||
for _, name := range []string{"bilibili", "youtube", "twitter"} {
|
|
||||||
t.Run(name, func(t *testing.T) {
|
|
||||||
var want any
|
|
||||||
for _, engine := range []string{"goja", "moejs"} {
|
|
||||||
t.Run(engine, func(t *testing.T) {
|
|
||||||
r := setup(t, engine, name)
|
|
||||||
v, e := r.Eval("lifecycleOnce()")
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
t.Log(v)
|
|
||||||
if engine == "goja" {
|
|
||||||
want = v
|
|
||||||
} else if want != nil && v != want {
|
|
||||||
t.Fatalf("lifecycle differs\ngoja: %s\nmoejs: %s", want, v)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
func BenchmarkBufferedTransport(b *testing.B) {
|
|
||||||
for _, name := range []string{"bilibili", "youtube", "twitter"} {
|
|
||||||
for _, engine := range []string{"goja", "moejs"} {
|
|
||||||
b.Run(name+"/"+engine, func(b *testing.B) {
|
|
||||||
r := setup(b, engine, name)
|
|
||||||
if _, e := r.Eval("Response = BufferedResponse; resolveOnce()"); e != nil {
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
b.ReportAllocs()
|
|
||||||
b.ResetTimer()
|
|
||||||
for i := 0; i < b.N; i++ {
|
|
||||||
if _, e := r.Eval("resolveOnce()"); e != nil {
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Fetch the unmodified published extension bundles at immutable commits."""
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
|
|
||||||
sources = {
|
|
||||||
'bilibili': ('monkeyWie/gopeed-extension-bilibili', 'b91db1ea52e975d8de9823f01a7af5f9ea83792f'),
|
|
||||||
'youtube': ('monkeyWie/gopeed-extension-youtube', 'eeb78d168e053e9a2aad1dbaa309fc11f6992f7c'),
|
|
||||||
'twitter': ('monkeyWie/gopeed-extension-twitter', 'a963428bbfdbee9510bf224b00b384e37514ce77'),
|
|
||||||
}
|
|
||||||
root = Path(__file__).resolve().parent / 'testdata' / 'extensions'
|
|
||||||
root.mkdir(parents=True, exist_ok=True)
|
|
||||||
expected = json.loads((root.parent.parent / 'results/sources.json').read_text())
|
|
||||||
manifest = {}
|
|
||||||
for name, (repo, commit) in sources.items():
|
|
||||||
url = f'https://raw.githubusercontent.com/{repo}/{commit}/dist/index.js'
|
|
||||||
with tempfile.TemporaryDirectory(prefix='gopeed-extension-') as checkout:
|
|
||||||
subprocess.run(['git', 'init', '-q', checkout], check=True)
|
|
||||||
subprocess.run(['git', '-C', checkout, 'fetch', '-q', '--depth=1', f'https://github.com/{repo}.git', commit], check=True)
|
|
||||||
data = subprocess.check_output(['git', '-C', checkout, 'show', 'FETCH_HEAD:dist/index.js'])
|
|
||||||
digest = hashlib.sha256(data).hexdigest()
|
|
||||||
if digest != expected[name]['sha256']:
|
|
||||||
raise RuntimeError(f'Unexpected bundle hash for {name}: {digest}')
|
|
||||||
(root / f'{name}.js').write_bytes(data)
|
|
||||||
manifest[name] = dict(repo=repo, commit=commit, url=url, bytes=len(data), sha256=hashlib.sha256(data).hexdigest())
|
|
||||||
print(name, len(data), manifest[name]['sha256'])
|
|
||||||
(root / 'sources.json').write_text(json.dumps(manifest, indent=2) + '\n')
|
|
||||||
@@ -1,16 +0,0 @@
|
|||||||
package evaluation
|
|
||||||
|
|
||||||
import "testing"
|
|
||||||
|
|
||||||
func TestFrozenExtensionReplay(t *testing.T) {
|
|
||||||
for _, name := range []string{"bilibili", "youtube", "twitter"} {
|
|
||||||
t.Run(name, func(t *testing.T) {
|
|
||||||
r := setup(t, "moejs-frozen", name)
|
|
||||||
v, e := r.Eval("resolveOnce()")
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
t.Log(v)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
module github.com/GopeedLab/gopeed/experiments/moejs
|
|
||||||
|
|
||||||
go 1.26.0
|
|
||||||
|
|
||||||
replace github.com/GopeedLab/gopeed => ../..
|
|
||||||
|
|
||||||
require (
|
|
||||||
github.com/GopeedLab/gopeed v0.0.0
|
|
||||||
github.com/MoeForge/moejs v0.0.0-20261113160646-b2c7fed9c42d
|
|
||||||
github.com/dop251/goja v0.0.0-20260106131823-651366fbe6e3
|
|
||||||
)
|
|
||||||
|
|
||||||
require (
|
|
||||||
codeberg.org/gruf/go-ffmpreg v0.6.23-0.20260827095416-df1836cb5be0 // indirect
|
|
||||||
github.com/MoeForge/moego v0.1.0-alpha.1 // indirect
|
|
||||||
github.com/andybalholm/brotli v1.2.0 // indirect
|
|
||||||
github.com/dlclark/regexp2 v1.11.5 // indirect
|
|
||||||
github.com/dop251/goja_nodejs v0.0.0-20240728170619-29b559befffc // indirect
|
|
||||||
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
|
|
||||||
github.com/google/go-querystring v1.1.0 // indirect
|
|
||||||
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 // indirect
|
|
||||||
github.com/icholy/digest v1.1.0 // indirect
|
|
||||||
github.com/imroc/req/v3 v3.58.0 // indirect
|
|
||||||
github.com/klauspost/compress v1.18.2 // indirect
|
|
||||||
github.com/mattn/go-ieproxy v0.0.12 // indirect
|
|
||||||
github.com/quic-go/qpack v0.6.0 // indirect
|
|
||||||
github.com/quic-go/quic-go v0.59.0 // indirect
|
|
||||||
github.com/refraction-networking/utls v1.8.2 // indirect
|
|
||||||
github.com/tetratelabs/wazero v1.12.0 // indirect
|
|
||||||
golang.org/x/crypto v0.57.0 // indirect
|
|
||||||
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
|
|
||||||
golang.org/x/net v0.60.0 // indirect
|
|
||||||
golang.org/x/sys v0.48.0 // indirect
|
|
||||||
golang.org/x/text v0.42.0 // indirect
|
|
||||||
)
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
codeberg.org/gruf/go-ffmpreg v0.6.23-0.20260827095416-df1836cb5be0 h1:AqNFl60o4xhTTOxmzAvSf7pazXHKBGuV5ATw7sETIRA=
|
|
||||||
codeberg.org/gruf/go-ffmpreg v0.6.23-0.20260827095416-df1836cb5be0/go.mod h1:XM6eW8CSgAuZRKRynm5nDiSNedm0rav30IychqygOHE=
|
|
||||||
github.com/Masterminds/semver/v3 v3.2.1 h1:RN9w6+7QoMeJVGyfmbcgs28Br8cvmnucEXnY0rYXWg0=
|
|
||||||
github.com/Masterminds/semver/v3 v3.2.1/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
|
|
||||||
github.com/MoeForge/moego v0.1.0-alpha.1 h1:d+asBZu+2oKnBj1kzdvdANMStbIKw9BLCpknuWjiR28=
|
|
||||||
github.com/MoeForge/moego v0.1.0-alpha.1/go.mod h1:SBNiQjCpWC3IEBMKFBOwOyhhQ3jywMrISqCwJTdr4aA=
|
|
||||||
github.com/MoeForge/moejs v0.0.0-20261113160646-b2c7fed9c42d h1:aM5l0SGzQaE3rPvqskuukTL8Iv1wKt+kBq06NjfSlm0=
|
|
||||||
github.com/MoeForge/moejs v0.0.0-20261113160646-b2c7fed9c42d/go.mod h1:GDaH0vQSxKDN7FCyBaLEZgqsGKRyIYBfje/p74aZQ3k=
|
|
||||||
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
|
|
||||||
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
|
|
||||||
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
|
|
||||||
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
|
|
||||||
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
|
|
||||||
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
|
|
||||||
github.com/dop251/goja v0.0.0-20260106131823-651366fbe6e3 h1:bVp3yUzvSAJzu9GqID+Z96P+eu5TKnIMJSV4QaZMauM=
|
|
||||||
github.com/dop251/goja v0.0.0-20260106131823-651366fbe6e3/go.mod h1:MxLav0peU43GgvwVgNbLAj1s/bSGboKkhuULvq/7hx4=
|
|
||||||
github.com/dop251/goja_nodejs v0.0.0-20240728170619-29b559befffc h1:MKYt39yZJi0Z9xEeRmDX2L4ocE0ETKcHKw6MVL3R+co=
|
|
||||||
github.com/dop251/goja_nodejs v0.0.0-20240728170619-29b559befffc/go.mod h1:VULptt4Q/fNzQUJlqY/GP3qHyU7ZH46mFkBZe0ZTokU=
|
|
||||||
github.com/go-sourcemap/sourcemap v2.1.4+incompatible h1:a+iTbH5auLKxaNwQFg0B+TCYl6lbukKPc7b5x0n1s6Q=
|
|
||||||
github.com/go-sourcemap/sourcemap v2.1.4+incompatible/go.mod h1:F8jJfvm2KbVjc5NqelyYJmf/v5J0dwNLS2mL4sNA1Jg=
|
|
||||||
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
|
||||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
|
||||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
|
||||||
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
|
|
||||||
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
|
|
||||||
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 h1:z2ogiKUYzX5Is6zr/vP9vJGqPwcdqsWjOt+V8J7+bTc=
|
|
||||||
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
|
|
||||||
github.com/icholy/digest v1.1.0 h1:HfGg9Irj7i+IX1o1QAmPfIBNu/Q5A5Tu3n/MED9k9H4=
|
|
||||||
github.com/icholy/digest v1.1.0/go.mod h1:QNrsSGQ5v7v9cReDI0+eyjsXGUoRSUZQHeQ5C4XLa0Y=
|
|
||||||
github.com/imroc/req/v3 v3.58.0 h1:rBqJhv2gZjMymGot4vKmNSUs/mAITMpGASqElhuMvS4=
|
|
||||||
github.com/imroc/req/v3 v3.58.0/go.mod h1:rC7NTGgfMSkVWR4GO46rMeJsUfaeOpa6pLsgpq9q3zI=
|
|
||||||
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
|
|
||||||
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
|
|
||||||
github.com/mattn/go-ieproxy v0.0.12 h1:OZkUFJC3ESNZPQ+6LzC3VJIFSnreeFLQyqvBWtvfL2M=
|
|
||||||
github.com/mattn/go-ieproxy v0.0.12/go.mod h1:Vn+N61199DAnVeTgaF8eoB9PvLO8P3OBnG95ENh7B7c=
|
|
||||||
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
|
|
||||||
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
|
|
||||||
github.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw=
|
|
||||||
github.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
|
|
||||||
github.com/refraction-networking/utls v1.8.2 h1:j4Q1gJj0xngdeH+Ox/qND11aEfhpgoEvV+S9iJ2IdQo=
|
|
||||||
github.com/refraction-networking/utls v1.8.2/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM=
|
|
||||||
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
|
|
||||||
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
|
|
||||||
github.com/tetratelabs/wazero v1.12.0 h1:DuWcpNu/FzgEXgGBDp8J1Spc+CWOvvtvVyjKlaZopYU=
|
|
||||||
github.com/tetratelabs/wazero v1.12.0/go.mod h1:LvKtzl2RqO4gyF27BiXU+nKAjcV8f38U+kP/q2vgxh0=
|
|
||||||
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
|
|
||||||
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
|
|
||||||
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
|
|
||||||
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
|
|
||||||
go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
|
|
||||||
go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
|
|
||||||
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
|
|
||||||
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
|
|
||||||
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
|
|
||||||
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
|
|
||||||
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
|
|
||||||
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
|
|
||||||
golang.org/x/net v0.60.0 h1:79p50tfZlm0J9YfoDsSi639qSXNGVwEzOPLCxM2FsYU=
|
|
||||||
golang.org/x/net v0.60.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
|
|
||||||
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
|
|
||||||
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
|
|
||||||
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
|
|
||||||
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
|
|
||||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
|
||||||
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
|
|
||||||
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
|
|
||||||
gotest.tools/v3 v3.5.1 h1:EENdUnS3pdur5nybKYIh2Vfgc8IUNBjxDPSjtiJcOzU=
|
|
||||||
gotest.tools/v3 v3.5.1/go.mod h1:isy3WKz7GK6uNw/sbHzfKBLvlvXwUyV06n6brMxxopU=
|
|
||||||
@@ -1,59 +0,0 @@
|
|||||||
package evaluation
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestAsyncHTTP(t *testing.T) {
|
|
||||||
cancelled := make(chan struct{}, 4)
|
|
||||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
if r.URL.Path == "/wait" {
|
|
||||||
<-r.Context().Done()
|
|
||||||
cancelled <- struct{}{}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
w.Header().Set("Content-Type", "application/json")
|
|
||||||
fmt.Fprint(w, `{"value":42}`)
|
|
||||||
}))
|
|
||||||
defer server.Close()
|
|
||||||
for _, engine := range []string{"goja", "moejs"} {
|
|
||||||
t.Run(engine, func(t *testing.T) {
|
|
||||||
r, e := newRuntime(engine, true)
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
defer r.Close()
|
|
||||||
v, e := r.Eval(fmt.Sprintf(`(async()=>{const r=await fetch(%q);return (await r.json()).value})()`, server.URL))
|
|
||||||
if e != nil || fmt.Sprint(v) != "42" {
|
|
||||||
t.Fatalf("fetch value=%v, error=%v", v, e)
|
|
||||||
}
|
|
||||||
v, e = r.Eval(fmt.Sprintf(`(async()=>{const c=new AbortController();const p=fetch(%q,{signal:c.signal});setTimeout(()=>c.abort(new Error('evaluation cancel')),10);try{await p;return false}catch(e){return e.name+': '+e.message}})()`, server.URL+"/wait"))
|
|
||||||
want := "evaluation cancel"
|
|
||||||
// Existing Gopeed fetch catch path constructs DOMException without a
|
|
||||||
// definition in this standalone host. Record this baseline divergence;
|
|
||||||
// independently require actual cancellation of the server request below.
|
|
||||||
if engine == "goja" {
|
|
||||||
want = "ReferenceError: DOMException is not defined"
|
|
||||||
}
|
|
||||||
if e != nil || !strings.Contains(fmt.Sprint(v), want) {
|
|
||||||
t.Fatalf("abort value=%v, error=%v", v, e)
|
|
||||||
}
|
|
||||||
t.Logf("abort rejection: %v", v)
|
|
||||||
select {
|
|
||||||
case <-cancelled:
|
|
||||||
case <-time.After(2 * time.Second):
|
|
||||||
t.Fatal("request context was not cancelled")
|
|
||||||
}
|
|
||||||
probe, e := r.Eval(`(()=>{try{const c=new AbortController();c.abort();return c.signal.reason?.name}catch(e){return e.name+': '+e.message}})()`)
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
t.Logf("default abort probe: %v", probe)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,107 +0,0 @@
|
|||||||
package evaluation
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/MoeForge/moejs"
|
|
||||||
"github.com/dop251/goja"
|
|
||||||
)
|
|
||||||
|
|
||||||
type request struct {
|
|
||||||
URL string `json:"url"`
|
|
||||||
}
|
|
||||||
type context struct {
|
|
||||||
Req *request `json:"req"`
|
|
||||||
Res map[string]any `json:"res"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func (r *request) SetURL(s string) { r.URL = s }
|
|
||||||
|
|
||||||
// These are migration probes, asserting the observed difference, not claiming
|
|
||||||
// the existing Gopeed Go bridge has been ported.
|
|
||||||
func TestHostInterop(t *testing.T) {
|
|
||||||
for _, name := range []string{"goja", "moejs"} {
|
|
||||||
t.Run(name, func(t *testing.T) {
|
|
||||||
c := &context{Req: &request{URL: "before"}}
|
|
||||||
code := `ctx.req.url = "after"; ctx.res = {files:[{name:"video.mp4"}]}; typeof ctx.req.setURL`
|
|
||||||
var v any
|
|
||||||
var e error
|
|
||||||
if name == "goja" {
|
|
||||||
rt := goja.New()
|
|
||||||
rt.SetFieldNameMapper(goja.TagFieldNameMapper("json", true))
|
|
||||||
if e = rt.Set("ctx", c); e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
value, err := rt.RunString(code)
|
|
||||||
e = err
|
|
||||||
if e == nil {
|
|
||||||
v = value.Export()
|
|
||||||
}
|
|
||||||
if c.Req.URL != "after" || c.Res == nil {
|
|
||||||
t.Fatal("goja write through failed")
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
rt := moejs.NewRuntime(moejs.Options{MutableIntrinsics: true})
|
|
||||||
if e = rt.SetGlobal("ctx", c); e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
s, err := moejs.CompileScript("interop", code)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
value, err := rt.RunScript(s)
|
|
||||||
e = err
|
|
||||||
if e == nil {
|
|
||||||
v, e = rt.ToGo(value)
|
|
||||||
}
|
|
||||||
if c.Req.URL != "before" || c.Res != nil {
|
|
||||||
t.Fatal("expected snapshot semantics")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
t.Logf("Go URL=%s, Go res=%v, JS method=%v", c.Req.URL, c.Res, v)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
func TestFrozenCompatibility(t *testing.T) {
|
|
||||||
r, e := New("moejs-frozen")
|
|
||||||
if e != nil {
|
|
||||||
t.Logf("shared frozen builtins reject host polyfill: %v", e)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer r.Close()
|
|
||||||
b, e := r.Eval(`Object.defineProperty(Array.prototype,"__probe",{value:1}); true`)
|
|
||||||
if e == nil {
|
|
||||||
t.Fatalf("expected frozen builtins to reject mutation, got %v", b)
|
|
||||||
}
|
|
||||||
t.Log(e)
|
|
||||||
}
|
|
||||||
func BenchmarkBareRuntime(b *testing.B) {
|
|
||||||
for _, name := range []string{"goja", "moejs-mutable", "moejs-shared"} {
|
|
||||||
b.Run(name, func(b *testing.B) {
|
|
||||||
b.ReportAllocs()
|
|
||||||
for i := 0; i < b.N; i++ {
|
|
||||||
if name == "goja" {
|
|
||||||
r := goja.New()
|
|
||||||
v, e := r.RunString("1+1")
|
|
||||||
if e != nil || v.ToInteger() != 2 {
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
r := moejs.NewRuntime(moejs.Options{MutableIntrinsics: name == "moejs-mutable"})
|
|
||||||
s, e := moejs.CompileScript("small", "1+1")
|
|
||||||
if e != nil {
|
|
||||||
b.Fatal(e)
|
|
||||||
}
|
|
||||||
v, e := r.RunScript(s)
|
|
||||||
if e != nil || v.String() != "2" {
|
|
||||||
b.Fatal(fmt.Sprint(v), e)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,49 +0,0 @@
|
|||||||
package evaluation
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Opt in: public endpoints are volatile and are not used in benchmark timings.
|
|
||||||
// Only URL resolution is attempted; blob registration and UA WebView stay mocked.
|
|
||||||
func TestLiveResolve(t *testing.T) {
|
|
||||||
if os.Getenv("GOPEED_MOEJS_LIVE") != "1" {
|
|
||||||
t.Skip("set GOPEED_MOEJS_LIVE=1 to contact public endpoints")
|
|
||||||
}
|
|
||||||
for _, name := range []string{"bilibili", "youtube", "twitter"} {
|
|
||||||
var want any
|
|
||||||
for _, engine := range []string{"goja", "moejs"} {
|
|
||||||
t.Run(name+"/"+engine, func(t *testing.T) {
|
|
||||||
r, e := newRuntime(engine, true)
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
defer r.Close()
|
|
||||||
p, e := os.ReadFile("replay.js")
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
for _, s := range []string{fmt.Sprintf("var extensionName=%q;", name), string(p), "liveNetwork = true;", source(t, name)} {
|
|
||||||
if _, e = r.Eval(s); e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
started := time.Now()
|
|
||||||
v, e := r.Eval("resolveOnce()")
|
|
||||||
t.Logf("elapsed=%s", time.Since(started))
|
|
||||||
if e != nil {
|
|
||||||
t.Fatal(e)
|
|
||||||
}
|
|
||||||
if engine == "goja" {
|
|
||||||
want = v
|
|
||||||
} else if want != nil && v != want {
|
|
||||||
t.Fatalf("live resolve outputs differ: %v / %v", want, v)
|
|
||||||
}
|
|
||||||
t.Log(v)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,88 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Measure equal extension work in fresh OS processes, never CPU percentage."""
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import statistics
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
|
|
||||||
root = Path(__file__).resolve().parent
|
|
||||||
os.chdir(root)
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument('--repeats', type=int, default=5)
|
|
||||||
options = parser.add_mutually_exclusive_group()
|
|
||||||
options.add_argument('--smoke', action='store_true')
|
|
||||||
options.add_argument('--memory-only', action='store_true', help='replace retained samples, keeping saved CPU samples')
|
|
||||||
options.add_argument('--summarize-only', action='store_true')
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.repeats < 1:
|
|
||||||
parser.error('--repeats must be positive')
|
|
||||||
all_workloads = [('cold',20),('warm',200),('retained',1),('retained',8)]
|
|
||||||
workloads = [('cold',2),('warm',2),('retained',2)] if args.smoke else all_workloads
|
|
||||||
filename = 'resources-smoke.json' if args.smoke else 'resources.json'
|
|
||||||
rows = []
|
|
||||||
if args.summarize_only:
|
|
||||||
rows = json.loads((root/'results'/filename).read_text())
|
|
||||||
else:
|
|
||||||
(root/'bin').mkdir(exist_ok=True)
|
|
||||||
binary = root/'bin/resource-worker.test'
|
|
||||||
subprocess.run(['go','test','-pgo=off','-c','-o',str(binary)],check=True)
|
|
||||||
if args.memory_only:
|
|
||||||
rows = [r for r in json.loads((root/'results'/filename).read_text()) if r['mode'] != 'retained']
|
|
||||||
workloads = [('retained',1),('retained',8)]
|
|
||||||
with tempfile.TemporaryDirectory(prefix='gopeed-resource-') as temp:
|
|
||||||
for extension in ['bilibili','youtube','twitter']:
|
|
||||||
for mode,count in workloads:
|
|
||||||
for repeat in range(args.repeats):
|
|
||||||
engines = ['goja','moejs'] if repeat % 2 == 0 else ['moejs','goja']
|
|
||||||
for engine in engines:
|
|
||||||
output = Path(temp)/'result.json'
|
|
||||||
env = {**os.environ,'GOMAXPROCS':'1','GOGC':'100','GOMEMLIMIT':'off',
|
|
||||||
'GOPEED_RESOURCE_WORKER':'1','RESOURCE_ENGINE':engine,
|
|
||||||
'RESOURCE_EXTENSION':extension,'RESOURCE_MODE':mode,
|
|
||||||
'RESOURCE_COUNT':str(count),'RESOURCE_OUTPUT':str(output)}
|
|
||||||
run = subprocess.run([str(binary),'-test.run=^TestResourceWorker$','-test.timeout=90s'],
|
|
||||||
env=env,text=True,capture_output=True,timeout=100)
|
|
||||||
if run.returncode:
|
|
||||||
raise RuntimeError(run.stdout+run.stderr)
|
|
||||||
row = json.loads(output.read_text())
|
|
||||||
row['repeat'] = repeat
|
|
||||||
rows.append(row)
|
|
||||||
hashes = {r['output_sha256'] for r in rows if r['extension']==extension}
|
|
||||||
if len(hashes)!=1:
|
|
||||||
raise RuntimeError(f'Output mismatch for {extension}: {hashes}')
|
|
||||||
cpu = (row['user_cpu_ns']+row['system_cpu_ns'])/count/1e6
|
|
||||||
print(f'{extension}/{mode}/{count}/{engine} repeat={repeat+1}: '
|
|
||||||
f'CPU={cpu:.3f} ms/op, peakRSS={row["after_work"]["process_peak_rss_bytes"]/2**20:.1f} MiB',flush=True)
|
|
||||||
(root/'results'/filename).write_text(json.dumps(rows,indent=2)+'\n')
|
|
||||||
|
|
||||||
summary = []
|
|
||||||
for extension in ['bilibili','youtube','twitter']:
|
|
||||||
for mode,count in (all_workloads if args.memory_only else workloads):
|
|
||||||
for engine in ['goja','moejs']:
|
|
||||||
group = [r for r in rows if (r['extension'],r['mode'],r['count'],r['engine']) == (extension,mode,count,engine)]
|
|
||||||
if not group:
|
|
||||||
raise RuntimeError(f'Missing samples for {extension}/{mode}/{count}/{engine}')
|
|
||||||
def med(fn): return statistics.median(fn(r) for r in group)
|
|
||||||
cpu = [(r['user_cpu_ns']+r['system_cpu_ns'])/count for r in group]
|
|
||||||
summary.append(dict(extension=extension,mode=mode,count=count,engine=engine,samples=len(group),
|
|
||||||
cpu_ns_per_op=statistics.median(cpu),cpu_ns_min=min(cpu),cpu_ns_max=max(cpu),
|
|
||||||
user_cpu_ns_per_op=med(lambda r:r['user_cpu_ns']/count),
|
|
||||||
system_cpu_ns_per_op=med(lambda r:r['system_cpu_ns']/count),
|
|
||||||
wall_ns_per_op=med(lambda r:r['wall_ns']/count),
|
|
||||||
allocated_bytes_per_op=med(lambda r:r['allocated_bytes']/count),
|
|
||||||
gc_cycles=med(lambda r:r['gc_cycles']),
|
|
||||||
peak_rss_bytes=med(lambda r:r['after_work']['process_peak_rss_bytes']),
|
|
||||||
peak_rss_min_bytes=min(r['after_work']['process_peak_rss_bytes'] for r in group),
|
|
||||||
peak_rss_max_bytes=max(r['after_work']['process_peak_rss_bytes'] for r in group),
|
|
||||||
rss_after_work_bytes=med(lambda r:r['after_work']['rss_bytes']),
|
|
||||||
live_heap_bytes=med(lambda r:r['after_gc']['heap_alloc_bytes']),
|
|
||||||
live_heap_delta_bytes=med(lambda r:r['after_gc']['heap_alloc_bytes']-r['baseline']['heap_alloc_bytes']),
|
|
||||||
retained_rss_bytes=med(lambda r:r['after_os_release']['rss_bytes']),
|
|
||||||
retained_rss_delta_bytes=med(lambda r:r['after_os_release']['rss_bytes']-r['baseline']['rss_bytes']),
|
|
||||||
after_close_heap_delta_bytes=med(lambda r:r['after_close_os_release']['heap_alloc_bytes']-r['baseline']['heap_alloc_bytes']),
|
|
||||||
after_close_rss_bytes=med(lambda r:r['after_close_os_release']['rss_bytes'])))
|
|
||||||
name = 'resources-smoke-summary.json' if args.smoke else 'resources-summary.json'
|
|
||||||
(root/'results'/name).write_text(json.dumps(summary,indent=2)+'\n')
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Build stripped interpreter probes plus real CLI baseline/coexistence."""
|
|
||||||
import gzip
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
|
|
||||||
root = Path(__file__).resolve().parent
|
|
||||||
os.chdir(root)
|
|
||||||
output = root / 'bin'
|
|
||||||
output.mkdir(exist_ok=True)
|
|
||||||
rows = []
|
|
||||||
for target in [('darwin','arm64'),('linux','amd64'),('windows','amd64'),('android','arm64')]:
|
|
||||||
goos, arch = target
|
|
||||||
env = {**os.environ, 'GOOS':goos, 'GOARCH':arch, 'CGO_ENABLED':'0'}
|
|
||||||
for engine in ['goja','moejs']:
|
|
||||||
path = output / f'{goos}-{arch}-{engine}'
|
|
||||||
cmd = ['go','build','-trimpath','-pgo=off','-ldflags=-s -w','-o',str(path)]
|
|
||||||
if engine == 'moejs':
|
|
||||||
cmd += ['-tags=moejs']
|
|
||||||
subprocess.run(cmd+['./cmd/engine-size'], env=env, check=True)
|
|
||||||
data=path.read_bytes()
|
|
||||||
rows.append(dict(scope='interpreter-only',target=f'{goos}/{arch}',engine=engine,bytes=len(data),gzip_bytes=len(gzip.compress(data,mtime=0))))
|
|
||||||
print(rows[-1], flush=True)
|
|
||||||
# No fake replacement: this is the actual CLI, with and without linking the
|
|
||||||
# candidate runtime. The overlay does not change the root module or source.
|
|
||||||
with tempfile.TemporaryDirectory(prefix='gopeed-moejs-size-') as tmp:
|
|
||||||
tmp=Path(tmp)
|
|
||||||
extra=tmp/'link.go'
|
|
||||||
extra.write_text('''package main
|
|
||||||
import (
|
|
||||||
"github.com/MoeForge/moejs"
|
|
||||||
"github.com/MoeForge/moejs/eventloop"
|
|
||||||
"github.com/MoeForge/moejs/webapi"
|
|
||||||
)
|
|
||||||
func init() {
|
|
||||||
rt:=moejs.NewRuntime(moejs.Options{MutableIntrinsics:true})
|
|
||||||
l,e:=eventloop.New(rt,eventloop.Options{});if e!=nil{panic(e)}
|
|
||||||
api,e:=webapi.Install(l,webapi.Options{});if e!=nil{panic(e)}
|
|
||||||
s,e:=moejs.CompileScript("probe","1+1");if e!=nil{panic(e)}
|
|
||||||
_,e=rt.RunScript(s);if e!=nil{panic(e)}
|
|
||||||
api.Close();l.Terminate()
|
|
||||||
}
|
|
||||||
''')
|
|
||||||
overlay=tmp/'overlay.json'
|
|
||||||
overlay.write_text(json.dumps({'Replace':{str(root.parent.parent/'cmd/gopeed/moejs_eval_link.go'):str(extra)}}))
|
|
||||||
for goos,arch in [('darwin','arm64'),('linux','amd64'),('windows','amd64')]:
|
|
||||||
env={**os.environ,'GOOS':goos,'GOARCH':arch,'CGO_ENABLED':'0'}
|
|
||||||
for engine in ['goja-production','goja-plus-moejs-webapi']:
|
|
||||||
path=output/f'cli-{goos}-{arch}-{engine}'
|
|
||||||
cmd=['go','build','-mod=mod','-trimpath','-pgo=off','-tags=nosqlite','-ldflags=-s -w','-o',str(path)]
|
|
||||||
if engine.endswith('webapi'):
|
|
||||||
cmd += ['-overlay',str(overlay)]
|
|
||||||
subprocess.run(cmd+['github.com/GopeedLab/gopeed/cmd/gopeed'],env=env,check=True)
|
|
||||||
data=path.read_bytes()
|
|
||||||
rows.append(dict(scope='real-cli',target=f'{goos}/{arch}',engine=engine,bytes=len(data),gzip_bytes=len(gzip.compress(data,mtime=0))))
|
|
||||||
print(rows[-1],flush=True)
|
|
||||||
(root/'results/size.json').write_text(json.dumps(rows,indent=2)+'\n')
|
|
||||||
# CLI builds may add sums/indirect requirements in this isolated module.
|
|
||||||
subprocess.run(['go','mod','tidy'],check=True)
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
// Synthetic HTTP responses at the transport boundary. Entire upstream bundles
|
|
||||||
// (including SDKs) execute unchanged; this is not a live website benchmark.
|
|
||||||
var nativeFetch = fetch;
|
|
||||||
var liveNetwork = false;
|
|
||||||
var requests = [];
|
|
||||||
var handlers = {};
|
|
||||||
var blobSerial = 0;
|
|
||||||
var gopeed = {
|
|
||||||
info: {identity: 'monkeyWie@' + extensionName},
|
|
||||||
host: {env:{version:'2.0.0-beta.3',os:'darwin',arch:'arm64'}},
|
|
||||||
settings:{quality:extensionName==='youtube'?'1080p':80,qualityFallback:'best',fallbackToBest:true,cookie:''},
|
|
||||||
logger:{debug(){},info(){},warn(){},error(){}},
|
|
||||||
storage:{get(){return null},set(){}},
|
|
||||||
events:Object.fromEntries(['onResolve','onStart','onError','onDone'].map(k=>[k,fn=>handlers[k]=fn])),
|
|
||||||
runtime:{
|
|
||||||
ffmpeg:{merge(){throw new Error('media merge is outside this replay')},supportsInputFactory:true,supportsProducerProgress:true},
|
|
||||||
blob:{async createObjectURL(){return 'blob:replay/'+(++blobSerial)},async revokeObjectURL(){}},
|
|
||||||
webview:{isAvailable(){return true},async open(){return {
|
|
||||||
async goto(){},async execute(expression){
|
|
||||||
if (expression !== '() => navigator.userAgent') throw new Error('unexpected WebView execution');
|
|
||||||
return 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 Version/18.0 Safari/605.1.15';
|
|
||||||
},async close(){},async getCookies(){return []},async setCookie(){},async deleteCookie(){}
|
|
||||||
}}}
|
|
||||||
}};
|
|
||||||
class MessageError extends Error {}
|
|
||||||
var route = (raw, init) => {
|
|
||||||
const url = new URL(String(raw));
|
|
||||||
requests.push(url.pathname);
|
|
||||||
const json = x => new Response(JSON.stringify(x),{status:200,headers:{'content-type':'application/json'}});
|
|
||||||
if (url.pathname === '/x/web-interface/nav') return json({code:0,data:{wbi_img:{img_url:'https://i0.hdslb.com/bfs/wbi/7cd084941338484aae1ad9425b84077c.png',sub_url:'https://i0.hdslb.com/bfs/wbi/4932caff0ff746eab6f01bf08b70ac45.png'}}});
|
|
||||||
if (url.pathname === '/x/web-interface/wbi/view' || url.pathname === '/x/web-interface/view') return json({code:0,data:{title:'真实扩展 Replay 测试',pages:Array.from({length:3},(_,i)=>({cid:100+i,part:'P'+(i+1)}))}});
|
|
||||||
if (url.pathname === '/x/player/wbi/playurl' || url.pathname === '/x/player/playurl') return json({code:0,data:{quality:80,format:'dash',dash:{video:[{id:64,width:1280,height:720,baseUrl:'https://upos-test.bilivideo.com/video64'},{id:80,width:1920,height:1080,baseUrl:'https://upos-test.bilivideo.com/video80',backupUrl:['https://backup.example.com/video80']}],audio:[{id:30280,baseUrl:'https://upos-test.bilivideo.com/audio'}]}}});
|
|
||||||
if (url.pathname === '/youtubei/v1/player') return json({responseContext:{serviceTrackingParams:[]},playabilityStatus:{status:'OK'},videoDetails:{videoId:'dQw4w9WgXcQ',title:'YouTube Replay 测试',lengthSeconds:'212',channelId:'UCfixture',shortDescription:'Replay fixture',viewCount:'100',author:'Fixture',isLiveContent:false},microformat:{playerMicroformatRenderer:{title:{simpleText:'YouTube Replay 测试'},description:{simpleText:'Replay'},category:'Music',publishDate:'2026-11-01',uploadDate:'2026-11-01',isFamilySafe:true}},streamingData:{expiresInSeconds:'3600',adaptiveFormats:[]}});
|
|
||||||
if (url.pathname.endsWith('/guest/activate.json')) return json({guest_token:'fixture-guest'});
|
|
||||||
if (url.hostname === 'pastebin.com' && url.pathname === '/raw/nz3ApKQM') return new Response('Bearer fixture-only', {status:200});
|
|
||||||
if (url.pathname.includes('/TweetResultByRestId')) {
|
|
||||||
const media = [{type:'photo',media_url_https:'https://pbs.twimg.com/media/replay.jpg'}, {type:'video',media_url_https:'https://pbs.twimg.com/media/replay.jpg',video_info:{duration_millis:10000,variants:[{bitrate:256000,content_type:'video/mp4',url:'https://video.twimg.com/640x360/low.mp4'},{bitrate:2048000,content_type:'video/mp4',url:'https://video.twimg.com/1920x1080/high.mp4'}]}}];
|
|
||||||
return json({data:{tweetResult:{result:{__typename:'Tweet',views:{count:'100'},core:{user_results:{result:{legacy:{screen_name:'fixture',name:'Replay'}}}},legacy:{id_str:'1734145541599383880',full_text:'Replay',created_at:'Tue Dec 12 00:00:00 +0000 2023',entities:{media}}}}}});
|
|
||||||
}
|
|
||||||
if (url.pathname === '/sw.js_data' || url.pathname === '/youtubei/v1/config') return new Response('Offline session: exercise YouTube.js local fallback', {status:503});
|
|
||||||
throw new Error('unmatched replay request: '+url);
|
|
||||||
};
|
|
||||||
fetch = async (input,init) => liveNetwork ? nativeFetch(input,init) : route(input?.url || input,init);
|
|
||||||
async function resolveOnce() {
|
|
||||||
requests = [];
|
|
||||||
blobSerial = 0;
|
|
||||||
const url = globalThis.inputURL || {bilibili:'https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3',youtube:'https://www.youtube.com/watch?v=dQw4w9WgXcQ',twitter:'https://x.com/bbcchinese/status/1734145541599383880'}[extensionName];
|
|
||||||
const ctx = {req:{url}};
|
|
||||||
await handlers.onResolve(ctx);
|
|
||||||
if (!ctx.res?.files?.length) throw new Error('resolve returned no files');
|
|
||||||
return JSON.stringify({res:ctx.res,requests});
|
|
||||||
}
|
|
||||||
|
|
||||||
async function lifecycleOnce() {
|
|
||||||
const resolved = JSON.parse(await resolveOnce());
|
|
||||||
let continued = 0;
|
|
||||||
for (const file of resolved.res.files) {
|
|
||||||
const req = file.req;
|
|
||||||
req.putLabel = async (key,value) => { (req.labels ||= {})[key] = value; };
|
|
||||||
req.setUrl = async url => { req.url = url; };
|
|
||||||
const task = {meta:{req},status:'ready',setUrl:req.setUrl,async continue(){continued++}};
|
|
||||||
if (handlers.onStart) await handlers.onStart({task});
|
|
||||||
task.status = 'error';
|
|
||||||
if (handlers.onError) await handlers.onError({task});
|
|
||||||
}
|
|
||||||
return JSON.stringify({res:resolved.res,continued,requests});
|
|
||||||
}
|
|
||||||
|
|
||||||
// Optional common buffered transport to separate SDK work from each runtime's
|
|
||||||
// Response/ReadableStream implementation. Both engines execute this same JS.
|
|
||||||
class BufferedResponse {
|
|
||||||
constructor(text,options={}) {
|
|
||||||
this._text = String(text); this.status = options.status ?? 200;
|
|
||||||
this.ok = this.status >= 200 && this.status < 300;
|
|
||||||
this.statusText = this.ok ? 'OK' : 'Error'; this.url = '';
|
|
||||||
this.headers = new Headers(options.headers);
|
|
||||||
const bytes = new TextEncoder().encode(this._text);
|
|
||||||
this.body = {getReader(){let sent=false;return {async read(){if(sent)return {done:true};sent=true;return {done:false,value:bytes}},async cancel(){},releaseLock(){}}},async cancel(){}};
|
|
||||||
}
|
|
||||||
async text(){return this._text}
|
|
||||||
async json(){return JSON.parse(this._text)}
|
|
||||||
async arrayBuffer(){return new TextEncoder().encode(this._text).buffer}
|
|
||||||
clone(){return new BufferedResponse(this._text,{status:this.status,headers:this.headers})}
|
|
||||||
}
|
|
||||||
@@ -1,232 +0,0 @@
|
|||||||
//go:build darwin || linux
|
|
||||||
|
|
||||||
package evaluation
|
|
||||||
|
|
||||||
import (
|
|
||||||
"crypto/sha256"
|
|
||||||
"encoding/hex"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"runtime"
|
|
||||||
"runtime/debug"
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
"syscall"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
type resourceSnapshot struct {
|
|
||||||
HeapAlloc uint64 `json:"heap_alloc_bytes"`
|
|
||||||
HeapInuse uint64 `json:"heap_inuse_bytes"`
|
|
||||||
HeapObjects uint64 `json:"heap_objects"`
|
|
||||||
RSS int64 `json:"rss_bytes"`
|
|
||||||
PeakRSS int64 `json:"process_peak_rss_bytes"`
|
|
||||||
Goroutines int `json:"goroutines"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type resourceResult struct {
|
|
||||||
Engine string `json:"engine"`
|
|
||||||
Extension string `json:"extension"`
|
|
||||||
Mode string `json:"mode"`
|
|
||||||
Count int `json:"count"`
|
|
||||||
UserNS int64 `json:"user_cpu_ns"`
|
|
||||||
SystemNS int64 `json:"system_cpu_ns"`
|
|
||||||
WallNS int64 `json:"wall_ns"`
|
|
||||||
Allocated uint64 `json:"allocated_bytes"`
|
|
||||||
GCs uint32 `json:"gc_cycles"`
|
|
||||||
GCPause uint64 `json:"gc_pause_ns"`
|
|
||||||
OutputHash string `json:"output_sha256"`
|
|
||||||
Baseline resourceSnapshot `json:"baseline"`
|
|
||||||
AfterWork resourceSnapshot `json:"after_work"`
|
|
||||||
AfterGC resourceSnapshot `json:"after_gc"`
|
|
||||||
AfterOSRelease resourceSnapshot `json:"after_os_release"`
|
|
||||||
AfterClose resourceSnapshot `json:"after_close_os_release"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// moejs releases weak-keyed compiled metadata via asynchronous Go cleanups.
|
|
||||||
// Multiple collections with scheduling opportunities avoid counting objects
|
|
||||||
// pending those cleanups as permanently live. This is outside CPU timing.
|
|
||||||
func settledGC(t *testing.T) {
|
|
||||||
t.Helper()
|
|
||||||
var previous uint64
|
|
||||||
for i := 0; i < 8; i++ {
|
|
||||||
runtime.GC()
|
|
||||||
time.Sleep(20 * time.Millisecond)
|
|
||||||
var m runtime.MemStats
|
|
||||||
runtime.ReadMemStats(&m)
|
|
||||||
delta := m.HeapAlloc
|
|
||||||
if previous > m.HeapAlloc {
|
|
||||||
delta = previous - m.HeapAlloc
|
|
||||||
} else {
|
|
||||||
delta = m.HeapAlloc - previous
|
|
||||||
}
|
|
||||||
if i >= 2 && delta < 64*1024 {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
previous = m.HeapAlloc
|
|
||||||
}
|
|
||||||
t.Fatal("live heap did not settle within eight GC rounds")
|
|
||||||
}
|
|
||||||
|
|
||||||
func selfUsage(t *testing.T) syscall.Rusage {
|
|
||||||
t.Helper()
|
|
||||||
var usage syscall.Rusage
|
|
||||||
if err := syscall.Getrusage(syscall.RUSAGE_SELF, &usage); err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
return usage
|
|
||||||
}
|
|
||||||
|
|
||||||
func timevalNS(v syscall.Timeval) int64 { return int64(v.Sec)*1e9 + int64(v.Usec)*1e3 }
|
|
||||||
|
|
||||||
func snapshot(t *testing.T) resourceSnapshot {
|
|
||||||
t.Helper()
|
|
||||||
var mem runtime.MemStats
|
|
||||||
runtime.ReadMemStats(&mem)
|
|
||||||
// ps runs only outside the timed CPU interval. RUSAGE_SELF excludes its CPU.
|
|
||||||
b, err := exec.Command("ps", "-o", "rss=", "-p", strconv.Itoa(os.Getpid())).Output()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
rss, err := strconv.ParseInt(strings.TrimSpace(string(b)), 10, 64)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
peak := int64(selfUsage(t).Maxrss)
|
|
||||||
if runtime.GOOS == "linux" {
|
|
||||||
peak *= 1024
|
|
||||||
}
|
|
||||||
return resourceSnapshot{mem.HeapAlloc, mem.HeapInuse, mem.HeapObjects, rss * 1024, peak, runtime.NumGoroutine()}
|
|
||||||
}
|
|
||||||
|
|
||||||
func resourceRuntime(t *testing.T, engine, name, src, host string) Runtime {
|
|
||||||
t.Helper()
|
|
||||||
r, err := New(engine)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
// Clone the bundle for each runtime, like a fresh production file read.
|
|
||||||
for _, s := range []string{fmt.Sprintf("var extensionName=%q;", name), host, strings.Clone(src)} {
|
|
||||||
if _, err = r.Eval(s); err != nil {
|
|
||||||
r.Close()
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return r
|
|
||||||
}
|
|
||||||
|
|
||||||
func resolveResource(t *testing.T, r Runtime) string {
|
|
||||||
t.Helper()
|
|
||||||
value, err := r.Eval("resolveOnce()")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
text, ok := value.(string)
|
|
||||||
if !ok {
|
|
||||||
t.Fatalf("unexpected resolve result %T", value)
|
|
||||||
}
|
|
||||||
sum := sha256.Sum256([]byte(text))
|
|
||||||
return hex.EncodeToString(sum[:])
|
|
||||||
}
|
|
||||||
|
|
||||||
// Each invocation is a fresh OS process. The Python driver repeats both engines
|
|
||||||
// in alternating order; the ordinary test suite skips this measurement worker.
|
|
||||||
func TestResourceWorker(t *testing.T) {
|
|
||||||
if os.Getenv("GOPEED_RESOURCE_WORKER") != "1" {
|
|
||||||
t.Skip("process resource measurement worker")
|
|
||||||
}
|
|
||||||
engine, name, mode := os.Getenv("RESOURCE_ENGINE"), os.Getenv("RESOURCE_EXTENSION"), os.Getenv("RESOURCE_MODE")
|
|
||||||
count, err := strconv.Atoi(os.Getenv("RESOURCE_COUNT"))
|
|
||||||
if err != nil || count < 1 || count > 10000 {
|
|
||||||
t.Fatal("invalid RESOURCE_COUNT")
|
|
||||||
}
|
|
||||||
src := source(t, name)
|
|
||||||
hostBytes, err := os.ReadFile("replay.js")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
host := string(hostBytes)
|
|
||||||
// Pay one-time global initialization only before timing repeat events.
|
|
||||||
// Retained-memory runs start without a priming runtime so their memory
|
|
||||||
// includes the selected engine's first-use caches and builtins.
|
|
||||||
var hash string
|
|
||||||
if mode != "retained" {
|
|
||||||
prime := resourceRuntime(t, engine, name, src, host)
|
|
||||||
hash = resolveResource(t, prime)
|
|
||||||
prime.Close()
|
|
||||||
prime = nil
|
|
||||||
}
|
|
||||||
debug.FreeOSMemory()
|
|
||||||
result := resourceResult{Engine: engine, Extension: name, Mode: mode, Count: count, Baseline: snapshot(t)}
|
|
||||||
var retained []Runtime
|
|
||||||
if mode == "warm" {
|
|
||||||
r := resourceRuntime(t, engine, name, src, host)
|
|
||||||
if h := resolveResource(t, r); h != hash {
|
|
||||||
t.Fatal("warm output differs")
|
|
||||||
}
|
|
||||||
retained = append(retained, r)
|
|
||||||
debug.FreeOSMemory()
|
|
||||||
}
|
|
||||||
var before, after runtime.MemStats
|
|
||||||
runtime.ReadMemStats(&before)
|
|
||||||
cpuBefore := selfUsage(t)
|
|
||||||
started := time.Now()
|
|
||||||
for i := 0; i < count; i++ {
|
|
||||||
var r Runtime
|
|
||||||
switch mode {
|
|
||||||
case "warm":
|
|
||||||
r = retained[0]
|
|
||||||
case "cold", "retained":
|
|
||||||
r = resourceRuntime(t, engine, name, src, host)
|
|
||||||
default:
|
|
||||||
t.Fatal("unknown RESOURCE_MODE")
|
|
||||||
}
|
|
||||||
h := resolveResource(t, r)
|
|
||||||
if hash == "" {
|
|
||||||
hash = h
|
|
||||||
} else if h != hash {
|
|
||||||
t.Fatal("resolve output changed")
|
|
||||||
}
|
|
||||||
if mode == "retained" {
|
|
||||||
retained = append(retained, r)
|
|
||||||
} else if mode == "cold" {
|
|
||||||
r.Close()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
result.WallNS = time.Since(started).Nanoseconds()
|
|
||||||
cpuAfter := selfUsage(t)
|
|
||||||
runtime.ReadMemStats(&after)
|
|
||||||
result.UserNS = timevalNS(cpuAfter.Utime) - timevalNS(cpuBefore.Utime)
|
|
||||||
result.SystemNS = timevalNS(cpuAfter.Stime) - timevalNS(cpuBefore.Stime)
|
|
||||||
result.Allocated = after.TotalAlloc - before.TotalAlloc
|
|
||||||
result.GCs = after.NumGC - before.NumGC
|
|
||||||
result.GCPause = after.PauseTotalNs - before.PauseTotalNs
|
|
||||||
result.AfterWork = snapshot(t)
|
|
||||||
settledGC(t)
|
|
||||||
result.AfterGC = snapshot(t)
|
|
||||||
debug.FreeOSMemory()
|
|
||||||
result.AfterOSRelease = snapshot(t)
|
|
||||||
runtime.KeepAlive(retained)
|
|
||||||
for _, r := range retained {
|
|
||||||
r.Close()
|
|
||||||
}
|
|
||||||
retained = nil
|
|
||||||
settledGC(t)
|
|
||||||
debug.FreeOSMemory()
|
|
||||||
result.AfterClose = snapshot(t)
|
|
||||||
// Keep input strings alive across all snapshots, including after closing
|
|
||||||
// runtimes, so memory deltas cannot accidentally subtract the input bundle.
|
|
||||||
runtime.KeepAlive(src)
|
|
||||||
runtime.KeepAlive(host)
|
|
||||||
result.OutputHash = hash
|
|
||||||
output, err := json.MarshalIndent(result, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if err = os.WriteFile(os.Getenv("RESOURCE_OUTPUT"), append(output, '\n'), 0644); err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
be0a395447d788419e83acdfd693e48a1a1706bf
|
|
||||||
@@ -1,170 +0,0 @@
|
|||||||
{
|
|
||||||
"BenchmarkExtension/bilibili/goja/warm": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 21188099.0,
|
|
||||||
"ns_min": 19941715.0,
|
|
||||||
"ns_max": 21722030.0,
|
|
||||||
"bytes_median": 8340744.0,
|
|
||||||
"allocs_median": 143394.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/bilibili/goja/cold": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 118861317.0,
|
|
||||||
"ns_min": 117766708.0,
|
|
||||||
"ns_max": 119968958.0,
|
|
||||||
"bytes_median": 76348716.0,
|
|
||||||
"allocs_median": 1292863.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/bilibili/moejs/warm": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 3389193.0,
|
|
||||||
"ns_min": 3263242.0,
|
|
||||||
"ns_max": 3448471.0,
|
|
||||||
"bytes_median": 1305526.0,
|
|
||||||
"allocs_median": 11007.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/bilibili/moejs/cold": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 104827183.0,
|
|
||||||
"ns_min": 104022715.0,
|
|
||||||
"ns_max": 106050611.0,
|
|
||||||
"bytes_median": 62157413.0,
|
|
||||||
"allocs_median": 752967.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/youtube/goja/warm": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 3193004.0,
|
|
||||||
"ns_min": 3146585.0,
|
|
||||||
"ns_max": 3260717.0,
|
|
||||||
"bytes_median": 1201206.0,
|
|
||||||
"allocs_median": 25233.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/youtube/goja/cold": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 283412271.0,
|
|
||||||
"ns_min": 283191208.0,
|
|
||||||
"ns_max": 297033125.0,
|
|
||||||
"bytes_median": 164280012.0,
|
|
||||||
"allocs_median": 4518344.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/youtube/moejs/warm": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 276599.0,
|
|
||||||
"ns_min": 269731.0,
|
|
||||||
"ns_max": 286438.0,
|
|
||||||
"bytes_median": 153548.0,
|
|
||||||
"allocs_median": 1289.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/youtube/moejs/cold": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 239646417.0,
|
|
||||||
"ns_min": 238944375.0,
|
|
||||||
"ns_max": 253687520.0,
|
|
||||||
"bytes_median": 150780424.0,
|
|
||||||
"allocs_median": 1628650.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/twitter/goja/warm": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 3574601.0,
|
|
||||||
"ns_min": 3518603.0,
|
|
||||||
"ns_max": 3591739.0,
|
|
||||||
"bytes_median": 1682900.0,
|
|
||||||
"allocs_median": 32486.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/twitter/goja/cold": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 17238709.0,
|
|
||||||
"ns_min": 16791614.0,
|
|
||||||
"ns_max": 17428731.0,
|
|
||||||
"bytes_median": 11428458.0,
|
|
||||||
"allocs_median": 195739.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/twitter/moejs/warm": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 505147.0,
|
|
||||||
"ns_min": 497912.0,
|
|
||||||
"ns_max": 507563.0,
|
|
||||||
"bytes_median": 253665.0,
|
|
||||||
"allocs_median": 2799.0
|
|
||||||
},
|
|
||||||
"BenchmarkExtension/twitter/moejs/cold": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 11849345.0,
|
|
||||||
"ns_min": 11735946.0,
|
|
||||||
"ns_max": 11875913.0,
|
|
||||||
"bytes_median": 8040679.0,
|
|
||||||
"allocs_median": 89924.0
|
|
||||||
},
|
|
||||||
"BenchmarkBufferedTransport/bilibili/goja": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 22076250.0,
|
|
||||||
"ns_min": 20500819.0,
|
|
||||||
"ns_max": 22754704.0,
|
|
||||||
"bytes_median": 8228846.0,
|
|
||||||
"allocs_median": 141672.0
|
|
||||||
},
|
|
||||||
"BenchmarkBufferedTransport/bilibili/moejs": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 3528287.0,
|
|
||||||
"ns_min": 3421821.0,
|
|
||||||
"ns_max": 3572089.0,
|
|
||||||
"bytes_median": 1297568.0,
|
|
||||||
"allocs_median": 10838.0
|
|
||||||
},
|
|
||||||
"BenchmarkBufferedTransport/youtube/goja": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 2895027.0,
|
|
||||||
"ns_min": 2554084.0,
|
|
||||||
"ns_max": 2944065.0,
|
|
||||||
"bytes_median": 1096766.0,
|
|
||||||
"allocs_median": 22358.0
|
|
||||||
},
|
|
||||||
"BenchmarkBufferedTransport/youtube/moejs": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 289619.0,
|
|
||||||
"ns_min": 274739.0,
|
|
||||||
"ns_max": 290746.0,
|
|
||||||
"bytes_median": 154919.0,
|
|
||||||
"allocs_median": 1299.0
|
|
||||||
},
|
|
||||||
"BenchmarkBufferedTransport/twitter/goja": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 3450359.0,
|
|
||||||
"ns_min": 3359477.0,
|
|
||||||
"ns_max": 3460980.0,
|
|
||||||
"bytes_median": 1564097.0,
|
|
||||||
"allocs_median": 30491.0
|
|
||||||
},
|
|
||||||
"BenchmarkBufferedTransport/twitter/moejs": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 491075.0,
|
|
||||||
"ns_min": 480354.0,
|
|
||||||
"ns_max": 495429.0,
|
|
||||||
"bytes_median": 243989.0,
|
|
||||||
"allocs_median": 2633.0
|
|
||||||
},
|
|
||||||
"BenchmarkBareRuntime/goja": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 2847.0,
|
|
||||||
"ns_min": 2823.0,
|
|
||||||
"ns_max": 2893.0,
|
|
||||||
"bytes_median": 5216.0,
|
|
||||||
"allocs_median": 50.0
|
|
||||||
},
|
|
||||||
"BenchmarkBareRuntime/moejs-mutable": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 48417.0,
|
|
||||||
"ns_min": 48202.0,
|
|
||||||
"ns_max": 50930.0,
|
|
||||||
"bytes_median": 106053.0,
|
|
||||||
"allocs_median": 145.0
|
|
||||||
},
|
|
||||||
"BenchmarkBareRuntime/moejs-shared": {
|
|
||||||
"samples": 5,
|
|
||||||
"ns_median": 3857.0,
|
|
||||||
"ns_min": 3839.0,
|
|
||||||
"ns_max": 3874.0,
|
|
||||||
"bytes_median": 5307.0,
|
|
||||||
"allocs_median": 42.0
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,111 +0,0 @@
|
|||||||
goos: darwin
|
|
||||||
goarch: arm64
|
|
||||||
pkg: github.com/GopeedLab/gopeed/experiments/moejs
|
|
||||||
cpu: Apple M1
|
|
||||||
BenchmarkExtension/bilibili/goja/warm 39 19941715 ns/op 8340738 B/op 143388 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/warm 40 21722030 ns/op 8340729 B/op 143393 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/warm 38 21124144 ns/op 8340791 B/op 143401 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/warm 38 21188099 ns/op 8340744 B/op 143394 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/warm 39 21522829 ns/op 8340776 B/op 143399 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/cold 5 117766708 ns/op 76348771 B/op 1292869 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/cold 5 119968958 ns/op 76348876 B/op 1292861 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/cold 5 117841117 ns/op 76348716 B/op 1292863 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/cold 5 118861317 ns/op 76341414 B/op 1292867 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/goja/cold 5 119065983 ns/op 76341294 B/op 1292854 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/warm 180 3263242 ns/op 1304595 B/op 10995 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/warm 180 3375801 ns/op 1305709 B/op 11011 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/warm 182 3389193 ns/op 1305065 B/op 11002 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/warm 180 3448471 ns/op 1305526 B/op 11007 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/warm 171 3436332 ns/op 1305533 B/op 11008 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/cold 5 104827183 ns/op 62224300 B/op 754651 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/cold 6 104631410 ns/op 62138065 B/op 752864 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/cold 6 106050611 ns/op 62145738 B/op 752980 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/cold 6 104022715 ns/op 62157413 B/op 752954 allocs/op
|
|
||||||
BenchmarkExtension/bilibili/moejs/cold 6 105753188 ns/op 62159202 B/op 752967 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/warm 195 3146585 ns/op 1201213 B/op 25233 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/warm 159 3174966 ns/op 1201205 B/op 25233 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/warm 163 3260717 ns/op 1201209 B/op 25233 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/warm 165 3202571 ns/op 1201205 B/op 25233 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/warm 158 3193004 ns/op 1201206 B/op 25233 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/cold 2 285443125 ns/op 164279960 B/op 4518343 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/cold 2 297033125 ns/op 164279944 B/op 4518343 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/cold 2 283191208 ns/op 164280164 B/op 4518344 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/cold 2 283412271 ns/op 164280012 B/op 4518344 allocs/op
|
|
||||||
BenchmarkExtension/youtube/goja/cold 2 283388230 ns/op 164280452 B/op 4518345 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/warm 2664 269731 ns/op 153550 B/op 1289 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/warm 2636 286438 ns/op 153550 B/op 1289 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/warm 2654 277087 ns/op 153548 B/op 1289 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/warm 2653 276165 ns/op 153545 B/op 1289 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/warm 2638 276599 ns/op 153547 B/op 1289 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/cold 3 238944375 ns/op 150707533 B/op 1628515 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/cold 3 243268042 ns/op 150800744 B/op 1628681 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/cold 3 239646417 ns/op 150709949 B/op 1628458 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/cold 2 253687520 ns/op 150780424 B/op 1631728 allocs/op
|
|
||||||
BenchmarkExtension/youtube/moejs/cold 3 239059042 ns/op 150786264 B/op 1628650 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/warm 163 3518603 ns/op 1682901 B/op 32486 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/warm 165 3570482 ns/op 1682898 B/op 32486 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/warm 169 3575004 ns/op 1682900 B/op 32486 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/warm 169 3574601 ns/op 1682900 B/op 32486 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/warm 170 3591739 ns/op 1682900 B/op 32486 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/cold 45 17377419 ns/op 11428470 B/op 195739 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/cold 45 17238709 ns/op 11428451 B/op 195739 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/cold 45 16791614 ns/op 11428452 B/op 195739 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/cold 45 17428731 ns/op 11428460 B/op 195739 allocs/op
|
|
||||||
BenchmarkExtension/twitter/goja/cold 45 16943216 ns/op 11428458 B/op 195739 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/warm 1179 497912 ns/op 253663 B/op 2799 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/warm 1136 506676 ns/op 253669 B/op 2799 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/warm 1129 507563 ns/op 253674 B/op 2799 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/warm 1161 501615 ns/op 253665 B/op 2799 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/warm 1166 505147 ns/op 253663 B/op 2799 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/cold 61 11869347 ns/op 8040641 B/op 89924 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/cold 61 11735946 ns/op 8041001 B/op 89926 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/cold 62 11751223 ns/op 8040679 B/op 89923 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/cold 63 11849345 ns/op 8040427 B/op 89915 allocs/op
|
|
||||||
BenchmarkExtension/twitter/moejs/cold 63 11875913 ns/op 8040909 B/op 89925 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/goja 40 20500819 ns/op 8228883 B/op 141673 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/goja 39 22754704 ns/op 8228914 B/op 141680 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/goja 26 21679981 ns/op 8228819 B/op 141668 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/goja 40 22076250 ns/op 8228846 B/op 141672 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/goja 39 22079190 ns/op 8228820 B/op 141668 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/moejs 171 3446011 ns/op 1298601 B/op 10854 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/moejs 176 3528287 ns/op 1297426 B/op 10837 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/moejs 177 3560739 ns/op 1297948 B/op 10844 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/moejs 177 3421821 ns/op 1297568 B/op 10838 allocs/op
|
|
||||||
BenchmarkBufferedTransport/bilibili/moejs 177 3572089 ns/op 1297237 B/op 10834 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/goja 223 2554084 ns/op 1096767 B/op 22358 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/goja 211 2895027 ns/op 1096766 B/op 22358 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/goja 212 2936390 ns/op 1096766 B/op 22358 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/goja 211 2944065 ns/op 1096766 B/op 22358 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/goja 222 2889798 ns/op 1096766 B/op 22358 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/moejs 2630 274739 ns/op 154922 B/op 1299 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/moejs 2602 289619 ns/op 154919 B/op 1299 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/moejs 2600 290746 ns/op 154920 B/op 1299 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/moejs 2618 290116 ns/op 154919 B/op 1299 allocs/op
|
|
||||||
BenchmarkBufferedTransport/youtube/moejs 2569 280026 ns/op 154914 B/op 1299 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/goja 172 3359477 ns/op 1564100 B/op 30491 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/goja 172 3450359 ns/op 1564097 B/op 30491 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/goja 170 3458607 ns/op 1564097 B/op 30491 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/goja 170 3401113 ns/op 1564097 B/op 30491 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/goja 170 3460980 ns/op 1564097 B/op 30491 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/moejs 1196 480354 ns/op 243983 B/op 2633 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/moejs 1146 490821 ns/op 243989 B/op 2633 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/moejs 1152 493628 ns/op 243989 B/op 2633 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/moejs 1144 491075 ns/op 243991 B/op 2633 allocs/op
|
|
||||||
BenchmarkBufferedTransport/twitter/moejs 1152 495429 ns/op 243989 B/op 2633 allocs/op
|
|
||||||
BenchmarkBareRuntime/goja 204234 2825 ns/op 5216 B/op 50 allocs/op
|
|
||||||
BenchmarkBareRuntime/goja 190192 2893 ns/op 5216 B/op 50 allocs/op
|
|
||||||
BenchmarkBareRuntime/goja 189585 2847 ns/op 5216 B/op 50 allocs/op
|
|
||||||
BenchmarkBareRuntime/goja 191311 2823 ns/op 5216 B/op 50 allocs/op
|
|
||||||
BenchmarkBareRuntime/goja 192636 2849 ns/op 5216 B/op 50 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-mutable 10000 50930 ns/op 106053 B/op 145 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-mutable 12385 48203 ns/op 106051 B/op 145 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-mutable 12442 48417 ns/op 106053 B/op 145 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-mutable 12380 48202 ns/op 106052 B/op 145 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-mutable 12277 48647 ns/op 106053 B/op 145 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-shared 181504 3863 ns/op 5307 B/op 42 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-shared 179941 3851 ns/op 5307 B/op 42 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-shared 181015 3839 ns/op 5307 B/op 42 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-shared 183372 3857 ns/op 5307 B/op 42 allocs/op
|
|
||||||
BenchmarkBareRuntime/moejs-shared 178731 3874 ns/op 5307 B/op 42 allocs/op
|
|
||||||
PASS
|
|
||||||
ok github.com/GopeedLab/gopeed/experiments/moejs 118.047s
|
|
||||||
@@ -1,112 +0,0 @@
|
|||||||
=== RUN TestMoeResolveWriteback
|
|
||||||
bridge_test.go:44: Go Resource.Validate passed, name=BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4
|
|
||||||
--- PASS: TestMoeResolveWriteback (0.09s)
|
|
||||||
=== RUN TestBufferedReplay
|
|
||||||
=== RUN TestBufferedReplay/bilibili
|
|
||||||
=== RUN TestBufferedReplay/bilibili/goja
|
|
||||||
=== RUN TestBufferedReplay/bilibili/moejs
|
|
||||||
=== RUN TestBufferedReplay/youtube
|
|
||||||
=== RUN TestBufferedReplay/youtube/goja
|
|
||||||
=== RUN TestBufferedReplay/youtube/moejs
|
|
||||||
=== RUN TestBufferedReplay/twitter
|
|
||||||
=== RUN TestBufferedReplay/twitter/goja
|
|
||||||
=== RUN TestBufferedReplay/twitter/moejs
|
|
||||||
--- PASS: TestBufferedReplay (0.60s)
|
|
||||||
--- PASS: TestBufferedReplay/bilibili (0.18s)
|
|
||||||
--- PASS: TestBufferedReplay/bilibili/goja (0.09s)
|
|
||||||
--- PASS: TestBufferedReplay/bilibili/moejs (0.09s)
|
|
||||||
--- PASS: TestBufferedReplay/youtube (0.40s)
|
|
||||||
--- PASS: TestBufferedReplay/youtube/goja (0.21s)
|
|
||||||
--- PASS: TestBufferedReplay/youtube/moejs (0.19s)
|
|
||||||
--- PASS: TestBufferedReplay/twitter (0.02s)
|
|
||||||
--- PASS: TestBufferedReplay/twitter/goja (0.01s)
|
|
||||||
--- PASS: TestBufferedReplay/twitter/moejs (0.01s)
|
|
||||||
=== RUN TestExtensionReplay
|
|
||||||
=== RUN TestExtensionReplay/bilibili
|
|
||||||
=== RUN TestExtensionReplay/bilibili/goja
|
|
||||||
evaluation_test.go:46: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/2","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/3","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}}]},"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestExtensionReplay/bilibili/moejs
|
|
||||||
evaluation_test.go:46: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/2","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/3","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}}]},"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestExtensionReplay/youtube
|
|
||||||
=== RUN TestExtensionReplay/youtube/goja
|
|
||||||
evaluation_test.go:46: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged"}}}]},"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestExtensionReplay/youtube/moejs
|
|
||||||
evaluation_test.go:46: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged"}}}]},"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestExtensionReplay/twitter
|
|
||||||
=== RUN TestExtensionReplay/twitter/goja
|
|
||||||
evaluation_test.go:46: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
=== RUN TestExtensionReplay/twitter/moejs
|
|
||||||
evaluation_test.go:46: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
--- PASS: TestExtensionReplay (0.59s)
|
|
||||||
--- PASS: TestExtensionReplay/bilibili (0.17s)
|
|
||||||
--- PASS: TestExtensionReplay/bilibili/goja (0.09s)
|
|
||||||
--- PASS: TestExtensionReplay/bilibili/moejs (0.09s)
|
|
||||||
--- PASS: TestExtensionReplay/youtube (0.39s)
|
|
||||||
--- PASS: TestExtensionReplay/youtube/goja (0.21s)
|
|
||||||
--- PASS: TestExtensionReplay/youtube/moejs (0.19s)
|
|
||||||
--- PASS: TestExtensionReplay/twitter (0.02s)
|
|
||||||
--- PASS: TestExtensionReplay/twitter/goja (0.01s)
|
|
||||||
--- PASS: TestExtensionReplay/twitter/moejs (0.01s)
|
|
||||||
=== RUN TestExtensionLifecycle
|
|
||||||
=== RUN TestExtensionLifecycle/bilibili
|
|
||||||
=== RUN TestExtensionLifecycle/bilibili/goja
|
|
||||||
evaluation_test.go:110: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/5","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/7","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/9","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}}]},"continued":3,"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestExtensionLifecycle/bilibili/moejs
|
|
||||||
evaluation_test.go:110: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/5","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/7","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/9","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}}]},"continued":3,"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestExtensionLifecycle/youtube
|
|
||||||
=== RUN TestExtensionLifecycle/youtube/goja
|
|
||||||
evaluation_test.go:110: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged","retried":"1"}}}]},"continued":1,"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestExtensionLifecycle/youtube/moejs
|
|
||||||
evaluation_test.go:110: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged","retried":"1"}}}]},"continued":1,"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestExtensionLifecycle/twitter
|
|
||||||
=== RUN TestExtensionLifecycle/twitter/goja
|
|
||||||
evaluation_test.go:110: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"continued":0,"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
=== RUN TestExtensionLifecycle/twitter/moejs
|
|
||||||
evaluation_test.go:110: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"continued":0,"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
--- PASS: TestExtensionLifecycle (0.63s)
|
|
||||||
--- PASS: TestExtensionLifecycle/bilibili (0.20s)
|
|
||||||
--- PASS: TestExtensionLifecycle/bilibili/goja (0.11s)
|
|
||||||
--- PASS: TestExtensionLifecycle/bilibili/moejs (0.09s)
|
|
||||||
--- PASS: TestExtensionLifecycle/youtube (0.40s)
|
|
||||||
--- PASS: TestExtensionLifecycle/youtube/goja (0.21s)
|
|
||||||
--- PASS: TestExtensionLifecycle/youtube/moejs (0.19s)
|
|
||||||
--- PASS: TestExtensionLifecycle/twitter (0.02s)
|
|
||||||
--- PASS: TestExtensionLifecycle/twitter/goja (0.01s)
|
|
||||||
--- PASS: TestExtensionLifecycle/twitter/moejs (0.01s)
|
|
||||||
=== RUN TestFrozenExtensionReplay
|
|
||||||
=== RUN TestFrozenExtensionReplay/bilibili
|
|
||||||
frozen_test.go:13: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/2","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/3","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}}]},"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestFrozenExtensionReplay/youtube
|
|
||||||
frozen_test.go:13: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged"}}}]},"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestFrozenExtensionReplay/twitter
|
|
||||||
frozen_test.go:13: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
--- PASS: TestFrozenExtensionReplay (0.28s)
|
|
||||||
--- PASS: TestFrozenExtensionReplay/bilibili (0.08s)
|
|
||||||
--- PASS: TestFrozenExtensionReplay/youtube (0.19s)
|
|
||||||
--- PASS: TestFrozenExtensionReplay/twitter (0.01s)
|
|
||||||
=== RUN TestAsyncHTTP
|
|
||||||
=== RUN TestAsyncHTTP/goja
|
|
||||||
http_test.go:46: abort rejection: ReferenceError: DOMException is not defined
|
|
||||||
http_test.go:56: default abort probe: AbortError
|
|
||||||
=== RUN TestAsyncHTTP/moejs
|
|
||||||
http_test.go:46: abort rejection: Error: evaluation cancel
|
|
||||||
http_test.go:56: default abort probe: AbortError
|
|
||||||
--- PASS: TestAsyncHTTP (0.04s)
|
|
||||||
--- PASS: TestAsyncHTTP/goja (0.02s)
|
|
||||||
--- PASS: TestAsyncHTTP/moejs (0.02s)
|
|
||||||
=== RUN TestHostInterop
|
|
||||||
=== RUN TestHostInterop/goja
|
|
||||||
interop_test.go:65: Go URL=after, Go res=map[files:[map[name:video.mp4]]], JS method=function
|
|
||||||
=== RUN TestHostInterop/moejs
|
|
||||||
interop_test.go:65: Go URL=before, Go res=map[], JS method=undefined
|
|
||||||
--- PASS: TestHostInterop (0.00s)
|
|
||||||
--- PASS: TestHostInterop/goja (0.00s)
|
|
||||||
--- PASS: TestHostInterop/moejs (0.00s)
|
|
||||||
=== RUN TestFrozenCompatibility
|
|
||||||
interop_test.go:80: TypeError: Cannot modify property '__probe' of shared intrinsic [object Array]
|
|
||||||
--- PASS: TestFrozenCompatibility (0.00s)
|
|
||||||
=== RUN TestLiveResolve
|
|
||||||
live_test.go:14: set GOPEED_MOEJS_LIVE=1 to contact public endpoints
|
|
||||||
--- SKIP: TestLiveResolve (0.00s)
|
|
||||||
PASS
|
|
||||||
ok github.com/GopeedLab/gopeed/experiments/moejs 2.824s
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
=== RUN TestAsyncHTTP
|
|
||||||
=== RUN TestAsyncHTTP/goja
|
|
||||||
http_test.go:46: abort rejection: ReferenceError: DOMException is not defined
|
|
||||||
http_test.go:56: default abort probe: AbortError
|
|
||||||
=== RUN TestAsyncHTTP/moejs
|
|
||||||
http_test.go:46: abort rejection: Error: evaluation cancel
|
|
||||||
http_test.go:56: default abort probe: AbortError
|
|
||||||
--- PASS: TestAsyncHTTP (0.04s)
|
|
||||||
--- PASS: TestAsyncHTTP/goja (0.02s)
|
|
||||||
--- PASS: TestAsyncHTTP/moejs (0.02s)
|
|
||||||
PASS
|
|
||||||
ok github.com/GopeedLab/gopeed/experiments/moejs 0.631s
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
=== RUN TestAsyncHTTP
|
|
||||||
=== RUN TestAsyncHTTP/goja
|
|
||||||
http_test.go:35: abort rejection: ReferenceError: DOMException is not defined
|
|
||||||
=== RUN TestAsyncHTTP/moejs
|
|
||||||
http_test.go:35: abort rejection: AbortError: This operation was aborted
|
|
||||||
--- PASS: TestAsyncHTTP (0.04s)
|
|
||||||
--- PASS: TestAsyncHTTP/goja (0.02s)
|
|
||||||
--- PASS: TestAsyncHTTP/moejs (0.02s)
|
|
||||||
=== RUN TestLiveResolve
|
|
||||||
=== RUN TestLiveResolve/bilibili/goja
|
|
||||||
live_test.go:36: elapsed=1.205312291s
|
|
||||||
live_test.go:45: {"res":{"name":"李宏毅2024春《生成式人工智能导论》","range":false,"files":[{"name":"BV1BJ4m1e7g8.李宏毅2024春《生成式人工智能导论》(P1).480p.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":1449518003,"p":0,"type":"merged","quality":80,"selectedQuality":32,"resolution":"480p"}}},{"name":"BV1BJ4m1e7g8.李宏毅2024春《生成式人工智能导论》(P2).480p.mp4","req":{"url":"blob:replay/2","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":1449532181,"p":1,"type":"merged","quality":80,"selectedQuality":32,"resolution":"480p"}}},{"name":"BV1BJ4m1e7g8.李宏毅2024春《生成式人工智能导论》(P3).480p.mp4","req":{"url":"blob:replay/3","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":1456854415,"p":2,"type":"merged","quality":80,"selectedQuality":32,"resolution":"480p"}}}]},"requests":[]}
|
|
||||||
=== RUN TestLiveResolve/bilibili/moejs
|
|
||||||
live_test.go:36: elapsed=521.812875ms
|
|
||||||
live_test.go:45: {"res":{"name":"李宏毅2024春《生成式人工智能导论》","range":false,"files":[{"name":"BV1BJ4m1e7g8.李宏毅2024春《生成式人工智能导论》(P1).480p.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":1449518003,"p":0,"type":"merged","quality":80,"selectedQuality":32,"resolution":"480p"}}},{"name":"BV1BJ4m1e7g8.李宏毅2024春《生成式人工智能导论》(P2).480p.mp4","req":{"url":"blob:replay/2","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":1449532181,"p":1,"type":"merged","quality":80,"selectedQuality":32,"resolution":"480p"}}},{"name":"BV1BJ4m1e7g8.李宏毅2024春《生成式人工智能导论》(P3).480p.mp4","req":{"url":"blob:replay/3","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":1456854415,"p":2,"type":"merged","quality":80,"selectedQuality":32,"resolution":"480p"}}}]},"requests":[]}
|
|
||||||
=== RUN TestLiveResolve/youtube/goja
|
|
||||||
live_test.go:36: elapsed=1.994355417s
|
|
||||||
live_test.go:45: {"res":{"range":false,"files":[{"name":"Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster).mp4","req":{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged"}}}]},"requests":[]}
|
|
||||||
=== RUN TestLiveResolve/youtube/moejs
|
|
||||||
live_test.go:36: elapsed=969.283625ms
|
|
||||||
live_test.go:45: {"res":{"range":false,"files":[{"name":"Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster).mp4","req":{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged"}}}]},"requests":[]}
|
|
||||||
=== RUN TestLiveResolve/twitter/goja
|
|
||||||
live_test.go:36: elapsed=2.249496s
|
|
||||||
live_test.go:45: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.mp4","req":{"url":"https://video.twimg.com/ext_tw_video/1734143688874561536/pu/vid/avc1/1280x720/hpx1Z8GQYa0FVgGb.mp4?tag=12"}}]},"requests":[]}
|
|
||||||
=== RUN TestLiveResolve/twitter/moejs
|
|
||||||
live_test.go:36: elapsed=2.231541167s
|
|
||||||
live_test.go:45: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.mp4","req":{"url":"https://video.twimg.com/ext_tw_video/1734143688874561536/pu/vid/avc1/1280x720/hpx1Z8GQYa0FVgGb.mp4?tag=12"}}]},"requests":[]}
|
|
||||||
--- PASS: TestLiveResolve (9.86s)
|
|
||||||
--- PASS: TestLiveResolve/bilibili/goja (1.28s)
|
|
||||||
--- PASS: TestLiveResolve/bilibili/moejs (0.61s)
|
|
||||||
--- PASS: TestLiveResolve/youtube/goja (2.24s)
|
|
||||||
--- PASS: TestLiveResolve/youtube/moejs (1.19s)
|
|
||||||
--- PASS: TestLiveResolve/twitter/goja (2.28s)
|
|
||||||
--- PASS: TestLiveResolve/twitter/moejs (2.26s)
|
|
||||||
PASS
|
|
||||||
ok github.com/GopeedLab/gopeed/experiments/moejs 10.511s
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
ok github.com/GopeedLab/gopeed/pkg/download/engine 15.886s
|
|
||||||
ok github.com/GopeedLab/gopeed/pkg/download/engine/ffmpeg 12.799s
|
|
||||||
? github.com/GopeedLab/gopeed/pkg/download/engine/inject/error [no test files]
|
|
||||||
? github.com/GopeedLab/gopeed/pkg/download/engine/inject/fetch [no test files]
|
|
||||||
? github.com/GopeedLab/gopeed/pkg/download/engine/inject/ffmpeg [no test files]
|
|
||||||
? github.com/GopeedLab/gopeed/pkg/download/engine/inject/file [no test files]
|
|
||||||
ok github.com/GopeedLab/gopeed/pkg/download/engine/inject/stream 2.423s
|
|
||||||
? github.com/GopeedLab/gopeed/pkg/download/engine/inject/vm [no test files]
|
|
||||||
? github.com/GopeedLab/gopeed/pkg/download/engine/inject/xhr [no test files]
|
|
||||||
? github.com/GopeedLab/gopeed/pkg/download/engine/util [no test files]
|
|
||||||
ok github.com/GopeedLab/gopeed/pkg/download/engine/webview 0.777s
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
bilibili/retained/1/goja repeat=1: CPU=128.924 ms/op, peakRSS=113.7 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=1: CPU=108.563 ms/op, peakRSS=109.7 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=2: CPU=103.893 ms/op, peakRSS=110.5 MiB
|
|
||||||
bilibili/retained/1/goja repeat=2: CPU=119.400 ms/op, peakRSS=113.9 MiB
|
|
||||||
bilibili/retained/1/goja repeat=3: CPU=125.350 ms/op, peakRSS=117.8 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=3: CPU=110.583 ms/op, peakRSS=110.8 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=4: CPU=104.953 ms/op, peakRSS=110.7 MiB
|
|
||||||
bilibili/retained/1/goja repeat=4: CPU=129.226 ms/op, peakRSS=112.7 MiB
|
|
||||||
bilibili/retained/1/goja repeat=5: CPU=117.278 ms/op, peakRSS=113.5 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=5: CPU=105.932 ms/op, peakRSS=110.6 MiB
|
|
||||||
bilibili/retained/8/goja repeat=1: CPU=135.193 ms/op, peakRSS=339.0 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=1: CPU=106.505 ms/op, peakRSS=291.5 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=2: CPU=108.446 ms/op, peakRSS=322.6 MiB
|
|
||||||
bilibili/retained/8/goja repeat=2: CPU=138.953 ms/op, peakRSS=361.2 MiB
|
|
||||||
bilibili/retained/8/goja repeat=3: CPU=137.391 ms/op, peakRSS=362.8 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=3: CPU=106.384 ms/op, peakRSS=276.2 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=4: CPU=106.178 ms/op, peakRSS=302.2 MiB
|
|
||||||
bilibili/retained/8/goja repeat=4: CPU=135.749 ms/op, peakRSS=363.1 MiB
|
|
||||||
bilibili/retained/8/goja repeat=5: CPU=136.385 ms/op, peakRSS=357.0 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=5: CPU=106.298 ms/op, peakRSS=309.8 MiB
|
|
||||||
youtube/retained/1/goja repeat=1: CPU=263.248 ms/op, peakRSS=202.4 MiB
|
|
||||||
youtube/retained/1/moejs repeat=1: CPU=249.762 ms/op, peakRSS=189.8 MiB
|
|
||||||
youtube/retained/1/moejs repeat=2: CPU=239.557 ms/op, peakRSS=194.6 MiB
|
|
||||||
youtube/retained/1/goja repeat=2: CPU=258.828 ms/op, peakRSS=204.4 MiB
|
|
||||||
youtube/retained/1/goja repeat=3: CPU=262.150 ms/op, peakRSS=203.1 MiB
|
|
||||||
youtube/retained/1/moejs repeat=3: CPU=242.040 ms/op, peakRSS=191.3 MiB
|
|
||||||
youtube/retained/1/moejs repeat=4: CPU=237.677 ms/op, peakRSS=194.4 MiB
|
|
||||||
youtube/retained/1/goja repeat=4: CPU=260.255 ms/op, peakRSS=201.9 MiB
|
|
||||||
youtube/retained/1/goja repeat=5: CPU=275.425 ms/op, peakRSS=205.3 MiB
|
|
||||||
youtube/retained/1/moejs repeat=5: CPU=254.674 ms/op, peakRSS=190.8 MiB
|
|
||||||
youtube/retained/8/goja repeat=1: CPU=293.475 ms/op, peakRSS=663.5 MiB
|
|
||||||
youtube/retained/8/moejs repeat=1: CPU=233.453 ms/op, peakRSS=699.2 MiB
|
|
||||||
youtube/retained/8/moejs repeat=2: CPU=232.512 ms/op, peakRSS=679.2 MiB
|
|
||||||
youtube/retained/8/goja repeat=2: CPU=295.749 ms/op, peakRSS=668.1 MiB
|
|
||||||
youtube/retained/8/goja repeat=3: CPU=293.476 ms/op, peakRSS=607.0 MiB
|
|
||||||
youtube/retained/8/moejs repeat=3: CPU=231.929 ms/op, peakRSS=623.0 MiB
|
|
||||||
youtube/retained/8/moejs repeat=4: CPU=233.816 ms/op, peakRSS=698.8 MiB
|
|
||||||
youtube/retained/8/goja repeat=4: CPU=284.988 ms/op, peakRSS=616.5 MiB
|
|
||||||
youtube/retained/8/goja repeat=5: CPU=299.837 ms/op, peakRSS=633.7 MiB
|
|
||||||
youtube/retained/8/moejs repeat=5: CPU=232.017 ms/op, peakRSS=623.5 MiB
|
|
||||||
twitter/retained/1/goja repeat=1: CPU=14.898 ms/op, peakRSS=75.3 MiB
|
|
||||||
twitter/retained/1/moejs repeat=1: CPU=11.448 ms/op, peakRSS=74.1 MiB
|
|
||||||
twitter/retained/1/moejs repeat=2: CPU=11.200 ms/op, peakRSS=75.0 MiB
|
|
||||||
twitter/retained/1/goja repeat=2: CPU=15.070 ms/op, peakRSS=95.0 MiB
|
|
||||||
twitter/retained/1/goja repeat=3: CPU=15.161 ms/op, peakRSS=75.4 MiB
|
|
||||||
twitter/retained/1/moejs repeat=3: CPU=11.442 ms/op, peakRSS=74.5 MiB
|
|
||||||
twitter/retained/1/moejs repeat=4: CPU=11.417 ms/op, peakRSS=74.4 MiB
|
|
||||||
twitter/retained/1/goja repeat=4: CPU=15.046 ms/op, peakRSS=75.3 MiB
|
|
||||||
twitter/retained/1/goja repeat=5: CPU=15.185 ms/op, peakRSS=75.2 MiB
|
|
||||||
twitter/retained/1/moejs repeat=5: CPU=12.036 ms/op, peakRSS=74.2 MiB
|
|
||||||
twitter/retained/8/goja repeat=1: CPU=19.321 ms/op, peakRSS=106.7 MiB
|
|
||||||
twitter/retained/8/moejs repeat=1: CPU=12.334 ms/op, peakRSS=93.3 MiB
|
|
||||||
twitter/retained/8/moejs repeat=2: CPU=12.144 ms/op, peakRSS=93.3 MiB
|
|
||||||
twitter/retained/8/goja repeat=2: CPU=19.295 ms/op, peakRSS=109.0 MiB
|
|
||||||
twitter/retained/8/goja repeat=3: CPU=18.405 ms/op, peakRSS=98.2 MiB
|
|
||||||
twitter/retained/8/moejs repeat=3: CPU=11.997 ms/op, peakRSS=93.2 MiB
|
|
||||||
twitter/retained/8/moejs repeat=4: CPU=12.301 ms/op, peakRSS=93.2 MiB
|
|
||||||
twitter/retained/8/goja repeat=4: CPU=19.434 ms/op, peakRSS=106.7 MiB
|
|
||||||
twitter/retained/8/goja repeat=5: CPU=18.415 ms/op, peakRSS=98.4 MiB
|
|
||||||
twitter/retained/8/moejs repeat=5: CPU=12.492 ms/op, peakRSS=92.9 MiB
|
|
||||||
@@ -1,120 +0,0 @@
|
|||||||
bilibili/cold/20/goja repeat=1: CPU=119.376 ms/op, peakRSS=154.7 MiB
|
|
||||||
bilibili/cold/20/moejs repeat=1: CPU=105.032 ms/op, peakRSS=171.3 MiB
|
|
||||||
bilibili/cold/20/moejs repeat=2: CPU=103.703 ms/op, peakRSS=178.6 MiB
|
|
||||||
bilibili/cold/20/goja repeat=2: CPU=118.962 ms/op, peakRSS=151.4 MiB
|
|
||||||
bilibili/cold/20/goja repeat=3: CPU=118.279 ms/op, peakRSS=143.4 MiB
|
|
||||||
bilibili/cold/20/moejs repeat=3: CPU=103.668 ms/op, peakRSS=172.3 MiB
|
|
||||||
bilibili/cold/20/moejs repeat=4: CPU=104.081 ms/op, peakRSS=174.6 MiB
|
|
||||||
bilibili/cold/20/goja repeat=4: CPU=116.987 ms/op, peakRSS=155.2 MiB
|
|
||||||
bilibili/cold/20/goja repeat=5: CPU=119.285 ms/op, peakRSS=158.0 MiB
|
|
||||||
bilibili/cold/20/moejs repeat=5: CPU=104.162 ms/op, peakRSS=167.5 MiB
|
|
||||||
bilibili/warm/200/goja repeat=1: CPU=19.869 ms/op, peakRSS=127.2 MiB
|
|
||||||
bilibili/warm/200/moejs repeat=1: CPU=3.173 ms/op, peakRSS=118.5 MiB
|
|
||||||
bilibili/warm/200/moejs repeat=2: CPU=3.166 ms/op, peakRSS=118.2 MiB
|
|
||||||
bilibili/warm/200/goja repeat=2: CPU=19.644 ms/op, peakRSS=124.0 MiB
|
|
||||||
bilibili/warm/200/goja repeat=3: CPU=19.543 ms/op, peakRSS=127.3 MiB
|
|
||||||
bilibili/warm/200/moejs repeat=3: CPU=3.171 ms/op, peakRSS=118.5 MiB
|
|
||||||
bilibili/warm/200/moejs repeat=4: CPU=3.202 ms/op, peakRSS=119.3 MiB
|
|
||||||
bilibili/warm/200/goja repeat=4: CPU=19.688 ms/op, peakRSS=124.5 MiB
|
|
||||||
bilibili/warm/200/goja repeat=5: CPU=19.852 ms/op, peakRSS=125.2 MiB
|
|
||||||
bilibili/warm/200/moejs repeat=5: CPU=3.208 ms/op, peakRSS=118.6 MiB
|
|
||||||
bilibili/retained/1/goja repeat=1: CPU=132.688 ms/op, peakRSS=121.9 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=1: CPU=108.554 ms/op, peakRSS=109.5 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=2: CPU=111.625 ms/op, peakRSS=110.8 MiB
|
|
||||||
bilibili/retained/1/goja repeat=2: CPU=134.311 ms/op, peakRSS=121.9 MiB
|
|
||||||
bilibili/retained/1/goja repeat=3: CPU=128.855 ms/op, peakRSS=114.1 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=3: CPU=109.405 ms/op, peakRSS=109.4 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=4: CPU=107.809 ms/op, peakRSS=110.5 MiB
|
|
||||||
bilibili/retained/1/goja repeat=4: CPU=118.308 ms/op, peakRSS=108.8 MiB
|
|
||||||
bilibili/retained/1/goja repeat=5: CPU=116.917 ms/op, peakRSS=108.7 MiB
|
|
||||||
bilibili/retained/1/moejs repeat=5: CPU=108.363 ms/op, peakRSS=108.9 MiB
|
|
||||||
bilibili/retained/8/goja repeat=1: CPU=140.169 ms/op, peakRSS=362.2 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=1: CPU=107.816 ms/op, peakRSS=306.9 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=2: CPU=105.929 ms/op, peakRSS=298.2 MiB
|
|
||||||
bilibili/retained/8/goja repeat=2: CPU=134.081 ms/op, peakRSS=329.5 MiB
|
|
||||||
bilibili/retained/8/goja repeat=3: CPU=133.975 ms/op, peakRSS=355.8 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=3: CPU=107.769 ms/op, peakRSS=316.6 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=4: CPU=105.431 ms/op, peakRSS=271.5 MiB
|
|
||||||
bilibili/retained/8/goja repeat=4: CPU=134.465 ms/op, peakRSS=346.5 MiB
|
|
||||||
bilibili/retained/8/goja repeat=5: CPU=142.183 ms/op, peakRSS=361.4 MiB
|
|
||||||
bilibili/retained/8/moejs repeat=5: CPU=107.668 ms/op, peakRSS=292.8 MiB
|
|
||||||
youtube/cold/20/goja repeat=1: CPU=280.323 ms/op, peakRSS=293.2 MiB
|
|
||||||
youtube/cold/20/moejs repeat=1: CPU=235.688 ms/op, peakRSS=339.6 MiB
|
|
||||||
youtube/cold/20/moejs repeat=2: CPU=239.981 ms/op, peakRSS=374.8 MiB
|
|
||||||
youtube/cold/20/goja repeat=2: CPU=278.476 ms/op, peakRSS=283.8 MiB
|
|
||||||
youtube/cold/20/goja repeat=3: CPU=269.321 ms/op, peakRSS=247.4 MiB
|
|
||||||
youtube/cold/20/moejs repeat=3: CPU=237.330 ms/op, peakRSS=353.6 MiB
|
|
||||||
youtube/cold/20/moejs repeat=4: CPU=239.973 ms/op, peakRSS=376.2 MiB
|
|
||||||
youtube/cold/20/goja repeat=4: CPU=279.683 ms/op, peakRSS=269.2 MiB
|
|
||||||
youtube/cold/20/goja repeat=5: CPU=275.033 ms/op, peakRSS=288.9 MiB
|
|
||||||
youtube/cold/20/moejs repeat=5: CPU=237.308 ms/op, peakRSS=358.9 MiB
|
|
||||||
youtube/warm/200/goja repeat=1: CPU=2.895 ms/op, peakRSS=205.4 MiB
|
|
||||||
youtube/warm/200/moejs repeat=1: CPU=0.215 ms/op, peakRSS=198.4 MiB
|
|
||||||
youtube/warm/200/moejs repeat=2: CPU=0.215 ms/op, peakRSS=193.8 MiB
|
|
||||||
youtube/warm/200/goja repeat=2: CPU=2.905 ms/op, peakRSS=203.9 MiB
|
|
||||||
youtube/warm/200/goja repeat=3: CPU=2.993 ms/op, peakRSS=204.5 MiB
|
|
||||||
youtube/warm/200/moejs repeat=3: CPU=0.216 ms/op, peakRSS=201.8 MiB
|
|
||||||
youtube/warm/200/moejs repeat=4: CPU=0.216 ms/op, peakRSS=199.2 MiB
|
|
||||||
youtube/warm/200/goja repeat=4: CPU=2.912 ms/op, peakRSS=204.5 MiB
|
|
||||||
youtube/warm/200/goja repeat=5: CPU=2.900 ms/op, peakRSS=205.4 MiB
|
|
||||||
youtube/warm/200/moejs repeat=5: CPU=0.214 ms/op, peakRSS=201.7 MiB
|
|
||||||
youtube/retained/1/goja repeat=1: CPU=266.054 ms/op, peakRSS=202.5 MiB
|
|
||||||
youtube/retained/1/moejs repeat=1: CPU=246.957 ms/op, peakRSS=187.4 MiB
|
|
||||||
youtube/retained/1/moejs repeat=2: CPU=238.809 ms/op, peakRSS=195.7 MiB
|
|
||||||
youtube/retained/1/goja repeat=2: CPU=264.583 ms/op, peakRSS=203.7 MiB
|
|
||||||
youtube/retained/1/goja repeat=3: CPU=276.460 ms/op, peakRSS=205.2 MiB
|
|
||||||
youtube/retained/1/moejs repeat=3: CPU=244.340 ms/op, peakRSS=193.2 MiB
|
|
||||||
youtube/retained/1/moejs repeat=4: CPU=242.499 ms/op, peakRSS=193.5 MiB
|
|
||||||
youtube/retained/1/goja repeat=4: CPU=266.238 ms/op, peakRSS=204.8 MiB
|
|
||||||
youtube/retained/1/goja repeat=5: CPU=271.543 ms/op, peakRSS=204.9 MiB
|
|
||||||
youtube/retained/1/moejs repeat=5: CPU=246.237 ms/op, peakRSS=189.5 MiB
|
|
||||||
youtube/retained/8/goja repeat=1: CPU=301.610 ms/op, peakRSS=617.3 MiB
|
|
||||||
youtube/retained/8/moejs repeat=1: CPU=235.625 ms/op, peakRSS=674.7 MiB
|
|
||||||
youtube/retained/8/moejs repeat=2: CPU=239.678 ms/op, peakRSS=670.2 MiB
|
|
||||||
youtube/retained/8/goja repeat=2: CPU=292.897 ms/op, peakRSS=615.8 MiB
|
|
||||||
youtube/retained/8/goja repeat=3: CPU=294.733 ms/op, peakRSS=674.6 MiB
|
|
||||||
youtube/retained/8/moejs repeat=3: CPU=233.436 ms/op, peakRSS=618.6 MiB
|
|
||||||
youtube/retained/8/moejs repeat=4: CPU=236.478 ms/op, peakRSS=679.0 MiB
|
|
||||||
youtube/retained/8/goja repeat=4: CPU=297.068 ms/op, peakRSS=668.8 MiB
|
|
||||||
youtube/retained/8/goja repeat=5: CPU=297.040 ms/op, peakRSS=676.7 MiB
|
|
||||||
youtube/retained/8/moejs repeat=5: CPU=241.736 ms/op, peakRSS=706.0 MiB
|
|
||||||
twitter/cold/20/goja repeat=1: CPU=15.356 ms/op, peakRSS=79.7 MiB
|
|
||||||
twitter/cold/20/moejs repeat=1: CPU=10.555 ms/op, peakRSS=97.0 MiB
|
|
||||||
twitter/cold/20/moejs repeat=2: CPU=10.767 ms/op, peakRSS=82.5 MiB
|
|
||||||
twitter/cold/20/goja repeat=2: CPU=15.595 ms/op, peakRSS=78.9 MiB
|
|
||||||
twitter/cold/20/goja repeat=3: CPU=15.555 ms/op, peakRSS=79.9 MiB
|
|
||||||
twitter/cold/20/moejs repeat=3: CPU=10.720 ms/op, peakRSS=97.0 MiB
|
|
||||||
twitter/cold/20/moejs repeat=4: CPU=10.594 ms/op, peakRSS=82.3 MiB
|
|
||||||
twitter/cold/20/goja repeat=4: CPU=15.477 ms/op, peakRSS=80.2 MiB
|
|
||||||
twitter/cold/20/goja repeat=5: CPU=15.898 ms/op, peakRSS=79.0 MiB
|
|
||||||
twitter/cold/20/moejs repeat=5: CPU=10.541 ms/op, peakRSS=96.8 MiB
|
|
||||||
twitter/warm/200/goja repeat=1: CPU=3.278 ms/op, peakRSS=78.7 MiB
|
|
||||||
twitter/warm/200/moejs repeat=1: CPU=0.455 ms/op, peakRSS=76.4 MiB
|
|
||||||
twitter/warm/200/moejs repeat=2: CPU=0.452 ms/op, peakRSS=96.2 MiB
|
|
||||||
twitter/warm/200/goja repeat=2: CPU=3.265 ms/op, peakRSS=78.8 MiB
|
|
||||||
twitter/warm/200/goja repeat=3: CPU=3.223 ms/op, peakRSS=97.6 MiB
|
|
||||||
twitter/warm/200/moejs repeat=3: CPU=0.453 ms/op, peakRSS=76.4 MiB
|
|
||||||
twitter/warm/200/moejs repeat=4: CPU=0.453 ms/op, peakRSS=96.1 MiB
|
|
||||||
twitter/warm/200/goja repeat=4: CPU=3.276 ms/op, peakRSS=78.8 MiB
|
|
||||||
twitter/warm/200/goja repeat=5: CPU=3.201 ms/op, peakRSS=78.7 MiB
|
|
||||||
twitter/warm/200/moejs repeat=5: CPU=0.455 ms/op, peakRSS=76.7 MiB
|
|
||||||
twitter/retained/1/goja repeat=1: CPU=16.125 ms/op, peakRSS=75.2 MiB
|
|
||||||
twitter/retained/1/moejs repeat=1: CPU=11.682 ms/op, peakRSS=74.5 MiB
|
|
||||||
twitter/retained/1/moejs repeat=2: CPU=11.636 ms/op, peakRSS=74.6 MiB
|
|
||||||
twitter/retained/1/goja repeat=2: CPU=15.417 ms/op, peakRSS=75.1 MiB
|
|
||||||
twitter/retained/1/goja repeat=3: CPU=14.978 ms/op, peakRSS=74.8 MiB
|
|
||||||
twitter/retained/1/moejs repeat=3: CPU=11.520 ms/op, peakRSS=94.1 MiB
|
|
||||||
twitter/retained/1/moejs repeat=4: CPU=11.398 ms/op, peakRSS=74.6 MiB
|
|
||||||
twitter/retained/1/goja repeat=4: CPU=15.235 ms/op, peakRSS=94.5 MiB
|
|
||||||
twitter/retained/1/goja repeat=5: CPU=15.355 ms/op, peakRSS=75.3 MiB
|
|
||||||
twitter/retained/1/moejs repeat=5: CPU=11.538 ms/op, peakRSS=94.1 MiB
|
|
||||||
twitter/retained/8/goja repeat=1: CPU=19.502 ms/op, peakRSS=106.9 MiB
|
|
||||||
twitter/retained/8/moejs repeat=1: CPU=12.182 ms/op, peakRSS=93.0 MiB
|
|
||||||
twitter/retained/8/moejs repeat=2: CPU=12.171 ms/op, peakRSS=92.9 MiB
|
|
||||||
twitter/retained/8/goja repeat=2: CPU=18.439 ms/op, peakRSS=98.1 MiB
|
|
||||||
twitter/retained/8/goja repeat=3: CPU=19.251 ms/op, peakRSS=108.3 MiB
|
|
||||||
twitter/retained/8/moejs repeat=3: CPU=12.528 ms/op, peakRSS=96.6 MiB
|
|
||||||
twitter/retained/8/moejs repeat=4: CPU=12.132 ms/op, peakRSS=93.3 MiB
|
|
||||||
twitter/retained/8/goja repeat=4: CPU=19.235 ms/op, peakRSS=108.2 MiB
|
|
||||||
twitter/retained/8/goja repeat=5: CPU=18.473 ms/op, peakRSS=98.1 MiB
|
|
||||||
twitter/retained/8/moejs repeat=5: CPU=12.258 ms/op, peakRSS=93.1 MiB
|
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -1,115 +0,0 @@
|
|||||||
=== RUN TestMoeResolveWriteback
|
|
||||||
bridge_test.go:44: Go Resource.Validate passed, name=BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4
|
|
||||||
--- PASS: TestMoeResolveWriteback (0.09s)
|
|
||||||
=== RUN TestBufferedReplay
|
|
||||||
=== RUN TestBufferedReplay/bilibili
|
|
||||||
=== RUN TestBufferedReplay/bilibili/goja
|
|
||||||
=== RUN TestBufferedReplay/bilibili/moejs
|
|
||||||
=== RUN TestBufferedReplay/youtube
|
|
||||||
=== RUN TestBufferedReplay/youtube/goja
|
|
||||||
=== RUN TestBufferedReplay/youtube/moejs
|
|
||||||
=== RUN TestBufferedReplay/twitter
|
|
||||||
=== RUN TestBufferedReplay/twitter/goja
|
|
||||||
=== RUN TestBufferedReplay/twitter/moejs
|
|
||||||
--- PASS: TestBufferedReplay (0.60s)
|
|
||||||
--- PASS: TestBufferedReplay/bilibili (0.18s)
|
|
||||||
--- PASS: TestBufferedReplay/bilibili/goja (0.09s)
|
|
||||||
--- PASS: TestBufferedReplay/bilibili/moejs (0.09s)
|
|
||||||
--- PASS: TestBufferedReplay/youtube (0.40s)
|
|
||||||
--- PASS: TestBufferedReplay/youtube/goja (0.21s)
|
|
||||||
--- PASS: TestBufferedReplay/youtube/moejs (0.19s)
|
|
||||||
--- PASS: TestBufferedReplay/twitter (0.02s)
|
|
||||||
--- PASS: TestBufferedReplay/twitter/goja (0.01s)
|
|
||||||
--- PASS: TestBufferedReplay/twitter/moejs (0.01s)
|
|
||||||
=== RUN TestExtensionReplay
|
|
||||||
=== RUN TestExtensionReplay/bilibili
|
|
||||||
=== RUN TestExtensionReplay/bilibili/goja
|
|
||||||
evaluation_test.go:46: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/2","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/3","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}}]},"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestExtensionReplay/bilibili/moejs
|
|
||||||
evaluation_test.go:46: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/2","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/3","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}}]},"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestExtensionReplay/youtube
|
|
||||||
=== RUN TestExtensionReplay/youtube/goja
|
|
||||||
evaluation_test.go:46: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged"}}}]},"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestExtensionReplay/youtube/moejs
|
|
||||||
evaluation_test.go:46: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged"}}}]},"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestExtensionReplay/twitter
|
|
||||||
=== RUN TestExtensionReplay/twitter/goja
|
|
||||||
evaluation_test.go:46: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
=== RUN TestExtensionReplay/twitter/moejs
|
|
||||||
evaluation_test.go:46: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
--- PASS: TestExtensionReplay (0.60s)
|
|
||||||
--- PASS: TestExtensionReplay/bilibili (0.17s)
|
|
||||||
--- PASS: TestExtensionReplay/bilibili/goja (0.09s)
|
|
||||||
--- PASS: TestExtensionReplay/bilibili/moejs (0.09s)
|
|
||||||
--- PASS: TestExtensionReplay/youtube (0.40s)
|
|
||||||
--- PASS: TestExtensionReplay/youtube/goja (0.21s)
|
|
||||||
--- PASS: TestExtensionReplay/youtube/moejs (0.19s)
|
|
||||||
--- PASS: TestExtensionReplay/twitter (0.02s)
|
|
||||||
--- PASS: TestExtensionReplay/twitter/goja (0.01s)
|
|
||||||
--- PASS: TestExtensionReplay/twitter/moejs (0.01s)
|
|
||||||
=== RUN TestExtensionLifecycle
|
|
||||||
=== RUN TestExtensionLifecycle/bilibili
|
|
||||||
=== RUN TestExtensionLifecycle/bilibili/goja
|
|
||||||
evaluation_test.go:110: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/5","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/7","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/9","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}}]},"continued":3,"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestExtensionLifecycle/bilibili/moejs
|
|
||||||
evaluation_test.go:110: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/5","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/7","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/9","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p","mergeCdnState":"{\"failed\":[{\"video\":\"https://upos-test.bilivideo.com\",\"audio\":\"https://upos-test.bilivideo.com\"}],\"retries\":1,\"current\":{\"video\":\"https://backup.example.com\",\"audio\":\"https://upos-test.bilivideo.com\"}}"}}}]},"continued":3,"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestExtensionLifecycle/youtube
|
|
||||||
=== RUN TestExtensionLifecycle/youtube/goja
|
|
||||||
evaluation_test.go:110: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged","retried":"1"}}}]},"continued":1,"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestExtensionLifecycle/youtube/moejs
|
|
||||||
evaluation_test.go:110: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged","retried":"1"}}}]},"continued":1,"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestExtensionLifecycle/twitter
|
|
||||||
=== RUN TestExtensionLifecycle/twitter/goja
|
|
||||||
evaluation_test.go:110: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"continued":0,"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
=== RUN TestExtensionLifecycle/twitter/moejs
|
|
||||||
evaluation_test.go:110: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"continued":0,"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
--- PASS: TestExtensionLifecycle (0.62s)
|
|
||||||
--- PASS: TestExtensionLifecycle/bilibili (0.20s)
|
|
||||||
--- PASS: TestExtensionLifecycle/bilibili/goja (0.11s)
|
|
||||||
--- PASS: TestExtensionLifecycle/bilibili/moejs (0.09s)
|
|
||||||
--- PASS: TestExtensionLifecycle/youtube (0.39s)
|
|
||||||
--- PASS: TestExtensionLifecycle/youtube/goja (0.20s)
|
|
||||||
--- PASS: TestExtensionLifecycle/youtube/moejs (0.19s)
|
|
||||||
--- PASS: TestExtensionLifecycle/twitter (0.02s)
|
|
||||||
--- PASS: TestExtensionLifecycle/twitter/goja (0.01s)
|
|
||||||
--- PASS: TestExtensionLifecycle/twitter/moejs (0.01s)
|
|
||||||
=== RUN TestFrozenExtensionReplay
|
|
||||||
=== RUN TestFrozenExtensionReplay/bilibili
|
|
||||||
frozen_test.go:13: {"res":{"name":"真实扩展 Replay 测试","range":false,"files":[{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P1).1080p.mp4","req":{"url":"blob:replay/1","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":100,"p":0,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P2).1080p.mp4","req":{"url":"blob:replay/2","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":101,"p":1,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}},{"name":"BV1BJ4m1e7g8.真实扩展 Replay 测试(P3).1080p.mp4","req":{"url":"blob:replay/3","rawUrl":"https://www.bilibili.com/video/BV1BJ4m1e7g8?p=1-3","labels":{"monkeyWie@bilibili":"1","bvid":"BV1BJ4m1e7g8","cid":102,"p":2,"type":"merged","quality":80,"selectedQuality":80,"resolution":"1080p"}}}]},"requests":["/x/web-interface/view","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl","/x/web-interface/nav","/x/player/wbi/playurl"]}
|
|
||||||
=== RUN TestFrozenExtensionReplay/youtube
|
|
||||||
frozen_test.go:13: {"res":{"range":false,"files":[{"name":"YouTube Replay 测试.mp4","req":{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","rawUrl":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","labels":{"monkeyWie@youtube":"1","input":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","quality":"1080p","fallbackToBest":"true","type":"merged"}}}]},"requests":["/sw.js_data","/youtubei/v1/config","/youtubei/v1/player"]}
|
|
||||||
=== RUN TestFrozenExtensionReplay/twitter
|
|
||||||
frozen_test.go:13: {"res":{"name":"twitter-status-1734145541599383880","files":[{"name":"twitter-status-1734145541599383880-1.jpg","req":{"url":"https://pbs.twimg.com/media/replay.jpg"}},{"name":"twitter-status-1734145541599383880-2.mp4","req":{"url":"https://video.twimg.com/1920x1080/high.mp4"}}]},"requests":["/raw/nz3ApKQM","/1.1/guest/activate.json","/raw/nz3ApKQM","/i/api/graphql/DJS3BdhUhcaEpZ7B7irJDg/TweetResultByRestId"]}
|
|
||||||
--- PASS: TestFrozenExtensionReplay (0.29s)
|
|
||||||
--- PASS: TestFrozenExtensionReplay/bilibili (0.08s)
|
|
||||||
--- PASS: TestFrozenExtensionReplay/youtube (0.19s)
|
|
||||||
--- PASS: TestFrozenExtensionReplay/twitter (0.01s)
|
|
||||||
=== RUN TestAsyncHTTP
|
|
||||||
=== RUN TestAsyncHTTP/goja
|
|
||||||
http_test.go:46: abort rejection: ReferenceError: DOMException is not defined
|
|
||||||
http_test.go:56: default abort probe: AbortError
|
|
||||||
=== RUN TestAsyncHTTP/moejs
|
|
||||||
http_test.go:46: abort rejection: Error: evaluation cancel
|
|
||||||
http_test.go:56: default abort probe: AbortError
|
|
||||||
--- PASS: TestAsyncHTTP (0.03s)
|
|
||||||
--- PASS: TestAsyncHTTP/goja (0.02s)
|
|
||||||
--- PASS: TestAsyncHTTP/moejs (0.02s)
|
|
||||||
=== RUN TestHostInterop
|
|
||||||
=== RUN TestHostInterop/goja
|
|
||||||
interop_test.go:65: Go URL=after, Go res=map[files:[map[name:video.mp4]]], JS method=function
|
|
||||||
=== RUN TestHostInterop/moejs
|
|
||||||
interop_test.go:65: Go URL=before, Go res=map[], JS method=undefined
|
|
||||||
--- PASS: TestHostInterop (0.00s)
|
|
||||||
--- PASS: TestHostInterop/goja (0.00s)
|
|
||||||
--- PASS: TestHostInterop/moejs (0.00s)
|
|
||||||
=== RUN TestFrozenCompatibility
|
|
||||||
interop_test.go:80: TypeError: Cannot modify property '__probe' of shared intrinsic [object Array]
|
|
||||||
--- PASS: TestFrozenCompatibility (0.00s)
|
|
||||||
=== RUN TestLiveResolve
|
|
||||||
live_test.go:14: set GOPEED_MOEJS_LIVE=1 to contact public endpoints
|
|
||||||
--- SKIP: TestLiveResolve (0.00s)
|
|
||||||
=== RUN TestResourceWorker
|
|
||||||
resource_worker_test.go:139: process resource measurement worker
|
|
||||||
--- SKIP: TestResourceWorker (0.00s)
|
|
||||||
PASS
|
|
||||||
ok github.com/GopeedLab/gopeed/experiments/moejs 3.094s
|
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -1,14 +0,0 @@
|
|||||||
{'scope': 'interpreter-only', 'target': 'darwin/arm64', 'engine': 'goja', 'bytes': 9287106, 'gzip_bytes': 3286296}
|
|
||||||
{'scope': 'interpreter-only', 'target': 'darwin/arm64', 'engine': 'moejs', 'bytes': 8113842, 'gzip_bytes': 3249483}
|
|
||||||
{'scope': 'interpreter-only', 'target': 'linux/amd64', 'engine': 'goja', 'bytes': 9580706, 'gzip_bytes': 3505372}
|
|
||||||
{'scope': 'interpreter-only', 'target': 'linux/amd64', 'engine': 'moejs', 'bytes': 8454306, 'gzip_bytes': 3492080}
|
|
||||||
{'scope': 'interpreter-only', 'target': 'windows/amd64', 'engine': 'goja', 'bytes': 9819136, 'gzip_bytes': 3583977}
|
|
||||||
{'scope': 'interpreter-only', 'target': 'windows/amd64', 'engine': 'moejs', 'bytes': 8700416, 'gzip_bytes': 3573843}
|
|
||||||
{'scope': 'interpreter-only', 'target': 'android/arm64', 'engine': 'goja', 'bytes': 10027304, 'gzip_bytes': 3321271}
|
|
||||||
{'scope': 'interpreter-only', 'target': 'android/arm64', 'engine': 'moejs', 'bytes': 8847656, 'gzip_bytes': 3283095}
|
|
||||||
{'scope': 'real-cli', 'target': 'darwin/arm64', 'engine': 'goja-production', 'bytes': 52037266, 'gzip_bytes': 23744742}
|
|
||||||
{'scope': 'real-cli', 'target': 'darwin/arm64', 'engine': 'goja-plus-moejs-webapi', 'bytes': 58783778, 'gzip_bytes': 26370331}
|
|
||||||
{'scope': 'real-cli', 'target': 'linux/amd64', 'engine': 'goja-production', 'bytes': 53735586, 'gzip_bytes': 24612765}
|
|
||||||
{'scope': 'real-cli', 'target': 'linux/amd64', 'engine': 'goja-plus-moejs-webapi', 'bytes': 60735650, 'gzip_bytes': 27436023}
|
|
||||||
{'scope': 'real-cli', 'target': 'windows/amd64', 'engine': 'goja-production', 'bytes': 54965248, 'gzip_bytes': 24968301}
|
|
||||||
{'scope': 'real-cli', 'target': 'windows/amd64', 'engine': 'goja-plus-moejs-webapi', 'bytes': 62106112, 'gzip_bytes': 27830298}
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
[
|
|
||||||
{
|
|
||||||
"scope": "interpreter-only",
|
|
||||||
"target": "darwin/arm64",
|
|
||||||
"engine": "goja",
|
|
||||||
"bytes": 9287106,
|
|
||||||
"gzip_bytes": 3286296
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "interpreter-only",
|
|
||||||
"target": "darwin/arm64",
|
|
||||||
"engine": "moejs",
|
|
||||||
"bytes": 8113842,
|
|
||||||
"gzip_bytes": 3249483
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "interpreter-only",
|
|
||||||
"target": "linux/amd64",
|
|
||||||
"engine": "goja",
|
|
||||||
"bytes": 9580706,
|
|
||||||
"gzip_bytes": 3505372
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "interpreter-only",
|
|
||||||
"target": "linux/amd64",
|
|
||||||
"engine": "moejs",
|
|
||||||
"bytes": 8454306,
|
|
||||||
"gzip_bytes": 3492080
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "interpreter-only",
|
|
||||||
"target": "windows/amd64",
|
|
||||||
"engine": "goja",
|
|
||||||
"bytes": 9819136,
|
|
||||||
"gzip_bytes": 3583977
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "interpreter-only",
|
|
||||||
"target": "windows/amd64",
|
|
||||||
"engine": "moejs",
|
|
||||||
"bytes": 8700416,
|
|
||||||
"gzip_bytes": 3573843
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "interpreter-only",
|
|
||||||
"target": "android/arm64",
|
|
||||||
"engine": "goja",
|
|
||||||
"bytes": 10027304,
|
|
||||||
"gzip_bytes": 3321271
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "interpreter-only",
|
|
||||||
"target": "android/arm64",
|
|
||||||
"engine": "moejs",
|
|
||||||
"bytes": 8847656,
|
|
||||||
"gzip_bytes": 3283095
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "real-cli",
|
|
||||||
"target": "darwin/arm64",
|
|
||||||
"engine": "goja-production",
|
|
||||||
"bytes": 52037266,
|
|
||||||
"gzip_bytes": 23744742
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "real-cli",
|
|
||||||
"target": "darwin/arm64",
|
|
||||||
"engine": "goja-plus-moejs-webapi",
|
|
||||||
"bytes": 58783778,
|
|
||||||
"gzip_bytes": 26370331
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "real-cli",
|
|
||||||
"target": "linux/amd64",
|
|
||||||
"engine": "goja-production",
|
|
||||||
"bytes": 53735586,
|
|
||||||
"gzip_bytes": 24612765
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "real-cli",
|
|
||||||
"target": "linux/amd64",
|
|
||||||
"engine": "goja-plus-moejs-webapi",
|
|
||||||
"bytes": 60735650,
|
|
||||||
"gzip_bytes": 27436023
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "real-cli",
|
|
||||||
"target": "windows/amd64",
|
|
||||||
"engine": "goja-production",
|
|
||||||
"bytes": 54965248,
|
|
||||||
"gzip_bytes": 24968301
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scope": "real-cli",
|
|
||||||
"target": "windows/amd64",
|
|
||||||
"engine": "goja-plus-moejs-webapi",
|
|
||||||
"bytes": 62106112,
|
|
||||||
"gzip_bytes": 27830298
|
|
||||||
}
|
|
||||||
]
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
{
|
|
||||||
"bilibili": {
|
|
||||||
"repo": "monkeyWie/gopeed-extension-bilibili",
|
|
||||||
"commit": "b91db1ea52e975d8de9823f01a7af5f9ea83792f",
|
|
||||||
"bytes": 920556,
|
|
||||||
"sha256": "47d78e7596dfac5524bfaa7c24ff802350432c4cc22ec2e5047380212afed1e0"
|
|
||||||
},
|
|
||||||
"youtube": {
|
|
||||||
"repo": "monkeyWie/gopeed-extension-youtube",
|
|
||||||
"commit": "eeb78d168e053e9a2aad1dbaa309fc11f6992f7c",
|
|
||||||
"bytes": 3409614,
|
|
||||||
"sha256": "d0716f768216ea6764d474115db15303b842a6e39eec3bde75db2b6175bd6aec"
|
|
||||||
},
|
|
||||||
"twitter": {
|
|
||||||
"repo": "monkeyWie/gopeed-extension-twitter",
|
|
||||||
"commit": "a963428bbfdbee9510bf224b00b384e37514ce77",
|
|
||||||
"bytes": 66243,
|
|
||||||
"sha256": "19cfc174be730fbd6129be3877cde46de00da40deb20684b8279af593a75bc86"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
go version go1.26.0 darwin/arm64
|
|
||||||
ProductName: macOS
|
|
||||||
ProductVersion: 14.6.1
|
|
||||||
BuildVersion: 23G93
|
|
||||||
8589934592
|
|
||||||
@@ -1,142 +0,0 @@
|
|||||||
// Experimental adapter only; this is not a production engine switch.
|
|
||||||
package evaluation
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
production "github.com/GopeedLab/gopeed/pkg/download/engine"
|
|
||||||
"github.com/MoeForge/moejs"
|
|
||||||
"github.com/MoeForge/moejs/eventloop"
|
|
||||||
"github.com/MoeForge/moejs/webapi"
|
|
||||||
)
|
|
||||||
|
|
||||||
type Runtime interface {
|
|
||||||
Eval(string) (any, error)
|
|
||||||
Close()
|
|
||||||
}
|
|
||||||
|
|
||||||
type Goja struct{ *production.Engine }
|
|
||||||
|
|
||||||
func (g *Goja) Eval(s string) (any, error) { return g.RunString(s) }
|
|
||||||
|
|
||||||
type Moe struct {
|
|
||||||
rt *moejs.Runtime
|
|
||||||
loop *eventloop.Loop
|
|
||||||
api *webapi.API
|
|
||||||
}
|
|
||||||
|
|
||||||
func readHostScript(path string) (string, error) {
|
|
||||||
b, err := os.ReadFile(filepath.Join("..", "..", "pkg", "download", "engine", path))
|
|
||||||
return string(b), err
|
|
||||||
}
|
|
||||||
|
|
||||||
func New(name string) (Runtime, error) { return newRuntime(name, false) }
|
|
||||||
|
|
||||||
func newRuntime(name string, network bool) (Runtime, error) {
|
|
||||||
if name == "goja" {
|
|
||||||
return &Goja{production.NewEngine(nil)}, nil
|
|
||||||
}
|
|
||||||
if name != "moejs" && name != "moejs-frozen" {
|
|
||||||
return nil, fmt.Errorf("unknown engine %s", name)
|
|
||||||
}
|
|
||||||
// Preserve goja's mutable-builtin contract; shared builtins are a separate probe.
|
|
||||||
rt := moejs.NewRuntime(moejs.Options{MutableIntrinsics: name != "moejs-frozen", MaxDynamicSource: -1})
|
|
||||||
loop, err := eventloop.New(rt, eventloop.Options{})
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
opts := webapi.Options{}
|
|
||||||
if network {
|
|
||||||
opts.Client = &http.Client{Timeout: 20 * time.Second}
|
|
||||||
}
|
|
||||||
api, err := webapi.Install(loop, opts)
|
|
||||||
if err != nil {
|
|
||||||
loop.Terminate()
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
m := &Moe{rt, loop, api}
|
|
||||||
if err = loop.Start(); err != nil {
|
|
||||||
m.Close()
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
scripts := []string{"globalThis.global = globalThis; globalThis.window = globalThis; globalThis.location = new URL('http://localhost');"}
|
|
||||||
for _, path := range []string{"polyfill/out/index.js", "inject/xhr/xhr.js"} {
|
|
||||||
s, e := readHostScript(path)
|
|
||||||
if e != nil {
|
|
||||||
m.Close()
|
|
||||||
return nil, e
|
|
||||||
}
|
|
||||||
scripts = append(scripts, s)
|
|
||||||
}
|
|
||||||
for _, s := range scripts {
|
|
||||||
if _, e := m.Eval(s); e != nil {
|
|
||||||
m.Close()
|
|
||||||
return nil, e
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return m, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (m *Moe) Close() { m.api.Close(); m.loop.Terminate() }
|
|
||||||
func (m *Moe) Eval(source string) (any, error) {
|
|
||||||
// Compile each evaluation, matching production Engine.RunString.
|
|
||||||
script, err := moejs.CompileScript("evaluation.js", source)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
type result struct {
|
|
||||||
value any
|
|
||||||
err error
|
|
||||||
}
|
|
||||||
ch := make(chan result, 1)
|
|
||||||
ok := m.loop.RunOnLoop(func(rt *moejs.Runtime) {
|
|
||||||
v, e := rt.RunScript(script)
|
|
||||||
if e != nil {
|
|
||||||
ch <- result{err: e}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if _, _, promise := moejs.PromiseResult(v); promise {
|
|
||||||
then, e := rt.Get(v, "then")
|
|
||||||
if e != nil {
|
|
||||||
ch <- result{err: e}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
fulfilled := rt.Function("fulfilled", 1, func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
|
|
||||||
out, e := rt.ToGo(moejs.Arg(args, 0))
|
|
||||||
ch <- result{out, e}
|
|
||||||
return moejs.Undefined(), nil
|
|
||||||
})
|
|
||||||
rejected := rt.Function("rejected", 1, func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
|
|
||||||
v := moejs.Arg(args, 0)
|
|
||||||
stack, e := rt.Get(v, "stack")
|
|
||||||
message := v.String()
|
|
||||||
if e == nil && !stack.IsUndefined() {
|
|
||||||
message = stack.String()
|
|
||||||
}
|
|
||||||
ch <- result{err: fmt.Errorf("%s", message)}
|
|
||||||
return moejs.Undefined(), nil
|
|
||||||
})
|
|
||||||
_, e = rt.CallFunction(then, v, fulfilled, rejected)
|
|
||||||
if e != nil {
|
|
||||||
ch <- result{err: e}
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
out, e := rt.ToGo(v)
|
|
||||||
ch <- result{out, e}
|
|
||||||
})
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("loop terminated")
|
|
||||||
}
|
|
||||||
select {
|
|
||||||
case r := <-ch:
|
|
||||||
return r.value, r.err
|
|
||||||
case <-time.After(30 * time.Second):
|
|
||||||
m.rt.Interrupt("evaluation timeout")
|
|
||||||
return nil, fmt.Errorf("evaluation timed out")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Summarize saved Go benchmark samples; times are medians, not best runs."""
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import re
|
|
||||||
import statistics
|
|
||||||
|
|
||||||
root = Path(__file__).resolve().parent
|
|
||||||
samples = {}
|
|
||||||
for line in (root/'results/bench.txt').read_text().splitlines():
|
|
||||||
match = re.match(r'(Benchmark\S+)\s+\d+\s+([\d.]+) ns/op\s+([\d.]+) B/op\s+([\d.]+) allocs/op',line)
|
|
||||||
if match:
|
|
||||||
key, *values = match.groups()
|
|
||||||
samples.setdefault(key, []).append(list(map(float,values)))
|
|
||||||
summary = {}
|
|
||||||
for key, rows in samples.items():
|
|
||||||
summary[key] = dict(samples=len(rows),ns_median=statistics.median(r[0] for r in rows),ns_min=min(r[0] for r in rows),ns_max=max(r[0] for r in rows),bytes_median=statistics.median(r[1] for r in rows),allocs_median=statistics.median(r[2] for r in rows))
|
|
||||||
(root/'results/bench-summary.json').write_text(json.dumps(summary,indent=2)+'\n')
|
|
||||||
for key, row in summary.items():
|
|
||||||
print(f'{key}: {row["ns_median"]/1e6:.3f} ms/op; {row["bytes_median"]/1024:.1f} KiB/op; samples={row["samples"]}')
|
|
||||||
@@ -0,0 +1,259 @@
|
|||||||
|
package download
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/GopeedLab/gopeed/internal/fetcher"
|
||||||
|
"github.com/GopeedLab/gopeed/pkg/base"
|
||||||
|
)
|
||||||
|
|
||||||
|
type deleteResolveManager struct {
|
||||||
|
*generationTestManager
|
||||||
|
returned chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *deleteResolveManager) Build() fetcher.Fetcher {
|
||||||
|
return &deleteResolveFetcher{generationTestFetcher: m.generationTestManager.Build().(*generationTestFetcher), returned: m.returned}
|
||||||
|
}
|
||||||
|
|
||||||
|
type deleteResolveFetcher struct {
|
||||||
|
*generationTestFetcher
|
||||||
|
returned chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *deleteResolveFetcher) Resolve(req *base.Request) error {
|
||||||
|
defer close(f.returned)
|
||||||
|
return f.generationTestFetcher.Resolve(req)
|
||||||
|
}
|
||||||
|
func waitDeleteTest(t *testing.T, ch <-chan struct{}) {
|
||||||
|
t.Helper()
|
||||||
|
select {
|
||||||
|
case <-ch:
|
||||||
|
case <-time.After(3 * time.Second):
|
||||||
|
t.Fatal("operation did not finish")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestDeleteDuringResolveCannotRestartOrPersist(t *testing.T) {
|
||||||
|
for _, force := range []bool{false, true} {
|
||||||
|
for _, resolveErr := range []error{nil, errors.New("late resolve error")} {
|
||||||
|
t.Run(fmt.Sprintf("force=%v/error=%v", force, resolveErr != nil), func(t *testing.T) {
|
||||||
|
release := make(chan struct{})
|
||||||
|
var once sync.Once
|
||||||
|
defer once.Do(func() { close(release) })
|
||||||
|
m := &deleteResolveManager{generationTestManager: &generationTestManager{resolveStarted: make(chan struct{}), resolveRelease: release, resolveErr: resolveErr}, returned: make(chan struct{})}
|
||||||
|
d := NewDownloader(&DownloaderConfig{Storage: NewMemStorage(), StorageDir: t.TempDir(), TempDir: t.TempDir(), FetchManagers: []fetcher.FetcherManager{m}})
|
||||||
|
if err := d.Setup(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer d.Close()
|
||||||
|
id, err := d.CreateDirect(&base.Request{URL: "generation://delete"}, &base.Options{Path: t.TempDir(), Name: "delete.bin"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
waitDeleteTest(t, m.resolveStarted)
|
||||||
|
task := d.GetTask(id)
|
||||||
|
if err := d.Delete(&TaskFilter{IDs: []string{id}}, force); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
once.Do(func() { close(release) })
|
||||||
|
waitDeleteTest(t, m.returned)
|
||||||
|
task.lock.Lock()
|
||||||
|
task.lock.Unlock()
|
||||||
|
// A stale error callback can run immediately after the startup handler exits.
|
||||||
|
d.doOnError(task, errors.New("late callback"))
|
||||||
|
if got := m.starts.Load(); got != 0 {
|
||||||
|
t.Errorf("deleted task started %d time(s)", got)
|
||||||
|
}
|
||||||
|
if d.GetTask(id) != nil {
|
||||||
|
t.Error("deleted task is still listed")
|
||||||
|
}
|
||||||
|
for _, bucket := range []string{bucketTask, bucketSave, bucketTaskStats} {
|
||||||
|
var value any
|
||||||
|
present, err := d.storage.Get(bucket, id, &value)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if present {
|
||||||
|
t.Errorf("deleted task resurrected in %s", bucket)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type blockedTaskStorage struct {
|
||||||
|
Storage
|
||||||
|
entered chan struct{}
|
||||||
|
release chan struct{}
|
||||||
|
blockID string
|
||||||
|
once sync.Once
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *blockedTaskStorage) Put(bucket, key string, value any) error {
|
||||||
|
if bucket == bucketTask && key == s.blockID {
|
||||||
|
s.once.Do(func() { close(s.entered); <-s.release })
|
||||||
|
}
|
||||||
|
return s.Storage.Put(bucket, key, value)
|
||||||
|
}
|
||||||
|
func TestDeleteSerializesInFlightTaskCheckpoint(t *testing.T) {
|
||||||
|
s := &blockedTaskStorage{Storage: NewMemStorage(), entered: make(chan struct{}), release: make(chan struct{})}
|
||||||
|
d := NewDownloader(&DownloaderConfig{Storage: s, StorageDir: t.TempDir(), TempDir: t.TempDir(), FetchManagers: []fetcher.FetcherManager{&generationTestManager{}}})
|
||||||
|
if err := d.Setup(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer d.Close()
|
||||||
|
d.cfg.MaxRunning = 0
|
||||||
|
id, err := d.CreateDirect(&base.Request{URL: "generation://checkpoint"}, &base.Options{Path: t.TempDir()})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task := d.GetTask(id)
|
||||||
|
s.blockID = id
|
||||||
|
saved := make(chan struct{})
|
||||||
|
go func() { defer close(saved); _ = d.saveTask(task) }()
|
||||||
|
waitDeleteTest(t, s.entered)
|
||||||
|
deleted := make(chan struct{})
|
||||||
|
go func() { defer close(deleted); _ = d.Delete(&TaskFilter{IDs: []string{id}}, false) }()
|
||||||
|
deadline := time.Now().Add(3 * time.Second)
|
||||||
|
for d.GetTask(id) != nil && time.Now().Before(deadline) {
|
||||||
|
time.Sleep(time.Millisecond)
|
||||||
|
}
|
||||||
|
close(s.release)
|
||||||
|
waitDeleteTest(t, saved)
|
||||||
|
waitDeleteTest(t, deleted)
|
||||||
|
var value any
|
||||||
|
present, err := s.Get(bucketTask, id, &value)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if present {
|
||||||
|
t.Fatal("in-flight checkpoint restored a deleted task")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteDuringPreparationCannotStartOrPersist(t *testing.T) {
|
||||||
|
s := &blockedTaskStorage{Storage: NewMemStorage(), entered: make(chan struct{}), release: make(chan struct{})}
|
||||||
|
m := &generationTestManager{}
|
||||||
|
d := NewDownloader(&DownloaderConfig{Storage: s, StorageDir: t.TempDir(), TempDir: t.TempDir(), FetchManagers: []fetcher.FetcherManager{m}})
|
||||||
|
if err := d.Setup(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer d.Close()
|
||||||
|
d.cfg.MaxRunning = 0
|
||||||
|
id, err := d.CreateDirect(&base.Request{URL: "generation://preparation"}, &base.Options{Path: t.TempDir()})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task := d.GetTask(id)
|
||||||
|
s.blockID = id
|
||||||
|
d.cfg.MaxRunning = 1
|
||||||
|
if err := d.Continue(&TaskFilter{IDs: []string{id}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
waitDeleteTest(t, s.entered)
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); _ = d.Delete(&TaskFilter{IDs: []string{id}}, true) }()
|
||||||
|
deadline := time.Now().Add(2 * time.Second)
|
||||||
|
for d.GetTask(id) != nil && time.Now().Before(deadline) {
|
||||||
|
time.Sleep(time.Millisecond)
|
||||||
|
}
|
||||||
|
close(s.release)
|
||||||
|
waitDeleteTest(t, done)
|
||||||
|
task.lock.Lock()
|
||||||
|
task.lock.Unlock()
|
||||||
|
if m.starts.Load() != 0 {
|
||||||
|
t.Fatal("deleted preparation started downloading")
|
||||||
|
}
|
||||||
|
var saved Task
|
||||||
|
if exists, err := s.Get(bucketTask, id, &saved); err != nil || exists {
|
||||||
|
t.Fatalf("deleted preparation persisted: exists=%v err=%v", exists, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestDeletedTaskRejectsLateSnapshotsAndCompletion(t *testing.T) {
|
||||||
|
d := NewDownloader(&DownloaderConfig{Storage: NewMemStorage(), StorageDir: t.TempDir(), TempDir: t.TempDir(), FetchManagers: []fetcher.FetcherManager{&generationTestManager{}}})
|
||||||
|
if err := d.Setup(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer d.Close()
|
||||||
|
d.cfg.MaxRunning = 0
|
||||||
|
id, err := d.CreateDirect(&base.Request{URL: "generation://late"}, &base.Options{Path: t.TempDir()})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task := d.GetTask(id)
|
||||||
|
snapshot := task.clone()
|
||||||
|
task.fetcher.(*generationTestFetcher).stats = &fetcher.Stats{Snapshot: map[string]any{"bytes": 1}}
|
||||||
|
if err := d.Delete(&TaskFilter{IDs: []string{id}}, false); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := d.putTaskRecord(task, snapshot); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := d.captureTaskStats(task, false, true); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := d.saveTask(task); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if d.markTaskDone(task) {
|
||||||
|
t.Error("deleted task accepted a late completion")
|
||||||
|
}
|
||||||
|
for _, bucket := range []string{bucketTask, bucketSave, bucketTaskStats} {
|
||||||
|
var value any
|
||||||
|
if exists, err := d.storage.Get(bucket, id, &value); err != nil || exists {
|
||||||
|
t.Errorf("late callback restored %s: exists=%v err=%v", bucket, exists, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type singleFailureTaskDeleteStorage struct {
|
||||||
|
Storage
|
||||||
|
fail atomic.Bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *singleFailureTaskDeleteStorage) Delete(bucket, key string) error {
|
||||||
|
if bucket == bucketTask && s.fail.Swap(false) {
|
||||||
|
return errors.New("task deletion failed")
|
||||||
|
}
|
||||||
|
return s.Storage.Delete(bucket, key)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFailedDeleteCanBeRetriedWithoutRevivingOldRun(t *testing.T) {
|
||||||
|
s := &singleFailureTaskDeleteStorage{Storage: NewMemStorage()}
|
||||||
|
m := &generationTestManager{holdOpen: true}
|
||||||
|
d := NewDownloader(&DownloaderConfig{Storage: s, StorageDir: t.TempDir(), TempDir: t.TempDir(), FetchManagers: []fetcher.FetcherManager{m}})
|
||||||
|
if err := d.Setup(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer d.Close()
|
||||||
|
d.cfg.MaxRunning = 0
|
||||||
|
id, err := d.CreateDirect(&base.Request{URL: "generation://failed-delete"}, &base.Options{Path: t.TempDir()})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.fail.Store(true)
|
||||||
|
if err := d.Delete(&TaskFilter{IDs: []string{id}}, false); err == nil {
|
||||||
|
t.Fatal("expected storage deletion failure")
|
||||||
|
}
|
||||||
|
task := d.GetTask(id)
|
||||||
|
if task == nil || d.taskStatus(task) != base.DownloadStatusWait {
|
||||||
|
t.Fatal("failed deletion did not restore queued task")
|
||||||
|
}
|
||||||
|
d.cfg.MaxRunning = 1
|
||||||
|
if err := d.Continue(&TaskFilter{IDs: []string{id}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
deadline := time.Now().Add(time.Second)
|
||||||
|
for m.starts.Load() < 1 && time.Now().Before(deadline) {
|
||||||
|
time.Sleep(time.Millisecond)
|
||||||
|
}
|
||||||
|
if m.starts.Load() != 1 {
|
||||||
|
t.Fatal("failed deletion could not be retried")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,253 @@
|
|||||||
|
package download
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/GopeedLab/gopeed/internal/fetcher"
|
||||||
|
"github.com/GopeedLab/gopeed/pkg/base"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestFailedDeleteDuringResolveCanContinue(t *testing.T) {
|
||||||
|
release := make(chan struct{})
|
||||||
|
var once sync.Once
|
||||||
|
defer once.Do(func() { close(release) })
|
||||||
|
m := &deleteResolveManager{generationTestManager: &generationTestManager{
|
||||||
|
resolveStarted: make(chan struct{}), resolveRelease: release, holdOpen: true,
|
||||||
|
}, returned: make(chan struct{})}
|
||||||
|
s := &singleFailureTaskDeleteStorage{Storage: NewMemStorage()}
|
||||||
|
d := NewDownloader(&DownloaderConfig{Storage: s, StorageDir: t.TempDir(), TempDir: t.TempDir(), FetchManagers: []fetcher.FetcherManager{m}})
|
||||||
|
if err := d.Setup(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer d.Close()
|
||||||
|
id, err := d.CreateDirect(&base.Request{URL: "generation://failed-delete-resolve"}, &base.Options{Path: t.TempDir()})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
waitDeleteTest(t, m.resolveStarted)
|
||||||
|
task := d.GetTask(id)
|
||||||
|
s.fail.Store(true)
|
||||||
|
if err := d.Delete(&TaskFilter{IDs: []string{id}}, false); err == nil {
|
||||||
|
t.Fatal("expected storage deletion failure")
|
||||||
|
}
|
||||||
|
once.Do(func() { close(release) })
|
||||||
|
waitDeleteTest(t, m.returned)
|
||||||
|
task.lock.Lock()
|
||||||
|
task.lock.Unlock()
|
||||||
|
if m.starts.Load() != 0 {
|
||||||
|
t.Fatal("invalidated startup must not reach Start")
|
||||||
|
}
|
||||||
|
if err := d.Continue(&TaskFilter{IDs: []string{id}}); err != nil {
|
||||||
|
t.Fatalf("failed deletion left task in %s, cannot continue: %v", d.taskStatus(task), err)
|
||||||
|
}
|
||||||
|
deadline := time.Now().Add(3 * time.Second)
|
||||||
|
for m.starts.Load() == 0 && time.Now().Before(deadline) {
|
||||||
|
time.Sleep(time.Millisecond)
|
||||||
|
}
|
||||||
|
if m.starts.Load() != 1 {
|
||||||
|
t.Fatal("continued task did not start exactly once")
|
||||||
|
}
|
||||||
|
task.lock.Lock()
|
||||||
|
task.lock.Unlock()
|
||||||
|
}
|
||||||
|
|
||||||
|
type deleteRecoveryStorage struct {
|
||||||
|
Storage
|
||||||
|
bucket string
|
||||||
|
fail atomic.Bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *deleteRecoveryStorage) Delete(bucket, key string) error {
|
||||||
|
if bucket == s.bucket && s.fail.Swap(false) {
|
||||||
|
return errors.New("injected deletion failure")
|
||||||
|
}
|
||||||
|
return s.Storage.Delete(bucket, key)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unlike the generation fixture, this fetcher stops its watcher on Pause and
|
||||||
|
// cannot be started again after Close, as with a dropped BT torrent.
|
||||||
|
type deleteRecoveryManager struct {
|
||||||
|
generationTestManager
|
||||||
|
failClose atomic.Bool
|
||||||
|
restored atomic.Int64
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *deleteRecoveryManager) Build() fetcher.Fetcher {
|
||||||
|
return m.wrap(m.generationTestManager.Build().(*generationTestFetcher))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *deleteRecoveryManager) wrap(f *generationTestFetcher) *deleteRecoveryFetcher {
|
||||||
|
return &deleteRecoveryFetcher{generationTestFetcher: f, manager: m, waiting: make(chan struct{})}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *deleteRecoveryManager) Store(fetcher.Fetcher) (any, error) {
|
||||||
|
return &recoveryTestData{SeedBytes: 42}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *deleteRecoveryManager) Restore() (any, func(*fetcher.FetcherMeta, any) fetcher.Fetcher) {
|
||||||
|
return &recoveryTestData{}, func(meta *fetcher.FetcherMeta, value any) fetcher.Fetcher {
|
||||||
|
m.restored.Store(value.(*recoveryTestData).SeedBytes)
|
||||||
|
return m.wrap(&generationTestFetcher{manager: &m.generationTestManager, meta: meta})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type deleteRecoveryFetcher struct {
|
||||||
|
*generationTestFetcher
|
||||||
|
manager *deleteRecoveryManager
|
||||||
|
closed atomic.Bool
|
||||||
|
running atomic.Bool
|
||||||
|
waiting chan struct{}
|
||||||
|
once sync.Once
|
||||||
|
waitRelease <-chan struct{}
|
||||||
|
waitSuccess bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *deleteRecoveryFetcher) Start() error {
|
||||||
|
if f.closed.Load() {
|
||||||
|
return errors.New("cannot restart a closed fetcher")
|
||||||
|
}
|
||||||
|
f.running.Store(true)
|
||||||
|
return f.generationTestFetcher.Start()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *deleteRecoveryFetcher) Pause() error {
|
||||||
|
if f.running.Swap(false) {
|
||||||
|
f.done <- errors.New("paused")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *deleteRecoveryFetcher) Close() error {
|
||||||
|
f.closed.Store(true)
|
||||||
|
_ = f.Pause()
|
||||||
|
if f.manager.failClose.Swap(false) {
|
||||||
|
return errors.New("close failed after stopping the fetcher")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *deleteRecoveryFetcher) Wait() error {
|
||||||
|
f.once.Do(func() { close(f.waiting) })
|
||||||
|
err := f.generationTestFetcher.Wait()
|
||||||
|
if f.waitRelease != nil {
|
||||||
|
<-f.waitRelease
|
||||||
|
if f.waitSuccess {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFailedDeleteRunningTaskCanResumeAndComplete(t *testing.T) {
|
||||||
|
for _, stage := range []string{bucketTask, bucketTaskStats, bucketSave, "close", "files", "late-close", "late-close-error"} {
|
||||||
|
t.Run(stage, func(t *testing.T) {
|
||||||
|
s := &deleteRecoveryStorage{Storage: NewMemStorage(), bucket: stage}
|
||||||
|
lateClose := stage == "late-close" || stage == "late-close-error"
|
||||||
|
if lateClose {
|
||||||
|
s.bucket = bucketSave
|
||||||
|
}
|
||||||
|
m := &deleteRecoveryManager{generationTestManager: generationTestManager{holdOpen: true}}
|
||||||
|
d := NewDownloader(&DownloaderConfig{Storage: s, StorageDir: t.TempDir(), TempDir: t.TempDir(), FetchManagers: []fetcher.FetcherManager{m}})
|
||||||
|
if err := d.Setup(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer d.Close()
|
||||||
|
id, err := d.CreateDirect(&base.Request{URL: "generation://recover-running"}, &base.Options{Path: t.TempDir()})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task := d.GetTask(id)
|
||||||
|
task.lock.Lock()
|
||||||
|
f := task.fetcher.(*deleteRecoveryFetcher)
|
||||||
|
task.lock.Unlock()
|
||||||
|
waitDeleteTest(t, f.waiting)
|
||||||
|
watcher, _ := d.watchedTasks.Load(id)
|
||||||
|
var release chan struct{}
|
||||||
|
var releaseOnce sync.Once
|
||||||
|
if lateClose {
|
||||||
|
release = make(chan struct{})
|
||||||
|
defer releaseOnce.Do(func() { close(release) })
|
||||||
|
f.waitRelease = release
|
||||||
|
f.waitSuccess = stage == "late-close"
|
||||||
|
}
|
||||||
|
var blockedPath string
|
||||||
|
if stage == "files" {
|
||||||
|
blockedPath = task.Meta.SingleFilepath()
|
||||||
|
if err := os.MkdirAll(blockedPath, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(blockedPath, "keep"), []byte("nonempty"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
s.fail.Store(true)
|
||||||
|
m.failClose.Store(stage == "close")
|
||||||
|
if err := d.Delete(&TaskFilter{IDs: []string{id}}, stage == "files"); err == nil {
|
||||||
|
t.Fatal("expected deletion failure")
|
||||||
|
}
|
||||||
|
if f.running.Load() || d.taskStatus(task) != base.DownloadStatusPause {
|
||||||
|
t.Fatal("failed deletion must restore a paused task and stop transmission")
|
||||||
|
}
|
||||||
|
if !lateClose {
|
||||||
|
waitDeleteTest(t, watcher.(*taskWatcher).done)
|
||||||
|
}
|
||||||
|
var saved Task
|
||||||
|
if exists, err := s.Get(bucketTask, id, &saved); err != nil || !exists || saved.Status != base.DownloadStatusPause {
|
||||||
|
t.Fatalf("failed deletion did not persist paused recovery: exists=%v err=%v status=%s", exists, err, saved.Status)
|
||||||
|
}
|
||||||
|
var checkpoint recoveryTestData
|
||||||
|
exists, err := s.Get(bucketSave, id, &checkpoint)
|
||||||
|
if stage == "files" {
|
||||||
|
if err != nil || exists {
|
||||||
|
t.Fatalf("file cleanup must not leave a checkpoint for removed bytes: exists=%v err=%v", exists, err)
|
||||||
|
}
|
||||||
|
} else if err != nil || !exists || checkpoint.SeedBytes != 42 {
|
||||||
|
t.Fatalf("failed deletion lost checkpoint: exists=%v err=%v checkpoint=%+v", exists, err, checkpoint)
|
||||||
|
}
|
||||||
|
if blockedPath != "" {
|
||||||
|
if err := os.RemoveAll(blockedPath); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := d.Continue(&TaskFilter{IDs: []string{id}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task.lock.Lock()
|
||||||
|
resumed := task.fetcher.(*deleteRecoveryFetcher)
|
||||||
|
task.lock.Unlock()
|
||||||
|
waitDeleteTest(t, resumed.waiting)
|
||||||
|
if stage == bucketSave || stage == "close" || stage == "files" || lateClose {
|
||||||
|
wantSeed := int64(42)
|
||||||
|
if stage == "files" {
|
||||||
|
wantSeed = 0
|
||||||
|
}
|
||||||
|
if resumed == f || m.restored.Load() != wantSeed {
|
||||||
|
t.Fatal("closed fetcher was not rebuilt from the retained checkpoint")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if lateClose {
|
||||||
|
releaseOnce.Do(func() { close(release) })
|
||||||
|
waitDeleteTest(t, watcher.(*taskWatcher).done)
|
||||||
|
if d.taskStatus(task) != base.DownloadStatusRunning {
|
||||||
|
t.Fatal("old completion changed the replacement run's status")
|
||||||
|
}
|
||||||
|
current, exists := d.watchedTasks.Load(id)
|
||||||
|
if !exists || current.(*taskWatcher).fetcher != resumed {
|
||||||
|
t.Fatal("old completion removed the replacement watcher")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
resumed.running.Store(false)
|
||||||
|
resumed.done <- nil
|
||||||
|
waitForTaskStatus(t, d, id, base.DownloadStatusDone, 3*time.Second)
|
||||||
|
if m.starts.Load() != 2 {
|
||||||
|
t.Fatal("resuming should start one new transfer")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
+183
-42
@@ -515,7 +515,36 @@ func (d *Downloader) setupFetcher(fm fetcher.FetcherManager, fetcher fetcher.Fet
|
|||||||
fetcher.Setup(ctl)
|
fetcher.Setup(ctl)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// putTaskValue serializes late progress/completion callbacks with deletion.
|
||||||
|
// A snapshot may have been captured before Delete removed the task.
|
||||||
|
func (d *Downloader) putTaskValue(task *Task, bucket string, value any) error {
|
||||||
|
if task.lifetime != nil {
|
||||||
|
task.lifetime.mu.Lock()
|
||||||
|
defer task.lifetime.mu.Unlock()
|
||||||
|
if task.lifetime.deleted.Load() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return d.storage.Put(bucket, task.ID, value)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *Downloader) putTaskRecord(task *Task, snapshot *Task) error {
|
||||||
|
return d.putTaskValue(task, bucketTask, snapshot)
|
||||||
|
}
|
||||||
|
|
||||||
func (d *Downloader) saveTask(task *Task) error {
|
func (d *Downloader) saveTask(task *Task) error {
|
||||||
|
if task.lifetime != nil {
|
||||||
|
task.lifetime.mu.Lock()
|
||||||
|
defer task.lifetime.mu.Unlock()
|
||||||
|
if task.lifetime.deleted.Load() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return d.saveTaskLocked(task)
|
||||||
|
}
|
||||||
|
|
||||||
|
// saveTaskLocked requires the lifetime mutex when the task has one.
|
||||||
|
func (d *Downloader) saveTaskLocked(task *Task) error {
|
||||||
data, err := task.fetcherManager.Store(task.fetcher)
|
data, err := task.fetcherManager.Store(task.fetcher)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
d.Logger.Error().Stack().Err(err).Msgf("serialize fetcher failed: %s", task.ID)
|
d.Logger.Error().Stack().Err(err).Msgf("serialize fetcher failed: %s", task.ID)
|
||||||
@@ -1227,7 +1256,7 @@ func (d *Downloader) Delete(filter *TaskFilter, force bool) error {
|
|||||||
for _, task := range tasks {
|
for _, task := range tasks {
|
||||||
if err := d.doDelete(task, force); err != nil {
|
if err := d.doDelete(task, force); err != nil {
|
||||||
// Deletion can fail after removing bucketTask. Keep recovery durable.
|
// Deletion can fail after removing bucketTask. Keep recovery durable.
|
||||||
if saveErr := d.storage.Put(bucketTask, task.ID, task.clone()); saveErr != nil {
|
if saveErr := d.putTaskRecord(task, task.clone()); saveErr != nil {
|
||||||
d.Logger.Error().Err(saveErr).Msg("restore failed task deletion failed")
|
d.Logger.Error().Err(saveErr).Msg("restore failed task deletion failed")
|
||||||
if firstErr == nil {
|
if firstErr == nil {
|
||||||
firstErr = saveErr
|
firstErr = saveErr
|
||||||
@@ -1419,7 +1448,7 @@ func (d *Downloader) captureTaskStats(task *Task, live, force bool) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
if err := d.storage.Put(bucketTaskStats, task.ID, &storedTaskStats{Snapshot: snapshot}); err != nil {
|
if err := d.putTaskValue(task, bucketTaskStats, &storedTaskStats{Snapshot: snapshot}); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
task.statsSnapshot = append(task.statsSnapshot[:0], snapshot...)
|
task.statsSnapshot = append(task.statsSnapshot[:0], snapshot...)
|
||||||
@@ -1525,23 +1554,55 @@ func (d *Downloader) RuntimeStatus(id string) (*TaskRuntimeStatus, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (d *Downloader) doDelete(task *Task, force bool) (err error) {
|
func (d *Downloader) doDelete(task *Task, force bool) (err error) {
|
||||||
|
task.lifetime.deleted.Store(true)
|
||||||
|
task.statusLock.Lock()
|
||||||
|
task.runGeneration++
|
||||||
|
if task.Status == base.DownloadStatusRunning {
|
||||||
|
// Close/Pause can wake the completion watcher. Make it observe a
|
||||||
|
// stopped run before touching the fetcher, including on deletion errors.
|
||||||
|
task.updateStatus(base.DownloadStatusPause)
|
||||||
|
task.Progress.Speed = 0
|
||||||
|
}
|
||||||
|
task.statusLock.Unlock()
|
||||||
|
task.lifetime.mu.Lock()
|
||||||
|
defer task.lifetime.mu.Unlock()
|
||||||
defer d.releaseBlobTask(task)
|
defer d.releaseBlobTask(task)
|
||||||
|
defer func() {
|
||||||
|
if err != nil {
|
||||||
|
// The startup generation stays invalid. Stop any physical transfer
|
||||||
|
// before restoring a state that Continue is allowed to restart.
|
||||||
|
var pauseErr error
|
||||||
|
if task.fetcher != nil {
|
||||||
|
pauseErr = task.fetcher.Pause()
|
||||||
|
}
|
||||||
|
task.statusLock.Lock()
|
||||||
|
if pauseErr != nil {
|
||||||
|
task.updateStatus(base.DownloadStatusError)
|
||||||
|
err = errors.Join(err, pauseErr)
|
||||||
|
}
|
||||||
|
task.statusLock.Unlock()
|
||||||
|
task.lifetime.deleted.Store(false)
|
||||||
|
}
|
||||||
|
}()
|
||||||
err = func() error {
|
err = func() error {
|
||||||
if err := d.storage.Delete(bucketTask, task.ID); err != nil {
|
if err := d.storage.Delete(bucketTask, task.ID); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if err := d.storage.Delete(bucketSave, task.ID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := d.storage.Delete(bucketTaskStats, task.ID); err != nil {
|
if err := d.storage.Delete(bucketTaskStats, task.ID); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
if task.fetcher != nil {
|
if task.fetcher != nil {
|
||||||
|
task.lifetime.restoreFetcher.Store(true)
|
||||||
if err := task.fetcher.Close(); err != nil {
|
if err := task.fetcher.Close(); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Retain recovery state through Close, which can make the fetcher
|
||||||
|
// unusable even on failure. Clear it before removing any file bytes.
|
||||||
|
if err := d.storage.Delete(bucketSave, task.ID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if force && task.Meta.Res != nil {
|
if force && task.Meta.Res != nil {
|
||||||
if task.Meta.Res.Name != "" {
|
if task.Meta.Res.Name != "" {
|
||||||
if err := os.RemoveAll(task.Meta.FolderPath()); err != nil {
|
if err := os.RemoveAll(task.Meta.FolderPath()); err != nil {
|
||||||
@@ -1861,18 +1922,34 @@ func (d *Downloader) getProtocolConfig(name string, v any) bool {
|
|||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
// wait task done
|
type taskWatcher struct {
|
||||||
func (d *Downloader) watch(task *Task) {
|
fetcher fetcher.Fetcher
|
||||||
if _, loaded := d.watchedTasks.LoadOrStore(task.ID, true); loaded {
|
done chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A replacement fetcher needs its own watcher even if Close has not yet
|
||||||
|
// returned the previous fetcher's final notification.
|
||||||
|
func (d *Downloader) watch(task *Task, f fetcher.Fetcher) {
|
||||||
|
task.lock.Lock()
|
||||||
|
if task.fetcher != f {
|
||||||
|
task.lock.Unlock()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if existing, loaded := d.watchedTasks.Load(task.ID); loaded && existing.(*taskWatcher).fetcher == f {
|
||||||
|
task.lock.Unlock()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
watcher := &taskWatcher{fetcher: f, done: make(chan struct{})}
|
||||||
|
d.watchedTasks.Store(task.ID, watcher)
|
||||||
|
task.lock.Unlock()
|
||||||
defer func() {
|
defer func() {
|
||||||
d.watchedTasks.Delete(task.ID)
|
d.watchedTasks.CompareAndDelete(task.ID, watcher)
|
||||||
|
close(watcher.done)
|
||||||
}()
|
}()
|
||||||
|
|
||||||
// wait task upload done
|
// wait task upload done
|
||||||
if task.Uploading {
|
if task.Uploading {
|
||||||
if uploader, ok := task.fetcher.(fetcher.Uploader); ok {
|
if uploader, ok := f.(fetcher.Uploader); ok {
|
||||||
go func() {
|
go func() {
|
||||||
err := uploader.WaitUpload()
|
err := uploader.WaitUpload()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -1880,9 +1957,19 @@ func (d *Downloader) watch(task *Task) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Check if the task is deleted
|
// Check if the task is deleted
|
||||||
if d.GetTask(task.ID) != nil {
|
listed := d.GetTask(task.ID) != nil
|
||||||
|
task.lock.Lock()
|
||||||
|
task.statusLock.Lock()
|
||||||
|
current := task.fetcher == f && listed
|
||||||
|
var snapshot *Task
|
||||||
|
if current {
|
||||||
task.Uploading = false
|
task.Uploading = false
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
snapshot = task.clone()
|
||||||
|
}
|
||||||
|
task.statusLock.Unlock()
|
||||||
|
task.lock.Unlock()
|
||||||
|
if current {
|
||||||
|
d.putTaskRecord(task, snapshot)
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
}
|
}
|
||||||
@@ -1893,7 +1980,13 @@ func (d *Downloader) watch(task *Task) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
for {
|
for {
|
||||||
err := task.fetcher.Wait()
|
task.lock.Lock()
|
||||||
|
current := task.fetcher == f
|
||||||
|
task.lock.Unlock()
|
||||||
|
if !current {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
err := f.Wait()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
status := d.taskStatus(task)
|
status := d.taskStatus(task)
|
||||||
if status == base.DownloadStatusPause || status == base.DownloadStatusWait {
|
if status == base.DownloadStatusPause || status == base.DownloadStatusWait {
|
||||||
@@ -1904,9 +1997,9 @@ func (d *Downloader) watch(task *Task) {
|
|||||||
sourceErr = d.blob.SourceError(task.Meta.Req.URL)
|
sourceErr = d.blob.SourceError(task.Meta.Req.URL)
|
||||||
}
|
}
|
||||||
if sourceErr != nil {
|
if sourceErr != nil {
|
||||||
d.doOnBlobSourceError(task, sourceErr)
|
d.handleOnErrorForFetcher(task, sourceErr, true, f)
|
||||||
} else {
|
} else {
|
||||||
d.doOnError(task, err)
|
d.handleOnErrorForFetcher(task, err, false, f)
|
||||||
}
|
}
|
||||||
if d.taskStatus(task) == base.DownloadStatusRunning {
|
if d.taskStatus(task) == base.DownloadStatusRunning {
|
||||||
continue
|
continue
|
||||||
@@ -1915,12 +2008,12 @@ func (d *Downloader) watch(task *Task) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// When delete a not resolved task, need check if the task resource is nil
|
// When delete a not resolved task, need check if the task resource is nil
|
||||||
if task.Meta.Res == nil || d.GetTask(task.ID) == nil {
|
if d.GetTask(task.ID) == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if d.blob != nil && task.Meta.Req != nil {
|
if d.blob != nil && task.Meta.Req != nil {
|
||||||
if err := d.blob.SourceError(task.Meta.Req.URL); err != nil {
|
if err := d.blob.SourceError(task.Meta.Req.URL); err != nil {
|
||||||
d.doOnBlobSourceError(task, err)
|
d.handleOnErrorForFetcher(task, err, true, f)
|
||||||
if d.taskStatus(task) == base.DownloadStatusRunning {
|
if d.taskStatus(task) == base.DownloadStatusRunning {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -1928,7 +2021,7 @@ func (d *Downloader) watch(task *Task) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if !d.markTaskDone(task) {
|
if !d.markTaskDoneForFetcher(task, f) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if err := d.captureTaskStats(task, false, true); err != nil {
|
if err := d.captureTaskStats(task, false, true); err != nil {
|
||||||
@@ -1937,7 +2030,7 @@ func (d *Downloader) watch(task *Task) {
|
|||||||
task.statusLock.Lock()
|
task.statusLock.Lock()
|
||||||
snapshot := task.clone()
|
snapshot := task.clone()
|
||||||
task.statusLock.Unlock()
|
task.statusLock.Unlock()
|
||||||
d.storage.Put(bucketTask, task.ID, snapshot)
|
d.putTaskRecord(task, snapshot)
|
||||||
d.emit(EventKeyDone, task)
|
d.emit(EventKeyDone, task)
|
||||||
d.emit(EventKeyFinally, task, err)
|
d.emit(EventKeyFinally, task, err)
|
||||||
d.notifyRunning()
|
d.notifyRunning()
|
||||||
@@ -2042,10 +2135,18 @@ func (d *Downloader) doOnBlobSourceError(task *Task, err error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (d *Downloader) handleOnError(task *Task, err error, resetFetcher bool) {
|
func (d *Downloader) handleOnError(task *Task, err error, resetFetcher bool) {
|
||||||
|
d.handleOnErrorForFetcher(task, err, resetFetcher, nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *Downloader) handleOnErrorForFetcher(task *Task, err error, resetFetcher bool, expected fetcher.Fetcher) {
|
||||||
if task == nil || task.lock == nil {
|
if task == nil || task.lock == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
task.lock.Lock()
|
task.lock.Lock()
|
||||||
|
if expected != nil && task.fetcher != expected {
|
||||||
|
task.lock.Unlock()
|
||||||
|
return
|
||||||
|
}
|
||||||
if !d.markTaskError(task) {
|
if !d.markTaskError(task) {
|
||||||
task.lock.Unlock()
|
task.lock.Unlock()
|
||||||
return
|
return
|
||||||
@@ -2080,7 +2181,7 @@ func (d *Downloader) handleOnError(task *Task, err error, resetFetcher bool) {
|
|||||||
d.Logger.Warn().Err(statsErr).Msgf("persist failed task stats failed, task id: %s", task.ID)
|
d.Logger.Warn().Err(statsErr).Msgf("persist failed task stats failed, task id: %s", task.ID)
|
||||||
}
|
}
|
||||||
if d.taskStatus(task) == base.DownloadStatusError {
|
if d.taskStatus(task) == base.DownloadStatusError {
|
||||||
if persistErr := d.storage.Put(bucketTask, task.ID, task.clone()); persistErr != nil {
|
if persistErr := d.putTaskRecord(task, task.clone()); persistErr != nil {
|
||||||
d.Logger.Error().Stack().Err(persistErr).Msgf("persist failed task failed, task id: %s", task.ID)
|
d.Logger.Error().Stack().Err(persistErr).Msgf("persist failed task failed, task id: %s", task.ID)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -2134,10 +2235,11 @@ func (d *Downloader) resetTaskFetcher(task *Task) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (d *Downloader) restoreTask(task *Task) error {
|
func (d *Downloader) restoreTask(task *Task) error {
|
||||||
if task.fetcher == nil {
|
if task.fetcher == nil || task.lifetime.restoreFetcher.Load() {
|
||||||
if err := d.restoreFetcher(task); err != nil {
|
if err := d.restoreFetcher(task); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
task.lifetime.restoreFetcher.Store(false)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -2145,10 +2247,16 @@ func (d *Downloader) restoreTask(task *Task) error {
|
|||||||
func (d *Downloader) restoreFetcher(task *Task) error {
|
func (d *Downloader) restoreFetcher(task *Task) error {
|
||||||
v, f := task.fetcherManager.Restore()
|
v, f := task.fetcherManager.Restore()
|
||||||
if v != nil {
|
if v != nil {
|
||||||
_, err := d.storage.Get(bucketSave, task.ID, v)
|
present, err := d.storage.Get(bucketSave, task.ID, v)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
if !present && task.lifetime.restoreFetcher.Load() {
|
||||||
|
// File cleanup can fail after the checkpoint was removed. Resolve
|
||||||
|
// afresh rather than starting a closed protocol with empty state.
|
||||||
|
task.Meta.Res = nil
|
||||||
|
task.Progress.Downloaded = 0
|
||||||
|
}
|
||||||
}
|
}
|
||||||
task.fetcher = f(task.Meta, v)
|
task.fetcher = f(task.Meta, v)
|
||||||
if task.fetcher == nil {
|
if task.fetcher == nil {
|
||||||
@@ -2208,7 +2316,7 @@ func (d *Downloader) doCreate(f fetcher.Fetcher, opts *base.Options, groupID str
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if err = d.storage.Put(bucketTask, task.ID, task.clone()); err != nil {
|
if err = d.putTaskRecord(task, task.clone()); err != nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
taskId = task.ID
|
taskId = task.ID
|
||||||
@@ -2362,7 +2470,17 @@ func (d *Downloader) prepareTask(task *Task) error {
|
|||||||
task.statusLock.Unlock()
|
task.statusLock.Unlock()
|
||||||
preparedTask.Meta = &preparedMeta
|
preparedTask.Meta = &preparedMeta
|
||||||
preparedTask.NeedsPreparation = false
|
preparedTask.NeedsPreparation = false
|
||||||
if err := d.saveTask(&preparedTask); err != nil {
|
// The candidate save and publication are one operation with respect to
|
||||||
|
// deletion, which reads the live metadata to decide which files to remove.
|
||||||
|
task.lifetime.mu.Lock()
|
||||||
|
defer task.lifetime.mu.Unlock()
|
||||||
|
task.statusLock.Lock()
|
||||||
|
current := !task.lifetime.deleted.Load() && task.runGeneration == preparedTask.runGeneration
|
||||||
|
task.statusLock.Unlock()
|
||||||
|
if !current {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := d.saveTaskLocked(&preparedTask); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
task.statusLock.Lock()
|
task.statusLock.Lock()
|
||||||
@@ -2375,7 +2493,7 @@ func (d *Downloader) prepareTask(task *Task) error {
|
|||||||
func (d *Downloader) doStart(task *Task) (err error) {
|
func (d *Downloader) doStart(task *Task) (err error) {
|
||||||
var generation uint64
|
var generation uint64
|
||||||
isReturn, err := d.statusMut(task, func() (isReturn bool, err error) {
|
isReturn, err := d.statusMut(task, func() (isReturn bool, err error) {
|
||||||
if task.Status == base.DownloadStatusRunning || task.Status == base.DownloadStatusDone {
|
if task.lifetime.deleted.Load() || task.Status == base.DownloadStatusRunning || task.Status == base.DownloadStatusDone {
|
||||||
isReturn = true
|
isReturn = true
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -2400,6 +2518,7 @@ func (d *Downloader) doStart(task *Task) (err error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
started := false
|
started := false
|
||||||
|
var startedFetcher fetcher.Fetcher
|
||||||
handler := func() error {
|
handler := func() error {
|
||||||
task.lock.Lock()
|
task.lock.Lock()
|
||||||
defer task.lock.Unlock()
|
defer task.lock.Unlock()
|
||||||
@@ -2432,6 +2551,9 @@ func (d *Downloader) doStart(task *Task) (err error) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
if !d.taskIsRunningGeneration(task, generation) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
task.Meta.Res = task.fetcher.Meta().Res
|
task.Meta.Res = task.fetcher.Meta().Res
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -2456,8 +2578,14 @@ func (d *Downloader) doStart(task *Task) (err error) {
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
task.lifetime.mu.Lock()
|
||||||
|
if task.lifetime.deleted.Load() {
|
||||||
|
task.lifetime.mu.Unlock()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
task.statusLock.Lock()
|
task.statusLock.Lock()
|
||||||
if task.Status != base.DownloadStatusRunning || task.runGeneration != generation {
|
if task.Status != base.DownloadStatusRunning || task.runGeneration != generation {
|
||||||
|
task.lifetime.mu.Unlock()
|
||||||
task.statusLock.Unlock()
|
task.statusLock.Unlock()
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -2466,10 +2594,13 @@ func (d *Downloader) doStart(task *Task) (err error) {
|
|||||||
task.downloadSampledAt = time.Time{}
|
task.downloadSampledAt = time.Time{}
|
||||||
task.timer.Start()
|
task.timer.Start()
|
||||||
task.statusLock.Unlock()
|
task.statusLock.Unlock()
|
||||||
if err := task.fetcher.Start(); err != nil {
|
startErr := task.fetcher.Start()
|
||||||
return err
|
task.lifetime.mu.Unlock()
|
||||||
|
if startErr != nil {
|
||||||
|
return startErr
|
||||||
}
|
}
|
||||||
started = true
|
started = true
|
||||||
|
startedFetcher = task.fetcher
|
||||||
if err := d.captureTaskStats(task, true, true); err != nil {
|
if err := d.captureTaskStats(task, true, true); err != nil {
|
||||||
// Statistics are auxiliary. A storage or serialization failure must not
|
// Statistics are auxiliary. A storage or serialization failure must not
|
||||||
// turn an already-started download into a failed task.
|
// turn an already-started download into a failed task.
|
||||||
@@ -2491,7 +2622,7 @@ func (d *Downloader) doStart(task *Task) (err error) {
|
|||||||
if d.taskIsRunningGeneration(task, generation) {
|
if d.taskIsRunningGeneration(task, generation) {
|
||||||
d.emit(EventKeyStart, task)
|
d.emit(EventKeyStart, task)
|
||||||
}
|
}
|
||||||
go d.watch(task)
|
go d.watch(task, startedFetcher)
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
|
|
||||||
@@ -2578,7 +2709,7 @@ func (d *Downloader) runPauseHandler(task *Task, generation uint64, pauseFetcher
|
|||||||
return false, err
|
return false, err
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
if err := d.storage.Put(bucketTask, task.ID, task.clone()); err != nil {
|
if err := d.putTaskRecord(task, task.clone()); err != nil {
|
||||||
return false, err
|
return false, err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -2591,7 +2722,7 @@ func (d *Downloader) taskIsRunningGeneration(task *Task, generation uint64) bool
|
|||||||
}
|
}
|
||||||
task.statusLock.Lock()
|
task.statusLock.Lock()
|
||||||
defer task.statusLock.Unlock()
|
defer task.statusLock.Unlock()
|
||||||
return task.Status == base.DownloadStatusRunning && task.runGeneration == generation
|
return !task.lifetime.deleted.Load() && task.Status == base.DownloadStatusRunning && task.runGeneration == generation
|
||||||
}
|
}
|
||||||
|
|
||||||
func (d *Downloader) taskIsPausedGeneration(task *Task, generation uint64) bool {
|
func (d *Downloader) taskIsPausedGeneration(task *Task, generation uint64) bool {
|
||||||
@@ -2600,7 +2731,7 @@ func (d *Downloader) taskIsPausedGeneration(task *Task, generation uint64) bool
|
|||||||
}
|
}
|
||||||
task.statusLock.Lock()
|
task.statusLock.Lock()
|
||||||
defer task.statusLock.Unlock()
|
defer task.statusLock.Unlock()
|
||||||
return (task.Status == base.DownloadStatusPause || task.Status == base.DownloadStatusWait) && task.runGeneration == generation
|
return !task.lifetime.deleted.Load() && (task.Status == base.DownloadStatusPause || task.Status == base.DownloadStatusWait) && task.runGeneration == generation
|
||||||
}
|
}
|
||||||
|
|
||||||
func (d *Downloader) taskStatus(task *Task) base.Status {
|
func (d *Downloader) taskStatus(task *Task) base.Status {
|
||||||
@@ -2613,14 +2744,21 @@ func (d *Downloader) taskStatus(task *Task) base.Status {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (d *Downloader) markTaskDone(task *Task) bool {
|
func (d *Downloader) markTaskDone(task *Task) bool {
|
||||||
|
return d.markTaskDoneForFetcher(task, nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *Downloader) markTaskDoneForFetcher(task *Task, expected fetcher.Fetcher) bool {
|
||||||
if task == nil || task.statusLock == nil {
|
if task == nil || task.statusLock == nil {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
task.lock.Lock()
|
task.lock.Lock()
|
||||||
defer task.lock.Unlock()
|
defer task.lock.Unlock()
|
||||||
|
if expected != nil && task.fetcher != expected {
|
||||||
|
return false
|
||||||
|
}
|
||||||
task.statusLock.Lock()
|
task.statusLock.Lock()
|
||||||
defer task.statusLock.Unlock()
|
defer task.statusLock.Unlock()
|
||||||
if task.Status != base.DownloadStatusRunning {
|
if task.lifetime.deleted.Load() || task.Status != base.DownloadStatusRunning || task.Meta.Res == nil {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
task.Progress.Used = task.timer.Used()
|
task.Progress.Used = task.timer.Used()
|
||||||
@@ -2648,7 +2786,7 @@ func (d *Downloader) markTaskError(task *Task) bool {
|
|||||||
}
|
}
|
||||||
task.statusLock.Lock()
|
task.statusLock.Lock()
|
||||||
defer task.statusLock.Unlock()
|
defer task.statusLock.Unlock()
|
||||||
if task.Status != base.DownloadStatusRunning {
|
if task.lifetime.deleted.Load() || task.Status != base.DownloadStatusRunning {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
task.updateStatus(base.DownloadStatusError)
|
task.updateStatus(base.DownloadStatusError)
|
||||||
@@ -2710,7 +2848,7 @@ func (d *Downloader) enqueueSingleExtraction(task *Task, downloadFilePath string
|
|||||||
// Set extraction status to queued
|
// Set extraction status to queued
|
||||||
task.Progress.ExtractStatus = ExtractStatusQueued
|
task.Progress.ExtractStatus = ExtractStatusQueued
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
d.Logger.Info().Msgf("extraction queued, task id: %s, job id: %s", task.ID, jobID)
|
d.Logger.Info().Msgf("extraction queued, task id: %s, job id: %s", task.ID, jobID)
|
||||||
|
|
||||||
// Create and enqueue the extraction job
|
// Create and enqueue the extraction job
|
||||||
@@ -2739,7 +2877,7 @@ func (d *Downloader) enqueueMultiPartExtraction(task *Task, downloadFilePath str
|
|||||||
// Not all parts are ready yet - just set status to waiting, don't queue anything
|
// Not all parts are ready yet - just set status to waiting, don't queue anything
|
||||||
task.Progress.ExtractStatus = ExtractStatusWaitingParts
|
task.Progress.ExtractStatus = ExtractStatusWaitingParts
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
d.Logger.Info().Msgf("multi-part archive waiting for other parts, task id: %s, missing: %v", task.ID, missingParts)
|
d.Logger.Info().Msgf("multi-part archive waiting for other parts, task id: %s, missing: %v", task.ID, missingParts)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -2755,14 +2893,14 @@ func (d *Downloader) enqueueMultiPartExtraction(task *Task, downloadFilePath str
|
|||||||
task.Progress.ExtractStatus = ExtractStatusDone
|
task.Progress.ExtractStatus = ExtractStatusDone
|
||||||
task.Progress.ExtractProgress = 100
|
task.Progress.ExtractProgress = 100
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
d.Logger.Info().Msgf("multi-part archive extraction already handled by another part, task id: %s", task.ID)
|
d.Logger.Info().Msgf("multi-part archive extraction already handled by another part, task id: %s", task.ID)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// This task claimed the extraction - status already set to queued in tryClaimMultiPartExtraction
|
// This task claimed the extraction - status already set to queued in tryClaimMultiPartExtraction
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
|
|
||||||
jobID := "multipart:" + fullBaseName
|
jobID := "multipart:" + fullBaseName
|
||||||
d.Logger.Info().Msgf("multi-part extraction queued, task id: %s, job id: %s", task.ID, jobID)
|
d.Logger.Info().Msgf("multi-part extraction queued, task id: %s, job id: %s", task.ID, jobID)
|
||||||
@@ -2851,7 +2989,7 @@ func (d *Downloader) performExtraction(task *Task, archivePath string, destDir s
|
|||||||
task.Progress.ExtractStatus = ExtractStatusExtracting
|
task.Progress.ExtractStatus = ExtractStatusExtracting
|
||||||
task.Progress.ExtractProgress = 0
|
task.Progress.ExtractProgress = 0
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
|
|
||||||
// Extract the archive
|
// Extract the archive
|
||||||
extractErr := extractArchive(archivePath, destDir, opts.ArchivePassword, func(extractedFiles int, totalFiles int, progress int) {
|
extractErr := extractArchive(archivePath, destDir, opts.ArchivePassword, func(extractedFiles int, totalFiles int, progress int) {
|
||||||
@@ -2871,7 +3009,7 @@ func (d *Downloader) performMultiPartExtraction(task *Task, firstPartPath string
|
|||||||
task.Progress.ExtractStatus = ExtractStatusExtracting
|
task.Progress.ExtractStatus = ExtractStatusExtracting
|
||||||
task.Progress.ExtractProgress = 0
|
task.Progress.ExtractProgress = 0
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
|
|
||||||
d.Logger.Info().Msgf("starting multi-part archive extraction, first part: %s, task id: %s", firstPartPath, task.ID)
|
d.Logger.Info().Msgf("starting multi-part archive extraction, first part: %s, task id: %s", firstPartPath, task.ID)
|
||||||
|
|
||||||
@@ -3015,13 +3153,13 @@ func (d *Downloader) handleExtractionResult(task *Task, extractErr error, archiv
|
|||||||
d.Logger.Error().Err(extractErr).Msgf("auto extract archive failed, task id: %s", task.ID)
|
d.Logger.Error().Err(extractErr).Msgf("auto extract archive failed, task id: %s", task.ID)
|
||||||
task.Progress.ExtractStatus = ExtractStatusError
|
task.Progress.ExtractStatus = ExtractStatusError
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
} else {
|
} else {
|
||||||
d.Logger.Info().Msgf("auto extract archive completed, task id: %s", task.ID)
|
d.Logger.Info().Msgf("auto extract archive completed, task id: %s", task.ID)
|
||||||
task.Progress.ExtractStatus = ExtractStatusDone
|
task.Progress.ExtractStatus = ExtractStatusDone
|
||||||
task.Progress.ExtractProgress = 100
|
task.Progress.ExtractProgress = 100
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
|
|
||||||
// Delete archive files after successful extraction if enabled
|
// Delete archive files after successful extraction if enabled
|
||||||
if deleteAfterExtract {
|
if deleteAfterExtract {
|
||||||
@@ -3061,12 +3199,15 @@ func (d *Downloader) updateMultiPartTasksStatus(sourceTask *Task, extractErr err
|
|||||||
task.Progress.ExtractStatus = status
|
task.Progress.ExtractStatus = status
|
||||||
task.Progress.ExtractProgress = progress
|
task.Progress.ExtractProgress = progress
|
||||||
d.emit(EventKeyProgress, task)
|
d.emit(EventKeyProgress, task)
|
||||||
d.storage.Put(bucketTask, task.ID, task.clone())
|
d.putTaskRecord(task, task.clone())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func initTask(task *Task) {
|
func initTask(task *Task) {
|
||||||
|
if task.lifetime == nil {
|
||||||
|
task.lifetime = &taskLifetime{}
|
||||||
|
}
|
||||||
task.timer = util.NewTimer(task.Progress.Used)
|
task.timer = util.NewTimer(task.Progress.Used)
|
||||||
|
|
||||||
task.statusLock = &sync.Mutex{}
|
task.statusLock = &sync.Mutex{}
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package download
|
|||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"sync"
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/GopeedLab/gopeed/internal/controller"
|
"github.com/GopeedLab/gopeed/internal/controller"
|
||||||
@@ -43,6 +44,16 @@ type CreateTaskBatchResult struct {
|
|||||||
ParentID string `json:"parentId,omitempty"`
|
ParentID string `json:"parentId,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// taskLifetime is shared with preparation candidates. Its mutex serializes
|
||||||
|
// physical starts and persistence with deletion; the tombstone also invalidates
|
||||||
|
// work that is currently outside that mutex (for example Resolve).
|
||||||
|
type taskLifetime struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
deleted atomic.Bool
|
||||||
|
// Close can make a fetcher unusable even when a later deletion step fails.
|
||||||
|
restoreFetcher atomic.Bool
|
||||||
|
}
|
||||||
|
|
||||||
// Task represents a download unit.
|
// Task represents a download unit.
|
||||||
// Options flow path:
|
// Options flow path:
|
||||||
// REST Request (model.CreateTask/ResolveTask) -> pkg/download Downloader -> Task.Meta.Opts (base.Options) -> Fetcher.Resolve/Start (e.g. internal/protocol/http/fetcher.go)
|
// REST Request (model.CreateTask/ResolveTask) -> pkg/download Downloader -> Task.Meta.Opts (base.Options) -> Fetcher.Resolve/Start (e.g. internal/protocol/http/fetcher.go)
|
||||||
@@ -87,6 +98,7 @@ type Task struct {
|
|||||||
blobRefLock *sync.Mutex
|
blobRefLock *sync.Mutex
|
||||||
blobURL string
|
blobURL string
|
||||||
runGeneration uint64
|
runGeneration uint64
|
||||||
|
lifetime *taskLifetime
|
||||||
speedArr []speedSample
|
speedArr []speedSample
|
||||||
uploadSpeedArr []speedSample
|
uploadSpeedArr []speedSample
|
||||||
downloadSampledAt time.Time
|
downloadSampledAt time.Time
|
||||||
@@ -126,6 +138,7 @@ func NewTask() *Task {
|
|||||||
}
|
}
|
||||||
return &Task{
|
return &Task{
|
||||||
ID: id,
|
ID: id,
|
||||||
|
lifetime: &taskLifetime{},
|
||||||
Status: base.DownloadStatusReady,
|
Status: base.DownloadStatusReady,
|
||||||
CreatedAt: time.Now(),
|
CreatedAt: time.Now(),
|
||||||
UpdatedAt: time.Now(),
|
UpdatedAt: time.Now(),
|
||||||
|
|||||||
Reference in new issue
Block a user