Compare commits

...
Author SHA1 Message Date
Levi e4b4062e81 fix(bt): upgrade torrent to v1.62.0 for WebTorrent announces 2026-11-11 09:36:43 +08:00
Lykin 392ee36137 fix(web): wait for the saved analytics preference (#1656) 2026-11-10 15:00:15 +08:00
Konstantin OstrovskiiandLevi a34383a7e2 fix(http): wait for the prefetch goroutine to exit before reusing its file (#1606)
* fix(http): wait for the prefetch goroutine to exit before reusing its file

stopPrefetchAndCopyData waited at most 10 s for the Resolve prefetch, but a
stalled body read only times out after readTimeout (15 s). The caller then
closed and removed the temporary file while the prefetch goroutine could
still write to it.

Stopping the prefetch now closes the Resolve body, which interrupts a
pending read, and waits on a channel closed when the goroutine exits. Pause
uses the same path. The stop channel is closed under resolveRespLock so a
concurrent Pause and first-connection takeover cannot close it twice.

The proxy test helper now calls Setup before Resolve, as the downloader
does, so the Resolve request also goes through the proxy.

Fixes #1563

* refactor(http): wait on prefetchExitCh in completeFromResolveFallback

Replaces the 10 ms polling loop with the exit channel added in the
previous commit, so every place that waits for the prefetch uses it.

---------

Co-authored-by: Levi <[email protected]>
2026-11-09 11:14:49 +08:00
Silas MariuszandLevi 98879569f0 chore(deps): update modules with known vulnerabilities (#1612)
govulncheck (-tags nosqlite,web ./cmd/web/... and ./cmd/api/...) reports
40 reachable vulnerabilities in 10 modules on main. This update fixes 34
of them and keeps the go directive at 1.25.4, the version CI uses.

- github.com/go-git/go-git/v5 v5.8.1 -> v5.19.2: GO-2024-2456,
  GO-2024-2466, GO-2025-3367, GO-2025-3368, GO-2026-4473, GO-2026-4909,
  GO-2026-4910, GO-2026-5074, GO-2026-5105, GO-2026-5336, GO-2026-5496,
  GO-2026-5693, GO-2026-6213, GO-2026-6214
- github.com/go-git/go-billy/v5 v5.4.1 -> v5.9.2: GO-2026-5490,
  GO-2026-5597
- golang.org/x/crypto v0.48.0 -> v0.55.0: GO-2026-5013, GO-2026-5015,
  GO-2026-5017, GO-2026-5018, GO-2026-5019, GO-2026-5020, GO-2026-5021
- golang.org/x/net v0.50.0 -> v0.58.0: GO-2026-4559, GO-2026-5026,
  GO-2026-5942
- golang.org/x/text v0.34.0 -> v0.41.0: GO-2026-5970
- github.com/imroc/req/v3 v3.52.2 -> v3.58.0, which brings
  github.com/quic-go/quic-go v0.51.0 -> v0.59.0 (GO-2025-4017) and
  github.com/refraction-networking/utls v1.6.7 -> v1.8.2 (GO-2025-3638,
  GO-2026-4509, GO-2026-4512). req v3.52.2 does not build against a
  quic-go with the fix.
- github.com/pion/dtls/v3 v3.0.10 -> v3.1.4: GO-2026-6165
- github.com/pion/stun/v3 v3.1.1 -> v3.1.5: GO-2026-6163
- github.com/cloudflare/circl v1.6.1 -> v1.6.3: GO-2026-4550

The other module changes are what go mod tidy needs for these.

Still open, because the fixed releases need Go 1.26: golang.org/x/net
GO-2026-6603, GO-2026-6610, GO-2026-6611, GO-2026-6612 and GO-2026-6617
(fixed in v0.60.0), and golang.org/x/crypto GO-2026-6354 and GO-2026-6355
(fixed in v0.56.0). GO-2026-6610 affects every x/net before v0.60.0;
govulncheck does not report it as reachable with v0.50.0, but does with
v0.56.0 and v0.58.0.

Co-authored-by: Levi <[email protected]>
2026-11-09 10:52:49 +08:00
Levi 8ef3b63e8d fix(ui): respect mobile safe areas in tablet layouts (#1641) 2026-11-06 09:32:45 +08:00
Konstantin Ostrovskii d873cd411c fix(http): copy a fully prefetched body when Start runs before prefetchDone (#1608)
The prefetch goroutine sets prefetchDone only after one more read and
closing the body. A Start in that window took the whole file as the
fallback snapshot, expandConnections marked the task done, and the temp
file was never copied, leaving a zero-filled target.

Read prefetchSize once and treat the prefetch as complete when it reached
the known size.
2026-11-06 09:24:19 +08:00
Levi 9e9bcd4b01 fix(ci): keep beta Docker images off latest (#1645) 2026-11-04 14:41:53 +08:00
Silas Mariusz 5bd57608d2 fix(script): keep the task name in the script snapshot; wait for the full env output in tests (#1568) 2026-11-03 10:40:55 +08:00
Codess ebfd113360 fix(ui): make download file deletion explicit in task dialog (#1622) (#1638) 2026-11-03 09:27:30 +08:00
Levi d69874e13e feat(ui): unify child surfaces and show web task creation in a modal (#1633) 2026-11-02 23:54:05 +08:00
Levi b0d45226d7 fix(macos): release closed child window engines (#1639) 2026-11-02 23:52:11 +08:00
Junker der Provinz cd19772099 fix(bt): races between building and closing the shared client (#1587) 2026-11-02 21:03:16 +08:00
WuFeng 1e36599d52 feat(tasks): group extension folder downloads (#1594) 2026-11-02 15:39:08 +08:00
Levi fc0a1e0ef9 fix(extension): restore deprecated request APIs for compatibility (#1628) 2026-10-31 17:51:56 +08:00
HouKangle 8719166dbf fix(web): prevent browser context menu over task menus (#1627) 2026-10-30 18:02:02 +08:00
Silas Mariusz c0b70d319d feat(protocol): add FTP, FTPS and FTPES downloads (#1569) 2026-10-30 17:01:46 +08:00
Levi e0951de270 fix(analytics): use Android ID and persist UUID fallbacks (#1624) 2026-10-28 22:33:13 +08:00
Levi be0fb05edd fix(mcp): describe dynamic outputs with explicit schemas (#1626) 2026-10-28 22:32:52 +08:00
Arpit Khandelwal 91edba9af6 perf(http): replace per-read goroutine in TimeoutReader with a timer (#1597) 2026-10-28 13:58:06 +08:00
OKAN TIKTAS 7d423e2182 fix(web): return startup errors instead of panicking (#1620)
Replace server startup panics with returned errors, print actionable CLI
messages with exit status 1, and clarify the API token flags.

Add regressions for missing web authentication credentials.

Fixes GopeedLab/gopeed#1614
2026-10-26 18:05:56 +08:00
HouKangleandHou KangLe 903c5c46c4 fix(extensions): match magnet URI patterns (#1619)
Co-authored-by: Hou KangLe <[email protected]>
2026-10-26 14:15:10 +08:00
Levi b8ff5ff97f fix(ci): restore Windows release asset uploads (#1605) 2026-10-24 11:38:10 +08:00
Levi c51c5b284d fix(ci): select the first Windows compiler for runtime packaging (#1604) 2026-10-24 00:39:11 +08:00
Levi 4d65948934 fix(ci): resolve Windows AMD64 runtime DLLs from compiler (#1603) 2026-10-23 23:38:15 +08:00
Levi c275bc2e46 release: v2.0.0-beta.4 2026-10-23 21:00:49 +08:00
WuFeng 9aae13453a feat(download): auto-categorize downloads into folders by file extension (#1572) 2026-10-23 20:38:25 +08:00
Levi d125d4ab75 feat(http): support advisory extension download sizes (#1599) 2026-10-22 23:35:01 +08:00
Levi 4214765d39 refactor: remove proxy API (#1596) 2026-10-21 22:48:17 +08:00
Arpit Khandelwal 47cafb7575 perf(http): use 64 KiB connection read buffers (#1588) 2026-10-21 14:19:01 +08:00
Levi de65149b29 refactor(downloader): remove opts parameter from Resolve (#1595) 2026-10-19 21:40:57 +08:00
Junker der Provinz 5462b14705 feat(bt): support disabling DHT and PEX (#1555) 2026-10-17 11:22:34 +08:00
Levi eee39e012f fix(ui): widen navigation rail for macOS traffic lights (#1582) 2026-10-14 20:09:08 +08:00
Levi 12860bb207 perf(extensions): load store summaries and check updates asynchronously (#1583) 2026-10-14 20:08:41 +08:00
Levi b86c2c18e7 fix(ci): restore Windows ARM builds with scoped MSVC compatibility (#1586) 2026-10-13 21:06:09 +08:00
Ashish Goswami 6f20790b17 feat(protocol/http, mcpserver, download): add checksum verification, MCP speed tool, pause webhook, and CLI flags (#1542) 2026-10-13 08:28:56 +08:00
Kunpeng Xie fa622a4063 fix(http): skip prefetch wait for restored downloads (#1584) 2026-10-12 11:13:42 +08:00
Levi c6619794fc fix: preserve Cookie headers in extension requests (#1581) 2026-10-10 22:10:08 +08:00
Levi 87a8f87b70 feat(ui): rework extension update detection and entry point (#1535) 2026-10-10 22:09:47 +08:00
WuFeng ba84a04a43 fix(settings): use Save as the primary label on settings form dialogs (#1571) 2026-10-08 16:31:16 +08:00
Levi 224b488087 feat(task): record and display task completion time (#1536) 2026-09-25 19:15:31 +08:00
Levi 7ce4fb7f0e fix(ui): smooth task progress bar updates (#1532) 2026-09-25 19:08:20 +08:00
UltraGeek0102 8d1ae93b03 refactor(ios): clean up continued processing task handling (#1547) 2026-09-25 16:21:09 +08:00
JiangJiang 363153fa33 fix(http): use Content-Range total size for 206 responses (#1545) 2026-09-23 09:52:59 +08:00
Levi f0dab90e09 feat(extension): expose request mutation methods on task.meta.req (#1538) 2026-09-23 09:33:28 +08:00
doctor-martin-sowa 6e60889ce8 feat(lang): Add Slovak localization (app_sk.arb) (#1541) 2026-09-23 09:32:07 +08:00
Levi 88d150b933 test(bt): isolate cleanup fixtures from part-file promotion (#1531) 2026-09-21 22:30:54 +08:00
Levi 694b43e3f9 docs: update beta download link to v2.0.0-beta.3 2026-09-19 08:16:57 +08:00
Levi fe25ccab1d release: v2.0.0-beta.3 2026-09-18 23:34:09 +08:00
Levi 8301b1f9ce feat(http): accelerate connection expansion with squared batches (#1530) 2026-09-18 23:22:57 +08:00
Levi 49aab4ea7c feat(extension): add host environment and WebView page events (#1527) 2026-09-18 23:21:53 +08:00
Levi 5d8267fa56 fix(ui): fix various UI issues (#1518) 2026-09-18 22:40:36 +08:00
Levi 1ab452ced8 fix(bt): honor selected files when creating resolved tasks (#1523) 2026-09-18 22:32:09 +08:00
Levi 743f356567 feat(flutter): add action buttons to desktop download notifications (#1524) 2026-09-18 22:18:47 +08:00
Levi b1dd46cc7c feat: spool media inputs to disk and report producer progress (#1520) 2026-09-18 22:17:22 +08:00
sysy 38750d8505 fix: sanitize archive entry paths before extraction to prevent path traversal (#1526) 2026-09-18 22:12:43 +08:00
UltraGeek0102 7c1447f659 fix(ios): recover continued processing after toggle (#1519) 2026-09-17 08:32:21 +08:00
Levi 21702d5d46 feat(webview): add proxy support and per-extension profile lifecycle (#1517) 2026-09-16 23:17:43 +08:00
Levi e99ef6c7f4 fix(flutter): keep waiting task progress static (#1516) 2026-09-16 00:39:18 +08:00
Levi c667f097dc feat: add FFmpeg WASM runtime (#1512) 2026-09-16 00:38:22 +08:00
sun 46c34f2e92 feat(hls): add native HLS (m3u8) download protocol (#1515) 2026-09-15 23:40:31 +08:00
UltraGeek0102 e47a20a1bd feat(ios): add iOS 26+ continued processing and Live Activities (#1514) 2026-09-15 22:49:11 +08:00
Levi cee5f3336e fix: extension install by deeplink (#1510) 2026-09-13 23:40:59 +08:00
Levi a760290213 fix(ui): enable task detail text selection and compact extension stats (#1511) 2026-09-13 23:30:45 +08:00
Levi a29f4bb42a fix(ui): use typed controls for extension settings (#1509) 2026-09-13 14:29:46 +08:00
Levi 62e8ab3cef fix(ui): show downloaded bytes for tasks with unknown size (#1508) 2026-09-13 11:57:26 +08:00
Levi 8fa9a1e902 fix(ui): preserve mobile home route and confirm Android exit (#1507) 2026-09-12 21:44:27 +08:00
Levi 46675f26af fix(ui): align history search placeholder and prevent clipping (#1506) 2026-09-12 20:04:28 +08:00
Levi 0d52b840a2 feat(ui): emphasize task creation and add empty-list actions (#1505) 2026-09-12 18:09:36 +08:00
Levi 091e7b8936 fix(ui): keep mobile create task actions in safe area (#1504) 2026-09-12 09:28:04 +08:00
Levi ff8f61423e fix(ui): refine app update dialog layout and scrolling (#1503) 2026-09-12 07:25:37 +08:00
Levi e5ac13cfdc docs: update beta download links 2026-09-11 08:36:01 +08:00
Levi 073e1413ab release: 2.0.0-beta.2 2026-09-11 08:20:45 +08:00
Wu Shuwen 04b9aa18ec fix(http): infer extension for opaque URLs (#1495) 2026-09-11 08:01:32 +08:00
Levi 08fe4eef4e fix(ui): refine create task actions and relocate icon sources (#1500) 2026-09-11 08:00:03 +08:00
Levi 8f9bac9e6b fix(ui): reuse shared file path picker for script settings (#1496) 2026-09-10 23:51:16 +08:00
494 changed files with 50722 additions and 4656 deletions

No files matched your search

+2
View File
@@ -0,0 +1,2 @@
# Keep upstream plugin sources recognizable and collapsed in GitHub reviews.
ui/flutter/third_party/** linguist-vendored whitespace=-blank-at-eol,-blank-at-eof
+21 -6
View File
@@ -13,7 +13,7 @@ on:
default: "false"
env:
GO_VERSION: "1.25.4"
GO_VERSION: "1.26.11"
FLUTTER_VERSION: "3.41.6"
GA4_MEASUREMENT_ID: ${{ secrets.GA4_MEASUREMENT_ID }}
GA4_API_SECRET: ${{ secrets.GA4_API_SECRET }}
@@ -106,17 +106,32 @@ jobs:
if ("${{ matrix.arch }}" -eq "amd64") {
# for amd64
$release = "build\windows\x64\runner\Release\"
$mingw = "C:\Program Files\Git\mingw64\bin"
cp $mingw\libstdc++-6.dll $release
cp $mingw\libgcc_s_seh-1.dll $release
# Use the same toolchain as cgo instead of Git's bundled DLLs.
$cc = (Get-Command -Name (go env CC) -CommandType Application -ErrorAction Stop | Select-Object -First 1).Source
foreach ($dll in @("libstdc++-6.dll", "libgcc_s_seh-1.dll", "libwinpthread-1.dll")) {
$runtimePath = [string](& $cc "-print-file-name=$dll")
if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($runtimePath)) {
throw "Failed to locate $dll using $cc"
}
$runtimePath = $runtimePath.Trim()
# Some MinGW distributions keep runtime DLLs beside gcc.exe.
if ($runtimePath -eq $dll) {
$runtimePath = Join-Path (Split-Path -Parent $cc) $dll
}
if (!(Test-Path -LiteralPath $runtimePath -PathType Leaf)) {
throw "MinGW runtime $dll was not found by $cc (reported path: $runtimePath)"
}
Write-Host "Copying MinGW runtime: $runtimePath"
Copy-Item -LiteralPath $runtimePath -Destination $release
}
} else {
# for ARM64
$release = "build\windows\arm64\runner\Release\"
$mingw = "$env:CLANGARM64_ROOT\aarch64-w64-mingw32\bin"
cp $mingw\libc++.dll $release
cp $mingw\libunwind.dll $release
cp $mingw\libwinpthread-1.dll $release
}
cp $mingw\libwinpthread-1.dll $release
cp $system\msvcp140.dll $release
cp $system\vcruntime140.dll $release
cp $system\vcruntime140_1.dll $release
@@ -783,5 +798,5 @@ jobs:
linux/arm64
linux/arm/v7
tags: |
liwei2633/gopeed:latest
${{ !contains(env.VERSION, 'beta') && 'liwei2633/gopeed:latest' || '' }}
liwei2633/gopeed:v${{ env.VERSION }}
+1 -1
View File
@@ -17,7 +17,7 @@ on:
- "Dockerfile"
env:
GO_VERSION: "1.25.4"
GO_VERSION: "1.26.11"
permissions:
contents: write
+63 -2
View File
@@ -28,7 +28,7 @@ on:
workflow_dispatch:
env:
GO_VERSION: "1.25.4"
GO_VERSION: "1.26.11"
FLUTTER_VERSION: "3.41.6"
jobs:
@@ -69,6 +69,49 @@ jobs:
with:
files: ./coverage.txt
token: ${{ secrets.CODECOV_UPLOAD_TOKEN }}
test-flutter-category:
# The settings editor and the backend have to read an extension list the
# same way, so both sides run the shared rule table. The Go side is covered
# by the test job above, this runs the ui side of it.
name: Test shared category rules
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: subosito/flutter-action@v2
with:
flutter-version: ${{ env.FLUTTER_VERSION }}
- name: Run the shared category rule table on the ui side
working-directory: ui/flutter
run: |
flutter pub get
flutter gen-l10n
flutter test test/api/downloader_config_test.dart
test-flutter-task-context-menu:
name: Test task context menus and text inputs
runs-on: ubuntu-latest
timeout-minutes: 20
env:
CHROME_EXECUTABLE: /usr/bin/google-chrome
steps:
- uses: actions/checkout@v3
- uses: subosito/flutter-action@v2
with:
flutter-version: ${{ env.FLUTTER_VERSION }}
- name: Prepare Flutter tests
working-directory: ui/flutter
run: |
flutter pub get
flutter gen-l10n
- name: Test task menu actions and native input policy
working-directory: ui/flutter
run: |
flutter test --no-pub test/features/tasks/presentation/widgets/task_context_menu_test.dart test/shared/widgets/app_text_field_test.dart
flutter test --no-pub test/widget_test.dart --plain-name 'task context menu exposes'
- name: Test browser menu cancellation and input policy
working-directory: ui/flutter
run: |
"$CHROME_EXECUTABLE" --version
flutter test --no-pub --platform chrome test/features/tasks/presentation/widgets/task_context_menu_test.dart test/shared/widgets/app_text_field_test.dart
test-webview-integration:
name: Test webview integration (${{ matrix.name }})
strategy:
@@ -77,6 +120,8 @@ jobs:
include:
- name: Windows
os: windows-latest
- name: Linux
os: ubuntu-latest
- name: macOS
os: macos-latest
runs-on: ${{ matrix.os }}
@@ -89,9 +134,25 @@ jobs:
if: runner.os == 'macOS'
with:
flutter-version: ${{ env.FLUTTER_VERSION }}
- name: Install Linux WebView dependencies
if: runner.os == 'Linux'
run: |
sudo apt-get update -y
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev xvfb
- name: Test webview
if: runner.os == 'Linux'
# Match the full suite's coverage instrumentation and repeat on the same
# GTK event loop to catch queued callbacks after engine destruction.
run: |
PACKAGES=$(go list ./... | awk '!/\/bind\// && (!/\/cmd\// || /\/cmd\/web$/)')
go test -c -tags webview -coverpkg=$(echo "$PACKAGES" | paste -sd "," -) -covermode=count -o "$RUNNER_TEMP/goprovider.test" ./internal/webview/goprovider
xvfb-run -a "$RUNNER_TEMP/goprovider.test" -test.v -test.count=5 -test.timeout=300s -test.coverprofile="$RUNNER_TEMP/webview-coverage.out"
env:
CGO_ENABLED: 1
- name: Test webview
if: runner.os == 'Windows'
run: go test -tags webview -v -count=1 -timeout 300s ./internal/webview/goprovider/
# Repeat close/reopen coverage to catch browser shutdown races.
run: go test -tags webview -v -count=3 -timeout 300s ./internal/webview/goprovider/
env:
CGO_ENABLED: 1
- name: Test webview
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.25.4-alpine3.22 AS go
FROM golang:1.26.11-alpine3.23 AS go
WORKDIR /app
COPY ./go.mod ./go.sum ./
RUN go mod download
+2 -2
View File
@@ -20,7 +20,7 @@ Beyond core download management, Gopeed offers browser integration, JavaScript e
Visit ✈ [Official Website](https://gopeed.com)
![Application screenshot](_docs/img/ui-concept-en.png)
![Application screenshot](_docs/img/ui-concept-en.webp)
## ✨ Features
@@ -63,7 +63,7 @@ Connect Gopeed to an AI agent and manage downloads with natural language. For ex
Gopeed 2.0.0 is currently in public beta, introducing a redesigned interface, a native communication architecture that connects desktop and mobile clients directly to the Go core through FFI, a more consistent cross-platform experience, improved task management, more flexible API support, and MCP-based AI agent integration. Some features may still be incomplete or unstable, so please try it and report any issues you encounter.
- [Download Gopeed 2.0.0 Beta 1](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.1)
- [Download Gopeed 2.0.0 Beta 3](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.3)
Once the features and stability meet our release standards, we will publish the official Gopeed 2.0.0 release. Beta users will be able to upgrade directly to the final release, while existing stable users will not be automatically moved onto the beta channel.
+2 -2
View File
@@ -20,7 +20,7 @@ Gopeed(**Go Speed** の略)は、Go と Flutter で開発された高速で
✈ [公式サイト](https://gopeed.com)
![アプリのスクリーンショット](_docs/img/ui-concept-en.png)
![アプリのスクリーンショット](_docs/img/ui-concept-en.webp)
## ✨ 主な機能
@@ -63,7 +63,7 @@ Gopeed を AI Agent と連携すると、自然言語でダウンロードを管
Gopeed 2.0.0 は現在パブリックベータ版です。刷新されたインターフェース、デスクトップおよびモバイルから FFI で Go コアへ直接接続するネイティブ通信アーキテクチャ、より統一されたクロスプラットフォーム体験、強化されたタスク管理、より柔軟な API、MCP による AI Agent 連携機能を導入しています。一部の機能はまだ不完全または不安定な場合がありますので、ぜひお試しいただき、問題があればフィードバックをお寄せください。
- [Gopeed 2.0.0 Beta 1 をダウンロード](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.1)
- [Gopeed 2.0.0 Beta 3 をダウンロード](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.3)
機能と安定性が正式リリースの基準に達した時点で、Gopeed 2.0.0 正式版を公開します。ベータ版のユーザーは正式版へ直接アップデートできます。既存の安定版ユーザーが自動的にベータチャンネルへ移行することはありません。
+2 -2
View File
@@ -20,7 +20,7 @@ Ngoài việc quản lý tải xuống hằng ngày, Gopeed còn hỗ trợ tíc
Truy cập ✈ [Trang web chính thức](https://gopeed.com)
![Ảnh chụp màn hình ứng dụng](_docs/img/ui-concept-en.png)
![Ảnh chụp màn hình ứng dụng](_docs/img/ui-concept-en.webp)
## ✨ Tính năng chính
@@ -63,7 +63,7 @@ Sau khi kết nối Gopeed với AI Agent, bạn có thể quản lý tải xu
Gopeed 2.0.0 hiện đang trong giai đoạn beta công khai, với giao diện được thiết kế lại, kiến trúc giao tiếp native kết nối trực tiếp ứng dụng desktop và di động với lõi Go qua FFI, trải nghiệm đa nền tảng nhất quán hơn, khả năng quản lý tác vụ được cải thiện, API linh hoạt hơn và khả năng tích hợp AI Agent qua MCP. Một số tính năng có thể vẫn chưa hoàn thiện hoặc chưa ổn định, vì vậy hãy dùng thử và gửi phản hồi nếu bạn gặp vấn đề.
- [Tải Gopeed 2.0.0 Beta 1](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.1)
- [Tải Gopeed 2.0.0 Beta 3](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.3)
Khi các tính năng và độ ổn định đáp ứng tiêu chuẩn phát hành, chúng tôi sẽ phát hành Gopeed 2.0.0 chính thức. Người dùng bản beta có thể nâng cấp trực tiếp lên bản chính thức, trong khi người dùng bản ổn định hiện tại sẽ không tự động được chuyển sang kênh beta.
+2 -2
View File
@@ -20,7 +20,7 @@ Gopeed(全称 **Go Speed**,也叫“够快下载器”)是一款使用 Go
访问 ✈ [官方网站](https://gopeed.com)
![应用截图](_docs/img/ui-concept-zh-CN.png)
![应用截图](_docs/img/ui-concept-zh-CN.webp)
## ✨ 主要特性
@@ -63,7 +63,7 @@ Gopeed(全称 **Go Speed**,也叫“够快下载器”)是一款使用 Go
Gopeed 2.0.0 目前处于公开 Beta 测试阶段,引入了全新界面、通过 FFI 直连 Go 核心的原生通信架构、更一致的跨平台体验、更完善的任务管理、更灵活的 API,以及用于 AI Agent 接入的 MCP 能力。部分功能仍可能不够完善或稳定,欢迎提前体验并向我们反馈问题。
- [下载 Gopeed 2.0.0 Beta 1](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.1)
- [下载 Gopeed 2.0.0 Beta 3](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.3)
当功能完整度和稳定性达到正式发布标准后,我们会发布 Gopeed 2.0.0 正式版。已安装 Beta 版本的用户可以直接升级到最终正式版,现有稳定版用户则不会被自动切换到 Beta 渠道。
+2 -2
View File
@@ -20,7 +20,7 @@ Gopeed(**Go Speed** 的縮寫)是一款以 Go 與 Flutter 開發的高速、
瀏覽 ✈ [官方網站](https://gopeed.com)
![應用程式截圖](_docs/img/ui-concept-en.png)
![應用程式截圖](_docs/img/ui-concept-en.webp)
## ✨ 主要功能
@@ -63,7 +63,7 @@ Gopeed(**Go Speed** 的縮寫)是一款以 Go 與 Flutter 開發的高速、
Gopeed 2.0.0 目前處於公開 Beta 測試階段,引入了全新介面、透過 FFI 直連 Go 核心的原生通訊架構、更一致的跨平台體驗、更完善的任務管理、更靈活的 API,以及透過 MCP 串接 AI Agent 的能力。部分功能仍可能不夠完善或穩定,歡迎搶先體驗並向我們回報問題。
- [下載 Gopeed 2.0.0 Beta 1](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.1)
- [下載 Gopeed 2.0.0 Beta 3](https://github.com/GopeedLab/gopeed/releases/tag/v2.0.0-beta.3)
當功能完整度與穩定性達到正式發布標準後,我們會發布 Gopeed 2.0.0 正式版。已安裝 Beta 版本的使用者可以直接升級至最終正式版,現有穩定版使用者則不會被自動切換至 Beta 頻道。
Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 391 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 792 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 357 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 787 KiB

+52
View File
@@ -0,0 +1,52 @@
# Batch task creation
`POST /api/v1/tasks/batch` accepts ordinary and grouped batches:
```json
{
"group": true,
"groupName": "album",
"opts": {"path": "/downloads"},
"reqs": [
{"rid": "resolved-resource-id", "opts": {"selectFiles": [1]}},
{"req": {"url": "https://example.com/cover.jpg"}}
]
}
```
Each member accepts `rid` or `req`. A nonempty `rid` takes priority and creates a
task from the cached resolve result. A member's `opts`, when present, replaces
batch `opts` as a whole; omitted fields do not inherit individual batch values.
Members without `opts` use batch `opts`. BT file selections use the indexes from
the resolve result. Requests without `rid` continue to work.
For groups, batch `opts.path` selects the parent directory. Members inheriting
batch options, or with an empty path, use the reserved group directory. Explicit
relative member paths are resolved under that directory; do not prefix them with
the batch parent path. For example, parent `downloads` and member path `nested`
place the member in `downloads/<group name>/nested`. Absolute paths under the
parent directory preserve their relative structure within the group. In ordinary
batches, an empty member path uses the configured default download directory.
All members are attempted. If at least one task is persisted, the response is
successful and contains the created task IDs. A task that fails to start after
being persisted is included; use the existing task controls to retry it.
Successful tasks are kept when other members fail. If every member fails, the
API returns an error and removes any newly reserved empty group and directory.
Both ordinary and grouped batches now return an object in `data`:
```json
{"code": 0, "msg": "", "data": {"taskIds": ["task-1", "task-2"], "parentId": "group-id"}}
```
Ordinary batches omit `parentId`. **This replaces the former array response for
ordinary batches.** The outer `code`/`msg`/`data` envelope is unchanged. Clients
must read `data.taskIds`; there are no per-member results. A successful response
completes the batch, including partial success, so clients must not automatically
resubmit missing members.
With `group: true`, the server generates the group ID and reserves its directory.
Creation does not accept a client group ID or append to an existing group. Group
metadata remains available through the existing task list and detail responses;
`parentId` identifies membership for group operations.
+12
View File
@@ -7,6 +7,7 @@ import (
nativebridge "github.com/GopeedLab/gopeed/bind/native"
"github.com/GopeedLab/gopeed/internal/webview/rpcprovider"
goapi "github.com/GopeedLab/gopeed/pkg/api"
"github.com/GopeedLab/gopeed/pkg/rest"
"github.com/GopeedLab/gopeed/pkg/rest/model"
)
@@ -90,3 +91,14 @@ func applyWebViewProvider(config *model.StartConfig) {
}
config.WebViewProvider = rpcprovider.New(*config.WebViewRPCConfig)
}
func LiveActivityTaskEventMask() int64 {
return int64(
goapi.TaskEventDone |
goapi.TaskEventError |
goapi.TaskEventStart |
goapi.TaskEventProgress |
goapi.TaskEventPause |
goapi.TaskEventDelete,
)
}
+7 -1
View File
@@ -1,6 +1,9 @@
package main
import (
"fmt"
"os"
"github.com/GopeedLab/gopeed/cmd"
"github.com/GopeedLab/gopeed/pkg/rest/model"
)
@@ -13,5 +16,8 @@ func main() {
Storage: model.StorageBolt,
WebEnable: true,
}
cmd.Start(cfg)
if err := cmd.Start(cfg); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
+2 -2
View File
@@ -166,7 +166,7 @@ func runResolve(args []string) error {
result, err := d.Resolve(&base.Request{
URL: url,
Labels: map[string]string(labels),
}, nil)
})
if err != nil {
return err
}
@@ -250,7 +250,7 @@ func runDownload(args []string) error {
result, err := d.Resolve(&base.Request{
URL: url,
Labels: map[string]string(labels),
}, nil)
})
if err != nil {
return err
}
+7 -3
View File
@@ -7,9 +7,11 @@ import (
)
type args struct {
url string
connections *int
dir *string
url string
connections *int
dir *string
checksum *string
checksumAlgo *string
}
func parse() *args {
@@ -20,6 +22,8 @@ func parse() *args {
var args args
args.connections = flag.Int("C", 16, "Concurrent connections.")
args.dir = flag.String("D", dir, "Store directory.")
args.checksum = flag.String("checksum", "", "Expected checksum hash (hex encoded).")
args.checksumAlgo = flag.String("checksum-algo", "md5", "Checksum algorithm: md5, sha1, or sha256.")
flag.Parse()
t := flag.Args()
if len(t) > 0 {
+17 -2
View File
@@ -19,6 +19,18 @@ func main() {
var wg sync.WaitGroup
wg.Add(1)
var checksumOpt *http.ChecksumOption
if args.checksum != nil && *args.checksum != "" {
algo := "md5"
if args.checksumAlgo != nil && *args.checksumAlgo != "" {
algo = *args.checksumAlgo
}
checksumOpt = &http.ChecksumOption{
Algorithm: algo,
Expected: *args.checksum,
}
}
_, err := download.Boot().
URL(args.url).
Listener(func(event *download.Event) {
@@ -43,8 +55,11 @@ func main() {
}
}).
Create(&base.Options{
Path: *args.dir,
Extra: http.OptsExtra{Connections: *args.connections},
Path: *args.dir,
Extra: http.OptsExtra{
Connections: *args.connections,
Checksum: checksumOpt,
},
})
if err != nil {
panic(err)
+14 -6
View File
@@ -16,17 +16,18 @@ import (
//go:embed banner.txt
var banner string
func Start(cfg *model.StartConfig) {
fmt.Println(banner)
func Start(cfg *model.StartConfig) error {
srv, listener, err := rest.BuildServer(cfg)
if err != nil {
panic(err)
return err
}
fmt.Println(banner)
downloadCfg, err := rest.Downloader.GetConfig()
if err != nil {
panic(err)
listener.Close()
return err
}
if downloadCfg.FirstLoad {
if rest.Downloader.IsFirstLoad() {
// Set default download config
if cfg.DownloadConfig != nil {
cfg.DownloadConfig.Merge(downloadCfg)
@@ -51,13 +52,20 @@ func Start(cfg *model.StartConfig) {
rest.Downloader.PutConfig(downloadCfg)
}
}
// The default download dir is resolved only after the core starts, so
// seed the built-in categories now that the dir is known. Init is
// idempotent and leaves an existing category list untouched.
downloadCfg.Init()
rest.Downloader.PutConfig(downloadCfg)
}
watchExit()
fmt.Printf("Server start success on http://%s\n", listener.Addr().String())
if err := srv.Serve(listener); err != nil && err != http.ErrServerClosed {
panic(err)
return err
}
return nil
}
func watchExit() {
+35
View File
@@ -0,0 +1,35 @@
package cmd
import (
"strings"
"testing"
"github.com/GopeedLab/gopeed/pkg/rest/model"
)
func TestStartInvalidWebAPITokenReturnsError(t *testing.T) {
tests := []struct {
name string
webAuth *model.WebAuth
}{
{name: "no credentials"},
{name: "missing username", webAuth: &model.WebAuth{Password: "secret"}},
{name: "missing password", webAuth: &model.WebAuth{Username: "gopeed"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := Start(&model.StartConfig{
WebEnable: true,
ApiToken: "test-token",
WebAuth: tt.webAuth,
})
if err == nil {
t.Fatal("Start() error = nil, want web authentication validation error")
}
if !strings.Contains(err.Error(), "web authentication username and password are required") {
t.Fatalf("Start() error = %q, want missing web authentication credentials", err)
}
})
}
}
+10 -4
View File
@@ -37,15 +37,17 @@ func installByInstaller(killSignalChan chan<- any, packagePath, destDir string)
}
defer reader.Close()
// Find the installer file
var installerPath string
for _, file := range reader.File {
cleanName := filepath.Clean(file.Name)
if strings.HasPrefix(cleanName, "..") || filepath.IsAbs(cleanName) {
continue
}
if file.FileInfo().IsDir() {
continue
}
// Extract file
path := filepath.Join(tempDir, file.Name)
path := filepath.Join(tempDir, cleanName)
if err := os.MkdirAll(filepath.Dir(path), os.ModePerm); err != nil {
return err
@@ -101,7 +103,11 @@ func installByPortable(killSignalChan chan<- any, packagePath, destDir string) e
defer reader.Close()
for _, file := range reader.File {
path := filepath.Join(destDir, file.Name)
cleanName := filepath.Clean(file.Name)
if strings.HasPrefix(cleanName, "..") || filepath.IsAbs(cleanName) {
continue
}
path := filepath.Join(destDir, cleanName)
if file.FileInfo().IsDir() {
os.MkdirAll(path, file.Mode())
+9 -2
View File
@@ -13,6 +13,7 @@ import (
)
type args struct {
TempDir *string `json:"tempDir"`
Address *string `json:"address"`
Port *int `json:"port"`
Username *string `json:"username"`
@@ -51,9 +52,10 @@ func loadCliArgs() *args {
flag.StringVar(cfg.Username, "username", "gopeed", "Web Authentication Username")
cfg.Password = flag.String("p", "", "Web Authentication Password, if no password is set, web authentication will not be enabled")
flag.StringVar(cfg.Password, "password", "", "Web Authentication Password, if no password is set, web authentication will not be enabled")
cfg.ApiToken = flag.String("T", "", "API token, it must be configured when using HTTP API in the case of enabling web authentication")
flag.StringVar(cfg.ApiToken, "api-token", "", "API token, it must be configured when using HTTP API in the case of enabling web authentication")
cfg.ApiToken = flag.String("T", "", "API token (requires web authentication with username and password)")
flag.StringVar(cfg.ApiToken, "api-token", "", "API token (requires web authentication with username and password)")
flag.BoolVar(&cfg.MCPEnable, "mcp-enable", false, "Enable MCP endpoint")
cfg.TempDir = flag.String("temp-dir", "", "Temporary directory (defaults to the OS temporary directory)")
cfg.StorageDir = flag.String("d", "", "Storage directory")
flag.StringVar(cfg.StorageDir, "storage-dir", "", "Storage directory")
whiteDownloadDirs := flag.String("white-download-dirs", "", "White download directories, comma-separated")
@@ -89,6 +91,8 @@ func overrideWithCliArgs(cfg *args, cliConfig *args) {
cfg.ApiToken = cliConfig.ApiToken
case "mcp-enable":
cfg.MCPEnable = cliConfig.MCPEnable
case "temp-dir":
cfg.TempDir = cliConfig.TempDir
case "d", "storage-dir":
cfg.StorageDir = cliConfig.StorageDir
case "white-download-dirs":
@@ -101,6 +105,9 @@ func overrideWithCliArgs(cfg *args, cliConfig *args) {
// setDefaults sets default values for any unset configuration fields
func setDefaults(cfg *args, cliConfig *args) {
if cfg.TempDir == nil {
cfg.TempDir = cliConfig.TempDir
}
if cfg.Address == nil {
cfg.Address = cliConfig.Address
}
+9 -1
View File
@@ -43,11 +43,16 @@ func main() {
storageDir = filepath.Join(filepath.Dir(exe), "storage")
}
tempDir := ""
if args.TempDir != nil {
tempDir = *args.TempDir
}
cfg := &model.StartConfig{
Network: "tcp",
Address: fmt.Sprintf("%s:%d", *args.Address, *args.Port),
Storage: model.StorageBolt,
StorageDir: storageDir,
TempDir: tempDir,
WhiteDownloadDirs: args.WhiteDownloadDirs,
ApiToken: *args.ApiToken,
MCPEnable: args.MCPEnable,
@@ -57,7 +62,10 @@ func main() {
WebFS: sub,
WebAuth: webAuth,
}
cmd.Start(cfg)
if err := cmd.Start(cfg); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
func isNotBlank(str *string) bool {
+51 -50
View File
@@ -1,18 +1,23 @@
module github.com/GopeedLab/gopeed
go 1.25.4
go 1.26.0
require (
github.com/GopeedLab/webview_go v0.0.0-20260423085439-7a2f88b6e9b5
github.com/anacrolix/torrent v1.60.1-0.20251217073903-486bcbe758e0
codeberg.org/gruf/go-ffmpreg v0.6.23-0.20260827095416-df1836cb5be0
github.com/GopeedLab/webview_go v0.0.0-20260918010912-0d4c27bb7914
github.com/RoaringBitmap/roaring/v2 v2.16.0
github.com/anacrolix/torrent v1.62.0
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5
github.com/bodgit/sevenzip v1.6.1
github.com/dop251/goja v0.0.0-20260106131823-651366fbe6e3
github.com/dop251/goja_nodejs v0.0.0-20240728170619-29b559befffc
github.com/go-git/go-git/v5 v5.8.1
github.com/fclairamb/ftpserverlib v0.31.0
github.com/go-git/go-git/v5 v5.19.2
github.com/google/jsonschema-go v0.4.3
github.com/gorilla/handlers v1.5.1
github.com/gorilla/mux v1.8.0
github.com/imroc/req/v3 v3.52.2
github.com/imroc/req/v3 v3.58.0
github.com/jlaffaye/ftp v0.2.4
github.com/matoous/go-nanoid/v2 v2.0.0
github.com/mattn/go-ieproxy v0.0.12
github.com/mholt/archives v0.1.5
@@ -21,65 +26,65 @@ require (
github.com/pkg/errors v0.9.1
github.com/rs/dnscache v0.0.0-20230804202142-fc85eb664529
github.com/rs/zerolog v1.31.0
github.com/spf13/afero v1.15.0
github.com/tetratelabs/wazero v1.12.0
github.com/xiaoqidun/setft v0.0.0-20220310121541-be86327699ad
go.etcd.io/bbolt v1.4.3
golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f
)
require (
github.com/cyphar/filepath-securejoin v0.6.1 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/onsi/gomega v1.36.3 // indirect
)
require (
github.com/STARRY-S/zip v0.2.3 // indirect
github.com/anacrolix/btree v0.1.1 // indirect
github.com/anacrolix/missinggo/v2 v2.10.0 // indirect
github.com/anacrolix/go-utp v0.0.0-20260908033909-9f1664acf866 // indirect
github.com/anacrolix/missinggo/v2 v2.11.0 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/bodgit/plumbing v1.3.0 // indirect
github.com/bodgit/windows v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/coder/websocket v1.8.15 // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/felixge/fgprof v0.9.5 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/jsonschema-go v0.4.3 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/icholy/digest v1.1.0 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/compress v1.18.2 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/mikelolasagasti/xz v1.0.1 // indirect
github.com/minio/minlz v1.0.1 // indirect
github.com/nwaples/rardecode/v2 v2.2.0 // indirect
github.com/onsi/ginkgo/v2 v2.23.4 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/pion/transport/v4 v4.0.1 // indirect
github.com/quic-go/qpack v0.5.1 // indirect
github.com/quic-go/quic-go v0.51.0 // indirect
github.com/refraction-networking/utls v1.6.7 // indirect
github.com/pion/transport/v4 v4.0.2 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/quic-go/quic-go v0.59.0 // indirect
github.com/refraction-networking/utls v1.8.2 // indirect
github.com/segmentio/asm v1.1.3 // indirect
github.com/segmentio/encoding v0.5.4 // indirect
github.com/sorairolake/lzip-go v0.3.8 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.uber.org/automaxprocs v1.6.0 // indirect
go.uber.org/mock v0.5.1 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/oauth2 v0.35.0 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/sync v0.23.0 // indirect
)
require (
dario.cat/mergo v1.0.0 // indirect
github.com/Microsoft/go-winio v0.6.1
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 // indirect
github.com/RoaringBitmap/roaring v1.9.4
github.com/acomagu/bufpipe v1.0.4 // indirect
github.com/Microsoft/go-winio v0.6.2
github.com/ProtonMail/go-crypto v1.1.6 // indirect
github.com/alecthomas/atomic v0.1.0-alpha2 // indirect
github.com/anacrolix/chansync v0.7.0 // indirect
github.com/anacrolix/dht/v2 v2.23.0 // indirect
github.com/anacrolix/chansync v0.8.0 // indirect
github.com/anacrolix/dht/v2 v2.23.1-0.20260525063928-ec3a9bd99456 // indirect
github.com/anacrolix/envpprof v1.5.0 // indirect
github.com/anacrolix/generics v0.2.0 // indirect
github.com/anacrolix/go-libutp v1.3.2 // indirect
github.com/anacrolix/generics v0.2.0
github.com/anacrolix/go-libutp v1.5.2-0.20260908013213-836dd42cdde6 // indirect
github.com/anacrolix/log v0.17.1-0.20251118025802-918f1157b7bb // indirect
github.com/anacrolix/missinggo v1.3.0 // indirect
github.com/anacrolix/missinggo/perf v1.0.0 // indirect
@@ -88,13 +93,12 @@ require (
github.com/anacrolix/stm v0.5.0 // indirect
github.com/anacrolix/sync v0.6.0 // indirect
github.com/anacrolix/upnp v0.1.4 // indirect
github.com/anacrolix/utp v0.2.0 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/benbjohnson/immutable v0.4.3 // indirect
github.com/bits-and-blooms/bitset v1.24.4 // indirect
github.com/bradfitz/iter v0.0.0-20191230175014-e8f45d346db8 // indirect
github.com/cespare/xxhash v1.1.0 // indirect
github.com/cloudflare/circl v1.6.1 // indirect
github.com/cloudflare/circl v1.6.3 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
@@ -102,17 +106,16 @@ require (
github.com/emirpasic/gods v1.18.1 // indirect
github.com/felixge/httpsnoop v1.0.1 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.4.1 // indirect
github.com/go-git/go-billy/v5 v5.9.2 // indirect
github.com/go-llsqlite/adapter v0.2.0 // indirect
github.com/go-llsqlite/crawshaw v0.6.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/logr v1.4.4 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/google/btree v1.1.3 // indirect
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/huandu/xstrings v1.5.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
@@ -126,7 +129,7 @@ require (
github.com/multiformats/go-varint v0.1.0 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/pion/datachannel v1.6.0 // indirect
github.com/pion/dtls/v3 v3.0.10 // indirect
github.com/pion/dtls/v3 v3.1.4 // indirect
github.com/pion/ice/v4 v4.2.0 // indirect
github.com/pion/interceptor v0.1.42 // indirect
github.com/pion/logging v0.2.4 // indirect
@@ -137,29 +140,27 @@ require (
github.com/pion/sctp v1.9.1 // indirect
github.com/pion/sdp/v3 v3.0.17 // indirect
github.com/pion/srtp/v3 v3.0.10 // indirect
github.com/pion/stun/v3 v3.1.1 // indirect
github.com/pion/stun/v3 v3.1.5 // indirect
github.com/pion/turn/v4 v4.1.4 // indirect
github.com/pion/webrtc/v4 v4.2.2-0.20260109001657-a5962f314db7 // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c
github.com/protolambda/ctxlock v0.1.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/sergi/go-diff v1.1.0 // indirect
github.com/skeema/knownhosts v1.2.0 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/tidwall/btree v1.8.1 // indirect
github.com/wlynxg/anet v0.0.5 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
go.opentelemetry.io/otel v1.39.0 // indirect
go.opentelemetry.io/otel/metric v1.39.0 // indirect
go.opentelemetry.io/otel/trace v1.39.0 // indirect
golang.org/x/crypto v0.48.0 // indirect
golang.org/x/mod v0.33.0 // indirect
golang.org/x/net v0.50.0
golang.org/x/sys v0.41.0 // indirect
golang.org/x/text v0.34.0
golang.org/x/time v0.15.0 // indirect
golang.org/x/tools v0.42.0 // indirect
go.opentelemetry.io/otel v1.45.0 // indirect
go.opentelemetry.io/otel/metric v1.45.0 // indirect
go.opentelemetry.io/otel/trace v1.45.0 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/net v0.60.0
golang.org/x/sys v0.48.0 // indirect
golang.org/x/text v0.42.0
golang.org/x/time v0.15.0
gopkg.in/warnings.v0 v0.1.2 // indirect
lukechampine.com/blake3 v1.4.1 // indirect
modernc.org/libc v1.67.4 // indirect
+107 -151
View File
File diff suppressed because it is too large. Load diff
+44
View File
@@ -7,6 +7,7 @@ import (
"encoding/hex"
"errors"
"fmt"
"github.com/GopeedLab/gopeed/internal/production"
"io"
"net"
"net/http"
@@ -61,6 +62,7 @@ type Metadata struct {
}
type Source struct {
producer production.Reader
ID string
URL string
ContentType string
@@ -359,11 +361,35 @@ func (r *Registry) ServeHTTP(w http.ResponseWriter, req *http.Request) {
return
}
defer reader.Close()
if producer, ok := reader.(production.Reader); ok {
src.mu.Lock()
src.producer = producer
src.mu.Unlock()
if err := producer.WaitProduction(req.Context()); err != nil {
return
}
if producer.Production() != nil {
meta.Size = 0
meta.Range = false
start = 0
end = -1
ranged = false
src.mu.Lock()
src.size = 0
src.rangeEnabled = false
src.mu.Unlock()
}
}
writeHeaders(w, meta, start, end, ranged)
if ranged {
w.WriteHeader(http.StatusPartialContent)
}
if producer, ok := reader.(production.Reader); ok && producer.Production() != nil {
if flusher, ok := w.(http.Flusher); ok {
flusher.Flush()
}
}
limit := int64(-1)
if end >= start {
limit = end - start + 1
@@ -675,3 +701,21 @@ func randomID(size int) (string, error) {
}
return hex.EncodeToString(buf), nil
}
// Production returns the active opener's input counter, if it generates output.
func (r *Registry) Production(raw string) production.Source {
if r == nil {
return nil
}
src, err := r.get(raw)
if err != nil {
return nil
}
src.mu.Lock()
reader := src.producer
src.mu.Unlock()
if reader == nil {
return nil
}
return reader.Production()
}
+10 -2
View File
@@ -1,7 +1,9 @@
package controller
import (
"github.com/GopeedLab/gopeed/internal/tempfiles"
"github.com/GopeedLab/gopeed/pkg/base"
"github.com/rs/zerolog"
"net/http"
"net/url"
"os"
@@ -9,8 +11,14 @@ import (
)
type Controller struct {
GetConfig func(v any)
GetProxy func(requestProxy *base.RequestProxy) func(*http.Request) (*url.URL, error)
TempFiles *tempfiles.Scope
ManagedProduction func(string) bool
TempDir string
GetConfig func(v any)
GetProxy func(requestProxy *base.RequestProxy) func(*http.Request) (*url.URL, error)
// Logger is the downloader's core logger. It may be nil, in which case
// fetchers log nothing.
Logger *zerolog.Logger
FileController
//ContextDialer() (proxy.Dialer, error)
}
+8 -2
View File
@@ -12,7 +12,7 @@ import (
// Each download task will have a corresponding Fetcher instance for the management of the download task
type Fetcher interface {
Setup(ctl *controller.Controller)
Resolve(req *base.Request, opts *base.Options) error
Resolve(req *base.Request) error
Start() error
// Patch modifies task-specific data based on the protocol.
// For HTTP: can modify Request info (URL, headers, etc.)
@@ -106,7 +106,13 @@ func (s *SchemeFilter) Match(uri string) bool {
case FilterTypeUrl:
return strings.HasPrefix(uriUpper, patternUpper+":")
case FilterTypeFile:
return strings.HasSuffix(uriUpper, "."+patternUpper)
// Strip query and fragment before matching so URLs like
// https://example.com/video.m3u8?sign=abc still route by extension.
clean := uri
if idx := strings.IndexAny(clean, "?#"); idx >= 0 {
clean = clean[:idx]
}
return strings.HasSuffix(strings.ToUpper(clean), "."+patternUpper)
case FilterTypeBase64:
return strings.HasPrefix(uriUpper, "DATA:"+patternUpper+";BASE64,")
}
+72
View File
@@ -0,0 +1,72 @@
package fetcher
import (
"context"
"encoding/json"
"errors"
"time"
)
type TriggerSource string
const (
TriggerScheduled TriggerSource = "scheduled"
TriggerManual TriggerSource = "manual"
)
var ErrJobSkipped = errors.New("protocol job skipped")
var ErrConfigChanged = errors.New("protocol configuration changed during execution")
// ConfigState embeds the common backend-owned state section in a protocol config.
// Public config PUTs always retain this section from the latest stored config.
type ConfigState[T any] struct {
State T `json:"state"`
}
// ProtocolHost is scoped to one manager. UpdateState runs against the latest
// config under Downloader's lock; callbacks must not perform network I/O.
type ProtocolHost struct {
Read func(any) error
UpdateState func(func(config, state json.RawMessage) (any, error)) error
Fetch func(context.Context, string, int64, func([]byte) error) ([]byte, error)
}
func ReadConfig[T any](host ProtocolHost) (*T, error) {
var cfg T
if err := host.Read(&cfg); err != nil {
return nil, err
}
return &cfg, nil
}
// UpdateState gives the callback a detached config snapshot and mutable state.
// Only the state is committed; edits to the config snapshot are discarded.
func UpdateState[C, S any](host ProtocolHost, update func(*C, *S) error) error {
return host.UpdateState(func(rawConfig, rawState json.RawMessage) (any, error) {
var cfg C
var state S
if err := json.Unmarshal(rawConfig, &cfg); err != nil {
return nil, err
}
if err := json.Unmarshal(rawState, &state); err != nil {
return nil, err
}
if err := update(&cfg, &state); err != nil {
return nil, err
}
return &state, nil
})
}
type CronJob struct {
Name string
Every time.Duration
Timeout time.Duration
Run func(context.Context, TriggerSource) error
}
type CronProvider interface{ CronJobs(ProtocolHost) []CronJob }
// ConfigNormalizer is optional. It owns protocol validation and derived fields.
type ConfigNormalizer interface {
NormalizeConfig(json.RawMessage) (any, error)
}
+73
View File
@@ -0,0 +1,73 @@
# GitHub mirror execution
This package owns GitHub URL rewriting and execution strategies. It depends only
on the standard library and does not depend on Downloader, HTTP clients, Git, or
storage. Downloader persists `Config` at its top-level `githubMirror` field;
`pkg/base` preserves the existing configuration names through aliases.
## Race: resource reads
```go
body, err := githubmirror.Race(ctx, cfg.GithubMirror, rawURL,
githubmirror.Options[[]byte]{
Kind: githubmirror.Resource,
Timeout: 30 * time.Second,
},
func(ctx context.Context, candidate string) ([]byte, error) {
return fetchAndValidate(ctx, client, candidate)
},
)
```
The callback owns HTTP requests, size limits, and content validation. The first
completed valid result wins. Other operations are canceled and drained. There is
no HEAD-probe assumption or client-specific HTTP behavior inside the executor.
## Fallback: repository operations
```go
repo, err := githubmirror.Fallback(ctx, cfg.GithubMirror, officialURL,
githubmirror.Options[*git.Repository]{
Kind: githubmirror.Repository,
},
func(ctx context.Context, candidate string) (*git.Repository, error) {
// Use a clean temporary checkout for each attempt.
return clone(ctx, candidate)
},
)
```
The official URL is attempted first; mirrors follow their configured order.
jsDelivr is excluded for repositories. Stop on the first success. Perform
persistent installation or other business state changes after success, rather
than inside a potentially repeated attempt.
## Results with resources
For temporary files, set `Options[*Download].Discard` to remove the file. Return
the partial `*Download` alongside errors, so cleanup also covers failed reads or
validation. The executor discards every failure and every race loser exactly
once. The caller owns and must track/release the winning result.
- `Timeout == 0` uses 30 seconds per resource attempt and no overall deadline
for repository attempts; a negative timeout is invalid. Extension Git HTTP
transport limits connections/TLS handshakes to 10 seconds and resets a 60-second
read deadline as data arrives. Slow continuous transfers can finish, while
stalled sources fail and allow ordered fallback. Parent cancellation still applies.
- Attempts must honor the supplied context; draining waits for them to return.
- Attempt contexts are canceled after return, so callbacks must finish reading
HTTP bodies before returning. Returning a still-streaming response is unsupported.
- Discard callbacks must accept zero values, must not panic, and run serially.
- Parent cancellation wins over a result selected while the parent is canceled;
the selected result is then discarded as well.
- Errors preserve their causes through wrapping/joining for `errors.Is`.
- `github.com` and `raw.githubusercontent.com` expand into eligible mirrors.
jsDelivr is only eligible for raw source resources. `api.github.com` is never
mirrored and is rejected by the public mirror candidates API.
- Client application update checks request the official GitHub API directly,
falling back to the independent Gopeed website API on request or validation
failure. Only package downloads obtain URLs from
`POST /api/v1/mirrors/github/candidates`. Candidate generation never performs
HTTP requests.
- Disabled mirrors and other URLs execute only the original URL.
- Config is a caller-owned immutable snapshot for the duration of a call.
+90
View File
@@ -0,0 +1,90 @@
// Package githubmirror rewrites GitHub URLs and executes mirrored operations.
package githubmirror
import (
"fmt"
"net/url"
"strings"
)
type Mirror struct {
Type string `json:"type"`
URL string `json:"url"`
IsBuiltIn bool `json:"isBuiltIn"`
IsDeleted bool `json:"isDeleted"`
}
type Config struct {
Enabled bool `json:"enabled"`
Mirrors []Mirror `json:"mirrors"`
}
func DefaultConfig() *Config {
return &Config{Enabled: true, Mirrors: []Mirror{
{Type: "jsdelivr", URL: "https://fastly.jsdelivr.net/gh", IsBuiltIn: true},
{Type: "ghProxy", URL: "https://fastgit.cc", IsBuiltIn: true},
{Type: "ghProxy", URL: "https://gh-proxy.com", IsBuiltIn: true},
{Type: "ghProxy", URL: "https://ghfast.top", IsBuiltIn: true},
}}
}
func (c *Config) Validate() error {
if len(c.Mirrors) > 32 {
return fmt.Errorf("too many GitHub mirrors")
}
for _, mirror := range c.Mirrors {
u, err := url.Parse(mirror.URL)
if err != nil || u == nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" || u.User != nil || u.RawQuery != "" || u.Fragment != "" {
return fmt.Errorf("invalid GitHub mirror URL")
}
if mirror.Type != "jsdelivr" && mirror.Type != "ghProxy" {
return fmt.Errorf("invalid GitHub mirror type: %s", mirror.Type)
}
}
return nil
}
// Kind describes what a candidate endpoint must support.
type Kind uint8
const (
Resource Kind = iota
Repository
)
// Candidates preserves the official URL first. jsDelivr is only eligible for
// raw source resources; Git operations use mirrors supporting Git Smart HTTP.
// GitHub API URLs are never mirrored.
func Candidates(c *Config, raw string, kind Kind) []string {
result := []string{raw}
u, err := url.Parse(raw)
if err != nil || c == nil || !c.Enabled || u.User != nil || (u.Scheme != "https" && u.Scheme != "http") {
return result
}
if u.Host != "github.com" && u.Host != "raw.githubusercontent.com" {
return result
}
seen := map[string]bool{raw: true}
for _, mirror := range c.Mirrors {
if mirror.IsDeleted {
continue
}
var candidate string
switch mirror.Type {
case "ghProxy":
candidate = strings.TrimRight(mirror.URL, "/") + "/" + raw
case "jsdelivr":
if kind == Repository || u.Host != "raw.githubusercontent.com" {
continue
}
parts := strings.SplitN(strings.TrimPrefix(u.EscapedPath(), "/"), "/", 4)
if len(parts) != 4 {
continue
}
candidate = strings.TrimRight(mirror.URL, "/") + "/" + parts[0] + "/" + parts[1] + "@" + parts[2] + "/" + parts[3]
}
if candidate != "" && !seen[candidate] {
result = append(result, candidate)
seen[candidate] = true
}
}
return result
}
+38
View File
@@ -0,0 +1,38 @@
package githubmirror
import (
"reflect"
"testing"
)
func TestMirrorCandidates(t *testing.T) {
cfg := &Config{Enabled: true, Mirrors: []Mirror{
{Type: "jsdelivr", URL: "https://cdn.example/gh/"},
{Type: "ghProxy", URL: "https://mirror.example/"},
{Type: "ghProxy", URL: "https://deleted.example", IsDeleted: true},
}}
source := "https://raw.githubusercontent.com/owner/repo/main/a.txt"
want := []string{source, "https://cdn.example/gh/owner/repo@main/a.txt", "https://mirror.example/" + source}
if got := Candidates(cfg, source, Resource); !reflect.DeepEqual(got, want) {
t.Fatalf("got %v", got)
}
repo := "https://github.com/owner/repo.git"
if got := Candidates(cfg, repo, Repository); !reflect.DeepEqual(got, []string{repo, "https://mirror.example/" + repo}) {
t.Fatalf("git candidates %v", got)
}
for _, raw := range []string{"https://github.com.evil.example/a", "https://example.com/a", "https://user:[email protected]/a"} {
if got := Candidates(cfg, raw, Resource); !reflect.DeepEqual(got, []string{raw}) {
t.Fatalf("unexpected mirror: %v", got)
}
}
apiURL := "https://api.github.com/repos/GopeedLab/gopeed/releases?per_page=10"
for _, kind := range []Kind{Resource, Repository} {
if got := Candidates(cfg, apiURL, kind); !reflect.DeepEqual(got, []string{apiURL}) {
t.Fatalf("GitHub API must never use mirrors: %v", got)
}
}
cfg.Enabled = false
if got := Candidates(cfg, source, Resource); !reflect.DeepEqual(got, []string{source}) {
t.Fatalf("disabled mirror: %v", got)
}
}
+160
View File
@@ -0,0 +1,160 @@
package githubmirror
import (
"context"
"errors"
"fmt"
"time"
)
// Options configures execution, not URL persistence. Discard releases results
// from failed attempts and successful race losers, including partial results
// returned together with an error. The caller owns the winning result.
type Options[T any] struct {
Kind Kind
// Timeout limits each attempt. Zero uses 30 seconds for resources and no
// overall deadline for repositories, whose transport owns idle timeouts.
Timeout time.Duration
Discard func(T)
}
// Attempt must honor ctx and return a completed, validated result. Its context
// is canceled after it returns, so live response streams must not be returned.
// Return any partially acquired resources along with errors so Discard can
// release them. Failed attempts must not mutate persistent business state.
type Attempt[T any] func(context.Context, string) (T, error)
const defaultTimeout = 30 * time.Second
func prepare[T any](ctx context.Context, config *Config, raw string, options Options[T], attempt Attempt[T]) ([]string, time.Duration, error) {
if err := ctx.Err(); err != nil {
return nil, 0, err
}
if raw == "" {
return nil, 0, errors.New("empty GitHub source URL")
}
if options.Kind != Resource && options.Kind != Repository {
return nil, 0, errors.New("invalid GitHub resource kind")
}
if options.Timeout < 0 {
return nil, 0, errors.New("invalid GitHub attempt timeout")
}
if attempt == nil {
return nil, 0, errors.New("GitHub mirror attempt is required")
}
if config != nil {
if err := config.Validate(); err != nil {
return nil, 0, err
}
}
timeout := options.Timeout
if timeout == 0 && options.Kind == Resource {
timeout = defaultTimeout
}
return Candidates(config, raw, options.Kind), timeout, nil
}
func invoke[T any](ctx context.Context, timeout time.Duration, candidate string, attempt Attempt[T]) (value T, err error) {
var cancel context.CancelFunc
if timeout > 0 {
ctx, cancel = context.WithTimeout(ctx, timeout)
} else {
ctx, cancel = context.WithCancel(ctx)
}
defer cancel()
defer func() {
if recovered := recover(); recovered != nil {
err = fmt.Errorf("GitHub mirror attempt panicked: %v", recovered)
}
}()
value, err = attempt(ctx, candidate)
if err == nil {
err = ctx.Err()
}
return
}
func discard[T any](options Options[T], value T) {
if options.Discard != nil {
options.Discard(value)
}
}
// Race executes the official URL and eligible mirrors concurrently. The first
// completed successful result wins; losing attempts are canceled and drained
// before return so no temporary files or other resources outlive the call.
// HTTP status/content validation is the attempt's responsibility.
func Race[T any](ctx context.Context, config *Config, raw string, options Options[T], attempt Attempt[T]) (T, error) {
var zero T
candidates, timeout, err := prepare(ctx, config, raw, options, attempt)
if err != nil {
return zero, err
}
raceCtx, cancel := context.WithCancel(ctx)
defer cancel()
type outcome struct {
index int
value T
err error
}
outcomes := make(chan outcome, len(candidates))
for index, candidate := range candidates {
go func(index int, candidate string) {
value, err := invoke(raceCtx, timeout, candidate, attempt)
outcomes <- outcome{index, value, err}
}(index, candidate)
}
var winner T
won := false
failures := make([]error, len(candidates))
for range candidates {
outcome := <-outcomes
if !won && outcome.err == nil && ctx.Err() == nil {
winner = outcome.value
won = true
cancel()
} else {
discard(options, outcome.value)
if outcome.err != nil {
failures[outcome.index] = fmt.Errorf("GitHub source %d: %w", outcome.index+1, outcome.err)
}
if ctx.Err() != nil {
cancel()
}
}
}
if err := ctx.Err(); err != nil {
if won {
discard(options, winner)
}
return zero, err
}
if won {
return winner, nil
}
return zero, fmt.Errorf("all GitHub sources failed: %w", errors.Join(failures...))
}
// Fallback executes the official URL first, then eligible mirrors in their
// configured order. It stops immediately on success and cleans every failure.
func Fallback[T any](ctx context.Context, config *Config, raw string, options Options[T], attempt Attempt[T]) (T, error) {
var zero T
candidates, timeout, err := prepare(ctx, config, raw, options, attempt)
if err != nil {
return zero, err
}
failures := make([]error, 0, len(candidates))
for index, candidate := range candidates {
if err := ctx.Err(); err != nil {
return zero, err
}
value, err := invoke(ctx, timeout, candidate, attempt)
if err == nil && ctx.Err() == nil {
return value, nil
}
discard(options, value)
if parentErr := ctx.Err(); parentErr != nil {
return zero, parentErr
}
failures = append(failures, fmt.Errorf("GitHub source %d: %w", index+1, err))
}
return zero, fmt.Errorf("all GitHub sources failed: %w", errors.Join(failures...))
}
+214
View File
@@ -0,0 +1,214 @@
package githubmirror
import (
"context"
"errors"
"reflect"
"strings"
"sync"
"testing"
"time"
)
const testSource = "https://github.com/owner/repo.git"
func testConfig() *Config {
return &Config{Enabled: true, Mirrors: []Mirror{
{Type: "ghProxy", URL: "https://first.example"},
{Type: "ghProxy", URL: "https://second.example"},
}}
}
func TestRaceWinsAndDrainsOwnedResults(t *testing.T) {
config := testConfig()
candidates := Candidates(config, testSource, Resource)
var started sync.WaitGroup
started.Add(len(candidates))
discarded := []string{}
winner, err := Race(context.Background(), config, testSource, Options[string]{
Discard: func(value string) { discarded = append(discarded, value) },
}, func(ctx context.Context, address string) (string, error) {
started.Done()
started.Wait()
switch address {
case candidates[0]:
return address, errors.New("official offline")
case candidates[1]:
return address, nil
default:
<-ctx.Done()
return address, ctx.Err()
}
})
if err != nil || winner != candidates[1] {
t.Fatalf("winner=%s err=%v", winner, err)
}
if len(discarded) != 2 || !contains(discarded, candidates[0]) || !contains(discarded, candidates[2]) {
t.Fatalf("discarded %v", discarded)
}
}
func contains(values []string, value string) bool {
for _, item := range values {
if item == value {
return true
}
}
return false
}
func TestRaceCleansSimultaneousSuccessfulLosers(t *testing.T) {
config := testConfig()
var started sync.WaitGroup
started.Add(3)
cleaned := []string{}
winner, err := Race(context.Background(), config, testSource, Options[string]{Discard: func(value string) { cleaned = append(cleaned, value) }}, func(_ context.Context, address string) (string, error) {
started.Done()
started.Wait()
return address, nil
})
if err != nil || winner == "" || len(cleaned) != 2 || contains(cleaned, winner) {
t.Fatalf("winner=%s discarded=%v err=%v", winner, cleaned, err)
}
}
func TestRaceParentCancellationCleansWinnerToo(t *testing.T) {
config := testConfig()
config.Mirrors = config.Mirrors[:1]
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
cleaned := []string{}
winner, err := Race(ctx, config, testSource, Options[string]{Discard: func(value string) { cleaned = append(cleaned, value) }}, func(ctx context.Context, address string) (string, error) {
if address == testSource {
return address, nil
}
<-ctx.Done()
cancel()
return address, ctx.Err()
})
if !errors.Is(err, context.Canceled) || winner != "" || len(cleaned) != 2 || !contains(cleaned, testSource) {
t.Fatalf("winner=%s cleanup=%v err=%v", winner, cleaned, err)
}
}
func TestRaceTimeoutAndErrorAggregation(t *testing.T) {
expected := errors.New("offline")
config := testConfig()
config.Mirrors = config.Mirrors[:1]
discarded := 0
_, err := Race(context.Background(), config, testSource, Options[string]{Timeout: 10 * time.Millisecond, Discard: func(string) { discarded++ }}, func(ctx context.Context, address string) (string, error) {
if address == testSource {
return address, expected
}
<-ctx.Done()
return address, ctx.Err()
})
if !errors.Is(err, expected) || !errors.Is(err, context.DeadlineExceeded) || discarded != 2 {
t.Fatalf("discarded=%d err=%v", discarded, err)
}
}
func TestFallbackPreservesOrderAndStopsOnSuccess(t *testing.T) {
config := testConfig()
candidates := Candidates(config, testSource, Repository)
tried := []string{}
cleaned := []string{}
winner, err := Fallback(context.Background(), config, testSource, Options[string]{Kind: Repository, Discard: func(value string) { cleaned = append(cleaned, value) }}, func(_ context.Context, address string) (string, error) {
tried = append(tried, address)
if address == candidates[1] {
return address, nil
}
return address, errors.New("offline")
})
if err != nil || winner != candidates[1] || !reflect.DeepEqual(tried, candidates[:2]) || !reflect.DeepEqual(cleaned, candidates[:1]) {
t.Fatalf("winner=%s tried=%v discarded=%v err=%v", winner, tried, cleaned, err)
}
tried = nil
_, err = Fallback(context.Background(), config, testSource, Options[string]{Kind: Repository}, func(_ context.Context, address string) (string, error) {
tried = append(tried, address)
return address, nil
})
if err != nil || !reflect.DeepEqual(tried, candidates[:1]) {
t.Fatalf("tried=%v err=%v", tried, err)
}
}
func TestFallbackTimeoutContinuesWithFreshContext(t *testing.T) {
discarded := []string{}
winner, err := Fallback(context.Background(), testConfig(), testSource, Options[string]{Kind: Repository, Timeout: 10 * time.Millisecond, Discard: func(value string) { discarded = append(discarded, value) }}, func(ctx context.Context, address string) (string, error) {
if address == testSource {
<-ctx.Done()
return address, ctx.Err()
}
if err := ctx.Err(); err != nil {
return address, err
}
return address, nil
})
if err != nil || !strings.HasPrefix(winner, "https://first.example/") || !reflect.DeepEqual(discarded, []string{testSource}) {
t.Fatalf("winner=%s cleanup=%v err=%v", winner, discarded, err)
}
}
func TestFallbackCancellationStopsBeforeNextMirror(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
calls := 0
cleaned := 0
_, err := Fallback(ctx, testConfig(), testSource, Options[string]{Kind: Repository, Discard: func(string) { cleaned++ }}, func(ctx context.Context, address string) (string, error) {
calls++
cancel()
return address, ctx.Err()
})
if !errors.Is(err, context.Canceled) || calls != 1 || cleaned != 1 {
t.Fatalf("calls=%d discarded=%d err=%v", calls, cleaned, err)
}
}
func TestExecutorsDoNotStartInvalidOrCanceledRequests(t *testing.T) {
canceled, cancel := context.WithCancel(context.Background())
cancel()
for _, test := range []struct {
ctx context.Context
config *Config
options Options[string]
attempt Attempt[string]
}{
{ctx: canceled, options: Options[string]{}, attempt: func(context.Context, string) (string, error) { t.Fatal("called after cancellation"); return "", nil }},
{ctx: context.Background(), options: Options[string]{Kind: 99}, attempt: func(context.Context, string) (string, error) { t.Fatal("invalid kind"); return "", nil }},
{ctx: context.Background(), options: Options[string]{Timeout: -time.Second}, attempt: func(context.Context, string) (string, error) { t.Fatal("invalid timeout"); return "", nil }},
{ctx: context.Background(), options: Options[string]{}, attempt: nil},
{ctx: context.Background(), config: &Config{Mirrors: []Mirror{{Type: "invalid", URL: "https://mirror.example"}}}, attempt: func(context.Context, string) (string, error) { t.Fatal("invalid config"); return "", nil }},
} {
if _, err := Race(test.ctx, test.config, testSource, test.options, test.attempt); err == nil {
t.Fatal("invalid race accepted")
}
if _, err := Fallback(test.ctx, test.config, testSource, test.options, test.attempt); err == nil {
t.Fatal("invalid fallback accepted")
}
}
}
func TestAttemptPanicBecomesError(t *testing.T) {
attempt := func(context.Context, string) (string, error) { panic("broken operation") }
if _, err := Race(context.Background(), nil, testSource, Options[string]{}, attempt); err == nil || !strings.Contains(err.Error(), "broken operation") {
t.Fatal(err)
}
if _, err := Fallback(context.Background(), nil, testSource, Options[string]{}, attempt); err == nil || !strings.Contains(err.Error(), "broken operation") {
t.Fatal(err)
}
}
func TestNonGithubResourcesAndDisabledMirrorsUseOnlyOriginal(t *testing.T) {
for _, test := range []struct {
config *Config
raw string
}{{testConfig(), "https://example.com/resource"}, {&Config{Enabled: false, Mirrors: testConfig().Mirrors}, testSource}} {
calls := 0
got, err := Race(context.Background(), test.config, test.raw, Options[string]{}, func(_ context.Context, address string) (string, error) { calls++; return address, nil })
if err != nil || got != test.raw || calls != 1 {
t.Fatalf("got=%s calls=%d err=%v", got, calls, err)
}
}
}
func TestRepositoryAttemptsHaveNoDefaultDeadline(t *testing.T) {
value, err := Fallback(context.Background(), nil, testSource, Options[string]{Kind: Repository}, func(ctx context.Context, raw string) (string, error) {
if _, ok := ctx.Deadline(); ok {
t.Fatal("repository attempt has a total deadline")
}
return raw, nil
})
if err != nil || value != testSource {
t.Fatalf("value=%s error=%v", value, err)
}
}
+35
View File
@@ -0,0 +1,35 @@
// Package production associates generated resources with their input progress.
package production
import (
"context"
"sync"
)
// Source counts unique input bytes and total received bytes (including refetches).
// Implementations must support concurrent snapshots.
type Source interface {
Progress() (downloaded, received int64)
}
// Reader resolves its optional producer before response headers are sent.
// A nil producer preserves ordinary Blob behavior.
type Reader interface {
Production() Source
WaitProduction(context.Context) error
}
type Registry struct {
mu sync.Mutex
sources map[string]Source
}
func (r *Registry) Set(id string, source Source) {
r.mu.Lock()
defer r.mu.Unlock()
if r.sources == nil {
r.sources = make(map[string]Source)
}
r.sources[id] = source
}
func (r *Registry) Get(id string) Source { r.mu.Lock(); defer r.mu.Unlock(); return r.sources[id] }
func (r *Registry) Remove(id string) { r.mu.Lock(); defer r.mu.Unlock(); delete(r.sources, id) }
+27 -2
View File
@@ -1,12 +1,37 @@
package bt
import (
"time"
"github.com/GopeedLab/gopeed/internal/fetcher"
)
type trackerSubscriptionStatus struct {
Success bool `json:"success"`
TrackerCount int `json:"trackerCount"`
LastFetchTime time.Time `json:"lastFetchTime"`
}
type state struct {
TrackerSubscriptionStatuses map[string]trackerSubscriptionStatus `json:"trackerSubscriptionStatuses"`
TrackerSources map[string][]string `json:"trackerSources"`
LastTrackerUpdateTime time.Time `json:"lastTrackerUpdateTime"`
}
type config struct {
ListenPort int `json:"listenPort"`
Trackers []string `json:"trackers"`
fetcher.ConfigState[state]
TrackerSubscribeURLs []string `json:"trackerSubscribeUrls"`
AutoUpdateTrackers bool `json:"autoUpdateTrackers"`
CustomTrackers []string `json:"customTrackers"`
ListenPort int `json:"listenPort"`
// SeedKeep is always keep seeding after downloading is complete, unless manually stopped.
SeedKeep bool `json:"seedKeep"`
// SeedRatio is the ratio of uploaded data to downloaded data to seed.
SeedRatio float64 `json:"seedRatio"`
// SeedTime is the time in seconds to seed after downloading is complete.
SeedTime int64 `json:"seedTime"`
// DisableDHT and DisablePEX turn off DHT and PEX peer discovery. Like ListenPort,
// they are read when the shared client is built.
DisableDHT bool `json:"disableDht"`
DisablePEX bool `json:"disablePex"`
}
+464
View File
@@ -0,0 +1,464 @@
package bt
import (
"encoding/json"
"net/url"
"os"
"path/filepath"
"strings"
"sync/atomic"
"testing"
"time"
"github.com/GopeedLab/gopeed/internal/controller"
"github.com/GopeedLab/gopeed/internal/fetcher"
"github.com/GopeedLab/gopeed/pkg/base"
btprotocol "github.com/GopeedLab/gopeed/pkg/protocol/bt"
"github.com/anacrolix/torrent"
pp "github.com/anacrolix/torrent/peer_protocol"
"golang.org/x/time/rate"
)
type btDownloadHarness struct {
*Fetcher
pendingRequests atomic.Int64
}
// These tests exercise the BT fetcher, native storage and real peer protocol.
// All content is generated locally and served by a loopback seeder.
func btDownloadFetcher(t *testing.T, f fetcher.Fetcher) *btDownloadHarness {
t.Helper()
if f == nil {
f = new(FetcherManager).Build()
}
ctl := controller.NewController()
ctl.GetConfig = func(v any) {
encoded, err := json.Marshal(new(FetcherManager).DefaultConfig())
if err != nil {
t.Fatal(err)
}
if err := json.Unmarshal(encoded, v); err != nil {
t.Fatal(err)
}
}
f.Setup(ctl)
t.Cleanup(func() {
if err := f.Close(); err != nil {
t.Error(err)
}
if err := closeClient(); err != nil {
t.Error(err)
}
})
h := &btDownloadHarness{Fetcher: f.(*Fetcher)}
// Keep the test client on loopback and observe real cancellation replies.
// Waiting for them avoids the pinned library's immediate cancel/resume
// request scheduling bug, without relying on arbitrary sleep durations.
cfg = torrent.TestingConfig(t)
cfg.DisableTrackers = false
cfg.Seed = true
cfg.DisableUTP = true
cfg.DisableIPv6 = true
cfg.DisableWebtorrent = true
cfg.KeepAliveTimeout = time.Minute
cfg.MaxAllocPeerRequestDataPerConn = 1 << 20
cfg.Callbacks.SentRequest = []func(torrent.PeerRequestEvent){func(torrent.PeerRequestEvent) {
h.pendingRequests.Add(1)
}}
cfg.Callbacks.ReadMessage = func(_ *torrent.PeerConn, msg *pp.Message) {
if !msg.Keepalive && (msg.Type == pp.Piece || msg.Type == pp.Reject) {
h.pendingRequests.Add(-1)
}
}
var err error
client, err = torrent.NewClient(cfg)
if err != nil {
t.Fatal(err)
}
return h
}
func btResolve(t *testing.T, f *btDownloadHarness, source string) {
t.Helper()
if err := f.Resolve(&base.Request{URL: source}); err != nil {
t.Fatal(err)
}
}
func btStart(t *testing.T, f *btDownloadHarness) {
t.Helper()
if err := f.Start(); err != nil {
t.Fatal(err)
}
}
func btAwaitDone(t *testing.T, f *btDownloadHarness) {
t.Helper()
done := make(chan error, 1)
go func() { done <- f.Wait() }()
timeout := time.NewTimer(20 * time.Second)
defer timeout.Stop()
select {
case err := <-done:
if err != nil {
t.Fatalf("BT download failed: %v", err)
}
if !f.isDone() {
t.Fatal("Wait returned before selected files were verified")
}
case <-timeout.C:
var status strings.Builder
client.WriteStatus(&status)
t.Log(status.String())
t.Fatalf("BT download did not finish: progress=%v stats=%+v pieces=%+v peers=%+v", f.Progress(), f.torrent.Stats(), f.torrent.PieceStateRuns(), f.Stats().Runtime)
}
}
func btAwaitPartial(t *testing.T, f *btDownloadHarness) {
t.Helper()
timeout := time.NewTimer(15 * time.Second)
defer timeout.Stop()
ticker := time.NewTicker(20 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-ticker.C:
progress := f.Progress().TotalDownloaded()
if progress >= 32*1024 && progress < f.Meta().Res.Size {
return
}
case <-timeout.C:
t.Fatal("BT transfer did not reach partial progress")
}
}
}
func btPause(t *testing.T, f *btDownloadHarness) {
t.Helper()
if err := f.Pause(); err != nil {
t.Fatal(err)
}
timeout := time.NewTimer(5 * time.Second)
defer timeout.Stop()
ticker := time.NewTicker(10 * time.Millisecond)
defer ticker.Stop()
for {
if f.pendingRequests.Load() == 0 {
// The message callback runs under the client lock. Acquiring that
// lock ensures the final cancellation reply has finished processing.
client.Stats()
return
}
select {
case <-ticker.C:
case <-timeout.C:
t.Fatal("peer did not acknowledge cancelled requests")
}
}
}
func TestBTDownloadSources(t *testing.T) {
for _, kind := range []string{"magnet", "torrent", "file URI", "data URI"} {
t.Run(kind, func(t *testing.T) {
fixture := newBTFixture(t, map[string][]byte{"": btPayload(96*1024, kind)}, 0)
source := fixture.Magnet
switch kind {
case "torrent":
source = fixture.TorrentFile
case "file URI":
source = fixture.FileURI()
case "data URI":
source = fixture.DataURI
}
f := btDownloadFetcher(t, nil)
btResolve(t, f, source)
if f.Meta().Opts != nil {
t.Fatal("Resolve assigned download options")
}
f.storage.mu.RLock()
bound := f.storage.owner != nil
f.storage.mu.RUnlock()
if bound {
t.Fatal("Resolve bound download storage")
}
resolvedTorrent := f.torrent
dir := filepath.Join(t.TempDir(), "chosen-after-resolve")
f.Meta().Opts = &base.Options{Path: dir}
if _, err := os.Stat(dir); !os.IsNotExist(err) {
t.Fatalf("output directory exists before Start: %v", err)
}
btStart(t, f)
if f.torrent != resolvedTorrent {
t.Fatal("Start replaced the resolved torrent")
}
btAwaitDone(t, f)
fixture.AssertFiles(t, dir)
})
}
}
func TestBTDirectDownloadWithExtraTrackers(t *testing.T) {
for _, source := range []string{"request", "config", "subscription"} {
t.Run(source, func(t *testing.T) {
fixture := makeBTFixture(t, map[string][]byte{
"a.bin": btPayload(20*1024, "direct-a"),
"nested/b.bin": btPayload(48*1024, "direct-b"),
}, 0, false, false)
magnet, err := url.Parse(fixture.Magnet)
if err != nil {
t.Fatal(err)
}
query := magnet.Query()
query.Del("tr")
query.Del("x.pe")
magnet.RawQuery = query.Encode()
req := &base.Request{URL: magnet.String()}
f := btDownloadFetcher(t, nil)
if source == "request" {
req.Extra = &btprotocol.ReqExtra{Trackers: []string{fixture.TrackerURL}}
} else {
backendConfig := new(FetcherManager).DefaultConfig().(*config)
if source == "config" {
backendConfig.CustomTrackers = []string{fixture.TrackerURL}
} else {
backendConfig.TrackerSubscribeURLs = []string{"https://subscription.example/list"}
backendConfig.State.TrackerSources[backendConfig.TrackerSubscribeURLs[0]] = []string{fixture.TrackerURL}
}
f.ctl.GetConfig = func(dst any) {
raw, err := json.Marshal(backendConfig)
if err != nil {
t.Error(err)
return
}
if err := json.Unmarshal(raw, dst); err != nil {
t.Error(err)
}
}
}
// CreateDirect supplies options before Downloader resolves the task.
dir := filepath.Join(t.TempDir(), "direct-download")
opts := &base.Options{Path: dir, SelectFiles: []int{}}
f.Meta().Opts = opts
resolved := make(chan error, 1)
go func() { resolved <- f.Resolve(req) }()
select {
case err := <-resolved:
if err != nil {
t.Fatal(err)
}
case <-time.After(5 * time.Second):
f.torrentDropFunc()
<-resolved
t.Fatal("direct magnet download did not obtain metadata from its extra tracker")
}
if f.Meta().Opts != opts || len(opts.SelectFiles) != len(fixture.Files) {
t.Fatal("resolution lost direct download options or did not select all files")
}
btStart(t, f)
btAwaitDone(t, f)
fixture.AssertFiles(t, dir)
})
}
}
func TestBTDownloadSelectedFiles(t *testing.T) {
fixture := newBTFixture(t, map[string][]byte{
"a.bin": btPayload(20*1024, "a"), "nested/b.bin": btPayload(48*1024, "b"),
}, 0)
f := btDownloadFetcher(t, nil)
dir := t.TempDir()
btResolve(t, f, fixture.Magnet)
selected := -1
for i, file := range f.Meta().Res.Files {
if file.Name == "b.bin" {
selected = i
}
}
if selected < 0 {
t.Fatal("selected file absent from metadata")
}
f.Meta().Opts = &base.Options{Path: dir}
if err := f.Patch(nil, &base.Options{SelectFiles: []int{selected}}); err != nil {
t.Fatal(err)
}
btStart(t, f)
btAwaitDone(t, f)
fixture.AssertFiles(t, dir, filepath.Join("bundle", "nested", "b.bin"))
if _, err := os.Stat(filepath.Join(dir, "bundle", "a.bin")); !os.IsNotExist(err) {
t.Fatalf("unselected output remains: %v", err)
}
if f.Meta().Res.Size != 48*1024 {
t.Fatal("selected resource size is incorrect")
}
}
func TestBTPauseAndResume(t *testing.T) {
fixture := newBTFixture(t, map[string][]byte{"": btPayload(512*1024, "pause")}, 64*1024)
f := btDownloadFetcher(t, nil)
dir := t.TempDir()
btResolve(t, f, fixture.Magnet)
f.Meta().Opts = &base.Options{Path: dir}
btStart(t, f)
btAwaitPartial(t, f)
btPause(t, f)
resolvedTorrent := f.torrent
fixture.Limiter.SetLimit(rate.Inf)
// Once cancelled requests have settled, no more data should arrive.
stats := f.torrent.Stats()
readWhilePaused := stats.BytesReadData.Int64()
time.Sleep(200 * time.Millisecond)
stats = f.torrent.Stats()
if stats.BytesReadData.Int64() != readWhilePaused || f.isDone() {
t.Fatal("data download continued while paused")
}
btStart(t, f)
if f.torrent != resolvedTorrent {
t.Fatal("resume replaced the resolved torrent")
}
btAwaitDone(t, f)
fixture.AssertFiles(t, dir)
}
type btFetcherCheckpoint struct {
Meta *fetcher.FetcherMeta `json:"meta"`
Data json.RawMessage `json:"data"`
}
func btStore(t *testing.T, f *btDownloadHarness) []byte {
t.Helper()
data, err := new(FetcherManager).Store(f.Fetcher)
if err != nil {
t.Fatal(err)
}
encodedData, err := json.Marshal(data)
if err != nil {
t.Fatal(err)
}
encoded, err := json.Marshal(btFetcherCheckpoint{Meta: f.Meta(), Data: encodedData})
if err != nil {
t.Fatal(err)
}
return encoded
}
func btRestore(t *testing.T, encoded []byte) *btDownloadHarness {
t.Helper()
var checkpoint btFetcherCheckpoint
if err := json.Unmarshal(encoded, &checkpoint); err != nil {
t.Fatal(err)
}
data, restore := new(FetcherManager).Restore()
if err := json.Unmarshal(checkpoint.Data, data); err != nil {
t.Fatal(err)
}
return btDownloadFetcher(t, restore(checkpoint.Meta, data))
}
func btCloseForRestart(t *testing.T, f *btDownloadHarness) {
t.Helper()
if err := f.Close(); err != nil {
t.Fatal(err)
}
if err := new(FetcherManager).Close(); err != nil {
t.Fatal(err)
}
}
func TestBTColdStartResume(t *testing.T) {
for _, kind := range []string{"v1", "hybrid"} {
t.Run(kind, func(t *testing.T) {
var fixture *btFixture
if kind == "hybrid" {
fixture = newBTHybridFixture(t, btPayload(512*1024, "restart-hybrid"))
fixture.Limiter.SetLimit(64 * 1024)
} else {
fixture = newBTFixture(t, map[string][]byte{"": btPayload(512*1024, "restart")}, 64*1024)
}
f := btDownloadFetcher(t, nil)
dir := t.TempDir()
btResolve(t, f, fixture.Magnet)
f.Meta().Opts = &base.Options{Path: dir}
btStart(t, f)
btAwaitPartial(t, f)
btPause(t, f)
checkpoint := btStore(t, f)
hash := f.Meta().Res.Hash
btCloseForRestart(t, f)
seedStats := fixture.Torrent.Stats()
sentBeforeRestart := seedStats.BytesWrittenData.Int64()
restored := btRestore(t, checkpoint)
if restored.Meta().Opts.Path != dir || restored.Meta().Res.Hash != hash {
t.Fatal("download directory or hash was lost")
}
fixture.Limiter.SetLimit(rate.Inf)
btStart(t, restored)
btAwaitDone(t, restored)
fixture.AssertFiles(t, dir)
seedStats = fixture.Torrent.Stats()
if sent := seedStats.BytesWrittenData.Int64() - sentBeforeRestart; sent >= 512*1024 {
t.Fatalf("restart redownloaded the entire file: %d bytes", sent)
}
})
}
}
func TestBTDownloadHybrid(t *testing.T) {
fixture := newBTHybridFixture(t, btPayload(80*1024, "hybrid-v2"))
f := btDownloadFetcher(t, nil)
dir := t.TempDir()
btResolve(t, f, fixture.Magnet)
f.Meta().Opts = &base.Options{Path: dir}
btStart(t, f)
btAwaitDone(t, f)
fixture.AssertFiles(t, dir)
}
func TestBTRepairCorruptedExistingFile(t *testing.T) {
fixture := newBTFixture(t, map[string][]byte{"": btPayload(128*1024, "repair")}, 0)
dir := t.TempDir()
damaged := append([]byte(nil), fixture.Files["payload.bin"]...)
damaged[20000] ^= 0xff
if err := os.WriteFile(filepath.Join(dir, "payload.bin"), damaged, 0600); err != nil {
t.Fatal(err)
}
f := btDownloadFetcher(t, nil)
btResolve(t, f, fixture.Magnet)
f.Meta().Opts = &base.Options{Path: dir}
btStart(t, f)
btAwaitDone(t, f)
fixture.AssertFiles(t, dir)
}
func TestBTPatchSelectionWhilePaused(t *testing.T) {
fixture := newBTFixture(t, map[string][]byte{"a.bin": btPayload(256*1024, "patch-a"), "b.bin": btPayload(256*1024, "patch-b")}, 64*1024)
f := btDownloadFetcher(t, nil)
dir := t.TempDir()
btResolve(t, f, fixture.Magnet)
f.Meta().Opts = &base.Options{Path: dir, SelectFiles: []int{0}}
btStart(t, f)
btAwaitPartial(t, f)
btPause(t, f)
if err := f.Patch(nil, &base.Options{SelectFiles: []int{1}}); err != nil {
t.Fatal(err)
}
fixture.Limiter.SetLimit(rate.Inf)
btStart(t, f)
btAwaitDone(t, f)
fixture.AssertFiles(t, dir, filepath.Join("bundle", "b.bin"))
for _, suffix := range []string{"", ".part"} {
if _, err := os.Stat(filepath.Join(dir, "bundle", "a.bin") + suffix); !os.IsNotExist(err) {
t.Fatalf("unselected output remains: %v", err)
}
}
}
func TestBTDownloadMultiFileWithEmptyFile(t *testing.T) {
fixture := newBTFixture(t, map[string][]byte{"empty.txt": {}, "nested/data.bin": btPayload(48*1024, "empty-file")}, 0)
f := btDownloadFetcher(t, nil)
dir := t.TempDir()
btResolve(t, f, fixture.Magnet)
f.Meta().Opts = &base.Options{Path: dir}
btStart(t, f)
btAwaitDone(t, f)
fixture.AssertFiles(t, dir)
}
File diff suppressed because it is too large. Load diff
+145 -17
View File
@@ -8,6 +8,7 @@ import (
"os"
"path/filepath"
"reflect"
"sync"
"testing"
"github.com/GopeedLab/gopeed/internal/controller"
@@ -18,11 +19,11 @@ import (
)
func TestFetcher_Resolve_Torrent(t *testing.T) {
doResolve(t, buildFetcher())
doResolve(t, buildFetcher(t))
}
func TestFetcher_Resolve_DataUri_Torrent(t *testing.T) {
fetcher := buildFetcher()
fetcher := buildFetcher(t)
buf, err := os.ReadFile("./testdata/ubuntu-22.04-live-server-amd64.iso.torrent")
if err != nil {
t.Fatal(err)
@@ -31,7 +32,7 @@ func TestFetcher_Resolve_DataUri_Torrent(t *testing.T) {
dataUri := "data:application/x-bittorrent;base64," + base64.StdEncoding.EncodeToString(buf)
err = fetcher.Resolve(&base.Request{
URL: dataUri,
}, nil)
})
if err != nil {
panic(err)
}
@@ -53,7 +54,7 @@ func TestFetcher_Resolve_DataUri_Torrent(t *testing.T) {
}
func TestFetcher_Config(t *testing.T) {
doResolve(t, buildConfigFetcher(nil))
doResolve(t, buildConfigFetcher(t, nil))
}
func TestFetcher_ResolveWithProxy(t *testing.T) {
@@ -61,7 +62,7 @@ func TestFetcher_ResolveWithProxy(t *testing.T) {
proxyListener := test.StartSocks5Server(usr, pwd)
defer proxyListener.Close()
doResolve(t, buildConfigFetcher(&base.DownloaderProxyConfig{
doResolve(t, buildConfigFetcher(t, &base.DownloaderProxyConfig{
Enable: true,
System: false,
Scheme: "socks5",
@@ -71,6 +72,82 @@ func TestFetcher_ResolveWithProxy(t *testing.T) {
}))
}
func TestFetcher_CloseAfterInit(t *testing.T) {
if err := closeClient(); err != nil {
t.Fatal(err)
}
for i := 0; i < 100; i++ {
f := buildFetcher(t).(*Fetcher)
f.meta.Req = &base.Request{}
lock.Lock()
err := f.initClient()
lock.Unlock()
if err != nil {
t.Fatal(err)
}
if err := closeClient(); err != nil {
t.Fatal(err)
}
}
}
func TestFetcher_CloseWithoutClient(t *testing.T) {
if err := closeClient(); err != nil {
t.Fatal(err)
}
t.Run("restored task", func(t *testing.T) {
_, build := new(FetcherManager).Restore()
f := build(&fetcher.FetcherMeta{
Req: &base.Request{URL: "./testdata/test.torrent"},
Opts: &base.Options{},
}, &fetcherData{})
f.Setup(controller.NewController())
if err := f.Close(); err != nil {
t.Errorf("Close() got = %v, want nil", err)
}
})
t.Run("failed resolve", func(t *testing.T) {
f := buildFetcher(t)
if err := f.Resolve(&base.Request{URL: "./testdata/not-exist.torrent"}); err == nil {
t.Fatal("Resolve() got = nil, want error")
}
if err := f.Close(); err != nil {
t.Errorf("Close() got = %v, want nil", err)
}
})
}
func TestFetcher_ResolveWhileClientCloses(t *testing.T) {
if err := closeClient(); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { closeClient() })
var wg sync.WaitGroup
for _, u := range []string{"./testdata/test.torrent", "./testdata/ubuntu-22.04-live-server-amd64.iso.torrent"} {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < 10; i++ {
f := buildFetcher(t)
// The other goroutine may close the client after the torrent was added.
if err := f.Resolve(&base.Request{URL: u}); err != nil && err.Error() != "torrent closed during resolution" {
t.Errorf("Resolve(%s) got = %v", u, err)
return
}
f.Close()
if err := closeClient(); err != nil {
t.Errorf("closeClient() got = %v", err)
return
}
}
}()
}
wg.Wait()
}
func doResolve(t *testing.T, fetcher fetcher.Fetcher) {
t.Run("Resolve Single File", func(t *testing.T) {
err := fetcher.Resolve(&base.Request{
@@ -81,7 +158,7 @@ func doResolve(t *testing.T, fetcher fetcher.Fetcher) {
"udp://tracker.bitsearch.to:1337/announce",
},
},
}, nil)
})
if err != nil {
panic(err)
}
@@ -111,7 +188,7 @@ func doResolve(t *testing.T, fetcher fetcher.Fetcher) {
"udp://tracker.bitsearch.to:1337/announce",
},
},
}, nil)
})
if err != nil {
panic(err)
}
@@ -145,7 +222,7 @@ func doResolve(t *testing.T, fetcher fetcher.Fetcher) {
t.Run("Resolve Unclean Torrent", func(t *testing.T) {
err := fetcher.Resolve(&base.Request{
URL: "./testdata/test.unclean.torrent",
}, nil)
})
if err != nil {
t.Errorf("Resolve Unclean Torrent Resolve() got = %v, want nil", err)
}
@@ -156,7 +233,7 @@ func doResolve(t *testing.T, fetcher fetcher.Fetcher) {
uri := "file:///" + file
err := fetcher.Resolve(&base.Request{
URL: uri,
}, nil)
})
if err != nil {
t.Errorf("Resolve file scheme Torrent Resolve() got = %v, want nil", err)
}
@@ -211,7 +288,7 @@ func TestFetcherManager_ParseName(t *testing.T) {
}
}
func buildFetcher() fetcher.Fetcher {
func buildFetcher(t testing.TB) fetcher.Fetcher {
fb := new(FetcherManager)
fetcher := fb.Build()
newController := controller.NewController()
@@ -219,14 +296,15 @@ func buildFetcher() fetcher.Fetcher {
json.Unmarshal([]byte(test.ToJson(fb.DefaultConfig())), v)
}
fetcher.Setup(newController)
t.Cleanup(func() { fetcher.Close(); closeClient() })
return fetcher
}
func buildConfigFetcher(proxyConfig *base.DownloaderProxyConfig) fetcher.Fetcher {
func buildConfigFetcher(t testing.TB, proxyConfig *base.DownloaderProxyConfig) fetcher.Fetcher {
fetcher := new(FetcherManager).Build()
newController := controller.NewController()
mockCfg := config{
Trackers: []string{
CustomTrackers: []string{
"udp://tracker.birkenwald.de:6969/announce",
"udp://tracker.bitsearch.to:1337/announce",
}}
@@ -237,29 +315,30 @@ func buildConfigFetcher(proxyConfig *base.DownloaderProxyConfig) fetcher.Fetcher
return proxyConfig.ToHandler()
}
fetcher.Setup(newController)
t.Cleanup(func() { fetcher.Close(); closeClient() })
return fetcher
}
// TestFetcher_Patch tests the Patch functionality for BT fetcher.
// It tests modifying selected files after Resolve (without downloading).
func TestFetcher_Patch(t *testing.T) {
f := buildFetcher()
f := buildFetcher(t)
// Resolve a multi-file torrent
err := f.Resolve(&base.Request{
URL: "./testdata/test.torrent",
}, nil)
})
if err != nil {
t.Fatalf("Resolve failed: %v", err)
}
// Verify initial state: 3 files, all selected by default
// Resolution describes files without choosing download options.
meta := f.Meta()
if len(meta.Res.Files) != 3 {
t.Fatalf("Expected 3 files, got %d", len(meta.Res.Files))
}
if len(meta.Opts.SelectFiles) != 3 {
t.Fatalf("Expected 3 selected files, got %d", len(meta.Opts.SelectFiles))
if meta.Opts != nil {
t.Fatal("Resolve should not assign download options")
}
// Total size of all files: 107484864
@@ -366,3 +445,52 @@ func TestFetcher_Patch(t *testing.T) {
}
})
}
func TestFetcher_PeerDiscoveryConfig(t *testing.T) {
tests := []struct {
name string
stored any
wantDisableDHT bool
wantDisablePEX bool
}{
{"default config", new(FetcherManager).DefaultConfig(), false, false},
{"stored config without the fields", map[string]any{"listenPort": 0, "seedRatio": 1.0}, false, false},
{"both disabled", map[string]any{"disableDht": true, "disablePex": true}, true, true},
{"only DHT disabled", map[string]any{"disableDht": true}, true, false},
{"only PEX disabled", map[string]any{"disablePex": true}, false, true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
f := buildFetcherWithConfig(tt.stored).(*Fetcher)
if f.config.DisableDHT != tt.wantDisableDHT || f.config.DisablePEX != tt.wantDisablePEX {
t.Errorf("config got DisableDHT = %v, DisablePEX = %v, want %v, %v", f.config.DisableDHT, f.config.DisablePEX, tt.wantDisableDHT, tt.wantDisablePEX)
}
})
}
}
func TestFetcher_PeerDiscoveryDisabledOnClient(t *testing.T) {
if err := closeClient(); err != nil {
t.Fatal(err)
}
f := buildFetcherWithConfig(map[string]any{"disableDht": true, "disablePex": true})
if err := f.Resolve(&base.Request{URL: "./testdata/test.torrent"}); err != nil {
t.Fatal(err)
}
if !cfg.NoDHT || !cfg.DisablePEX {
t.Errorf("client config got NoDHT = %v, DisablePEX = %v, want true", cfg.NoDHT, cfg.DisablePEX)
}
if n := len(client.DhtServers()); n != 0 {
t.Errorf("client got %d DHT servers, want 0", n)
}
}
func buildFetcherWithConfig(stored any) fetcher.Fetcher {
fetcher := new(FetcherManager).Build()
newController := controller.NewController()
newController.GetConfig = func(v any) {
json.Unmarshal([]byte(test.ToJson(stored)), v)
}
fetcher.Setup(newController)
return fetcher
}
+212
View File
@@ -0,0 +1,212 @@
package bt
import (
"bytes"
"context"
"encoding/base64"
"encoding/binary"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/anacrolix/torrent"
"github.com/anacrolix/torrent/bencode"
"github.com/anacrolix/torrent/merkle"
"github.com/anacrolix/torrent/metainfo"
"golang.org/x/time/rate"
)
// btFixture serves generated, trackerless content over a real loopback TCP peer.
// Its HTTP tracker exercises .torrent downloads; magnet links also carry the
// explicit x.pe address.
type btFixture struct {
MetaInfo *metainfo.MetaInfo
Magnet string
TorrentFile string
DataURI string
TrackerURL string
Files map[string][]byte // Paths relative to the selected download directory.
Seeder *torrent.Client
Torrent *torrent.Torrent
Limiter *rate.Limiter
}
func newBTFixture(t testing.TB, files map[string][]byte, bytesPerSecond int) *btFixture {
return makeBTFixture(t, files, bytesPerSecond, false, true)
}
func newBTHybridFixture(t testing.TB, payload []byte) *btFixture {
return makeBTFixture(t, map[string][]byte{"": payload}, 0, true, true)
}
func makeBTFixture(t testing.TB, files map[string][]byte, bytesPerSecond int, v2, private bool) *btFixture {
t.Helper()
source := t.TempDir()
name := "payload.bin"
if _, single := files[""]; !single {
name = "bundle"
}
root := filepath.Join(source, name)
expected := make(map[string][]byte, len(files))
for path, data := range files {
target := filepath.Join(root, filepath.FromSlash(path))
if err := os.MkdirAll(filepath.Dir(target), 0700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(target, data, 0600); err != nil {
t.Fatal(err)
}
expected[filepath.Join(name, filepath.FromSlash(path))] = append([]byte(nil), data...)
}
info := metainfo.Info{PieceLength: 16 * 1024}
if err := info.BuildFromFilePath(root); err != nil {
t.Fatal(err)
}
var layers map[string]string
if v2 {
data := files[""]
h := merkle.NewHash()
h.Write(data)
root := h.Sum(nil)
var pieceHashes []byte
for off := 0; off < len(data); off += int(info.PieceLength) {
end := min(off+int(info.PieceLength), len(data))
ph := merkle.NewHash()
ph.Write(data[off:end])
pieceHashes = append(pieceHashes, ph.SumMinLength(nil, int(info.PieceLength))...)
}
info = metainfo.Info{Name: name, PieceLength: info.PieceLength, Length: int64(len(data)), Pieces: info.Pieces, MetaVersion: 2, FileTree: metainfo.FileTree{File: metainfo.FileTreeFile{Length: int64(len(data)), PiecesRoot: string(root)}}}
layers = map[string]string{string(root): string(pieceHashes)}
}
// Private torrents avoid public DHT discovery once metadata is available.
info.Private = &private
raw, err := bencode.Marshal(&info)
if err != nil {
t.Fatal(err)
}
mi := &metainfo.MetaInfo{InfoBytes: raw, PieceLayers: layers}
cfg := torrent.TestingConfig(t)
cfg.DataDir = source
cfg.Seed = true
cfg.DisableUTP = true
cfg.DisableIPv6 = true
cfg.DisableWebtorrent = true
cfg.KeepAliveTimeout = time.Minute
cfg.MaxAllocPeerRequestDataPerConn = 1 << 20
limiter := rate.NewLimiter(rate.Inf, 16*1024)
if bytesPerSecond > 0 {
limiter.SetLimit(rate.Limit(bytesPerSecond))
}
cfg.UploadRateLimiter = limiter
seed, err := torrent.NewClient(cfg)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { seed.Close() })
seeded, _, err := seed.AddTorrentSpec(torrent.TorrentSpecFromMetaInfo(mi))
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := seeded.VerifyDataContext(ctx); err != nil {
t.Fatal(err)
}
select {
case <-seeded.Complete().On():
case <-ctx.Done():
t.Fatal("local seeder did not finish verification")
}
addr := seed.ListenAddrs()[0].String()
host, portString, err := net.SplitHostPort(addr)
if err != nil {
t.Fatal(err)
}
resolved, err := net.ResolveTCPAddr("tcp4", net.JoinHostPort(host, portString))
if err != nil {
t.Fatal(err)
}
compact := make([]byte, 6)
copy(compact, resolved.IP.To4())
binary.BigEndian.PutUint16(compact[4:], uint16(resolved.Port))
tracker := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
response, err := bencode.Marshal(map[string]any{"interval": 3600, "peers": string(compact)})
if err != nil {
http.Error(w, err.Error(), 500)
return
}
w.Write(response)
}))
t.Cleanup(tracker.Close)
// The pinned library has a hybrid tracker-dispatch assertion bug.
// Hybrid fixtures use their explicit loopback peer rather than a tracker.
if !v2 {
mi.Announce = tracker.URL
}
var encoded bytes.Buffer
if err := mi.Write(&encoded); err != nil {
t.Fatal(err)
}
torrentFile := filepath.Join(t.TempDir(), "fixture.torrent")
if err := os.WriteFile(torrentFile, encoded.Bytes(), 0600); err != nil {
t.Fatal(err)
}
m, err := mi.MagnetV2()
if err != nil {
t.Fatal(err)
}
magnet := m.String()
u, err := url.Parse(magnet)
if err != nil {
t.Fatal(err)
}
q := u.Query()
q.Add("x.pe", addr)
u.RawQuery = q.Encode()
return &btFixture{MetaInfo: mi, Magnet: u.String(), TorrentFile: torrentFile, DataURI: "data:application/x-bittorrent;base64," + base64.StdEncoding.EncodeToString(encoded.Bytes()), TrackerURL: tracker.URL, Files: expected, Seeder: seed, Torrent: seeded, Limiter: limiter}
}
func btPayload(size int, label string) []byte {
data := make([]byte, size)
for i := range data {
data[i] = byte((i*31 + i/251) % 251)
}
copy(data, label)
return data
}
func (f *btFixture) AssertFiles(t testing.TB, dir string, paths ...string) {
t.Helper()
if len(paths) == 0 {
for path := range f.Files {
paths = append(paths, path)
}
}
for _, path := range paths {
got, err := os.ReadFile(filepath.Join(dir, path))
if err != nil {
t.Fatal(err)
}
want, ok := f.Files[path]
if !ok {
t.Fatalf("unknown fixture file %q", path)
}
if !bytes.Equal(got, want) {
t.Fatalf("BT file %s differs: received %d bytes, want %d", path, len(got), len(want))
}
}
}
func (f *btFixture) FileURI() string {
path := filepath.ToSlash(f.TorrentFile)
if !strings.HasPrefix(path, "/") {
path = "/" + path
}
return (&url.URL{Scheme: "file", Path: path}).String()
}
+367
View File
@@ -0,0 +1,367 @@
package bt
import (
"context"
"crypto/sha1"
"fmt"
"os"
"path/filepath"
"testing"
"time"
"github.com/GopeedLab/gopeed/internal/fetcher"
"github.com/GopeedLab/gopeed/pkg/base"
"github.com/anacrolix/torrent"
"github.com/anacrolix/torrent/bencode"
"github.com/anacrolix/torrent/metainfo"
"github.com/anacrolix/torrent/storage"
)
func TestStartDoesNotPublishUnverifiedCompletion(t *testing.T) {
cfg := torrent.TestingConfig(t)
cfg.DisableTCP = true
cfg.DisableUTP = true
c, err := torrent.NewClient(cfg)
if err != nil {
t.Fatal(err)
}
defer c.Close()
payload := btPayload(32*1024, "unverified")
info := metainfo.Info{Name: "payload.bin", Length: int64(len(payload)), PieceLength: 16 * 1024}
for offset := 0; offset < len(payload); offset += int(info.PieceLength) {
hash := sha1.Sum(payload[offset : offset+int(info.PieceLength)])
info.Pieces = append(info.Pieces, hash[:]...)
}
raw, err := bencode.Marshal(info)
if err != nil {
t.Fatal(err)
}
store := &deferredFileStorage{}
spec := torrent.TorrentSpecFromMetaInfo(&metainfo.MetaInfo{InfoBytes: raw})
spec.Storage = store
tor, _, err := c.AddTorrentSpec(spec)
if err != nil {
t.Fatal(err)
}
tor.DisallowDataDownload()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
f := &Fetcher{torrent: tor, storage: store, meta: &fetcher.FetcherMeta{}, data: &fetcherData{}, torrentDropCtx: ctx, torrentDropFunc: cancel, config: &config{SeedKeep: true}}
f.torrentReady.Store(true)
f.updateRes()
dir := t.TempDir()
payload[20000] ^= 0xff
if err := os.WriteFile(filepath.Join(dir, info.Name), payload, 0600); err != nil {
t.Fatal(err)
}
f.meta.Opts = &base.Options{Path: dir}
// Retain the original event channel to detect even a transient completion.
// With no network transports, the damaged piece cannot have been repaired.
completed := tor.Complete().On()
if err := f.Start(); err != nil {
t.Fatal(err)
}
select {
case <-completed:
t.Fatal("Start published completion before verifying the existing file")
default:
}
}
func TestStartUsesSelectionSubmittedAfterResolve(t *testing.T) {
cfg := torrent.TestingConfig(t)
cfg.DataDir = t.TempDir()
cfg.DisableTCP = true
cfg.DisableUTP = true
c, err := torrent.NewClient(cfg)
if err != nil {
t.Fatal(err)
}
defer c.Close()
store := &deferredFileStorage{}
mi, err := metainfo.LoadFromFile("testdata/test.torrent")
if err != nil {
t.Fatal(err)
}
spec := torrent.TorrentSpecFromMetaInfo(mi)
spec.Storage = store
tor, _, err := c.AddTorrentSpec(spec)
if err != nil {
t.Fatal(err)
}
tor.DisallowDataDownload()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
f := &Fetcher{torrent: tor, storage: store, meta: &fetcher.FetcherMeta{}, data: &fetcherData{}, torrentDropCtx: ctx, torrentDropFunc: cancel, config: &config{SeedKeep: true}}
f.torrentReady.Store(true)
f.updateRes()
f.meta.Opts = &base.Options{Path: t.TempDir(), SelectFiles: []int{2}}
if err := f.Start(); err != nil {
t.Fatal(err)
}
for i, file := range tor.Files() {
want := torrent.PiecePriorityNone
if i == 2 {
want = torrent.PiecePriorityNormal
}
if file.Priority() != want {
t.Errorf("file %d priority = %v, want %v", i, file.Priority(), want)
}
}
if len(f.data.Progress) != 1 {
t.Fatalf("progress tracks %d files, want 1", len(f.data.Progress))
}
if err := f.Pause(); err != nil {
t.Fatal(err)
}
if err := f.Start(); err != nil {
t.Fatal(err)
}
if tor.Files()[0].Priority() != torrent.PiecePriorityNone {
t.Fatal("resume selected an unselected file")
}
}
func TestWaitCleansUnselectedFilesFromSharedPiece(t *testing.T) {
for _, mode := range []string{"partial", "part only", "both paths", "all", "already removed", "remove error"} {
t.Run(mode, func(t *testing.T) {
dir := t.TempDir()
keepPath := filepath.Join(dir, "bundle", "keep.txt")
skipPath := filepath.Join(dir, "bundle", "nested", "skip.txt")
if err := os.MkdirAll(filepath.Dir(skipPath), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(keepPath, []byte("keep"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(skipPath, []byte("skip"), 0644); err != nil {
t.Fatal(err)
}
// Both files occupy the same piece, so downloading keep.txt also writes skip.txt.
hash := sha1.Sum([]byte("keepskip"))
info := metainfo.Info{Name: "bundle", PieceLength: 16, Pieces: hash[:], Files: []metainfo.FileInfo{
{Length: 4, Path: []string{"keep.txt"}},
{Length: 4, Path: []string{"nested", "skip.txt"}},
}}
storageOpts := storage.NewFileClientOpts{
ClientBaseDir: dir,
TorrentDirMaker: func(baseDir string, _ *metainfo.Info, _ metainfo.Hash) string { return baseDir },
}
// VerifyDataContext can return before storage finishes marking the piece
// complete. Disable automatic part-file promotion so it cannot rename
// the .part fixtures that this test creates to exercise cleanup.
storageOpts.UsePartFiles.Set(false)
store := storage.NewFileOpts(storageOpts)
defer store.Close()
config := torrent.TestingConfig(t)
config.DisableTCP = true
config.DisableUTP = true
c, err := torrent.NewClient(config)
if err != nil {
t.Fatal(err)
}
defer c.Close()
spec, err := torrent.TorrentSpecFromMetaInfoErr(&metainfo.MetaInfo{InfoBytes: bencode.MustMarshal(&info)})
if err != nil {
t.Fatal(err)
}
spec.Storage = store
tor, _, err := c.AddTorrentSpec(spec)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := tor.VerifyDataContext(ctx); err != nil {
t.Fatal(err)
}
f := &Fetcher{torrent: tor, meta: &fetcher.FetcherMeta{Opts: &base.Options{Path: dir, SelectFiles: []int{0}}}, data: &fetcherData{}, torrentDropCtx: ctx}
f.torrentReady.Store(true)
f.updateRes()
if mode == "all" {
f.meta.Opts.SelectFiles = []int{0, 1}
}
if !f.isDone() {
t.Fatal("selected file is not complete")
}
if mode == "part only" || mode == "both paths" || mode == "all" {
if err := os.WriteFile(skipPath+".part", []byte("skip"), 0644); err != nil {
t.Fatal(err)
}
}
if mode == "part only" || mode == "already removed" || mode == "remove error" {
if err := os.Remove(skipPath); err != nil {
t.Fatal(err)
}
}
if mode == "remove error" {
if err := os.Mkdir(skipPath, 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(skipPath, "child"), []byte("preserve"), 0644); err != nil {
t.Fatal(err)
}
}
err = f.Wait()
if ctx.Err() != nil {
t.Fatal(ctx.Err())
}
if mode == "remove error" {
if err == nil {
t.Fatal("cleanup failure was silently ignored")
}
} else if err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(keepPath)
if err != nil || string(data) != "keep" {
t.Fatalf("selected file changed: %q, %v", data, err)
}
if mode == "all" {
if _, err := os.Stat(skipPath + ".part"); err != nil {
t.Fatalf("selected part file was removed: %v", err)
}
data, err := os.ReadFile(skipPath)
if err != nil || string(data) != "skip" {
t.Fatalf("selected file changed: %q, %v", data, err)
}
} else if mode != "remove error" {
for _, removed := range []string{skipPath, skipPath + ".part", filepath.Dir(skipPath)} {
if _, err := os.Stat(removed); !os.IsNotExist(err) {
t.Fatalf("unselected file or empty directory remains: %s: %v", removed, err)
}
}
}
})
}
}
func TestRemoveUnselectedFilePrunesOnlyEmptyParents(t *testing.T) {
for _, preserveSibling := range []bool{false, true} {
t.Run(fmt.Sprint(preserveSibling), func(t *testing.T) {
root := t.TempDir()
relative := filepath.Join("bundle", "nested", "deep", "skip.txt")
name := filepath.Join(root, relative)
if err := os.MkdirAll(filepath.Dir(name), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(name+".part", []byte("partial"), 0644); err != nil {
t.Fatal(err)
}
sibling := filepath.Join(root, "bundle", "keep.txt")
if preserveSibling {
if err := os.WriteFile(sibling, []byte("keep"), 0644); err != nil {
t.Fatal(err)
}
}
if err := removeUnselectedFile(root, "bundle", relative); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(filepath.Join(root, "bundle", "nested")); !os.IsNotExist(err) {
t.Fatalf("empty ancestors remain: %v", err)
}
if preserveSibling {
data, err := os.ReadFile(sibling)
if err != nil || string(data) != "keep" {
t.Fatalf("sibling changed: %q, %v", data, err)
}
}
if info, err := os.Stat(filepath.Join(root, "bundle")); err != nil || !info.IsDir() {
t.Fatalf("torrent root directory was removed: %v", err)
}
if _, err := os.Stat(root); err != nil {
t.Fatalf("download directory was removed: %v", err)
}
// A file directly inside the torrent root must not cause that root to be pruned.
rootFile := filepath.Join("bundle", "root.txt")
if err := os.WriteFile(filepath.Join(root, rootFile)+".part", []byte("partial"), 0644); err != nil {
t.Fatal(err)
}
if err := removeUnselectedFile(root, "bundle", rootFile); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(filepath.Join(root, "bundle")); err != nil {
t.Fatalf("torrent root directory was removed after root-level cleanup: %v", err)
}
// Repeating cleanup after the file and its subdirectories are gone is harmless.
if err := removeUnselectedFile(root, "bundle", relative); err != nil {
t.Fatal(err)
}
})
}
}
func TestCleanupMatchesNamedAndRootlessStorage(t *testing.T) {
for _, version := range []int64{1, 2} {
for _, layout := range []string{"rootless", "wrapped", "file-tree-root"} {
if version == 1 && layout == "file-tree-root" {
continue
}
t.Run(fmt.Sprintf("v%d/%s", version, layout), func(t *testing.T) {
dir := t.TempDir()
info := metainfo.Info{Name: metainfo.NoName, PieceLength: 16384}
root := dir
if layout == "wrapped" {
info.Name = "bundle"
root = filepath.Join(dir, "bundle")
}
if version == 1 {
info.Files = []metainfo.FileInfo{{Path: []string{"keep.txt"}}, {Path: []string{"nested", "deep", "skip.txt"}}}
} else {
info.MetaVersion = 2
info.FileTree = metainfo.FileTree{Dir: map[string]metainfo.FileTree{
"keep.txt": {},
"nested": {Dir: map[string]metainfo.FileTree{"deep": {Dir: map[string]metainfo.FileTree{"skip.txt": {}}}}},
}}
}
if layout == "file-tree-root" {
info.FileTree = metainfo.FileTree{Dir: map[string]metainfo.FileTree{"tree-root": info.FileTree}}
root = filepath.Join(dir, "tree-root")
}
// Keep the library's default FilePathMaker here to independently verify
// that cleanup agrees with its real disk layout, including NoName.
store := storage.NewFileOpts(storage.NewFileClientOpts{
ClientBaseDir: dir,
TorrentDirMaker: func(baseDir string, _ *metainfo.Info, _ metainfo.Hash) string { return baseDir },
})
defer store.Close()
torrentStorage, err := store.OpenTorrent(context.Background(), &info, metainfo.Hash{1})
if err != nil {
t.Fatal(err)
}
defer torrentStorage.Close()
skip := filepath.Join(root, "nested", "deep", "skip.txt")
if _, err := os.Stat(skip); err != nil {
t.Fatalf("unexpected storage layout: %v", err)
}
if err := os.WriteFile(skip+".part", []byte("partial"), 0644); err != nil {
t.Fatal(err)
}
files := info.UpvertedFiles()
boundary, name := torrentFileLayout(&info, files[1])
if err := removeUnselectedFile(dir, boundary, name); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(filepath.Join(root, "nested")); !os.IsNotExist(err) {
t.Fatalf("empty subtree remains: %v", err)
}
if _, err := os.Stat(filepath.Join(root, "keep.txt")); err != nil {
t.Fatalf("selected file removed: %v", err)
}
// Also verify the boundary when the protected directory becomes empty.
if err := os.Remove(filepath.Join(root, "keep.txt")); err != nil {
t.Fatal(err)
}
for _, file := range files {
boundary, name := torrentFileLayout(&info, file)
if err := removeUnselectedFile(dir, boundary, name); err != nil {
t.Fatal(err)
}
}
if stat, err := os.Stat(root); err != nil || !stat.IsDir() {
t.Fatalf("cleanup boundary was removed: %v", err)
}
})
}
}
}
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"github.com/GopeedLab/gopeed/internal/fetcher"
"github.com/GopeedLab/gopeed/pkg/base"
protocolbt "github.com/GopeedLab/gopeed/pkg/protocol/bt"
"github.com/RoaringBitmap/roaring"
"github.com/RoaringBitmap/roaring/v2"
"github.com/anacrolix/torrent"
"github.com/anacrolix/torrent/storage"
)
+116
View File
@@ -0,0 +1,116 @@
package bt
import (
"context"
"errors"
"os"
"sync"
g "github.com/anacrolix/generics"
"github.com/anacrolix/torrent/metainfo"
"github.com/anacrolix/torrent/storage"
)
// deferredFileStorage lets metadata resolve without opening the user's files.
// It binds once. After binding, pieces are supplied directly by file storage.
type deferredFileStorage struct {
mu sync.RWMutex
ctx context.Context
info *metainfo.Info
hash metainfo.Hash
knownHashes []bool
owner storage.ClientImplCloser
torrent storage.TorrentImpl
closed bool
}
func (s *deferredFileStorage) OpenTorrent(ctx context.Context, info *metainfo.Info, hash metainfo.Hash) (storage.TorrentImpl, error) {
s.mu.Lock()
defer s.mu.Unlock()
if s.closed || s.info != nil {
return storage.TorrentImpl{}, errors.New("torrent storage already opened or closed")
}
s.ctx, s.info, s.hash = ctx, info, hash
s.knownHashes = make([]bool, info.NumPieces())
return storage.TorrentImpl{
PieceWithHash: func(p metainfo.Piece, hash g.Option[[]byte]) storage.PieceImpl {
s.mu.Lock()
defer s.mu.Unlock()
if s.closed {
return metadataPiece{err: os.ErrClosed}
}
if s.owner == nil {
s.knownHashes[p.Index()] = hash.Ok
// A missing v2 hash must remain unknown until its arrival changes
// completion and updates the library's piece request priorities.
return metadataPiece{known: hash.Ok}
}
if s.torrent.PieceWithHash != nil {
return s.torrent.PieceWithHash(p, hash)
}
return s.torrent.Piece(p)
},
Close: s.Close,
}, nil
}
func (s *deferredFileStorage) bind(opts storage.NewFileClientOpts) ([]bool, error) {
s.mu.Lock()
defer s.mu.Unlock()
if s.closed {
return nil, os.ErrClosed
}
if s.info == nil {
return nil, errors.New("torrent metadata is not ready")
}
if s.owner != nil {
return nil, nil
}
if opts.FilePathMaker == nil {
opts.FilePathMaker = func(o storage.FilePathMakerOpts) string { _, name := torrentFileLayout(o.Info, *o.File); return name }
}
owner := storage.NewFileOpts(opts)
t, err := owner.OpenTorrent(s.ctx, s.info, s.hash)
if err != nil {
owner.Close()
return nil, err
}
s.owner, s.torrent = owner, t
return append([]bool(nil), s.knownHashes...), nil
}
func (s *deferredFileStorage) Close() error {
s.mu.Lock()
defer s.mu.Unlock()
if s.closed {
return nil
}
s.closed = true
if s.owner == nil {
return nil
}
var err error
if s.torrent.Close != nil {
err = s.torrent.Close()
}
return errors.Join(err, s.owner.Close())
}
type metadataPiece struct {
known bool
err error
}
func (p metadataPiece) Completion() storage.Completion {
return storage.Completion{Ok: p.known, Err: p.err}
}
func (p metadataPiece) ReadAt([]byte, int64) (int, error) {
return 0, errors.New("download storage is not initialized")
}
func (p metadataPiece) WriteAt([]byte, int64) (int, error) {
return 0, errors.New("download storage is not initialized")
}
func (p metadataPiece) MarkComplete() error { return errors.New("download storage is not initialized") }
func (p metadataPiece) MarkNotComplete() error {
return errors.New("download storage is not initialized")
}
+234
View File
@@ -0,0 +1,234 @@
package bt
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net/url"
"slices"
"strings"
"sync"
"time"
"github.com/GopeedLab/gopeed/internal/fetcher"
)
var defaultTrackerSubscribeURLs = []string{
"https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_all.txt",
"https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_all_http.txt",
"https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_all_https.txt",
"https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_all_ip.txt",
"https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_all_udp.txt",
"https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_all_ws.txt",
"https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_best.txt",
"https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_best_ip.txt",
"https://raw.githubusercontent.com/XIU2/TrackersListCollection/master/all.txt",
"https://raw.githubusercontent.com/XIU2/TrackersListCollection/master/best.txt",
"https://raw.githubusercontent.com/XIU2/TrackersListCollection/master/http.txt",
}
func validTracker(raw string) bool {
u, err := url.Parse(raw)
if err != nil || u.Hostname() == "" || u.User != nil {
return false
}
switch u.Scheme {
case "udp", "http", "https", "ws", "wss":
return true
}
return false
}
func uniqueTrackers(groups ...[]string) []string {
result := []string{}
seen := map[string]bool{}
for _, group := range groups {
for _, raw := range group {
value := strings.TrimSpace(raw)
if value != "" && !seen[value] {
seen[value] = true
result = append(result, value)
}
}
}
return result
}
// effectiveTrackers derives the client list without persisting duplicate lists.
func (c *config) effectiveTrackers() []string {
groups := [][]string{c.CustomTrackers}
for _, source := range c.TrackerSubscribeURLs {
groups = append(groups, c.State.TrackerSources[source])
}
return uniqueTrackers(groups...)
}
func (fm *FetcherManager) NormalizeConfig(raw json.RawMessage) (any, error) {
c := fm.DefaultConfig().(*config)
if err := json.Unmarshal(raw, c); err != nil {
return nil, err
}
if c.ListenPort < 0 || c.ListenPort > 65535 || c.SeedRatio < 0 || c.SeedTime < 0 {
return nil, errors.New("invalid BT configuration")
}
if len(c.TrackerSubscribeURLs) > 64 {
return nil, errors.New("too many tracker subscriptions")
}
c.TrackerSubscribeURLs = uniqueTrackers(c.TrackerSubscribeURLs)
for _, source := range c.TrackerSubscribeURLs {
u, err := url.Parse(source)
if err != nil || u == nil || (u.Scheme != "https" && u.Scheme != "http") || u.Hostname() == "" || u.User != nil {
return nil, errors.New("invalid tracker subscription URL")
}
}
c.CustomTrackers = uniqueTrackers(c.CustomTrackers)
for _, tracker := range c.CustomTrackers {
if !validTracker(tracker) {
return nil, fmt.Errorf("invalid tracker URL: %s", tracker)
}
}
if c.State.TrackerSources == nil {
c.State.TrackerSources = map[string][]string{}
}
for source := range c.State.TrackerSources {
if !slices.Contains(c.TrackerSubscribeURLs, source) {
delete(c.State.TrackerSources, source)
}
}
if c.State.TrackerSubscriptionStatuses == nil {
c.State.TrackerSubscriptionStatuses = map[string]trackerSubscriptionStatus{}
}
for source := range c.State.TrackerSubscriptionStatuses {
if !slices.Contains(c.TrackerSubscribeURLs, source) {
delete(c.State.TrackerSubscriptionStatuses, source)
}
}
return c, nil
}
func (fm *FetcherManager) CronJobs(host fetcher.ProtocolHost) []fetcher.CronJob {
return []fetcher.CronJob{{
Name: "update-trackers",
Every: 24 * time.Hour,
Timeout: 90 * time.Second,
Run: func(ctx context.Context, source fetcher.TriggerSource) error {
return fm.updateTrackers(ctx, host, source)
},
}}
}
func parseTrackers(data []byte) ([]string, error) {
scanner := bufio.NewScanner(bytes.NewReader(data))
scanner.Buffer(make([]byte, 4096), 64*1024)
result := []string{}
for scanner.Scan() {
line := strings.TrimSpace(strings.TrimPrefix(scanner.Text(), "\ufeff"))
if validTracker(line) {
result = append(result, line)
}
if len(result) > 20000 {
return nil, errors.New("too many trackers")
}
}
if err := scanner.Err(); err != nil {
return nil, err
}
if len(result) == 0 {
return nil, errors.New("subscription contains no valid trackers")
}
return uniqueTrackers(result), nil
}
func (fm *FetcherManager) updateTrackers(ctx context.Context, host fetcher.ProtocolHost, source fetcher.TriggerSource) error {
snapshot, err := fetcher.ReadConfig[config](host)
if err != nil {
return err
}
if (source == fetcher.TriggerScheduled && !snapshot.AutoUpdateTrackers) || len(snapshot.TrackerSubscribeURLs) == 0 {
return fetcher.ErrJobSkipped
}
fetched := map[string][]string{}
statuses := map[string]trackerSubscriptionStatus{}
var failures []error
var mu sync.Mutex
var wg sync.WaitGroup
slots := make(chan struct{}, 4)
for _, address := range snapshot.TrackerSubscribeURLs {
wg.Add(1)
go func(address string) {
defer wg.Done()
select {
case slots <- struct{}{}:
defer func() { <-slots }()
case <-ctx.Done():
return
}
data, err := host.Fetch(ctx, address, 2<<20, func(body []byte) error { _, err := parseTrackers(body); return err })
var trackers []string
if err == nil {
trackers, err = parseTrackers(data)
}
mu.Lock()
defer mu.Unlock()
statuses[address] = trackerSubscriptionStatus{Success: err == nil, TrackerCount: len(trackers), LastFetchTime: time.Now().UTC()}
if err != nil {
failures = append(failures, fmt.Errorf("subscription %s: %w", address, err))
} else {
fetched[address] = trackers
}
}(address)
}
wg.Wait()
if err := ctx.Err(); err != nil {
failures = append(failures, err)
}
if len(statuses) == 0 {
return errors.Join(failures...)
}
err = fetcher.UpdateState[config, state](host, func(current *config, state *state) error {
if !slices.Equal(current.TrackerSubscribeURLs, snapshot.TrackerSubscribeURLs) {
return fetcher.ErrConfigChanged
}
if state.TrackerSources == nil {
state.TrackerSources = map[string][]string{}
}
if state.TrackerSubscriptionStatuses == nil {
state.TrackerSubscriptionStatuses = map[string]trackerSubscriptionStatus{}
}
for address, status := range statuses {
state.TrackerSubscriptionStatuses[address] = status
}
for address, trackers := range fetched {
state.TrackerSources[address] = trackers
}
if len(failures) == 0 {
state.LastTrackerUpdateTime = time.Now().UTC()
}
return nil
})
if err != nil {
return err
}
current, err := fetcher.ReadConfig[config](host)
if err != nil {
return err
}
fm.applyTrackers(current.effectiveTrackers())
return errors.Join(failures...)
}
func (fm *FetcherManager) applyTrackers(trackers []string) {
lock.Lock()
defer lock.Unlock()
if client == nil {
return
}
announce := make([][]string, 0, len(trackers))
for _, tracker := range trackers {
announce = append(announce, []string{tracker})
}
for _, t := range client.Torrents() {
info := t.Info()
if info == nil || (info.Private != nil && *info.Private) {
continue
}
t.AddTrackers(announce)
}
}
+217
View File
@@ -0,0 +1,217 @@
package bt
import (
"context"
"encoding/json"
"errors"
"reflect"
"sync"
"testing"
"time"
"github.com/GopeedLab/gopeed/internal/fetcher"
)
func trackerTestHost(t *testing.T, c *config, fetch func(context.Context, string, int64, func([]byte) error) ([]byte, error)) (fetcher.ProtocolHost, *sync.Mutex) {
t.Helper()
mu := &sync.Mutex{}
return fetcher.ProtocolHost{
Read: func(dst any) error {
mu.Lock()
defer mu.Unlock()
raw, err := json.Marshal(c)
if err != nil {
return err
}
return json.Unmarshal(raw, dst)
},
UpdateState: func(change func(config, state json.RawMessage) (any, error)) error {
mu.Lock()
defer mu.Unlock()
raw, _ := json.Marshal(c)
currentState, _ := json.Marshal(c.State)
next, err := change(raw, currentState)
if err != nil {
return err
}
raw, err = json.Marshal(next)
if err != nil {
return err
}
return json.Unmarshal(raw, &c.State)
},
Fetch: fetch,
}, mu
}
func TestTrackerRefreshPartialSuccessUsesLatestConfig(t *testing.T) {
fm := &FetcherManager{}
cfg := fm.DefaultConfig().(*config)
cfg.TrackerSubscribeURLs = []string{"https://a.example/list", "https://b.example/list"}
cfg.CustomTrackers = []string{"udp://custom.example:80/announce"}
cfg.State.TrackerSources = map[string][]string{cfg.TrackerSubscribeURLs[0]: {"udp://old.example:80/announce"}, cfg.TrackerSubscribeURLs[1]: {"udp://cached.example:80/announce"}}
host, mu := trackerTestHost(t, cfg, nil)
host.Fetch = func(ctx context.Context, address string, _ int64, validate func([]byte) error) ([]byte, error) {
if address == "https://b.example/list" {
return nil, errors.New("offline")
}
// Simulate a frontend update while the network request is in flight.
mu.Lock()
cfg.CustomTrackers = []string{"udp://new-custom.example:80/announce"}
mu.Unlock()
data := []byte("udp://fresh.example:80/announce\n# ignored\nudp://fresh.example:80/announce\n")
return data, validate(data)
}
err := fm.updateTrackers(context.Background(), host, fetcher.TriggerManual)
if err == nil {
t.Fatal("partial failure must be reported")
}
want := []string{"udp://new-custom.example:80/announce", "udp://fresh.example:80/announce", "udp://cached.example:80/announce"}
if !reflect.DeepEqual(cfg.effectiveTrackers(), want) {
t.Fatalf("got %v", cfg.effectiveTrackers())
}
success := cfg.State.TrackerSubscriptionStatuses["https://a.example/list"]
failure := cfg.State.TrackerSubscriptionStatuses["https://b.example/list"]
if !success.Success || success.TrackerCount != 1 || success.LastFetchTime.IsZero() || failure.Success || failure.TrackerCount != 0 || failure.LastFetchTime.IsZero() {
t.Fatalf("unexpected subscription statuses: %+v", cfg.State.TrackerSubscriptionStatuses)
}
if !cfg.State.LastTrackerUpdateTime.IsZero() {
t.Fatal("partial refresh marked fully successful")
}
}
func TestTrackerRefreshDiscardsChangedSubscriptions(t *testing.T) {
fm := &FetcherManager{}
cfg := fm.DefaultConfig().(*config)
cfg.TrackerSubscribeURLs = []string{"https://a.example/list"}
host, mu := trackerTestHost(t, cfg, nil)
host.Fetch = func(context.Context, string, int64, func([]byte) error) ([]byte, error) {
mu.Lock()
cfg.TrackerSubscribeURLs = []string{}
mu.Unlock()
return []byte("udp://fresh.example:80/announce"), nil
}
if err := fm.updateTrackers(context.Background(), host, fetcher.TriggerManual); !errors.Is(err, fetcher.ErrConfigChanged) {
t.Fatal(err)
}
if len(cfg.State.TrackerSources) != 0 {
t.Fatal("outdated source was committed")
}
}
func TestTrackerRefreshManualWorksWhenAutomaticDisabled(t *testing.T) {
fm := &FetcherManager{}
cfg := fm.DefaultConfig().(*config)
cfg.AutoUpdateTrackers = false
cfg.TrackerSubscribeURLs = []string{"https://a.example/list"}
calls := 0
host, _ := trackerTestHost(t, cfg, func(context.Context, string, int64, func([]byte) error) ([]byte, error) {
calls++
return []byte("udp://fresh.example:80/announce"), nil
})
if err := fm.updateTrackers(context.Background(), host, fetcher.TriggerScheduled); !errors.Is(err, fetcher.ErrJobSkipped) {
t.Fatal(err)
}
if calls != 0 {
t.Fatal("scheduled job fetched while disabled")
}
if err := fm.updateTrackers(context.Background(), host, fetcher.TriggerManual); err != nil {
t.Fatal(err)
}
if calls != 1 || cfg.State.LastTrackerUpdateTime.IsZero() {
t.Fatal("manual refresh did not succeed")
}
}
func TestTrackerConfigPreservesExplicitSettings(t *testing.T) {
value, err := (&FetcherManager{}).NormalizeConfig(json.RawMessage(`{
"listenPort":6881, "trackerSubscribeUrls":[], "autoUpdateTrackers":false,
"customTrackers":["udp://custom.example:80/announce"]
}`))
if err != nil {
t.Fatal(err)
}
cfg := value.(*config)
if cfg.AutoUpdateTrackers || len(cfg.TrackerSubscribeURLs) != 0 || cfg.ListenPort != 6881 {
t.Fatalf("%+v", cfg)
}
if !reflect.DeepEqual(cfg.effectiveTrackers(), cfg.CustomTrackers) {
t.Fatalf("derived trackers: %v", cfg.effectiveTrackers())
}
}
func TestParseTrackerList(t *testing.T) {
got, err := parseTrackers([]byte("\ufeffudp://tracker.example:80/announce\r\n\n<html>error</html>\nhttps://tracker.example/announce\n"))
if err != nil || len(got) != 2 {
t.Fatalf("%v %v", got, err)
}
if _, err := parseTrackers([]byte("<html>mirror error</html>")); err == nil {
t.Fatal("invalid mirror body accepted")
}
}
func TestTrackerDefaultsRegisterDailyJob(t *testing.T) {
jobs := (&FetcherManager{}).CronJobs(fetcher.ProtocolHost{})
if len(jobs) != 1 || jobs[0].Name != "update-trackers" || jobs[0].Every != 24*time.Hour {
t.Fatal(jobs)
}
}
func TestTrackerRefreshAllFailuresPersistStatusesAndKeepCache(t *testing.T) {
fm := &FetcherManager{}
cfg := fm.DefaultConfig().(*config)
address := "https://custom.example/trackers.txt"
cfg.TrackerSubscribeURLs = []string{address}
cfg.State.TrackerSources[address] = []string{"udp://cached.example:80/announce"}
cfg.State.TrackerSubscriptionStatuses[address] = trackerSubscriptionStatus{Success: true, TrackerCount: 1, LastFetchTime: time.Now().Add(-time.Hour)}
host, _ := trackerTestHost(t, cfg, func(context.Context, string, int64, func([]byte) error) ([]byte, error) {
return nil, errors.New("offline")
})
before := time.Now()
if err := fm.updateTrackers(context.Background(), host, fetcher.TriggerManual); err == nil {
t.Fatal("failure was not reported")
}
status, ok := cfg.State.TrackerSubscriptionStatuses[address]
if !ok || status.Success || status.TrackerCount != 0 || status.LastFetchTime.Before(before) {
t.Fatalf("failed fetch status not stored: %+v", status)
}
if !reflect.DeepEqual(cfg.State.TrackerSources[address], []string{"udp://cached.example:80/announce"}) {
t.Fatal("last good cache lost")
}
if !reflect.DeepEqual(cfg.effectiveTrackers(), cfg.State.TrackerSources[address]) {
t.Fatal("cached trackers not retained")
}
}
func TestTrackerConfigPrunesRemovedSubscriptions(t *testing.T) {
fm := &FetcherManager{}
cfg := fm.DefaultConfig().(*config)
address := "https://custom.example/trackers.txt"
cfg.TrackerSubscribeURLs = []string{}
cfg.State.TrackerSources[address] = []string{"udp://cached.example:80/announce"}
cfg.State.TrackerSubscriptionStatuses[address] = trackerSubscriptionStatus{Success: true, TrackerCount: 1, LastFetchTime: time.Now()}
raw, _ := json.Marshal(cfg)
value, err := fm.NormalizeConfig(raw)
if err != nil {
t.Fatal(err)
}
next := value.(*config)
if len(next.State.TrackerSources) != 0 || len(next.State.TrackerSubscriptionStatuses) != 0 || len(next.effectiveTrackers()) != 0 {
t.Fatal("removed subscription still owns cached results")
}
}
func TestTrackerRefreshTimeoutPersistsCompletedAttemptStatus(t *testing.T) {
fm := &FetcherManager{}
cfg := fm.DefaultConfig().(*config)
address := "https://custom.example/trackers.txt"
cfg.TrackerSubscribeURLs = []string{address}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
host, _ := trackerTestHost(t, cfg, func(context.Context, string, int64, func([]byte) error) ([]byte, error) {
cancel()
return nil, context.DeadlineExceeded
})
if err := fm.updateTrackers(ctx, host, fetcher.TriggerManual); !errors.Is(err, context.DeadlineExceeded) {
t.Fatal(err)
}
status, ok := cfg.State.TrackerSubscriptionStatuses[address]
if !ok || status.Success || status.TrackerCount != 0 || status.LastFetchTime.IsZero() {
t.Fatalf("timeout status lost: %+v", status)
}
}
+2 -5
View File
@@ -67,17 +67,14 @@ func (f *Fetcher) Setup(ctl *controller.Controller) {
f.ctl.GetConfig(&f.config)
}
func (f *Fetcher) Resolve(req *base.Request, opts *base.Options) error {
func (f *Fetcher) Resolve(req *base.Request) error {
link, err := parseLink(req.URL)
if err != nil {
return err
}
f.meta.Req = req
f.meta.Opts = opts
if f.meta.Opts == nil {
f.meta.Opts = &base.Options{}
}
f.meta.Res = buildResource(link)
return nil
}
Loaded 50 of 494 files, more files were not shown because too many files have changed in this diff. Show more