Compare commits

..
14 Commits
Author SHA1 Message Date
Levi 31228e3edf release: v1.5.3 2023-12-15 19:23:16 +08:00
Levi 31a32d7e14 fix: handle api panic (#302) 2023-12-15 19:20:56 +08:00
Levi 65fd6fb489 fix: http pause handle (#303) 2023-12-15 19:15:38 +08:00
Levi 953b878a89 fix: adapt mobile proxy settings ui (#301) 2023-12-15 19:09:48 +08:00
Elliot ae4ca3da58 feat: improve zh-TW translation (#300) 2023-12-14 18:06:49 +08:00
Levi d09bd180af release: v1.5.2 2023-12-13 09:38:34 +08:00
liwei 3d24e353c5 docs: docker data path mounted 2023-12-13 08:57:57 +08:00
Levi ce7489b0c0 feat: support volume storage dir on docker (#291) 2023-12-13 08:51:22 +08:00
Levi 2b5091ba21 feat: improve extension engine (#298)
1. support onError event
2. add built-in MessageError
2023-12-12 19:00:42 +08:00
Levi ff32971aca fix: repair invalid filename (#296) 2023-12-12 18:40:36 +08:00
Levi f2af61f9aa fix: sort of completed tasks (#297) 2023-12-12 18:36:40 +08:00
Elliot d935b9d631 feat: improve zh_TW translate (#287) 2023-12-04 23:01:49 +08:00
liwei ae66d1d83d docs: fix contributing 2023-12-04 13:41:27 +08:00
madoka773 aa91ca1d50 ci: linux build use libappindicator (#286) 2023-12-04 13:36:27 +08:00
42 changed files with 492 additions and 177 deletions

No files matched your search

+2
View File
@@ -19,6 +19,8 @@ categories:
label: 'doc'
- title: '🚀 CI/CD'
label: 'ci'
- title: '🌎 Internationalization'
label: 'translation'
change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
version-resolver:
major:
+2 -2
View File
@@ -224,7 +224,7 @@ jobs:
flutter-version: ${{ env.FLUTTER_VERSION }}
- run: |
sudo apt-get update -y
sudo apt-get install -y ninja-build libgtk-3-dev libayatana-appindicator3-dev
sudo apt-get install -y ninja-build libgtk-3-dev libappindicator3-dev
- name: Build
env:
VERSION: ${{ needs.get-release.outputs.tag_name }}
@@ -405,7 +405,7 @@ jobs:
plugin: flutter
stage-packages:
- libgtk-3-dev
- libayatana-appindicator3-dev
- libappindicator3-dev
flutter-target: lib/main.dart
EOF
+1 -1
View File
@@ -16,7 +16,7 @@ flutter 项目即可运行。
Gopeed 的国际化文件位于 `ui/flutter/lib/i18n/langs` 目录下,只需要在该目录下添加对应的语言文件即可。
请注意以 `en_US.json` 为参照进行翻译。
请注意以 `en_us.dart` 为参照进行翻译。
## flutter开发
+1 -1
View File
@@ -15,7 +15,7 @@
Gopeed 的翻譯文件位於 `ui/flutter/lib/i18n/langs` 目錄中,只需要修改或新建翻譯文件即可。
請以 `en_US.json` 作為參照。
請以 `en_us.dart` 作為參照。
## flutter開發
+1
View File
@@ -17,5 +17,6 @@ FROM alpine:3.14.2
LABEL maintainer="monkeyWie"
WORKDIR /app
COPY --from=go /app/dist/gopeed ./
VOLUME ["/app/storage"]
EXPOSE 9999
ENTRYPOINT ["./gopeed"]
+4 -3
View File
@@ -16,7 +16,7 @@
Gopeed (full name Go Speed), a high-speed downloader developed by `Golang` + `Flutter`, supports (HTTP, BitTorrent, Magnet) protocol, and supports all platforms. In addition to basic download functions, Gopeed is also a highly customizable downloader that supports implementing more features through integration with [APIs](https://docs.gopeed.com/dev-api.html) or installation and development of [extensions](https://docs.gopeed.com/dev-extension.html).
Visit ✈ [Official Website](https://gopeed.com) | 📖 [Develop Docs](https://docs.gopeed.com)
Visit ✈ [Official Website](https://gopeed.com) | 📖 [Official Docs](https://docs.gopeed.com)
## Install
@@ -47,9 +47,11 @@ go install github.com/GopeedLab/gopeed/cmd/gopeed@latest
#### Start directly
```bash
docker run -d -p 9999:9999 -v /path/to/download:/root/Downloads liwei2633/gopeed
docker run -d -p 9999:9999 -v /path/to/download:/root/Downloads -v /path/to/storage:/app/storage liwei2633/gopeed
```
[More details for docker](https://docs.gopeed.com/install.html#docker-installation)
#### Using Docker Compose
```bash
@@ -160,7 +162,6 @@ cd ../../
rm -rf cmd/web/dist
cp -r ui/flutter/build/web cmd/web/dist
go build -tags nosqlite,web -ldflags="-s -w" -o bin/ github.com/GopeedLab/gopeed/cmd/web
go run cmd/web/main.go
```
## Credits
+3 -2
View File
@@ -46,9 +46,11 @@ go install github.com/GopeedLab/gopeed/cmd/gopeed@latest
#### 直接始める
```bash
docker run -d -p 9999:9999 -v /path/to/download:/root/Downloads liwei2633/gopeed
docker run -d -p 9999:9999 -v /path/to/download:/root/Downloads -v /path/to/storage:/app/storage liwei2633/gopeed
```
[Dockerの使用については](https://docs.gopeed.com/install.html#docker-installation)
#### Docker Compose の使用
```bash
@@ -159,7 +161,6 @@ cd ../../
rm -rf cmd/web/dist
cp -r ui/flutter/build/web cmd/web/dist
go build -tags nosqlite,web -ldflags="-s -w" -o bin/ github.com/GopeedLab/gopeed/cmd/web
go run cmd/web/main.go
```
## 感謝
+4 -3
View File
@@ -15,7 +15,7 @@
Gopeed(全称 Go Speed),直译过来中文名叫做`够快下载器`(不是狗屁下载器!),是一款由`Golang`+`Flutter`开发的高速下载器,支持(HTTP、BitTorrent、Magnet)协议下载,并且支持全平台使用。除了基本的下载功能外,Gopeed 还是一款高度可定制化的下载器,支持通过对接[APIs](https://docs.gopeed.com/zh/dev-api.html)或者安装和开发[扩展](https://docs.gopeed.com/zh/dev-extension.html)来实现更多的功能。
访问 ✈ [官方网站](https://gopeed.com/zh-CN) | 📖 [开发文档](https://docs.gopeed.com/zh/)
访问 ✈ [官方网站](https://gopeed.com/zh-CN) | 📖 [官方文档](https://docs.gopeed.com/zh/)
## 安装
@@ -46,9 +46,11 @@ go install github.com/GopeedLab/gopeed/cmd/gopeed@latest
#### 直接运行
```bash
docker run -d -p 9999:9999 -v /path/to/download:/root/Downloads liwei2633/gopeed
docker run -d -p 9999:9999 -v /path/to/download:/root/Downloads -v /path/to/storage:/app/storage liwei2633/gopeed
```
[更多关于 Docker 的使用](https://docs.gopeed.com/zh/install.html#docker-%E5%AE%89%E8%A3%85)
#### 使用 Docker Compose
```bash
@@ -159,7 +161,6 @@ cd ../../
rm -rf cmd/web/dist
cp -r ui/flutter/build/web cmd/web/dist
go build -tags nosqlite,web -ldflags="-s -w" -o bin/ github.com/GopeedLab/gopeed/cmd/web
go run cmd/web/main.go
```
## 感谢
+3 -2
View File
@@ -46,9 +46,11 @@ go install github.com/GopeedLab/gopeed/cmd/gopeed@latest
#### 直接執行
```bash
docker run -d -p 9999:9999 -v /path/to/download:/root/Downloads liwei2633/gopeed
docker run -d -p 9999:9999 -v /path/to/download:/root/Downloads -v /path/to/storage:/app/storage liwei2633/gopeed
```
[更多關於 Docker 的使用](https://docs.gopeed.com/zh/install.html#docker-%E5%AE%89%E8%A3%85)
#### 使用 Docker Compose
```bash
@@ -159,7 +161,6 @@ cd ../../
rm -rf cmd/web/dist
cp -r ui/flutter/build/web cmd/web/dist
go build -tags nosqlite,web -ldflags="-s -w" -o bin/ github.com/GopeedLab/gopeed/cmd/web
go run cmd/web/main.go
```
## 感謝
+9
View File
@@ -9,6 +9,8 @@ import (
"github.com/GopeedLab/gopeed/cmd"
"github.com/GopeedLab/gopeed/pkg/rest/model"
"io/fs"
"os"
"path/filepath"
)
//go:embed dist/*
@@ -29,10 +31,17 @@ func main() {
}
}
exe, err := os.Executable()
if err != nil {
panic(err)
}
dir := filepath.Dir(exe)
cfg := &model.StartConfig{
Network: "tcp",
Address: fmt.Sprintf("%s:%d", *args.Address, *args.Port),
Storage: model.StorageBolt,
StorageDir: filepath.Join(dir, "storage"),
ApiToken: *args.ApiToken,
ProductionMode: true,
WebEnable: true,
+3 -2
View File
@@ -4,8 +4,9 @@ services:
gopeed:
container_name: gopeed
ports:
- 9999:9999 # HTTP port (host:container)
- "9999:9999" # HTTP port (host:container)
image: liwei2633/gopeed
volumes:
- ~/Downloads:/root/Downloads # replace with your prefer path
- ~/gopeed/Downloads:/root/Downloads # mount download path
#- ~/gopeed/storage:/app/storage # if you need to mount storage path, uncomment this line
restart: unless-stopped
+14 -16
View File
@@ -306,26 +306,13 @@ func (f *Fetcher) fetchChunk(index int, ctx context.Context) (err error) {
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != base.HttpCodeOK && resp.StatusCode != base.HttpCodePartialContent {
err = NewRequestError(resp.StatusCode, resp.Status)
return err
}
return nil
}()
if err != nil {
// If canceled, do not retry
if errors.Is(err, context.Canceled) {
return
}
// retry request after 1 second
chunk.retryTimes = chunk.retryTimes + 1
time.Sleep(time.Second)
continue
}
// Http request success, reset retry times
chunk.retryTimes = 0
err = func() error {
defer resp.Body.Close()
// Http request success, reset retry times
chunk.retryTimes = 0
for {
n, err := resp.Body.Read(buf)
if n > 0 {
@@ -342,7 +329,18 @@ func (f *Fetcher) fetchChunk(index int, ctx context.Context) (err error) {
return err
}
}
return nil
}()
if err != nil {
// If canceled, do not retry
if errors.Is(err, context.Canceled) {
return
}
// retry request after 1 second
chunk.retryTimes = chunk.retryTimes + 1
time.Sleep(time.Second)
continue
}
// if chunk is completed, reset other not complete chunks retry times
if err == nil {
for _, c := range f.chunks {
+42 -34
View File
@@ -8,6 +8,8 @@ import (
"github.com/GopeedLab/gopeed/pkg/base"
"github.com/GopeedLab/gopeed/pkg/util"
gonanoid "github.com/matoous/go-nanoid/v2"
"github.com/rs/zerolog"
"github.com/rs/zerolog/pkgerrors"
"github.com/virtuald/go-paniclog"
"os"
"path/filepath"
@@ -91,6 +93,7 @@ func NewDownloader(cfg *DownloaderConfig) *Downloader {
}
}
zerolog.ErrorStackMarshaler = pkgerrors.MarshalStack
d.Logger = logger.NewLogger(cfg.ProductionMode, filepath.Join(cfg.StorageDir, "logs", "core.log"))
d.ExtensionLogger = logger.NewLogger(cfg.ProductionMode, filepath.Join(cfg.StorageDir, "logs", "extension.log"))
if cfg.ProductionMode {
@@ -225,7 +228,10 @@ func (d *Downloader) Resolve(req *base.Request) (rr *ResolveResult, err error) {
return
}
res := d.triggerOnResolve(req)
res, err := d.triggerOnResolve(req)
if err != nil {
return
}
if res != nil && len(res.Files) > 0 {
rr = &ResolveResult{
Res: res,
@@ -525,9 +531,6 @@ func (d *Downloader) emit(eventKey EventKey, task *Task, errs ...error) {
Task: task,
Err: err,
})
if eventKey == EventKeyError {
d.emit(EventKeyFinally, task, err)
}
}
}
@@ -572,28 +575,38 @@ func (d *Downloader) getProtocolConfig(name string, v any) bool {
func (d *Downloader) watch(task *Task) {
err := task.fetcher.Wait()
if err != nil {
task.Status = base.DownloadStatusError
d.emit(EventKeyError, task, err)
} else {
task.Progress.Used = task.timer.Used()
if task.Meta.Res.Size == 0 {
task.Meta.Res.Size = task.fetcher.Progress().TotalDownloaded()
}
used := task.Progress.Used / int64(time.Second)
if used == 0 {
used = 1
}
totalSize := task.Meta.Res.Size
task.Progress.Speed = totalSize / used
task.Progress.Downloaded = totalSize
task.Status = base.DownloadStatusDone
d.storage.Put(bucketTask, task.ID, task.clone())
d.emit(EventKeyDone, task)
d.emit(EventKeyFinally, task, err)
d.doOnError(task, err)
return
}
task.Progress.Used = task.timer.Used()
if task.Meta.Res.Size == 0 {
task.Meta.Res.Size = task.fetcher.Progress().TotalDownloaded()
}
used := task.Progress.Used / int64(time.Second)
if used == 0 {
used = 1
}
totalSize := task.Meta.Res.Size
task.Progress.Speed = totalSize / used
task.Progress.Downloaded = totalSize
task.updateStatus(base.DownloadStatusDone)
d.storage.Put(bucketTask, task.ID, task.clone())
d.emit(EventKeyDone, task)
d.emit(EventKeyFinally, task, err)
d.notifyRunning()
}
func (d *Downloader) doOnError(task *Task, err error) {
d.Logger.Warn().Err(err).Msgf("task download failed, task id: %s", task.ID)
task.updateStatus(base.DownloadStatusError)
d.triggerOnError(task, err)
if task.Status == base.DownloadStatusError {
d.emit(EventKeyError, task, err)
d.emit(EventKeyFinally, task, err)
d.notifyRunning()
}
}
func (d *Downloader) restoreFetcher(task *Task) error {
if task.fetcher == nil {
fb, err := d.parseFb(task.Meta.Req.URL)
@@ -658,7 +671,6 @@ func (d *Downloader) doCreate(fetcher fetcher.Fetcher, opts *base.Options) (task
task.fetcher = fetcher
task.Meta = fetcher.Meta()
task.Progress = &Progress{}
task.Status = base.DownloadStatusReady
initTask(task)
if err = fetcher.Create(opts); err != nil {
return
@@ -697,25 +709,18 @@ func (d *Downloader) doStart(task *Task) (err error) {
}
}
cloneTask := task.clone()
isCreate := task.Status == base.DownloadStatusReady
task.Status = base.DownloadStatusRunning
doStart := func() error {
task.lock.Lock()
defer task.lock.Unlock()
req := d.triggerOnStart(cloneTask)
if req != nil {
task.Meta.Req = req
task.fetcher.Meta().Req = req
}
d.triggerOnStart(task)
task.updateStatus(base.DownloadStatusRunning)
if task.Meta.Res == nil {
err := task.fetcher.Resolve(task.Meta.Req)
if err != nil {
task.Status = base.DownloadStatusError
d.emit(EventKeyError, task, err)
return err
}
task.Meta.Res = task.fetcher.Meta().Res
@@ -724,8 +729,10 @@ func (d *Downloader) doStart(task *Task) (err error) {
if isCreate {
d.checkDuplicateLock.Lock()
defer d.checkDuplicateLock.Unlock()
task.Meta.Opts.Name = util.ReplaceInvalidFilename(task.Meta.Opts.Name)
// check if the download file is duplicated and rename it automatically.
if task.Meta.Res.Name != "" {
task.Meta.Res.Name = util.ReplaceInvalidFilename(task.Meta.Res.Name)
fullDirPath := task.Meta.FolderPath()
newName, err := util.CheckDuplicateAndRename(fullDirPath)
if err != nil {
@@ -733,6 +740,7 @@ func (d *Downloader) doStart(task *Task) (err error) {
}
task.Meta.Opts.Name = newName
} else {
task.Meta.Res.Files[0].Name = util.ReplaceInvalidFilename(task.Meta.Res.Files[0].Name)
fullFilePath := task.Meta.SingleFilepath()
newName, err := util.CheckDuplicateAndRename(fullFilePath)
if err != nil {
@@ -758,7 +766,7 @@ func (d *Downloader) doStart(task *Task) (err error) {
go func() {
err := doStart()
if err != nil {
d.Logger.Error().Stack().Err(err).Msgf("start task failed, task id: %s", task.ID)
d.doOnError(task, err)
}
}()
@@ -768,7 +776,7 @@ func (d *Downloader) doStart(task *Task) (err error) {
func (d *Downloader) doPause(task *Task) (err error) {
err = func() error {
if task.Status != base.DownloadStatusDone {
task.Status = base.DownloadStatusPause
task.updateStatus(base.DownloadStatusPause)
task.timer.Pause()
if task.fetcher != nil {
if err := task.fetcher.Pause(); err != nil {
+9 -1
View File
@@ -3,6 +3,7 @@ package engine
import (
_ "embed"
"errors"
gojaerror "github.com/GopeedLab/gopeed/pkg/download/engine/inject/error"
"github.com/GopeedLab/gopeed/pkg/download/engine/inject/file"
"github.com/GopeedLab/gopeed/pkg/download/engine/inject/formdata"
"github.com/GopeedLab/gopeed/pkg/download/engine/inject/vm"
@@ -125,6 +126,9 @@ func NewEngine(cfg *Config) *Engine {
runtime.SetFieldNameMapper(goja.TagFieldNameMapper("json", true))
vm.Enable(runtime)
gojaurl.Enable(runtime)
if err := gojaerror.Enable(runtime); err != nil {
return
}
if err := file.Enable(runtime); err != nil {
return
}
@@ -167,7 +171,11 @@ func resolveResult(value goja.Value) (any, error) {
case goja.PromiseStateFulfilled:
return p.Result().Export(), nil
case goja.PromiseStateRejected:
return nil, errors.New(p.Result().String())
if err, ok := p.Result().Export().(error); ok {
return nil, err
} else {
return nil, errors.New(p.Result().String())
}
}
}
return export, nil
+13
View File
@@ -7,7 +7,9 @@ import (
"errors"
"fmt"
"github.com/GopeedLab/gopeed/internal/test"
gojaerror "github.com/GopeedLab/gopeed/pkg/download/engine/inject/error"
"github.com/GopeedLab/gopeed/pkg/download/engine/inject/file"
gojautil "github.com/GopeedLab/gopeed/pkg/download/engine/util"
"github.com/GopeedLab/gopeed/pkg/util"
"github.com/dop251/goja"
"io"
@@ -20,6 +22,7 @@ import (
)
func TestPolyfill(t *testing.T) {
doTestPolyfill(t, "MessageError")
doTestPolyfill(t, "XMLHttpRequest")
doTestPolyfill(t, "Blob")
doTestPolyfill(t, "FormData")
@@ -27,6 +30,16 @@ func TestPolyfill(t *testing.T) {
doTestPolyfill(t, "__gopeed_create_vm")
}
func TestError(t *testing.T) {
engine := NewEngine(nil)
_, err := engine.RunString(`
throw new MessageError('test');
`)
if me, ok := gojautil.AssertError[*gojaerror.MessageError](err); !ok {
t.Fatalf("expect MessageError, but got %v", me)
}
}
func TestFetch(t *testing.T) {
server := startServer()
defer server.Close()
@@ -0,0 +1,29 @@
package error
import (
"github.com/dop251/goja"
)
type MessageError struct {
Message string `json:"message"`
}
func (e *MessageError) Error() string {
return e.Message
}
func Enable(runtime *goja.Runtime) error {
messageError := runtime.ToValue(func(call goja.ConstructorCall) *goja.Object {
var message string
if len(call.Arguments) > 0 {
message = call.Arguments[0].String()
}
instance := &MessageError{
Message: message,
}
instanceValue := runtime.ToValue(instance).(*goja.Object)
instanceValue.SetPrototype(call.This.Prototype())
return instanceValue
})
return runtime.Set("MessageError", messageError)
}
-9
View File
@@ -1,9 +0,0 @@
package inject
import (
"github.com/dop251/goja"
)
func ThrowTypeError(vm *goja.Runtime, msg string) {
panic(vm.NewTypeError(msg))
}
+2 -2
View File
@@ -2,9 +2,9 @@ package xhr
import (
"bytes"
"github.com/GopeedLab/gopeed/pkg/download/engine/inject"
"github.com/GopeedLab/gopeed/pkg/download/engine/inject/file"
"github.com/GopeedLab/gopeed/pkg/download/engine/inject/formdata"
"github.com/GopeedLab/gopeed/pkg/download/engine/util"
"github.com/dop251/goja"
"io"
"mime/multipart"
@@ -316,7 +316,7 @@ func (xhr *XMLHttpRequest) parseData(data goja.Value) any {
func Enable(runtime *goja.Runtime, proxyUrl *url.URL) error {
progressEvent := runtime.ToValue(func(call goja.ConstructorCall) *goja.Object {
if len(call.Arguments) < 1 {
inject.ThrowTypeError(runtime, "Failed to construct 'ProgressEvent': 1 argument required, but only 0 present.")
util.ThrowTypeError(runtime, "Failed to construct 'ProgressEvent': 1 argument required, but only 0 present.")
}
instance := &ProgressEvent{
Type: call.Argument(0).String(),
+24
View File
@@ -0,0 +1,24 @@
package util
import (
"github.com/dop251/goja"
)
func ThrowTypeError(vm *goja.Runtime, msg string) {
panic(vm.NewTypeError(msg))
}
func AssertError[T error](err error) (t T, r bool) {
if err == nil {
return
}
if e, ok := err.(T); ok {
return e, true
}
if e, ok := err.(*goja.Exception); ok {
if ee, okk := e.Value().Export().(T); okk {
return ee, true
}
}
return
}
+64 -14
View File
@@ -6,6 +6,8 @@ import (
"github.com/GopeedLab/gopeed/internal/logger"
"github.com/GopeedLab/gopeed/pkg/base"
"github.com/GopeedLab/gopeed/pkg/download/engine"
gojaerror "github.com/GopeedLab/gopeed/pkg/download/engine/inject/error"
gojautil "github.com/GopeedLab/gopeed/pkg/download/engine/util"
"github.com/GopeedLab/gopeed/pkg/util"
"github.com/dop251/goja"
"github.com/go-git/go-git/v5"
@@ -35,7 +37,7 @@ type ActivationEvent string
const (
EventOnResolve ActivationEvent = "onResolve"
EventOnStart ActivationEvent = "onStart"
//EventOnError ActivationEvent = "onError"
EventOnError ActivationEvent = "onError"
//EventOnDone ActivationEvent = "onDone"
)
@@ -238,8 +240,8 @@ func (d *Downloader) parseExtensionByPath(path string) (*Extension, error) {
return &ext, nil
}
func (d *Downloader) triggerOnResolve(req *base.Request) (res *base.Resource) {
doTrigger(d,
func (d *Downloader) triggerOnResolve(req *base.Request) (res *base.Resource, err error) {
err = doTrigger(d,
EventOnResolve,
req,
&OnResolveContext{
@@ -252,6 +254,10 @@ func (d *Downloader) triggerOnResolve(req *base.Request) (res *base.Resource) {
gopeed.Logger.logger.Warn().Err(err).Msgf("[%s] resource invalid", ext.buildIdentity())
return
}
ctx.Res.Name = util.ReplaceInvalidFilename(ctx.Res.Name)
for _, file := range ctx.Res.Files {
file.Name = util.ReplaceInvalidFilename(file.Name)
}
ctx.Res.CalcSize(nil)
}
res = ctx.Res
@@ -260,12 +266,12 @@ func (d *Downloader) triggerOnResolve(req *base.Request) (res *base.Resource) {
return
}
func (d *Downloader) triggerOnStart(task *Task) (req *base.Request) {
func (d *Downloader) triggerOnStart(task *Task) {
doTrigger(d,
EventOnStart,
task.Meta.Req,
&OnStartContext{
Task: task,
Task: NewExtensionTask(d, task),
},
func(ext *Extension, gopeed *Instance, ctx *OnStartContext) {
// Validate request structure
@@ -274,14 +280,25 @@ func (d *Downloader) triggerOnStart(task *Task) (req *base.Request) {
gopeed.Logger.logger.Warn().Err(err).Msgf("[%s] request invalid", ext.buildIdentity())
return
}
req = ctx.Task.Meta.Req
}
},
)
return
}
func doTrigger[T any](d *Downloader, event ActivationEvent, req *base.Request, ctx T, handler func(ext *Extension, gopeed *Instance, ctx T)) {
func (d *Downloader) triggerOnError(task *Task, err error) {
doTrigger(d,
EventOnError,
task.Meta.Req,
&OnErrorContext{
Task: NewExtensionTask(d, task),
Error: err,
},
nil,
)
}
func doTrigger[T any](d *Downloader, event ActivationEvent, req *base.Request, ctx T, handler func(ext *Extension, gopeed *Instance, ctx T)) error {
// init extension global object
gopeed := &Instance{
Events: make(InstanceEvents),
@@ -343,13 +360,20 @@ func doTrigger[T any](d *Downloader, event ActivationEvent, req *base.Request, c
gopeed.Logger.logger.Error().Err(err).Msgf("[%s] call function failed: %s", ext.buildIdentity(), event)
return
}
handler(ext, gopeed, ctx)
if handler != nil {
handler(ext, gopeed, ctx)
}
}
}()
}
}
}
return
// Only return MessageError
if me, ok := gojautil.AssertError[*gojaerror.MessageError](err); ok {
return me
}
return nil
}
func (d *Downloader) ExtensionPath(ext *Extension) string {
@@ -577,10 +601,10 @@ func (h InstanceEvents) OnStart(fn goja.Callable) {
h.register(EventOnStart, fn)
}
//func (h InstanceEvents) OnError(fn goja.Callable) {
// h.register(HookEventOnError, fn)
//}
//
func (h InstanceEvents) OnError(fn goja.Callable) {
h.register(EventOnError, fn)
}
//func (h InstanceEvents) OnDone(fn goja.Callable) {
// h.register(HookEventOnDone, fn)
//}
@@ -649,7 +673,33 @@ type OnResolveContext struct {
}
type OnStartContext struct {
Task *Task `json:"task"`
Task *ExtensionTask `json:"task"`
}
type OnErrorContext struct {
Task *ExtensionTask `json:"task"`
Error error `json:"error"`
}
type ExtensionTask struct {
*Task
download *Downloader
}
func NewExtensionTask(download *Downloader, task *Task) *ExtensionTask {
return &ExtensionTask{
Task: task.clone(),
download: download,
}
}
func (t *ExtensionTask) Continue() error {
return t.download.Continue(t.ID)
}
func (t *ExtensionTask) Pause() error {
return t.download.Pause(t.ID)
}
func parseSettings(settings []*Setting) map[string]any {
+66 -20
View File
@@ -4,6 +4,7 @@ import (
"errors"
"github.com/GopeedLab/gopeed/internal/logger"
"github.com/GopeedLab/gopeed/pkg/base"
gojaerror "github.com/GopeedLab/gopeed/pkg/download/engine/inject/error"
"github.com/dop251/goja"
"os"
"testing"
@@ -165,7 +166,7 @@ func TestDownloader_UpgradeExtension(t *testing.T) {
})
}
func TestDownloader_ExtensionByOnStart(t *testing.T) {
func TestDownloader_Extension_OnStart(t *testing.T) {
downloadAndCheck := func(req *base.Request) {
setupDownloader(func(downloader *Downloader) {
if _, err := downloader.InstallExtensionByFolder("./testdata/extensions/on_start", false); err != nil {
@@ -192,10 +193,10 @@ func TestDownloader_ExtensionByOnStart(t *testing.T) {
}
task := downloader.GetTask(id)
if task.Meta.Req.URL != "https://github.com" {
panic("extension on start modify url error")
t.Fatalf("except url: https://github.com, actual: %s", task.Meta.Req.URL)
}
if task.Meta.Req.Labels["modified"] != "true" {
panic("extension on start modify label error")
t.Fatalf("except label: modified=true, actual: %s", task.Meta.Req.Labels["modified"])
}
})
}
@@ -207,13 +208,54 @@ func TestDownloader_ExtensionByOnStart(t *testing.T) {
// label match
downloadAndCheck(&base.Request{
URL: "https://xxx.com",
URL: "https://test.com",
Labels: map[string]string{
"test": "true",
},
})
}
func TestDownloader_Extension_OnError(t *testing.T) {
setupDownloader(func(downloader *Downloader) {
if _, err := downloader.InstallExtensionByFolder("./testdata/extensions/on_error", false); err != nil {
t.Fatal(err)
}
errCh := make(chan error, 1)
downloader.Listener(func(event *Event) {
if event.Key == EventKeyFinally {
errCh <- event.Err
}
})
id, err := downloader.CreateDirect(&base.Request{
URL: "https://github.com/gopeed/test/404",
Labels: map[string]string{
"test": "true",
},
}, nil)
if err != nil {
t.Fatal(err)
}
select {
case err = <-errCh:
break
case <-time.After(time.Second * 10):
err = errors.New("timeout")
}
if err != nil {
panic("extension on error download error: " + err.Error())
}
// extension on error modify url and continue download
task := downloader.GetTask(id)
if task.Meta.Req.URL != "https://github.com" {
t.Fatalf("except url: https://github.com, actual: %s", task.Meta.Req.URL)
}
if task.Status != base.DownloadStatusDone {
t.Fatalf("except status is done, actual: %s", task.Status)
}
})
}
func TestDownloader_Extension_Errors(t *testing.T) {
setupDownloader(func(downloader *Downloader) {
if _, err := downloader.InstallExtensionByFolder("./testdata/extensions/script_error", false); err != nil {
@@ -244,6 +286,26 @@ func TestDownloader_Extension_Errors(t *testing.T) {
t.Fatal("function error catch failed")
}
})
setupDownloader(func(downloader *Downloader) {
if _, err := downloader.InstallExtensionByFolder("./testdata/extensions/message_error", false); err != nil {
t.Fatal(err)
}
_, err := downloader.Resolve(&base.Request{
URL: "https://github.com/test",
})
if err == nil {
t.Fatalf("except error, but got nil")
}
me, ok := err.(*gojaerror.MessageError)
if !ok {
t.Fatalf("except MessageError type, but got %s", err)
}
want := "test"
if me.Error() != want {
t.Fatalf("except MessageError message %s, but got %s", want, me.Message)
}
})
}
func TestDownloader_Extension_Settings(t *testing.T) {
@@ -336,22 +398,6 @@ func TestDownloader_DeleteExtension(t *testing.T) {
})
}
func TestDownloader_Extension_OnResolve(t *testing.T) {
setupDownloader(func(downloader *Downloader) {
installedExt, err := downloader.InstallExtensionByFolder("./testdata/extensions/settings_all", false)
if err != nil {
t.Fatal(err)
}
if err := downloader.DeleteExtension(installedExt.Identity); err != nil {
t.Fatal(err)
}
extensions := downloader.GetExtensions()
if len(extensions) != 0 {
t.Fatal("extension delete fail")
}
})
}
func TestDownloader_Extension_Logger(t *testing.T) {
logger := logger.NewLogger(false, "")
il := newInstanceLogger(&Extension{
+8
View File
@@ -24,6 +24,7 @@ type Task struct {
Status base.Status `json:"status"`
Progress *Progress `json:"progress"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
fetcherBuilder fetcher.FetcherBuilder
fetcher fetcher.Fetcher
@@ -41,9 +42,15 @@ func NewTask() *Task {
ID: id,
Status: base.DownloadStatusReady,
CreatedAt: time.Now(),
UpdatedAt: time.Now(),
}
}
func (t *Task) updateStatus(status base.Status) {
t.UpdatedAt = time.Now()
t.Status = status
}
func (t *Task) clone() *Task {
return &Task{
ID: t.ID,
@@ -51,6 +58,7 @@ func (t *Task) clone() *Task {
Status: t.Status,
Progress: t.Progress,
CreatedAt: t.CreatedAt,
UpdatedAt: t.UpdatedAt,
}
}
@@ -1,5 +1,6 @@
gopeed.events.onResolve(async function (ctx) {
const aaa = {};
// access undefined property
gopeed.logger.info(aaa.bbb.ccc);
ctx.res = {
@@ -0,0 +1,3 @@
gopeed.events.onResolve(async function (ctx) {
throw new MessageError("test");
});
@@ -0,0 +1,16 @@
{
"name": "message-error",
"title": "gopeed extension message error test",
"version": "0.0.1",
"scripts": [
{
"event": "onResolve",
"match": {
"urls": [
"*://github.com/*"
]
},
"entry": "index.js"
}
]
}
+8
View File
@@ -0,0 +1,8 @@
gopeed.events.onError(async function (ctx) {
gopeed.logger.info("url", ctx.task.meta.req.url);
gopeed.logger.info("error", ctx.error);
ctx.task.meta.req.url = "https://github.com";
ctx.task.pause();
ctx.task.continue();
});
+16
View File
@@ -0,0 +1,16 @@
{
"name": "on-error",
"title": "gopeed extension on error event test",
"version": "0.0.1",
"scripts": [
{
"event": "onError",
"match": {
"labels": [
"test"
]
},
"entry": "index.js"
}
]
}
+1
View File
@@ -1,4 +1,5 @@
gopeed.events.onStart(async function (ctx) {
gopeed.logger.info("url", ctx.task.meta.req.url);
ctx.task.meta.req.url = "https://github.com";
ctx.task.meta.req.labels['modified'] = 'true';
});
+16
View File
@@ -3,11 +3,13 @@ package rest
import (
"context"
"encoding/json"
"fmt"
"github.com/GopeedLab/gopeed/pkg/download"
"github.com/GopeedLab/gopeed/pkg/rest/model"
"github.com/GopeedLab/gopeed/pkg/util"
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
"github.com/pkg/errors"
"net"
"net/http"
"os"
@@ -131,6 +133,20 @@ func BuildServer(startCfg *model.StartConfig) (*http.Server, net.Listener, error
})
}
// recover panic
r.Use(func(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
if v := recover(); v != nil {
err := errors.WithStack(fmt.Errorf("%v", v))
Downloader.Logger.Error().Stack().Err(err).Msg("http server panic")
WriteJson(w, model.NewErrorResult(err.Error(), model.CodeError))
}
}()
h.ServeHTTP(w, r)
})
})
srv = &http.Server{Handler: handlers.CORS(
handlers.AllowedHeaders([]string{"Content-Type", "X-Api-Token", "X-Target-Uri"}),
handlers.AllowedMethods([]string{"GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"}),
+2
View File
@@ -204,12 +204,14 @@ func TestPauseAllAndContinueALLTasks(t *testing.T) {
// continue all
httpRequestCheckOk[any](http.MethodPut, "/api/v1/tasks/continue", nil)
time.Sleep(time.Millisecond * 100)
tasks := httpRequestCheckOk[[]*download.Task](http.MethodGet, fmt.Sprintf("/api/v1/tasks?status=%s", base.DownloadStatusRunning), nil)
if len(tasks) != cfg.MaxRunning {
t.Errorf("ContinueAllTasks() got = %v, want %v", len(tasks), cfg.MaxRunning)
}
// pause all
httpRequestCheckOk[any](http.MethodPut, "/api/v1/tasks/pause", nil)
time.Sleep(time.Millisecond * 100)
tasks = httpRequestCheckOk[[]*download.Task](http.MethodGet, fmt.Sprintf("/api/v1/tasks?status=%s", base.DownloadStatusPause), nil)
if len(tasks) != total {
t.Errorf("PauseAllTasks() got = %v, want %v", len(tasks), total)
+11
View File
@@ -183,3 +183,14 @@ func IsExistsFile(path string) bool {
}
return false
}
// ReplaceInvalidFilename replace invalid path characters
func ReplaceInvalidFilename(path string) string {
if path == "" {
return ""
}
for _, char := range invalidPathChars {
path = strings.ReplaceAll(path, char, "_")
}
return path
}
+6
View File
@@ -0,0 +1,6 @@
//go:build !windows
// +build !windows
package util
var invalidPathChars = []string{`/`, `:`}
+47
View File
@@ -178,3 +178,50 @@ func TestIsExistsFile(t *testing.T) {
})
}
}
func TestReplaceInvalidFilename(t *testing.T) {
type args struct {
path string
}
tests := []struct {
name string
args args
want string
}{
{
name: "blank",
args: args{
path: "",
},
want: "",
},
{
name: "normal",
args: args{
path: "test.txt",
},
want: "test.txt",
},
{
name: "case1",
args: args{
path: "te/st.txt",
},
want: "te_st.txt",
},
{
name: "case2",
args: args{
path: "te/st:.txt",
},
want: "te_st_.txt",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ReplaceInvalidFilename(tt.args.path); got != tt.want {
t.Errorf("ReplaceInvalidFilename() = %v, want %v", got, tt.want)
}
})
}
}
+3
View File
@@ -0,0 +1,3 @@
package util
var invalidPathChars = []string{`\`, `/`, `:`, `*`, `?`, `"`, `<`, `>`, `|`}
+2
View File
@@ -13,6 +13,7 @@ class Task {
Status status;
Progress progress;
DateTime createdAt;
DateTime updatedAt;
Task({
required this.id,
@@ -20,6 +21,7 @@ class Task {
required this.status,
required this.progress,
required this.createdAt,
required this.updatedAt,
});
factory Task.fromJson(Map<String, dynamic> json) => _$TaskFromJson(json);
+2
View File
@@ -12,6 +12,7 @@ Task _$TaskFromJson(Map<String, dynamic> json) => Task(
status: $enumDecode(_$StatusEnumMap, json['status']),
progress: Progress.fromJson(json['progress'] as Map<String, dynamic>),
createdAt: DateTime.parse(json['createdAt'] as String),
updatedAt: DateTime.parse(json['updatedAt'] as String),
);
Map<String, dynamic> _$TaskToJson(Task instance) => <String, dynamic>{
@@ -20,6 +21,7 @@ Map<String, dynamic> _$TaskToJson(Task instance) => <String, dynamic>{
'status': _$StatusEnumMap[instance.status]!,
'progress': instance.progress.toJson(),
'createdAt': instance.createdAt.toIso8601String(),
'updatedAt': instance.updatedAt.toIso8601String(),
};
const _$StatusEnumMap = {
@@ -387,35 +387,33 @@ class SettingView extends GetView<SettingController> {
},
);
final items = <Widget>[
SizedBox(
width: 150,
child: DropdownButtonFormField<String>(
value: proxy.scheme,
onChanged: (value) async {
if (value != null && value != proxy.scheme) {
proxy.scheme = value;
final scheme = SizedBox(
width: 100,
child: DropdownButtonFormField<String>(
value: proxy.scheme,
onChanged: (value) async {
if (value != null && value != proxy.scheme) {
proxy.scheme = value;
await debounceSave();
}
},
items: const [
DropdownMenuItem<String>(
value: 'http',
child: Text('HTTP'),
),
DropdownMenuItem<String>(
value: 'https',
child: Text('HTTPS'),
),
DropdownMenuItem<String>(
value: 'socks5',
child: Text('SOCKS5'),
),
],
),
)
];
await debounceSave();
}
},
items: const [
DropdownMenuItem<String>(
value: 'http',
child: Text('HTTP'),
),
DropdownMenuItem<String>(
value: 'https',
child: Text('HTTPS'),
),
DropdownMenuItem<String>(
value: 'socks5',
child: Text('SOCKS5'),
),
],
),
);
final arr = proxy.host.split(':');
var ip = '';
@@ -438,10 +436,8 @@ class SettingView extends GetView<SettingController> {
ipController.addListener(updateAddress);
portController.addListener(updateAddress);
items.addAll([
const Padding(padding: EdgeInsets.only(left: 20)),
SizedBox(
width: 150,
final host = [
Flexible(
child: TextFormField(
controller: ipController,
decoration: const InputDecoration(
@@ -455,8 +451,7 @@ class SettingView extends GetView<SettingController> {
),
),
const Padding(padding: EdgeInsets.only(left: 10)),
SizedBox(
width: 150,
Flexible(
child: TextFormField(
controller: portController,
decoration: InputDecoration(
@@ -470,14 +465,13 @@ class SettingView extends GetView<SettingController> {
],
),
),
]);
];
final usrController = TextEditingController(text: proxy.usr);
final pwdController = TextEditingController(text: proxy.pwd);
final auth = [
SizedBox(
width: 150,
Flexible(
child: TextFormField(
controller: usrController,
decoration: InputDecoration(
@@ -491,8 +485,7 @@ class SettingView extends GetView<SettingController> {
),
),
const Padding(padding: EdgeInsets.only(left: 10)),
SizedBox(
width: 150,
Flexible(
child: TextFormField(
controller: pwdController,
decoration: InputDecoration(
@@ -513,8 +506,9 @@ class SettingView extends GetView<SettingController> {
crossAxisAlignment: CrossAxisAlignment.start,
children: _addPadding([
switcher,
scheme,
Row(
children: items,
children: host,
),
Row(
children: auth,
@@ -534,7 +528,7 @@ class SettingView extends GetView<SettingController> {
(Key key) {
final items = <Widget>[
SizedBox(
width: 150,
width: 80,
child: DropdownButtonFormField<String>(
value: startCfg.value.network,
onChanged: Util.isDesktop()
@@ -585,8 +579,7 @@ class SettingView extends GetView<SettingController> {
portController.addListener(updateAddress);
items.addAll([
const Padding(padding: EdgeInsets.only(left: 20)),
SizedBox(
width: 150,
Flexible(
child: TextFormField(
controller: ipController,
decoration: const InputDecoration(
@@ -600,8 +593,7 @@ class SettingView extends GetView<SettingController> {
),
),
const Padding(padding: EdgeInsets.only(left: 10)),
SizedBox(
width: 150,
Flexible(
child: TextFormField(
controller: portController,
decoration: InputDecoration(
@@ -3,5 +3,6 @@ import 'package:gopeed/app/modules/task/controllers/task_list_controller.dart';
import '../../../../api/model/task.dart';
class TaskDownloadedController extends TaskListController {
TaskDownloadedController() : super([Status.done], SortDirection.asc);
TaskDownloadedController()
: super([Status.done], (a, b) => b.updatedAt.compareTo(a.updatedAt));
}
@@ -9,5 +9,5 @@ class TaskDownloadingController extends TaskListController {
Status.pause,
Status.wait,
Status.error
], SortDirection.desc);
], (a, b) => b.createdAt.compareTo(a.createdAt));
}
@@ -5,13 +5,11 @@ import 'package:get/get.dart';
import '../../../../api/api.dart';
import '../../../../api/model/task.dart';
enum SortDirection { asc, desc }
abstract class TaskListController extends GetxController {
List<Status> statuses;
SortDirection sortDirection;
int Function(Task a, Task b) compare;
TaskListController(this.statuses, this.sortDirection);
TaskListController(this.statuses, this.compare);
final tasks = <Task>[].obs;
final isRunning = false.obs;
@@ -48,13 +46,7 @@ abstract class TaskListController extends GetxController {
getTasksState() async {
final tasks = await getTasks(statuses);
// sort tasks by create time
tasks.sort((a, b) {
if (sortDirection == SortDirection.asc) {
return a.createdAt.compareTo(b.createdAt);
} else {
return b.createdAt.compareTo(a.createdAt);
}
});
tasks.sort(compare);
this.tasks.value = tasks;
}
}
+10 -6
View File
@@ -59,21 +59,25 @@ const zhTW = {
'deleteTask': '刪除任務',
'deleteTaskTip': '保留已經下載的文件',
'delete': '刪除',
'newVersionTitle': '發現新版本 @version',
'newVersionUpdate': '立刻更新',
'newVersionLater': '稍後更新',
'extensions': '外掛程序',
'newVersionTitle': '現新版本 @version 已發佈',
'newVersionUpdate': '下載',
'newVersionLater': '取消',
'extensions': '擴充程序',
'extensionInstallUrl': '安裝URL',
'extensionInstallSuccess': '安裝成功',
'extensionUpdateSuccess': '更新完成',
'extensionDelete': '解除安裝',
'extensionAlreadyLatest': '已是最新版本',
'extensionFind': '取得外掛',
'extensionDevelop': '開發外掛',
'extensionFind': '取得擴充程序',
'extensionDevelop': '開發擴充程序',
'history': '歷史記錄',
'clearHistory': '清空記錄',
'noHistoryFound': '還沒有記錄',
'serviceTitle': '下載服務',
'serviceText': '執行中',
'network': '網路',
'proxy': '代理',
'username': '名稱',
'password': '密碼',
}
};
+1 -1
View File
@@ -15,7 +15,7 @@ publish_to: "none" # Remove this line if you wish to publish to pub.dev
# In iOS, build-name is used as CFBundleShortVersionString while build-number used as CFBundleVersion.
# Read more about iOS versioning at
# https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CoreFoundationKeys.html
version: 1.5.1+1
version: 1.5.3+1
environment:
sdk: '>=3.0.0 <4.0.0'