Compare commits

..
4 Commits
Author SHA1 Message Date
silverwind 0d84ec6dab fix: remove the job network even when container removal overruns (#1318)
Job teardown ran every step on one shared minute. When removing the job container took longer, the network removal ran on an expired context and the network stayed behind until the orphan reclaim, which only takes networks older than `workdir_cleanup_age`, 24h by default. All that time it held a subnet of the daemon's address pool.

The job and service containers are now removed in parallel, and the docker proxy sweep, the network and the volumes get a fresh budget after them. The idle reclaim takes unused networks and volumes once they are 10 minutes old, and network removal treats a network that is already gone as removed.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1318
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-15 14:15:41 +00:00
silverwind b769a9765f perf: download actions as archives once per job (#1316)
Each step refreshed its own clone of an action, so a composite using one action repository ten times fetched it ten times per job, and a single network error failed the job.

Like actions/runner, a job now downloads each action repository and ref once, as an archive extracted into a directory per commit. Each action still gets its own copy in the job container. This restores the per-job dedupe of https://gitea.com/gitea/runner/pulls/1178, which only reached a since-deleted branch.

How a download works:

1. The action URL is taken after `insteadOf` rewriting, as git would use it.
2. The archive comes from `<url>/archive/<ref>.tar.gz`, or from codeload.github.com for github.com, as for actions/runner.
3. A cached tag or branch is revalidated with its ETag, which GitHub and Gitea 28+ answer with a 304. A cached commit pin needs no request.
4. Transient failures (5xx, 408, 429 honouring `Retry-After`, dropped transfers) are retried three times with a 10 to 30 second backoff.
5. Git takes over, cloning and exporting the commit into the same directory layout, when the URL is not HTTP(S), the ref is a short commit, the host serves no usable archive, the host cannot be connected to, or the retries run out.

Median of 3 runs, git (shallow clone, then fetch) against archives, on a cold and a warm cache:

| Action | git cold | git warm | archive cold | archive warm |
|---|---|---|---|---|
| github.com actions/checkout@v4 | 937ms | 726ms | 286ms | 171ms |
| github.com actions/cache@v4 | 1006ms | 801ms | 516ms | 161ms |
| github.com actions/setup-node@v4 | 1046ms | 786ms | 463ms | 156ms |
| gitea.com actions/checkout@v4 | 2007ms | 1165ms | 667ms | 222ms |

The action cache layout changes, so existing caches download once more.

*Written by Claude.*

Fixes: https://gitea.com/gitea/runner/issues/1220
Reviewed-on: https://gitea.com/gitea/runner/pulls/1316
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-15 12:49:37 +00:00
9c4c66e70f enhance(report): add runner.log_report_level for the uploaded runner messages (#1310)
The runner's own messages between steps (not step output) were uploaded to the job log at `log.level`, so raising `log.level` for local debugging also changed what workflow authors see in Gitea, and lowering it hid messages from them.

Add `runner.log_report_level` to set that independently. Unset, it still follows `log.level`, so existing setups behave as before. An invalid value is rejected when the config loads.

Assisted-by: Codet
Assisted-by: Claude Code:claude-opus-5-5
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1310
Co-authored-by: Lunny Xiao <[email protected]>
2026-10-14 14:15:26 +00:00
silverwind 42858c61df fix: stop pruning the cached action ref on refresh (#1312)
The action cache refresh fetched with `--prune` and the unqualified refspec `+<ref>:refs/runner/<hash>`. Git cannot map that ref back to a remote ref, so it deleted it on every other refresh. The first job after a branch or tag moved then failed with `cannot lock ref`, and a moved tag in a full clone could silently run its old commit. `--no-prune` fixes this and also overrides an inherited `fetch.prune`.

Fixes: https://gitea.com/gitea/runner/issues/1311
Reviewed-on: https://gitea.com/gitea/runner/pulls/1312
Reviewed-by: Lunny Xiao <[email protected]>
Reviewed-by: Zettat123 <[email protected]>
2026-10-14 13:15:01 +00:00
33 changed files with 906 additions and 336 deletions

No files matched your search

+3
View File
@@ -40,6 +40,9 @@ fields.
- The process-wide logger masks the secrets of every live task, the job logger masks its own.
Going around either leaks them.
- The job logger's output can reach the job log workflow authors see, the process-wide logger's
stays on the runner host, so keep operator-only detail (host paths, internal addresses) off the
job logger.
- Libraries that log through logrus's or slog's global logger are forwarded to the process-wide
one, so they follow its level and masking too.
- That forwarding only works while logrus's global logger is left as `logging` configures it. A
+2 -2
View File
@@ -6,7 +6,7 @@
On the runner:
- `git` 2.34.1 or newer on `PATH`, to download actions and reusable workflows.
- `git` 2.34.1 or newer on `PATH`, to clone reusable workflows, and actions whose host serves no archive of them.
- `docker` or `podman` with `podman.socket` enabled, for docker mode, Docker actions and `docker://` steps. To install Docker CE, follow the official [install instructions](https://docs.docker.com/engine/install/).
Where steps run, in the job image or on the runner in host mode:
@@ -368,7 +368,7 @@ While the runner is idle it cleans up after earlier jobs:
- when `container.bind_workdir` is enabled, stale task workspace directories older than `runner.workdir_cleanup_age` are removed (default: `24h`; set `0` to disable)
- only purely numeric subdirectories under `container.workdir_parent` are treated as task workspaces and may be removed
- cleanup assumes `container.workdir_parent` is not shared across multiple runners
- on runners that use docker, per-job networks left behind by jobs the runner did not live to tear down are removed, identified by the `com.gitea.runner.uuid` label carrying this runner's uuid
- on runners that use docker, per-job networks and volumes left behind by jobs the runner did not live to tear down are removed once unused and 10 minutes old, identified by the `com.gitea.runner.uuid` label carrying this runner's uuid
- on runners with `kubernetes` labels, job pods with that label older than `runner.workdir_cleanup_age` are removed the same way, together with their Secrets
- cleanup runs every `runner.idle_cleanup_interval` (default: `10m`; set `0` to disable), and setting either knob to `0` disables all of the above
+238
View File
@@ -0,0 +1,238 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package git
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"errors"
"io"
"math/rand/v2"
"net"
"net/http"
"net/url"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
"time"
"gitea.com/gitea/runner/act/common"
)
var archiveRetryDelay = func(retryAfter string) time.Duration { // as actions/runner backs off
if seconds, err := strconv.Atoi(retryAfter); err == nil {
return min(max(time.Duration(seconds)*time.Second, 10*time.Second), 10*time.Minute)
}
return time.Duration(10+rand.IntN(21)) * time.Second
}
func FetchAction(ctx context.Context, input NewGitCloneExecutorInput) (string, string, error) { // input.Dir is one clone URL and ref's, holding a directory per commit
unlock, err := AcquireCloneLock(ctx, input.Dir)
if err != nil {
return "", "", err
}
defer unlock()
if err := os.MkdirAll(input.Dir, 0o755); err != nil {
return "", "", err
}
tree := func(sha string) string { return filepath.Join(input.Dir, sha) }
record := filepath.Join(input.Dir, "ref")
data, _ := os.ReadFile(record)
sha, etag, _ := strings.Cut(string(data), "\n")
if !isFullObjectID(sha) || !exists(tree(sha)) {
sha, etag = "", ""
} else if input.OfflineMode || isFullObjectID(input.Ref) {
return tree(sha), sha, nil
}
newSha, newEtag, err := downloadArchive(ctx, input, sha, etag)
if err != nil && ctx.Err() == nil {
common.Logger(ctx).Debugf("Cloning %s@%s: %v", input.URL, input.Ref, err)
newEtag = ""
newSha, err = exportClone(ctx, input)
}
if err != nil {
return "", "", err
}
if newSha != sha || newEtag != etag {
if err := os.WriteFile(record, []byte(newSha+"\n"+newEtag), 0o600); err != nil {
return "", "", err
}
}
return tree(newSha), newSha, nil
}
func downloadArchive(ctx context.Context, input NewGitCloneExecutorInput, sha, etag string) (string, string, error) {
rewritten, err := gitOutput(ctx, "", "ls-remote", "--get-url", input.URL)
if err != nil {
return "", "", err
}
source, err := url.Parse(rewritten)
if err != nil || source.Scheme != "http" && source.Scheme != "https" || hexRefRegex.MatchString(input.Ref) && !isFullObjectID(input.Ref) { // only git tells a short commit from a branch
return "", "", errors.New("no archive")
}
archive := source.JoinPath("archive", input.Ref+".tar.gz")
if source.Host == "github.com" {
source.Host = "codeload.github.com"
archive = source.JoinPath("tar.gz", input.Ref)
}
client := &http.Client{Transport: input.Transport, Timeout: 5 * time.Minute, CheckRedirect: func(request *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return errors.New("stopped after 10 redirects")
}
if request.URL.Host != via[0].URL.Host || request.URL.Scheme != via[0].URL.Scheme { // the token stays on its origin
request.Header.Del("Authorization")
}
return nil
}}
for attempt := 1; ; attempt++ {
newSha, newEtag, retryAfter, err := downloadArchiveOnce(ctx, client, archive.String(), input.Token, sha, etag, input.Dir)
if retryAfter == 0 || attempt == 3 || ctx.Err() != nil {
return newSha, newEtag, err
}
common.Logger(ctx).Warnf("Failed to download action '%s'. Error: %v. Back off %s before retry.", archive.Redacted(), err, retryAfter)
select {
case <-ctx.Done():
return "", "", ctx.Err()
case <-time.After(retryAfter):
}
}
}
func downloadArchiveOnce(ctx context.Context, client *http.Client, archive, token, sha, etag, dir string) (string, string, time.Duration, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodGet, archive, nil)
if err != nil {
return "", "", 0, err
}
if etag != "" {
request.Header.Set("If-None-Match", etag)
}
if token != "" {
request.SetBasicAuth("token", token)
}
response, err := client.Do(request)
if err != nil {
var opErr *net.OpError
if errors.As(err, &opErr) && opErr.Op == "dial" {
return "", "", 0, err
}
return "", "", archiveRetryDelay(""), err
}
defer response.Body.Close()
switch status := response.StatusCode; {
case status == http.StatusNotModified && sha != "":
return sha, etag, 0, nil
case status == http.StatusOK:
newEtag := response.Header.Get("ETag")
gz, err := gzip.NewReader(response.Body)
if err == nil {
sha, err = extractTree(gz, dir, !strings.HasSuffix(newEtag, `-false"`)) // Gitea's mark of an unprefixed archive
}
var netErr net.Error
if errors.Is(err, io.ErrUnexpectedEOF) || errors.As(err, &netErr) {
return "", "", archiveRetryDelay(""), err
}
return sha, newEtag, 0, err
case status == http.StatusRequestTimeout || status == http.StatusTooManyRequests || status >= http.StatusInternalServerError:
return "", "", archiveRetryDelay(response.Header.Get("Retry-After")), errors.New(response.Status)
default:
return "", "", 0, errors.New(response.Status)
}
}
func exportClone(ctx context.Context, input NewGitCloneExecutorInput) (string, error) {
clone := input
clone.Dir = filepath.Join(input.Dir, "git")
sha, _, err := resolveClone(ctx, clone)
if err != nil || exists(filepath.Join(input.Dir, sha)) {
return sha, err
}
archive, err := runGit(ctx, clone.Dir, nil, "archive", "--format=tar", sha)
if err != nil {
return "", err
}
_, err = extractTree(bytes.NewReader(archive), input.Dir, false)
return sha, err
}
func extractTree(archive io.Reader, dir string, strip bool) (string, error) { // into dir/<commit of the pax header>, strip dropping the directory wrapping the files
temp, err := os.MkdirTemp(dir, "tmp-")
if err != nil {
return "", err
}
defer os.RemoveAll(temp)
root, err := os.OpenRoot(temp)
if err != nil {
return "", err
}
defer root.Close()
sha := ""
reader := tar.NewReader(archive)
for {
header, err := reader.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return "", err
}
if header.Typeflag == tar.TypeXGlobalHeader {
if sha = header.PAXRecords["comment"]; isFullObjectID(sha) && exists(filepath.Join(dir, sha)) {
return sha, nil
}
continue
}
name := header.Name
if strip {
_, name, _ = strings.Cut(name, "/")
}
if name = strings.TrimSuffix(name, "/"); name == "" {
continue
}
switch header.Typeflag {
case tar.TypeDir:
err = root.MkdirAll(name, 0o755)
case tar.TypeReg:
err = writeTreeFile(root, name, header.Mode, reader)
case tar.TypeSymlink:
if err = root.Symlink(header.Linkname, name); err != nil && runtime.GOOS == "windows" {
err = writeTreeFile(root, name, 0, strings.NewReader(header.Linkname))
}
}
if err != nil {
return "", err
}
}
if !isFullObjectID(sha) {
return "", errors.New("the archive names no commit")
}
if err := root.Close(); err != nil {
return "", err
}
return sha, os.Rename(temp, filepath.Join(dir, sha))
}
func writeTreeFile(root *os.Root, name string, mode int64, content io.Reader) error {
perm := os.FileMode(0o644)
if mode&0o111 != 0 {
perm = 0o755
}
file, err := root.OpenFile(name, os.O_CREATE|os.O_EXCL|os.O_WRONLY, perm)
if err != nil {
return err
}
_, err = io.Copy(file, content)
return errors.Join(err, file.Close())
}
func exists(name string) bool {
_, err := os.Stat(name)
return err == nil
}
+225
View File
@@ -0,0 +1,225 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package git
import (
"archive/tar"
"bytes"
"encoding/base64"
"fmt"
"net/http"
"net/http/cgi"
"net/http/httptest"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestFetchAction(t *testing.T) {
original := archiveRetryDelay
archiveRetryDelay = func(_retryAfter string) time.Duration { return time.Nanosecond }
t.Cleanup(func() { archiveRetryDelay = original })
root := t.TempDir()
remote := filepath.Join(root, "org", "action")
require.NoError(t, gitCmd("init", "--initial-branch=main", remote))
require.NoError(t, os.WriteFile(filepath.Join(remote, "action.yml"), []byte("name: action\n"), 0o644))
require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755))
require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link")))
require.NoError(t, gitCmd("-C", remote, "add", "."))
require.NoError(t, gitCmd("-C", remote, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", remote, "tag", "v1"))
gitPath, err := exec.LookPath("git")
require.NoError(t, err)
backend := &cgi.Handler{Path: gitPath, Args: []string{"http-backend"}, Env: []string{"GIT_PROJECT_ROOT=" + root, "GIT_HTTP_EXPORT_ALL=1"}}
var mu sync.Mutex
var archiveRequests []http.Header
var mirror *httptest.Server
requests, archives, failures := 0, "", 0
handler := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
mu.Lock()
defer mu.Unlock()
requests++
ref, ok := strings.CutPrefix(request.URL.Path, "/org/action/archive/")
if !ok {
backend.ServeHTTP(writer, request)
return
}
archiveRequests = append(archiveRequests, request.Header.Clone())
switch {
case failures > 0:
failures--
writer.WriteHeader(http.StatusServiceUnavailable)
return
case archives == "none":
http.NotFound(writer, request)
return
case archives == "unreachable":
http.Redirect(writer, request, "http://127.0.0.1:1"+request.URL.Path, http.StatusFound)
return
case archives == "redirect" && request.Host != mirror.Listener.Addr().String():
http.Redirect(writer, request, mirror.URL+request.URL.Path, http.StatusFound)
return
}
sha, _ := gitOutput(request.Context(), remote, "rev-parse", strings.TrimSuffix(ref, ".tar.gz"))
etag := fmt.Sprintf(`W/"%s-tar.gz-%t"`, sha, archives != "unprefixed")
if request.Header.Get("If-None-Match") == etag {
writer.WriteHeader(http.StatusNotModified)
return
}
args := []string{"archive", "--format=tar.gz", sha}
if archives != "unprefixed" {
args = append(args, "--prefix=action/")
}
archive, _ := runGit(request.Context(), remote, nil, args...)
writer.Header().Set("ETag", etag)
_, _ = writer.Write(archive)
})
server := httptest.NewServer(handler)
mirror = httptest.NewServer(handler)
t.Cleanup(server.Close)
t.Cleanup(mirror.Close)
serve := func(mode string, failing int) {
mu.Lock()
defer mu.Unlock()
requests, archives, failures, archiveRequests = 0, mode, failing, nil
}
served := func() (int, []http.Header) {
mu.Lock()
defer mu.Unlock()
return requests, append([]http.Header(nil), archiveRequests...)
}
newInput := func(ref string) NewGitCloneExecutorInput {
return NewGitCloneExecutorInput{URL: server.URL + "/org/action", Ref: ref, Dir: t.TempDir(), Token: "secret"}
}
t.Run("downloads, revalidates and follows a moved tag", func(t *testing.T) {
serve("prefixed", 0)
input := newInput("v1")
tree, sha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, gitRevParse(t, remote, "v1"), sha)
assert.Equal(t, filepath.Join(input.Dir, sha), tree)
assert.NoDirExists(t, filepath.Join(tree, ".git"))
info, err := os.Stat(filepath.Join(tree, "run.sh"))
require.NoError(t, err)
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm())
link, err := os.Readlink(filepath.Join(tree, "link"))
require.NoError(t, err)
assert.Equal(t, "action.yml", link)
again, _, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, tree, again)
require.NoError(t, gitCmd("-C", remote, "commit", "--allow-empty", "-m", "moved"))
require.NoError(t, gitCmd("-C", remote, "tag", "--force", "v1"))
moved, movedSha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, gitRevParse(t, remote, "v1"), movedSha)
assert.NotEqual(t, tree, moved)
assert.FileExists(t, filepath.Join(tree, "action.yml"))
input.OfflineMode = true
_, offlineSha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, movedSha, offlineSha)
requests, headers := served()
assert.Equal(t, 3, requests)
require.Len(t, headers, 3)
assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), headers[0].Get("Authorization"))
assert.Empty(t, headers[0].Get("If-None-Match"))
assert.Equal(t, fmt.Sprintf(`W/"%s-tar.gz-true"`, sha), headers[1].Get("If-None-Match"))
assert.Equal(t, headers[1].Get("If-None-Match"), headers[2].Get("If-None-Match"))
})
t.Run("downloads a pinned commit once and rejects a short one", func(t *testing.T) {
serve("prefixed", 0)
sha := gitRevParse(t, remote, "main")
input := newInput(sha)
for range 2 {
tree, got, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, sha, got)
assert.FileExists(t, filepath.Join(tree, "action.yml"))
}
_, _, err := FetchAction(t.Context(), newInput(sha[:7]))
require.ErrorIs(t, err, ErrShortRef)
_, headers := served()
assert.Len(t, headers, 1)
assert.NoDirExists(t, filepath.Join(input.Dir, "git"))
})
t.Run("retries a failing download and drops the token on a redirect to another origin", func(t *testing.T) {
serve("redirect", 2)
input := newInput("main")
_, sha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, gitRevParse(t, remote, "main"), sha)
assert.NoDirExists(t, filepath.Join(input.Dir, "git"))
_, headers := served()
require.Len(t, headers, 4)
assert.NotEmpty(t, headers[2].Get("Authorization"))
assert.Empty(t, headers[3].Get("Authorization"))
})
t.Run("keeps an unprefixed archive whole, downloads where insteadOf points and exports a clone without an archive or from an unreachable host", func(t *testing.T) {
for _, mode := range []string{"unprefixed", "none", "unreachable"} {
serve(mode, 0)
tree, sha, err := FetchAction(t.Context(), newInput("main"))
require.NoError(t, err, mode)
assert.Equal(t, gitRevParse(t, remote, "main"), sha, mode)
assert.FileExists(t, filepath.Join(tree, "action.yml"), mode)
assert.NoDirExists(t, filepath.Join(tree, ".git"), mode)
_, headers := served()
assert.Len(t, headers, 1, mode)
}
serve("prefixed", 0)
config := filepath.Join(t.TempDir(), "gitconfig")
require.NoError(t, os.WriteFile(config, []byte("[url \""+server.URL+"/org/\"]\n\tinsteadOf = https://example.invalid/org/\n"), 0o644))
t.Setenv("GIT_CONFIG_GLOBAL", config)
input := newInput("main")
input.URL = "https://example.invalid/org/action"
_, sha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, gitRevParse(t, remote, "main"), sha)
_, headers := served()
assert.Len(t, headers, 1)
assert.NoDirExists(t, filepath.Join(input.Dir, "git"))
})
}
func TestExtractTreeConfinesEntries(t *testing.T) {
outside := t.TempDir()
for name, entries := range map[string][]*tar.Header{
"parent path": {{Name: "../escape", Typeflag: tar.TypeReg}},
"through a symlink": {{Name: "out", Typeflag: tar.TypeSymlink, Linkname: outside}, {Name: "out/escape", Typeflag: tar.TypeReg}},
} {
t.Run(name, func(t *testing.T) {
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
require.NoError(t, writer.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, PAXRecords: map[string]string{"comment": strings.Repeat("a", 40)}}))
for _, entry := range entries {
require.NoError(t, writer.WriteHeader(entry))
}
require.NoError(t, writer.Close())
dir := t.TempDir()
_, err := extractTree(&archive, dir, false)
require.Error(t, err)
assert.NoFileExists(t, filepath.Join(dir, "escape"))
assert.NoFileExists(t, filepath.Join(outside, "escape"))
})
}
}
+28 -26
View File
@@ -10,6 +10,7 @@ import (
"encoding/base64"
"errors"
"fmt"
"net/http"
"net/url"
"os"
"os/exec"
@@ -178,13 +179,10 @@ type NewGitCloneExecutorInput struct {
// 0 for full clone.
Depth int
// Quiet drops the informational clone line to debug level, for callers that log their own
// download summary (the setup section's action report).
Quiet bool
// For Gitea
InsecureSkipTLS bool
ClientCertFile, ClientKeyFile string // PEM, absolute as git runs in other directories
ClientCertFile, ClientKeyFile string // PEM, absolute as git runs in other directories
Transport http.RoundTripper // for FetchAction's archive downloads, nil for http.DefaultTransport
}
// CloneIfRequired reports whether an existing local clone was reused.
@@ -221,11 +219,7 @@ func CloneIfRequired(ctx context.Context, input NewGitCloneExecutorInput, logger
func NewGitCloneExecutor(input NewGitCloneExecutorInput) common.Executor {
return func(ctx context.Context) error {
logger := common.Logger(ctx)
if input.Quiet {
logger.Debugf("git clone '%s' # ref=%s", input.URL, input.Ref)
} else {
logger.Infof("git clone '%s' # ref=%s", input.URL, input.Ref)
}
logger.Infof("git clone '%s' # ref=%s", input.URL, input.Ref)
logger.Debugf(" cloning %s to %s", input.URL, input.Dir)
unlock, err := AcquireCloneLock(ctx, input.Dir)
if err != nil {
@@ -233,24 +227,10 @@ func NewGitCloneExecutor(input NewGitCloneExecutorInput) common.Executor {
}
defer unlock()
reused, err := CloneIfRequired(ctx, input, logger)
resolved, reused, err := resolveClone(ctx, input)
if err != nil {
return err
}
resolved, err := resolveRef(ctx, input.Dir, input.Ref)
if errors.Is(err, ErrShortRef) || (err != nil && input.OfflineMode) {
return err
}
if !input.OfflineMode && (err != nil || (reused && !isFullHash(input.Ref, resolved))) {
if err := fetch(ctx, input); err != nil {
return err
}
resolved, err = resolveRef(ctx, input.Dir, input.Ref)
}
if err != nil {
logger.Errorf("Unable to resolve %s: %v", input.Ref, err)
return err
}
if _, err := runGit(ctx, input.Dir, nil, "checkout", "--force", "--detach", resolved); err != nil {
return fmt.Errorf("checkout %s: %w", resolved, err)
}
@@ -264,6 +244,28 @@ func NewGitCloneExecutor(input NewGitCloneExecutorInput) common.Executor {
}
}
func resolveClone(ctx context.Context, input NewGitCloneExecutorInput) (string, bool, error) { // the caller holds input.Dir's lock
logger := common.Logger(ctx)
reused, err := CloneIfRequired(ctx, input, logger)
if err != nil {
return "", false, err
}
resolved, err := resolveRef(ctx, input.Dir, input.Ref)
if errors.Is(err, ErrShortRef) || (err != nil && input.OfflineMode) {
return "", false, err
}
if !input.OfflineMode && (err != nil || (reused && !isFullHash(input.Ref, resolved))) {
if err := fetch(ctx, input); err != nil {
return "", false, err
}
resolved, err = resolveRef(ctx, input.Dir, input.Ref)
}
if err != nil {
logger.Errorf("Unable to resolve %s: %v", input.Ref, err)
}
return resolved, reused, err
}
func clone(ctx context.Context, input NewGitCloneExecutorInput) error {
if input.Depth > 0 && isFullObjectID(input.Ref) {
if err := shallowPinnedClone(ctx, input); err == nil {
@@ -304,7 +306,7 @@ func shallowPinnedClone(ctx context.Context, input NewGitCloneExecutorInput) err
}
func fetch(ctx context.Context, input NewGitCloneExecutorInput) error {
args := []string{"fetch", "--force", "--prune"}
args := []string{"fetch", "--force", "--no-prune"}
if _, err := os.Stat(filepath.Join(input.Dir, ".git", "shallow")); err == nil {
args = append(args, "--depth", "1", "--no-tags")
}
+12 -41
View File
@@ -21,7 +21,6 @@ import (
"testing/synctest"
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.com/gitea/runner/internal/pkg/lock"
@@ -354,6 +353,8 @@ func TestGitCloneExecutorFollowsRemoteRefChanges(t *testing.T) {
dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remoteDir, Ref: "main", Dir: dir})
require.NoError(t, clone(t.Context()))
require.NoError(t, gitCmd("-C", dir, "config", "fetch.prune", "true"))
require.NoError(t, clone(t.Context()))
require.NoError(t, gitCmd("-C", workDir, "reset", "--hard", "HEAD~1"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "feature-rewritten"))
@@ -361,11 +362,20 @@ func TestGitCloneExecutorFollowsRemoteRefChanges(t *testing.T) {
require.NoError(t, clone(t.Context()), "a non-fast-forward ref must update")
assert.Equal(t, "feature-rewritten", gitHeadSubject(t, dir))
tagClone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remoteDir, Ref: "v9", Dir: t.TempDir()})
tagDir := t.TempDir()
tagClone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remoteDir, Ref: "v9", Dir: tagDir})
require.Error(t, tagClone(t.Context()))
require.NoError(t, gitCmd("-C", workDir, "tag", "v9"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "v9"))
require.NoError(t, tagClone(t.Context()), "a tag published after a failed lookup must be fetched")
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "tag-moved"))
require.NoError(t, gitCmd("-C", workDir, "tag", "--force", "v9"))
require.NoError(t, gitCmd("-C", workDir, "push", "--force", "origin", "v9"))
for range 2 {
require.NoError(t, tagClone(t.Context()), "a moved tag must update and stay current")
assert.Equal(t, "tag-moved", gitHeadSubject(t, tagDir))
}
}
func TestGitCloneExecutorOfflineMode(t *testing.T) {
@@ -415,45 +425,6 @@ func TestGitCloneExecutorOfflineMode(t *testing.T) {
})
}
func TestGitCloneExecutorQuietDemotesCloneLine(t *testing.T) {
t.Parallel()
remoteDir := t.TempDir()
require.NoError(t, gitCmd("init", "--bare", "--initial-branch=main", remoteDir))
workDir := t.TempDir()
require.NoError(t, gitCmd("clone", remoteDir, workDir))
require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "main"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "initial"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
// Quiet callers report the download themselves, so the clone line must not reach the job log.
for name, quiet := range map[string]bool{"quiet": true, "not quiet": false} {
t.Run(name, func(t *testing.T) {
hook := logtest.New()
logger := logging.New(logging.NewLevelHandler(hook, logging.LevelInfo))
ctx := common.WithLogger(t.Context(), logger.With("job", "j1"))
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir,
Ref: "main",
Dir: t.TempDir(),
Quiet: quiet,
})(ctx))
var cloneLines int
for _, entry := range hook.AllRecords() {
if strings.HasPrefix(entry.Message, "git clone ") {
cloneLines++
}
}
if quiet {
assert.Zero(t, cloneLines)
} else {
assert.Equal(t, 1, cloneLines)
}
})
}
}
func TestGitCloneExecutorShallow(t *testing.T) {
t.Parallel()
// Build a local "remote" with several commits on main plus a tag, so a full clone would pull noticeably more history than a shallow one.
+30 -27
View File
@@ -53,7 +53,9 @@ func removeOrphanNetworks(ctx context.Context, cli client.APIClient, runnerUUID,
networks:
for _, n := range networks.Items {
result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{})
if err != nil {
if cerrdefs.IsNotFound(err) {
continue
} else if err != nil {
errs = append(errs, fmt.Errorf("failed to inspect network %s: %w", n.Name, err))
continue
}
@@ -71,7 +73,9 @@ networks:
continue networks
}
}
if _, err := cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil {
if _, err := cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); cerrdefs.IsNotFound(err) {
continue
} else if err != nil {
errs = append(errs, fmt.Errorf("failed to remove network %s: %w", n.Name, err))
continue
}
@@ -203,36 +207,35 @@ func NewDockerNetworkRemoveExecutor(name string) common.Executor {
return err
}
defer cli.Close()
return removeNetwork(ctx, cli, name)
}
}
// Make sure that all network of the specified name are removed
// cli.NetworkRemove refuses to remove a network if there are duplicates
networks, err := cli.NetworkList(ctx, client.NetworkListOptions{})
if err != nil {
func removeNetwork(ctx context.Context, cli client.APIClient, name string) error {
networks, err := cli.NetworkList(ctx, client.NetworkListOptions{}) // NetworkRemove by name refuses duplicates, so remove each by ID
if err != nil {
return err
}
var errs []error
for _, n := range networks.Items {
if n.Name != name {
continue
}
result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{})
if cerrdefs.IsNotFound(err) {
continue
} else if err != nil {
return err
}
// For Gitea, reduce log noise
// common.Logger(ctx).Debugf("%v", networks)
var errs []error
for _, n := range networks.Items {
if n.Name == name {
result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{})
if err != nil {
return err
}
// it holds a subnet out of the daemon's pool until something reclaims it
if len(result.Network.Containers) != 0 {
common.Logger(ctx).Warnf("Refusing to remove network %s because it still has active endpoints, the idle cleanup reclaims it once they are gone", name)
continue
}
if _, err = cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil {
errs = append(errs, fmt.Errorf("failed to remove network %s: %w", name, err))
}
}
if len(result.Network.Containers) != 0 {
common.Logger(ctx).Warnf("Refusing to remove network %s because it still has active endpoints, the orphan cleanup reclaims it once they are gone", name)
continue
}
if _, err = cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil && !cerrdefs.IsNotFound(err) {
errs = append(errs, fmt.Errorf("failed to remove network %s: %w", name, err))
}
return errors.Join(errs...)
}
return errors.Join(errs...)
}
// IsolatedCacheContainer returns the container holding addr, when jobs on jobNetwork cannot reach it there.
+21 -4
View File
@@ -116,7 +116,7 @@ func TestConnectAndDisconnectNetwork(t *testing.T) {
// Of this runner's networks, only the ones nothing but its own cache container is attached to
// and old enough to predate any job now starting are the runner's to reclaim. An unexpected
// NetworkRemove fails the test on its own, since testify has no expectation to match it against.
func TestRemoveOrphanNetworks(t *testing.T) {
func TestRemoveNetworks(t *testing.T) {
ctx := t.Context()
cutoff := time.Date(2026, time.April, 29, 20, 0, 0, 0, time.UTC)
client := &mockDockerClient{}
@@ -128,11 +128,13 @@ func TestRemoveOrphanNetworks(t *testing.T) {
{ID: "starting"},
{ID: "cached"},
{ID: "shared"},
{ID: "gone"},
{ID: "raced"},
}}, nil)
client.On("NetworkInspect", ctx, "orphan", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{}, nil)
Return(mobyclient.NetworkInspectResult{}, nil).Twice()
client.On("NetworkInspect", ctx, "busy", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"c": {}}}}, nil)
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"c": {}}}}, nil).Twice()
client.On("NetworkInspect", ctx, "starting", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{
Network: network.Network{Created: cutoff.Add(time.Second)},
@@ -142,13 +144,28 @@ func TestRemoveOrphanNetworks(t *testing.T) {
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"0123456789abcdef": {}}}}, nil)
client.On("NetworkInspect", ctx, "shared", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"0123456789abcdef": {}, "job": {}}}}, nil)
client.On("NetworkInspect", ctx, "gone", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{}, cerrdefs.ErrNotFound).Twice()
client.On("NetworkInspect", ctx, "raced", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{}, nil).Twice()
client.On("NetworkRemove", ctx, "raced", mobyclient.NetworkRemoveOptions{}).
Return(mobyclient.NetworkRemoveResult{}, cerrdefs.ErrNotFound).Twice()
client.On("NetworkRemove", ctx, "orphan", mobyclient.NetworkRemoveOptions{}).
Return(mobyclient.NetworkRemoveResult{}, nil)
Return(mobyclient.NetworkRemoveResult{}, nil).Twice()
client.On("NetworkDisconnect", ctx, "cached", mobyclient.NetworkDisconnectOptions{Container: "0123456789abcdef", Force: true}).
Return(nil).Once()
client.On("NetworkRemove", ctx, "cached", mobyclient.NetworkRemoveOptions{}).
Return(mobyclient.NetworkRemoveResult{}, nil).Once()
require.NoError(t, removeOrphanNetworks(ctx, client, "runner-1", "0123456789ab", cutoff))
client.On("NetworkList", ctx, mobyclient.NetworkListOptions{}).Return(mobyclient.NetworkListResult{Items: []network.Summary{
{ID: "orphan", Name: "job"},
{ID: "busy", Name: "job"},
{ID: "gone", Name: "job"},
{ID: "raced", Name: "job"},
{ID: "shared", Name: "other"},
}}, nil)
require.NoError(t, removeNetwork(ctx, client, "job"))
client.AssertExpectations(t)
}
+13 -24
View File
@@ -91,7 +91,7 @@ func readActionImpl(ctx context.Context, step *model.Step, actionDir, actionPath
if b, err = trampoline.ReadFile("res/trampoline.js"); err != nil {
return nil, err
}
err2 := writeFile(filepath.Join(actionDir, actionPath, "trampoline.js"), b, 0o400)
err2 := writeFile(filepath.Join(actionDir, actionPath, "trampoline.js"), b, 0o644)
if err2 != nil {
return nil, err2
}
@@ -152,11 +152,11 @@ func maybeCopyToActionDir(ctx context.Context, step actionStep, actionDir, actio
defer unlock()
if !rc.Config.NoActionPatch {
// A concurrent job's prepare resets this directory, so patch under the copy's lock.
// Other jobs share the directory, so patch under the copy's lock.
defer patchActions(ctx, actionScriptPaths(filepath.Join(actionDir, actionPath), step.getActionModel()))()
}
return rc.JobContainer.CopyDir(containerActionDirCopy, actionDir+"/", rc.Config.UseGitIgnore, true)(ctx)
return rc.JobContainer.CopyDir(containerActionDirCopy, actionDir+"/", false, true)(ctx) // an exported commit has no .git to apply gitignore with
}
func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction) common.Executor {
@@ -179,7 +179,7 @@ func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction
}
actionLocation := path.Join(actionDir, actionPath)
actionName, containerActionDir := getContainerActionPaths(stepModel, actionLocation, rc)
actionName, containerActionDir := getContainerActionPaths(step)
logger.Debugf("type=%v actionDir=%s actionPath=%s workdir=%s actionCacheDir=%s actionName=%s containerActionDir=%s", stepModel.Type(), actionDir, actionPath, rc.Config.Workdir, rc.ActionCacheDir(), actionName, containerActionDir)
@@ -509,25 +509,14 @@ func populateEnvsFromInput(ctx context.Context, env *map[string]string, action *
return nil
}
func getContainerActionPaths(step *model.Step, actionDir string, rc *RunContext) (string, string) {
actionName := ""
containerActionDir := "."
if step.Type() != model.StepTypeUsesActionRemote {
actionName = getOsSafeRelativePath(actionDir, rc.Config.Workdir)
containerActionDir = rc.JobContainer.ToContainerPath(rc.Config.Workdir) + "/" + actionName
actionName = "./" + actionName
} else if step.Type() == model.StepTypeUsesActionRemote {
actionName = getOsSafeRelativePath(actionDir, rc.ActionCacheDir())
containerActionDir = rc.JobContainer.GetActPath() + "/actions/" + actionName
func getContainerActionPaths(step actionStep) (string, string) {
rc := step.getRunContext()
if sar, ok := step.(*stepActionRemote); ok { // a copy per action, as its repository's directory is shared
actionName := path.Join(sar.containerKey(), sar.remoteAction.Path)
return actionName, rc.JobContainer.GetActPath() + "/actions/" + actionName
}
if actionName == "" {
actionName = filepath.Base(actionDir)
if runtime.GOOS == "windows" {
actionName = strings.ReplaceAll(actionName, "\\", "/")
}
}
return actionName, containerActionDir
actionName := getOsSafeRelativePath(filepath.Join(rc.Config.Workdir, step.getStepModel().Uses), rc.Config.Workdir)
return "./" + actionName, rc.JobContainer.ToContainerPath(rc.Config.Workdir) + "/" + actionName
}
func getOsSafeRelativePath(s, prefix string) string {
@@ -573,13 +562,13 @@ func actionStagePaths(step actionStep) (actionDir, actionPath, actionName, conta
stepModel := step.getStepModel()
if sar, ok := step.(*stepActionRemote); ok {
actionDir = sar.actionDir()
actionDir = sar.actionDir
actionPath = sar.remoteAction.Path
} else {
actionDir = filepath.Join(rc.Config.Workdir, stepModel.Uses)
}
actionName, containerActionDir = getContainerActionPaths(stepModel, path.Join(actionDir, actionPath), rc)
actionName, containerActionDir = getContainerActionPaths(step)
return actionDir, actionPath, actionName, containerActionDir
}
+4 -2
View File
@@ -4,9 +4,9 @@
package runner
import (
"context"
"testing"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/git"
"gitea.dev/actionslib/pkg/model"
@@ -50,7 +50,9 @@ func TestCompositeActionParity(t *testing.T) {
})
t.Run("propagates pre failures", func(t *testing.T) {
setCloneExecutor(t, func(git.NewGitCloneExecutorInput) common.Executor { return common.NewErrorExecutor(assert.AnError) })
setFetch(t, func(_ctx context.Context, _input git.NewGitCloneExecutorInput) (string, string, error) {
return "", "", assert.AnError
})
rc := &RunContext{
Config: &Config{GitHubInstance: "github.com", ActionCacheDir: t.TempDir()},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},
+8 -5
View File
@@ -128,7 +128,7 @@ runs:
writeFile := func(filename string, data []byte, perm fs.FileMode) error {
assert.Equal(t, "actionDir/actionPath/trampoline.js", filename)
assert.Equal(t, fs.FileMode(0o400), perm)
assert.Equal(t, fs.FileMode(0o644), perm)
return nil
}
@@ -178,7 +178,8 @@ func TestActionRunner(t *testing.T) {
Using: "node16",
},
},
env: map[string]string{},
env: map[string]string{},
remoteAction: mustNewRemoteAction(t, "org/repo/path@ref"),
},
expectedEnv: map[string]string{"INPUT_KEY": "default value"},
},
@@ -217,7 +218,8 @@ func TestActionRunner(t *testing.T) {
Using: "node16",
},
},
env: map[string]string{},
env: map[string]string{},
remoteAction: mustNewRemoteAction(t, "org/repo/path@ref"),
},
expectedEnv: map[string]string{"STATE_name": "state value"},
},
@@ -228,7 +230,8 @@ func TestActionRunner(t *testing.T) {
ctx := t.Context()
cm := &containerMock{}
cm.On("CopyDir", "/var/run/act/actions/dir/", "dir/", false, true).Return(func(ctx context.Context) error { return nil })
containerActionDir := "/var/run/act/actions/" + model.UsesHash("/org/repo/path@ref") + "/"
cm.On("CopyDir", containerActionDir, "dir/", false, true).Return(func(ctx context.Context) error { return nil })
envMatcher := mock.MatchedBy(func(env map[string]string) bool {
for k, v := range tt.expectedEnv {
@@ -239,7 +242,7 @@ func TestActionRunner(t *testing.T) {
return true
})
cm.On("Exec", []string{"node", "--preserve-symlinks-main", "/var/run/act/actions/dir/path"}, envMatcher, "", "").Return(func(ctx context.Context) error { return nil })
cm.On("Exec", []string{"node", "--preserve-symlinks-main", containerActionDir + "path"}, envMatcher, "", "").Return(func(ctx context.Context) error { return nil })
tt.step.getRunContext().JobContainer = cm
+4 -8
View File
@@ -80,7 +80,7 @@ func reportStepError(ctx context.Context, rc *RunContext, err error) {
// executor can fetch all of them up front.
type actionPreparer interface {
prepareActionExecutor() common.Executor
actionDownloadInfo() (reference, sha string, ok bool)
actionDownloadInfo() (reference, sha string)
}
// printPrepareActions downloads every action the job uses before its first step runs and reports
@@ -101,15 +101,11 @@ func printPrepareActions(rc *RunContext, preparers []actionPreparer) common.Exec
reportStepError(ctx, rc, err)
return setupError{err}
}
reference, sha, ok := preparer.actionDownloadInfo()
if !ok {
reference, sha := preparer.actionDownloadInfo()
if sha == "" {
continue
}
if sha == "" {
rawLogger.Infof("Download action repository '%s'", reference)
} else {
rawLogger.Infof("Download action repository '%s' (SHA:%s)", reference, sha)
}
rawLogger.Infof("Download action repository '%s' (SHA:%s)", reference, sha)
}
return nil
}
+8 -12
View File
@@ -118,7 +118,6 @@ func (sfm *stepFactoryMock) newStep(model *model.Step, rc *RunContext) (step, er
type actionPreparerMock struct {
reference string
sha string
ok bool
err error
prepared int
}
@@ -130,8 +129,8 @@ func (apm *actionPreparerMock) prepareActionExecutor() common.Executor {
}
}
func (apm *actionPreparerMock) actionDownloadInfo() (string, string, bool) {
return apm.reference, apm.sha, apm.ok
func (apm *actionPreparerMock) actionDownloadInfo() (string, string) {
return apm.reference, apm.sha
}
func TestPrintPrepareActionsGolden(t *testing.T) {
@@ -140,18 +139,15 @@ func TestPrintPrepareActionsGolden(t *testing.T) {
ctx := common.WithLogger(t.Context(), logger.With("job", "j1"))
preparers := []actionPreparer{
&actionPreparerMock{reference: "actions/checkout@v7", sha: "9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", ok: true},
// A resolved commit is best effort; the ref alone is reported when it is unknown.
&actionPreparerMock{reference: "actions/setup-go@v6", ok: true},
&actionPreparerMock{reference: "actions/checkout@v7", sha: "9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0"},
// A step that downloads nothing, such as the checkout of the workflow's own repository.
&actionPreparerMock{ok: false},
&actionPreparerMock{},
}
require.NoError(t, printPrepareActions(&RunContext{}, preparers)(ctx))
want := strings.Join([]string{
"[j1] | Prepare all required actions",
"[j1] | Download action repository 'actions/checkout@v7' (SHA:9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0)",
"[j1] | Download action repository 'actions/setup-go@v6'",
"",
}, "\n")
assert.Equal(t, want, buf.String())
@@ -173,7 +169,7 @@ func TestPrintPrepareActionsFailsJobOnDownloadError(t *testing.T) {
downloadErr := errors.New("failed to fetch \"actions/checkout\"")
rc := &RunContext{}
remaining := &actionPreparerMock{reference: "actions/setup-go@v6", ok: true}
remaining := &actionPreparerMock{reference: "actions/setup-go@v6", sha: "sha"}
err := printPrepareActions(rc, []actionPreparer{
&actionPreparerMock{err: downloadErr},
@@ -267,7 +263,7 @@ func TestNewJobExecutorDownloadsAllActionsBeforeTheFirstStep(t *testing.T) {
for _, stepModel := range steps {
sm := &stepMock{}
apm := &actionPreparerMock{reference: "actions/checkout@v" + stepModel.ID, ok: true}
apm := &actionPreparerMock{reference: "actions/checkout@v" + stepModel.ID, sha: "sha" + stepModel.ID}
sfm.On("newStep", stepModel, rc).Return(&actionStepMock{stepMock: sm, actionPreparerMock: apm}, nil)
sm.On("pre").Return(func(ctx context.Context) error {
@@ -304,8 +300,8 @@ func TestNewJobExecutorDownloadsAllActionsBeforeTheFirstStep(t *testing.T) {
}
assert.Equal(t, []string{
"Prepare all required actions",
"Download action repository 'actions/checkout@v1'",
"Download action repository 'actions/checkout@v2'",
"Download action repository 'actions/checkout@v1' (SHA:sha1)",
"Download action repository 'actions/checkout@v2' (SHA:sha2)",
"Complete job name: test",
"Run Pre 1",
"Run Pre 2",
+3 -4
View File
@@ -223,8 +223,6 @@ func TestActionScriptPaths(t *testing.T) {
assert.Nil(t, actionScriptPaths("/a", escaping))
}
// The bundle has to be patched whatever state the shared action directory is in, because a
// concurrent job's prepare checks the action out again and resets it.
func TestPatchActionsAtTheContainerCopy(t *testing.T) {
copiedBundle := func(t *testing.T, noPatch bool) string {
t.Helper()
@@ -234,12 +232,13 @@ func TestPatchActionsAtTheContainerCopy(t *testing.T) {
Step: &model.Step{Uses: "owner/repo/sub@v1"},
remoteAction: &remoteAction{Org: "owner", Repo: "repo", Path: "sub", Ref: "v1"},
action: &model.Action{Runs: model.ActionRuns{Using: "node20", Main: "index.js"}},
actionDir: t.TempDir(),
RunContext: &RunContext{
Config: &Config{ActionCacheDir: t.TempDir(), NoActionPatch: noPatch},
JobContainer: cm,
},
}
script := filepath.Join(sar.actionDir(), "sub", "index.js")
script := filepath.Join(sar.actionDir, "sub", "index.js")
require.NoError(t, os.MkdirAll(filepath.Dir(script), 0o755))
require.NoError(t, os.WriteFile(script, []byte(gateTSC), 0o600))
@@ -250,7 +249,7 @@ func TestPatchActionsAtTheContainerCopy(t *testing.T) {
copied = string(body)
return nil
})
require.NoError(t, maybeCopyToActionDir(t.Context(), sar, sar.actionDir(), "sub", "/var/run/act/actions/repo/sub"))
require.NoError(t, maybeCopyToActionDir(t.Context(), sar, sar.actionDir, "sub", "/var/run/act/actions/repo/sub"))
source, err := os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, gateTSC, string(source), "the shared copy is left as it was found")
+3 -1
View File
@@ -209,7 +209,9 @@ func (c Config) withInstanceAuth(input git.NewGitCloneExecutorInput, token strin
return input
}
input.ClientCertFile, input.ClientKeyFile = c.ClientCertFile, c.ClientKeyFile
input.InsecureSkipTLS = c.InsecureSkipTLS && onInstanceHost(u.Host, c.GitHubInstance)
if onInstanceHost(u.Host, c.GitHubInstance) {
input.InsecureSkipTLS, input.Transport = c.InsecureSkipTLS, c.InstanceTransport
}
if u.User == nil {
input.Token = token
}
+4
View File
@@ -185,8 +185,12 @@ func TestInstanceAuthAndTransportOnlyApplyToTheInstance(t *testing.T) {
InsecureSkipTLS: true,
ClientCertFile: "/cert",
ClientKeyFile: "/key",
InstanceTransport: &http.Transport{},
}
want := git.NewGitCloneExecutorInput{URL: tt.cloneURL, InsecureSkipTLS: tt.insecure}
if tt.insecure {
want.Transport = conf.InstanceTransport
}
if tt.token {
want.Token = "token"
}
+36 -10
View File
@@ -22,6 +22,7 @@ import (
"path/filepath"
"regexp"
"runtime"
"runtime/debug"
"slices"
"strconv"
"strings"
@@ -83,8 +84,9 @@ type RunContext struct {
platformImage string // container.image or the runs-on pick, resolved once by isEnabled
jobIndex, jobTotal int // strategy.job-index and job-total, total 0 when unknown
jobEnv map[string]string // workflow and job env as evaluated for the container and service specs
dockerActionImages map[*model.Step]string // Dockerfile action images built for this job, removed when it ends
jobEnv map[string]string // workflow and job env as evaluated for the container and service specs
dockerActionImages map[*model.Step]string // Dockerfile action images built for this job, removed when it ends
actionDownloads map[string]actionDownload // remote action repositories by clone URL and ref, fetched once per job
// summaryFileInitialized tracks which per-step summary files (workflow/step-summary-N.md)
// have already been created on the JobContainer. The runner sets up file-command files
@@ -718,31 +720,55 @@ func (rc *RunContext) cleanupJobResources(networkName string, createAndDeleteNet
logger := common.Logger(ctx)
rc.printServiceLogs(ctx)
errs := []error{rc.closeDockerProxy(ctx)}
var containers []common.Executor
if rc.JobContainer != nil {
errs = append(errs, rc.JobContainer.Remove()(ctx))
containers = append(containers, rc.JobContainer.Remove())
}
if len(rc.serviceContainers) > 0 {
logger.Infof("Cleaning up services for job %s", rc.JobName)
errs = append(errs, rc.stopServiceContainers()(ctx))
containers = append(containers, rc.stopServiceContainers())
}
errs = append(errs, runAll(ctx, containers))
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), jobCleanupTimeout)
defer cancel()
if !common.Dryrun(ctx) && !rc.Config.NoSweep && (preclean || rc.hadDockerProxy) {
errs = append(errs, container.RemoveDockerJobResources(ctx, rc.jobContainerName()))
errs = append(errs, container.RemoveDockerJobResources(ctx, rc.jobContainerName())) // after the containers, whose endpoints would block removing the job networks
}
var rest []common.Executor
if preclean || rc.deferVolumeCleanup == nil {
errs = append(errs, rc.cleanupJobVolumes(ctx))
rest = append(rest, rc.cleanupJobVolumes)
} else {
rc.deferVolumeCleanup(rc.cleanupJobVolumes)
}
if createAndDeleteNetwork {
logger.Infof("Cleaning up network for job %s, and network name is: %s", rc.JobName, networkName)
errs = append(errs, container.NewDockerNetworkDisconnectExecutor(networkName, rc.Config.CacheContainer).
IfBool(rc.Config.CacheContainer != "")(ctx))
errs = append(errs, container.NewDockerNetworkRemoveExecutor(networkName)(ctx))
rest = append(rest, func(ctx context.Context) error {
return errors.Join(
container.NewDockerNetworkDisconnectExecutor(networkName, rc.Config.CacheContainer).IfBool(rc.Config.CacheContainer != "")(ctx),
container.NewDockerNetworkRemoveExecutor(networkName)(ctx))
})
}
return errors.Join(errs...)
return errors.Join(append(errs, runAll(ctx, rest))...)
}
}
func runAll(ctx context.Context, executors []common.Executor) error {
errs := make([]error, len(executors))
var wg sync.WaitGroup
for i, executor := range executors {
wg.Go(func() {
defer func() {
if recovered := recover(); recovered != nil {
errs[i] = fmt.Errorf("panic: %v\n%s", recovered, debug.Stack())
}
}()
errs[i] = executor(ctx)
})
}
wg.Wait()
return errors.Join(errs...)
}
func (rc *RunContext) createJobVolumes(ctx context.Context, mounts map[string]string) error {
runnerUUID := rc.Config.ContainerNetworkCreateOptions.RunnerUUID
if runnerUUID == "" || common.Dryrun(ctx) {
+28 -14
View File
@@ -20,6 +20,7 @@ import (
"runtime"
"slices"
"strings"
"sync"
"testing"
"testing/synctest"
"time"
@@ -775,10 +776,10 @@ func TestRemoveDockerActionImagesByTag(t *testing.T) {
assert.Equal(t, []string{"act-action-dockeraction:latest-job"}, removed)
}
func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
func TestCleanupJobResources(t *testing.T) {
t.Setenv("TMPDIR", "/tmp")
proxyDir := t.TempDir()
for _, name := range []string{"synchronous", "proxy", "closed proxy", "kept proxy", "deferred", "preclean deferred", "canceled"} {
for _, name := range []string{"synchronous", "proxy", "closed proxy", "kept proxy", "deferred", "preclean deferred", "canceled proxy"} {
t.Run(name, func(t *testing.T) {
proxy, preclean, deferred := strings.HasSuffix(name, "proxy"), strings.HasPrefix(name, "preclean"), strings.HasSuffix(name, "deferred")
removesLabelled := (proxy || preclean) && name != "kept proxy"
@@ -787,8 +788,20 @@ func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
}
jobError, removeError, closeError := errors.New("remove job"), errors.New("remove service"), errors.New("close service")
job, service := &containerMock{}, &containerMock{}
job.On("Remove").Return(func(context.Context) error { return jobError }).Once()
service.On("Remove").Return(func(context.Context) error { return removeError }).Once()
removeAlongside := func(started, other chan struct{}, failure error) func(context.Context) error {
return func(context.Context) error {
close(started)
select {
case <-other:
case <-time.After(10 * time.Second):
t.Error("the containers are removed one after the other instead of alongside each other")
}
return failure
}
}
jobRemoving, serviceRemoving := make(chan struct{}), make(chan struct{})
job.On("Remove").Return(removeAlongside(jobRemoving, serviceRemoving, jobError)).Once()
service.On("Remove").Return(removeAlongside(serviceRemoving, jobRemoving, removeError)).Once()
service.On("Close").Return(func(context.Context) error { return closeError }).Once()
rc := &RunContext{
Config: &Config{CacheContainer: "cache-container", NoSweep: name == "kept proxy"},
@@ -825,7 +838,7 @@ func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
}
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
if name == "canceled" {
if name == "canceled proxy" {
cancel()
}
err := rc.cleanupJobResources("job-network", true, preclean)(ctx)
@@ -839,14 +852,10 @@ func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
for _, failure := range []error{jobError, removeError, closeError} {
require.ErrorIs(t, err, failure)
}
if name == "canceled" {
require.ErrorIs(t, err, context.Canceled)
} else {
for _, operation := range []string{"DELETE /volumes/" + rc.jobContainerName(), "DELETE /volumes/" + rc.jobContainerName() + "-env", "POST /networks/job-network/disconnect", "GET /networks"} {
require.ErrorContains(t, err, operation)
}
assert.Equal(t, 2, volumeRemovals)
for _, operation := range []string{"DELETE /volumes/" + rc.jobContainerName(), "DELETE /volumes/" + rc.jobContainerName() + "-env", "POST /networks/job-network/disconnect", "GET /networks"} {
require.ErrorContains(t, err, operation)
}
assert.Equal(t, 2, volumeRemovals)
assert.Equal(t, removesLabelled, strings.Contains(err.Error(), "labelled GET /containers/json"))
if removesLabelled {
require.ErrorContains(t, err, "labelled GET /networks")
@@ -1360,8 +1369,11 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
} {
t.Run(testcase.name, func(t *testing.T) {
var operations []string
var operationsMu sync.Mutex
record := func(operation string, failure error) func(context.Context) error {
return func(context.Context) error {
operationsMu.Lock()
defer operationsMu.Unlock()
operations = append(operations, operation)
return failure
}
@@ -1403,7 +1415,7 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
ctx := common.WithLogger(common.WithDryrun(t.Context(), true), logger)
rc.ExprEval = rc.NewExpressionEvaluator(ctx)
err = rc.startContainer().Then(rc.stopContainer()).Finally(rc.closeContainer())(ctx)
want := "job.Remove service.Remove service.Close service.Pull job.Pull"
want := "job.Remove service.Close service.Remove service.Pull job.Pull"
switch {
case testcase.pullError != nil:
require.ErrorIs(t, err, testcase.pullError)
@@ -1418,7 +1430,9 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
require.NoError(t, err)
}
}
assert.Equal(t, strings.Fields(want+" job.Remove service.Remove service.Close job.Close"), operations)
slices.Sort(operations[:3])
slices.Sort(operations[len(operations)-4 : len(operations)-1])
assert.Equal(t, strings.Fields(want+" job.Remove service.Close service.Remove job.Close"), operations)
assert.Empty(t, rc.Masks)
assert.Equal(t, testcase.pullError == nil && testcase.health != container.HealthUnhealthy, slices.ContainsFunc(hook.AllRecords(), func(entry logging.Record) bool {
return strings.HasPrefix(entry.Message, "::group::Print service container logs")
+2 -3
View File
@@ -47,7 +47,7 @@ func (sal *stepActionLocal) main() common.Executor {
defer rawLogger.Infof("::endgroup::")
actionDir := filepath.Join(sal.getRunContext().Config.Workdir, sal.Step.Uses)
_, containerActionPath := getContainerActionPaths(sal.Step, path.Join(actionDir, ""), sal.RunContext)
_, containerActionPath := getContainerActionPaths(sal)
localReader := func(filename string) (io.Reader, io.Closer, error) {
spath := path.Join(containerActionPath, filename)
@@ -127,8 +127,7 @@ func (sal *stepActionLocal) getActionModel() *model.Action {
func (sal *stepActionLocal) getCompositeRunContext(ctx context.Context) (*RunContext, error) {
if sal.compositeRunContext == nil {
actionDir := filepath.Join(sal.RunContext.Config.Workdir, sal.Step.Uses)
_, containerActionDir := getContainerActionPaths(sal.getStepModel(), actionDir, sal.RunContext)
_, containerActionDir := getContainerActionPaths(sal)
compositeRunContext, err := newCompositeRunContext(ctx, sal.RunContext, sal, containerActionDir)
if err != nil {
+44 -52
View File
@@ -30,13 +30,16 @@ type stepActionRemote struct {
action *model.Action
env map[string]string
remoteAction *remoteAction
resolvedSha string
actionDir string // the repository at the resolved commit, shared by the job's actions in it
resolvedSha string // set on the step that made the job's download of the repository
}
var stepActionRemoteNewCloneExecutor = git.NewGitCloneExecutor
type actionDownload struct {
dir string
err error
}
// selfRepoPrefix introduces a self-repository reference: the action lives in the repo holding the file that wrote the `uses:`.
const selfRepoPrefix = "$/"
var stepActionRemoteFetch = git.FetchAction
func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
return func(ctx context.Context) error {
@@ -64,61 +67,57 @@ func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
common.Logger(ctx).Debugf("Skipping local actions/checkout because workdir was already copied")
return nil
}
actionDir := sar.actionDir()
defaultActionURL := sar.RunContext.Config.DefaultActionURL()
// For Gitea
// github.Token, unlike Config.GetToken, survives the composite config copy, which nils Config.Secrets.
gitClone := stepActionRemoteNewCloneExecutor(sar.RunContext.Config.withInstanceAuth(git.NewGitCloneExecutorInput{
URL: sar.remoteAction.CloneURL(defaultActionURL),
Ref: sar.remoteAction.Ref,
Dir: actionDir,
OfflineMode: sar.RunContext.Config.ActionOfflineMode,
Depth: sar.RunContext.Config.ActionCloneDepth,
// printPrepareActions reports the download with its resolved commit.
Quiet: true,
}, github.Token))
if err := gitClone(ctx); err != nil {
var refErr *git.Error
switch {
case errors.As(err, &refErr) && errors.Is(err, git.ErrShortRef):
return fmt.Errorf("unable to resolve action `%s`, the provided ref `%s` is the shortened version of a commit SHA, which is not supported. Please use the full commit SHA `%s` instead",
sar.Step.Uses, sar.remoteAction.Ref, refErr.Commit())
default:
return err
cloneURL := sar.remoteAction.CloneURL(sar.RunContext.Config.DefaultActionURL())
job := sar.RunContext.topLevelRunContext()
key := cloneURL + "@" + sar.remoteAction.Ref
download, ok := job.actionDownloads[key]
if !ok {
// For Gitea
// github.Token, unlike Config.GetToken, survives the composite config copy, which nils Config.Secrets.
download.dir, sar.resolvedSha, download.err = stepActionRemoteFetch(ctx, sar.RunContext.Config.withInstanceAuth(git.NewGitCloneExecutorInput{
URL: cloneURL,
Ref: sar.remoteAction.Ref,
Dir: filepath.Join(sar.RunContext.ActionCacheDir(), model.UsesHash(key)),
OfflineMode: sar.RunContext.Config.ActionOfflineMode,
Depth: sar.RunContext.Config.ActionCloneDepth,
}, github.Token))
if job.actionDownloads == nil {
job.actionDownloads = map[string]actionDownload{}
}
job.actionDownloads[key] = download
}
var refErr *git.Error
switch {
case errors.As(download.err, &refErr) && errors.Is(download.err, git.ErrShortRef):
return fmt.Errorf("unable to resolve action `%s`, the provided ref `%s` is the shortened version of a commit SHA, which is not supported. Please use the full commit SHA `%s` instead",
sar.Step.Uses, sar.remoteAction.Ref, refErr.Commit())
case download.err != nil:
return download.err
}
sar.actionDir = download.dir
unlock, err := git.AcquireCloneLock(ctx, actionDir)
unlock, err := git.AcquireCloneLock(ctx, sar.actionDir)
if err != nil {
return err
}
defer unlock()
// Best effort: the download report falls back to the ref alone when the commit is unknown.
if _, sha, err := git.FindGitRevision(ctx, actionDir); err != nil {
common.Logger(ctx).Debugf("unable to resolve the commit of %s: %v", sar.remoteAction.Reference(), err)
} else {
sar.resolvedSha = sha
}
remoteReader := func(filename string) (io.Reader, io.Closer, error) {
f, err := os.Open(filepath.Join(actionDir, sar.remoteAction.Path, filename))
f, err := os.Open(filepath.Join(sar.actionDir, sar.remoteAction.Path, filename))
return f, f, err
}
actionModel, err := sar.readAction(ctx, sar.Step, actionDir, sar.remoteAction.Path, remoteReader, os.WriteFile)
actionModel, err := sar.readAction(ctx, sar.Step, sar.actionDir, sar.remoteAction.Path, remoteReader, os.WriteFile)
sar.action = actionModel
return err
}
}
// actionDownloadInfo reports the action this step downloaded and the commit it resolved to. ok is
// false when nothing was fetched, as for the local checkout of the workflow's own repository.
func (sar *stepActionRemote) actionDownloadInfo() (reference, sha string, ok bool) {
if sar.remoteAction == nil || sar.action == nil {
return "", "", false
func (sar *stepActionRemote) actionDownloadInfo() (reference, sha string) { // once per job, as actions/runner reports
if sar.resolvedSha == "" {
return "", ""
}
return sar.remoteAction.Reference(), sar.resolvedSha, true
return fmt.Sprintf("%s/%s@%s", sar.remoteAction.Org, sar.remoteAction.Repo, sar.remoteAction.Ref), sar.resolvedSha
}
func (sar *stepActionRemote) pre() common.Executor {
@@ -147,7 +146,7 @@ func (sar *stepActionRemote) main() common.Executor {
return sar.RunContext.JobContainer.CopyDir(copyToPath, sar.RunContext.Config.Workdir+string(filepath.Separator)+".", sar.RunContext.Config.UseGitIgnore, false)(ctx)
}
return sar.runAction(sar, sar.actionDir(), sar.remoteAction)(ctx)
return sar.runAction(sar, sar.actionDir, sar.remoteAction)(ctx)
}),
)
}
@@ -156,13 +155,8 @@ func (sar *stepActionRemote) post() common.Executor {
return runStepExecutor(sar, stepStagePost, runPostStep(sar)).If(hasPostStep(sar)).If(shouldRunPostStep(sar))
}
func (sar *stepActionRemote) actionDir() string {
uses := sar.Step.Uses
if strings.HasPrefix(uses, selfRepoPrefix) {
// The same `$/x` names a different action per enclosing repo, so key the cache on what it resolved to.
uses = sar.remoteAction.URL + "/" + sar.remoteAction.Reference()
}
return fmt.Sprintf("%s/%s", sar.RunContext.ActionCacheDir(), model.UsesHash(uses))
func (sar *stepActionRemote) containerKey() string {
return model.UsesHash(sar.remoteAction.URL + "/" + sar.remoteAction.Reference())
}
func (sar *stepActionRemote) getRunContext() *RunContext {
@@ -213,9 +207,7 @@ func (sar *stepActionRemote) getActionModel() *model.Action {
func (sar *stepActionRemote) getCompositeRunContext(ctx context.Context) (*RunContext, error) {
if sar.compositeRunContext == nil {
actionDir := sar.actionDir()
actionLocation := path.Join(actionDir, sar.remoteAction.Path)
_, containerActionDir := getContainerActionPaths(sar.getStepModel(), actionLocation, sar.RunContext)
_, containerActionDir := getContainerActionPaths(sar)
compositeRunContext, err := newCompositeRunContext(ctx, sar.RunContext, sar, containerActionDir)
if err != nil {
+73 -85
View File
@@ -12,6 +12,7 @@ import (
"io"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
"testing"
@@ -31,14 +32,10 @@ type stepActionRemoteMocks struct {
mock.Mock
}
func actionDirSuffix(suffix string) any {
return mock.MatchedBy(func(actionDir string) bool { return strings.HasSuffix(actionDir, suffix) })
}
func setCloneExecutor(t *testing.T, executor func(git.NewGitCloneExecutorInput) common.Executor) {
original := stepActionRemoteNewCloneExecutor
stepActionRemoteNewCloneExecutor = executor
t.Cleanup(func() { stepActionRemoteNewCloneExecutor = original })
func setFetch(t *testing.T, fetch func(context.Context, git.NewGitCloneExecutorInput) (string, string, error)) {
original := stepActionRemoteFetch
stepActionRemoteFetch = fetch
t.Cleanup(func() { stepActionRemoteFetch = original })
}
func TestShortSHAActionRejected(t *testing.T) {
@@ -181,12 +178,10 @@ func TestStepActionRemote(t *testing.T) {
sarm := &stepActionRemoteMocks{}
clonedAction := false
setCloneExecutor(t, func(input git.NewGitCloneExecutorInput) common.Executor {
return func(ctx context.Context) error {
clonedAction = true
return nil
}
actionDir := t.TempDir()
setFetch(t, func(_ctx context.Context, _input git.NewGitCloneExecutorInput) (string, string, error) {
clonedAction = true
return actionDir, "", nil
})
sar := &stepActionRemote{
@@ -213,10 +208,10 @@ func TestStepActionRemote(t *testing.T) {
sar.RunContext.ExprEval = sar.RunContext.NewExpressionEvaluator(ctx)
if tt.mocks.read {
sarm.On("readAction", sar.Step, actionDirSuffix(sar.Step.UsesHash()), "", mock.Anything, mock.Anything).Return(&model.Action{}, nil)
sarm.On("readAction", sar.Step, actionDir, "", mock.Anything, mock.Anything).Return(&model.Action{}, nil)
}
if tt.mocks.run {
sarm.On("runAction", sar, actionDirSuffix(sar.Step.UsesHash()), mustNewRemoteAction(t, sar.Step.Uses)).Return(func(ctx context.Context) error { return tt.runError })
sarm.On("runAction", sar, actionDir, mustNewRemoteAction(t, sar.Step.Uses)).Return(func(ctx context.Context) error { return tt.runError })
cm.On("Copy", "/var/run/act", mock.AnythingOfType("[]*container.FileEntry")).Return(noopExecutor)
@@ -263,40 +258,60 @@ func TestStepActionRemote(t *testing.T) {
}
func TestStepActionRemotePrepare(t *testing.T) {
for _, test := range []struct {
name, uses, instance, actionPath, wantURL string
}{
{name: "nested action", uses: "org/repo/path@ref", instance: "https://github.com", actionPath: "path", wantURL: "https://github.com/org/repo"},
{name: "instance fallback", uses: "actions/setup-go@v4", instance: "gitea.example", wantURL: "https://gitea.example/actions/setup-go"},
} {
t.Run(test.name, func(t *testing.T) {
var actualURL string
setCloneExecutor(t, func(input git.NewGitCloneExecutorInput) common.Executor {
return func(context.Context) error {
actualURL = input.URL
return nil
}
})
actionMocks := &stepActionRemoteMocks{}
action := &stepActionRemote{
Step: &model.Step{Uses: test.uses},
RunContext: &RunContext{
Config: &Config{GitHubInstance: test.instance, ActionCacheDir: t.TempDir()},
Run: &model.Run{JobID: "1", Workflow: &model.Workflow{
Jobs: map[string]*model.Job{"1": {}},
}},
},
readAction: actionMocks.readAction,
}
actionMocks.On("readAction", action.Step, actionDirSuffix(action.Step.UsesHash()), test.actionPath,
mock.Anything, mock.Anything).Return(&model.Action{}, nil)
require.NoError(t, action.prepareActionExecutor()(t.Context()))
assert.Equal(t, test.wantURL, actualURL)
actionMocks.AssertExpectations(t)
})
var fetched, read []string
setFetch(t, func(_ctx context.Context, input git.NewGitCloneExecutorInput) (string, string, error) {
fetched = append(fetched, input.URL+"@"+input.Ref)
if input.Ref == "missing" {
return "", "", assert.AnError
}
return filepath.Join(input.Dir, "commit"), "sha-" + input.Ref, nil
})
newJob := func(instance string) *RunContext {
return &RunContext{
Config: &Config{GitHubInstance: instance, ActionCacheDir: t.TempDir()},
Run: &model.Run{JobID: "1", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"1": {}}}},
JobContainer: &jobContainerMock{},
}
}
prepare := func(rc *RunContext, uses string) (*stepActionRemote, error) {
sar := &stepActionRemote{Step: &model.Step{Uses: uses}, RunContext: rc, readAction: func(_ctx context.Context, _step *model.Step, _actionDir, actionPath string, _readFile actionYamlReader, _writeFile fileWriter) (*model.Action, error) {
read = append(read, actionPath)
return &model.Action{}, nil
}}
return sar, sar.prepareActionExecutor()(t.Context())
}
job := newJob("https://github.com")
restore, err := prepare(job, "org/repo/restore@v1")
require.NoError(t, err)
root, err := prepare(job, "org/repo@v1")
require.NoError(t, err)
nested, err := prepare(&RunContext{Config: job.Config, Run: job.Run, JobContainer: job.JobContainer, Parent: job}, "org/repo/save@v1")
require.NoError(t, err)
other, err := prepare(job, "org/repo@v2")
require.NoError(t, err)
_, err = prepare(newJob("gitea.example"), "org/repo@v1")
require.NoError(t, err)
for range 2 {
_, err = prepare(job, "org/repo@missing")
require.ErrorIs(t, err, assert.AnError)
}
assert.Equal(t, []string{"https://github.com/org/repo@v1", "https://github.com/org/repo@v2", "https://gitea.example/org/repo@v1", "https://github.com/org/repo@missing"}, fetched)
assert.Equal(t, []string{"restore", "", "save", "", ""}, read)
assert.Equal(t, root.actionDir, restore.actionDir)
assert.Equal(t, root.actionDir, nested.actionDir)
assert.NotEqual(t, root.actionDir, other.actionDir)
reference, sha := restore.actionDownloadInfo()
assert.Equal(t, []string{"org/repo@v1", "sha-v1"}, []string{reference, sha})
_, sha = root.actionDownloadInfo()
assert.Empty(t, sha)
_, rootContainerDir := getContainerActionPaths(root)
_, restoreContainerDir := getContainerActionPaths(restore)
assert.NotEqual(t, path.Dir(rootContainerDir), path.Dir(restoreContainerDir))
assert.Equal(t, "restore", path.Base(restoreContainerDir))
}
func TestStepActionRemotePost(t *testing.T) {
@@ -701,17 +716,15 @@ func Test_newRemoteActionSelfRepo(t *testing.T) {
}
}
func Test_stepActionRemoteSelfRepoActionDir(t *testing.T) {
dirFor := func(repo string) string {
func Test_stepActionRemoteSelfRepoContainerKeyDiffersPerRepo(t *testing.T) {
keyFor := func(repo string) string {
sar := &stepActionRemote{
Step: &model.Step{Uses: "$/.gitea/actions/build"},
RunContext: &RunContext{Config: &Config{ActionCacheDir: "/cache"}},
remoteAction: &remoteAction{Org: "owner", Repo: repo, Ref: "v1"},
}
return sar.actionDir()
return sar.containerKey()
}
// The same `$/x` in two repos must not share a cache directory.
assert.NotEqual(t, dirFor("one"), dirFor("two"))
assert.NotEqual(t, keyFor("one"), keyFor("two"))
}
func Test_remoteActionReference(t *testing.T) {
@@ -774,37 +787,11 @@ func TestStepActionRemotePreResolvesDownloadedCommit(t *testing.T) {
require.NoError(t, sar.prepareActionExecutor()(t.Context()))
reference, sha, ok := sar.actionDownloadInfo()
assert.True(t, ok)
reference, sha := sar.actionDownloadInfo()
assert.Equal(t, "actions/setup-go@main", reference)
assert.Equal(t, wantSha, sha)
}
func TestStepActionRemoteActionDownloadInfo(t *testing.T) {
t.Run("reports the action and its resolved commit", func(t *testing.T) {
sar := &stepActionRemote{
remoteAction: mustNewRemoteAction(t, "actions/checkout@v7"),
action: &model.Action{},
resolvedSha: "9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0",
}
reference, sha, ok := sar.actionDownloadInfo()
assert.True(t, ok)
assert.Equal(t, "actions/checkout@v7", reference)
assert.Equal(t, "9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", sha)
})
t.Run("reports nothing when no action was downloaded", func(t *testing.T) {
// The local checkout of the workflow's own repository resolves no action.
sar := &stepActionRemote{remoteAction: mustNewRemoteAction(t, "actions/checkout@v7")}
_, _, ok := sar.actionDownloadInfo()
assert.False(t, ok)
})
}
func Test_safeFilename(t *testing.T) {
tests := []struct {
s string
@@ -856,9 +843,10 @@ func TestStepActionRemoteCloneTokenSurvivesNilSecrets(t *testing.T) {
ctx := t.Context()
var capturedToken string
setCloneExecutor(t, func(input git.NewGitCloneExecutorInput) common.Executor {
actionDir := t.TempDir()
setFetch(t, func(_ctx context.Context, input git.NewGitCloneExecutorInput) (string, string, error) {
capturedToken = input.Token
return func(ctx context.Context) error { return nil }
return actionDir, "", nil
})
sarm := &stepActionRemoteMocks{}
@@ -886,7 +874,7 @@ func TestStepActionRemoteCloneTokenSurvivesNilSecrets(t *testing.T) {
}
sar.RunContext.ExprEval = sar.RunContext.NewExpressionEvaluator(ctx)
sarm.On("readAction", sar.Step, actionDirSuffix(sar.Step.UsesHash()), "", mock.Anything, mock.Anything).Return(&model.Action{}, nil)
sarm.On("readAction", sar.Step, actionDir, "", mock.Anything, mock.Anything).Return(&model.Action{}, nil)
err := sar.prepareActionExecutor()(ctx)
require.NoError(t, err)
+48
View File
@@ -0,0 +1,48 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build e2e
package e2e
import (
"fmt"
"os"
"testing"
)
func testMovingActionRef(t *testing.T) {
t.Parallel()
ctx := t.Context()
api, repo, _ := startIsolatedScenario(t, "action_ref.yml", "e2e-action-ref", runnerOptions{})
const actionRepo = "e2e-moving-action"
if err := api.CreateRepo(ctx, actionRepo); err != nil {
t.Fatalf("create action repo: %v", err)
}
action, err := os.ReadFile("testdata/actions/moving.yml")
if err != nil {
t.Fatalf("read action fixture: %v", err)
}
if err := api.CreateFile(ctx, actionRepo, "action.yml", string(action), "add action"); err != nil {
t.Fatalf("push action: %v", err)
}
branch, err := api.DefaultBranch(ctx, repo)
if err != nil {
t.Fatalf("get default branch: %v", err)
}
var previousID int64
for revision := range 3 {
name := fmt.Sprintf("revision-%d", revision)
if err := api.CreateFile(ctx, actionRepo, name, name, "add "+name); err != nil {
t.Fatalf("move the action ref: %v", err)
}
if err := waitForDispatch(ctx, api, repo, "action_ref.yml", branch, map[string]string{"revision": name}); err != nil {
t.Fatalf("dispatch workflow: %v", err)
}
wfRun := waitForRunAfter(t, api, repo, previousID)
requireSuccess(t, api, repo, wfRun.ID)
previousID = wfRun.ID
}
}
+1
View File
@@ -31,6 +31,7 @@ func TestCompatibility(t *testing.T) {
t.Run("cancellation_and_log_streaming", testRunCancellation)
t.Run("dispatch", testWorkflowDispatch)
t.Run("builtin_checkout", testBuiltinCheckout)
t.Run("moving_action_ref", testMovingActionRef)
t.Run("ephemeral", testEphemeralRunner)
t.Run("kubernetes", testKubernetes)
}
+3 -3
View File
@@ -23,7 +23,7 @@ func testWorkflowDispatch(t *testing.T) {
t.Fatalf("get default branch: %v", err)
}
inputs := map[string]string{"subject": "dispatch-input-value"}
if err := waitForDispatch(ctx, api, repo, branch, inputs); err != nil {
if err := waitForDispatch(ctx, api, repo, "dispatch.yml", branch, inputs); err != nil {
t.Fatalf("dispatch workflow: %v", err)
}
@@ -35,13 +35,13 @@ func testWorkflowDispatch(t *testing.T) {
}
// Gitea indexes workflows asynchronously after the Contents API commit.
func waitForDispatch(ctx context.Context, api *GiteaAPI, repo, branch string, inputs map[string]string) error {
func waitForDispatch(ctx context.Context, api *GiteaAPI, repo, workflow, branch string, inputs map[string]string) error {
ctx, cancel := context.WithTimeout(ctx, time.Minute)
defer cancel()
var lastErr error
for {
lastErr = api.DispatchWorkflow(ctx, repo, "dispatch.yml", branch, inputs)
lastErr = api.DispatchWorkflow(ctx, repo, workflow, branch, inputs)
if lastErr == nil {
return nil
}
+9
View File
@@ -0,0 +1,9 @@
name: moving
inputs:
revision:
required: true
runs:
using: composite
steps:
- run: test -e "$GITHUB_ACTION_PATH/${{ inputs.revision }}"
shell: sh
+13
View File
@@ -0,0 +1,13 @@
name: action-ref
on:
workflow_dispatch:
inputs:
revision:
required: true
jobs:
moving:
runs-on: e2e-action-ref
steps:
- uses: self:e2e-admin/e2e-moving-action@main
with:
revision: ${{ inputs.revision }}
+5 -1
View File
@@ -157,6 +157,8 @@ var removeOrphanJobVolumes = container.RemoveOrphanJobVolumes
var removeOrphanKubernetesResources = container.RemoveOrphanKubernetesResources
const orphanDockerResourceAge = 10 * time.Minute // spares the network and volumes of a job still starting its first container
// OnIdle performs lightweight maintenance during polling idle windows.
// It runs synchronously on the poller goroutine; shouldRunIdleCleanup
// throttles invocations to runner.idle_cleanup_interval so the impact on
@@ -189,10 +191,12 @@ func (r *Runner) OnIdle(ctx context.Context) {
}
func (r *Runner) cleanupOrphanDockerResources(ctx context.Context) {
ctx, cancel := context.WithTimeout(ctx, time.Minute)
defer cancel()
if r.uuid == "" || (!r.requiresDocker() && !dockerReachable(ctx)) {
return
}
cutoff := r.now().Add(-r.cfg.Runner.WorkdirCleanupAge)
cutoff := r.now().Add(-orphanDockerResourceAge)
if err := removeOrphanNetworks(ctx, r.uuid, r.isolatedCacheContainer(), cutoff); err != nil {
logging.Warnf("failed to clean up networks left behind by earlier jobs: %v", err)
}
+5 -3
View File
@@ -317,7 +317,9 @@ func TestRunnerOnIdleRemovesOrphanNetworks(t *testing.T) {
var sweptVolumes []string
var sweptCutoff time.Time
origRemoveOrphanNetworks := removeOrphanNetworks
removeOrphanNetworks = func(_ context.Context, runnerUUID, detach string, createdBefore time.Time) error {
removeOrphanNetworks = func(ctx context.Context, runnerUUID, detach string, createdBefore time.Time) error {
_, hasDeadline := ctx.Deadline()
assert.True(t, hasDeadline)
swept = append(swept, runnerUUID)
detached = append(detached, detach)
sweptCutoff = createdBefore
@@ -327,7 +329,7 @@ func TestRunnerOnIdleRemovesOrphanNetworks(t *testing.T) {
origRemoveOrphanJobVolumes := removeOrphanJobVolumes
removeOrphanJobVolumes = func(_ context.Context, runnerUUID string, createdBefore time.Time) error {
sweptVolumes = append(sweptVolumes, runnerUUID)
assert.Equal(t, now.Add(-24*time.Hour), createdBefore)
assert.Equal(t, now.Add(-orphanDockerResourceAge), createdBefore)
return nil
}
t.Cleanup(func() { removeOrphanJobVolumes = origRemoveOrphanJobVolumes })
@@ -350,7 +352,7 @@ func TestRunnerOnIdleRemovesOrphanNetworks(t *testing.T) {
assert.Equal(t, []string{"runner-1"}, swept)
assert.Equal(t, swept, sweptVolumes)
// a network of a job starting during the pass is younger than this and so out of scope
assert.Equal(t, now.Add(-24*time.Hour), sweptCutoff)
assert.Equal(t, now.Add(-orphanDockerResourceAge), sweptCutoff)
// a host-only runner has no daemon to sweep
origDockerReachable := dockerReachable
+9 -5
View File
@@ -61,9 +61,9 @@ runner:
#workdir_cleanup_age: 24h
# Cadence for the idle cleanup pass. Besides the directories above, on runners that use
# docker it removes the per-job networks and volumes of jobs this runner did not live to
# tear down, which would otherwise hold a subnet of the daemon address pool and the job's
# workspace contents until the host is rebuilt. On runners with kubernetes labels it
# removes such jobs' pods and Secrets.
# tear down once unused and 10 minutes old, which would otherwise hold a subnet of the
# daemon address pool and the job's workspace contents until the host is rebuilt. On
# runners with kubernetes labels it removes such jobs' pods and Secrets.
#idle_cleanup_interval: 10m
# The base interval for periodic log flush to the Gitea instance.
# Logs may be sent earlier if the buffer reaches log_report_batch_size
@@ -76,6 +76,9 @@ runner:
# Flush logs immediately when the buffer reaches this many rows.
# This ensures bursty output (e.g., npm install) is delivered promptly.
#log_report_batch_size: 100
# The lowest severity of the runner's own messages (not step output) that are sent to the
# job log between steps: trace, debug, info, warn, error. Default if omitted: follow log.level.
#log_report_level: info
# The interval for reporting task state (step status, timing) to the Gitea instance.
# State is also reported on step transitions (start/stop) and after each log flush.
#state_report_interval: 5s
@@ -88,11 +91,12 @@ runner:
# it replaces https://github.com with the value here, which is useful for some special network environments.
#github_mirror: ''
# Extra HTTP headers for runner API requests to Gitea, e.g. reverse proxy authentication.
# Not used for action clones, artifacts, job summaries or jobs. Runner headers take precedence.
# Not used for action downloads, artifacts, job summaries or jobs. Runner headers take precedence.
# Dropped on redirects to another hostname, or to HTTP if the instance URL uses HTTPS.
#extra_headers:
# X-Proxy-Token: 'example'
# When true (the default), fetch only the requested ref of an action repository (e.g. actions/checkout@v4) at depth 1 instead of cloning every branch's full history.
# When true (the default), clone only the requested ref at depth 1 instead of every branch's full history. Actions download as
# archives, so this applies to reusable workflows and to actions whose host serves no archive of them.
# Set to false to clone the full history.
#action_shallow_clone: true
# When true (the default), inject the ACT=true environment variable into jobs.
+8 -2
View File
@@ -71,16 +71,17 @@ type Runner struct {
FetchTimeout time.Duration `yaml:"fetch_timeout"` // FetchTimeout specifies the timeout duration for fetching resources.
FetchInterval time.Duration `yaml:"fetch_interval"` // FetchInterval specifies the interval duration for fetching resources.
FetchIntervalMax time.Duration `yaml:"fetch_interval_max"` // FetchIntervalMax specifies the maximum backoff interval when idle.
WorkdirCleanupAge time.Duration `yaml:"workdir_cleanup_age"` // WorkdirCleanupAge removes stale bind-workdir task directories, orphaned host-mode scratch dirs and orphaned docker and Kubernetes job resources older than this duration during idle cleanup.
WorkdirCleanupAge time.Duration `yaml:"workdir_cleanup_age"` // WorkdirCleanupAge removes stale bind-workdir task directories, orphaned host-mode scratch dirs and orphaned Kubernetes job resources older than this duration during idle cleanup.
IdleCleanupInterval time.Duration `yaml:"idle_cleanup_interval"` // IdleCleanupInterval runs the idle cleanup (stale directories, orphaned docker networks and volumes, orphaned job pods) periodically while the runner is idle. Set to 0 to disable cleanup cadence.
LogReportInterval time.Duration `yaml:"log_report_interval"` // LogReportInterval specifies the base interval for periodic log flush.
LogReportMaxLatency time.Duration `yaml:"log_report_max_latency"` // LogReportMaxLatency specifies the max time a log row can wait before being sent.
LogReportBatchSize int `yaml:"log_report_batch_size"` // LogReportBatchSize triggers immediate log flush when buffer reaches this size.
LogReportLevel string `yaml:"log_report_level"` // LogReportLevel is the lowest level of the runner's own messages sent to the job log between steps. Empty, the default, follows log.level.
StateReportInterval time.Duration `yaml:"state_report_interval"` // StateReportInterval specifies the interval for state reporting.
ReportCloseTimeout time.Duration `yaml:"report_close_timeout"` // ReportCloseTimeout caps each RPC attempt when flushing the final logs and task state at job completion, on a detached context so a server cancel can't block the acknowledgement.
Labels []string `yaml:"labels"` // Labels specify the labels of the runner. Labels are declared on each startup
GithubMirror string `yaml:"github_mirror"` // GithubMirror defines what mirrors should be used when using github
ActionShallowClone *bool `yaml:"action_shallow_clone"` // ActionShallowClone fetches only the requested ref of an action repository at depth 1 instead of cloning every branch's full history. It is a pointer to distinguish between false and not set; if not set, it defaults to true.
ActionShallowClone *bool `yaml:"action_shallow_clone"` // ActionShallowClone clones only the requested ref at depth 1 instead of every branch's full history, for reusable workflows and actions without an archive. It is a pointer to distinguish between false and not set; if not set, it defaults to true.
SetActEnv *bool `yaml:"set_act_env"` // SetActEnv controls whether the ACT=true environment variable is injected into jobs. It is a pointer to distinguish between false and not set; if not set, it defaults to true. Set it to false so workflows gated on `if: ${{ !env.ACT }}` behave like on GitHub.
PatchActions *bool `yaml:"patch_actions"` // PatchActions applies compatibility patches to the actions a job runs, so actions written for GitHub work against Gitea, see act/runner/patch_actions.go. It is a pointer to distinguish between false and not set; if not set, it defaults to true. Set it to false to run every action exactly as published, at the price of the artifact actions refusing.
AllocatePTY bool `yaml:"allocate_pty"` // AllocatePTY allocates a pseudo-TTY for each step's process. Default is false, matching GitHub's actions/runner. Enable only for jobs that need an interactive terminal; tools like docker build emit redrawing progress frames into the captured log when a TTY is present. Applies to both host and docker backends.
@@ -348,6 +349,11 @@ func LoadDefault(file string) (*Config, error) {
if !slices.Contains(ToolCacheModes, cfg.Runner.ToolCacheMode) {
return nil, fmt.Errorf("invalid runner.tool_cache_mode %q: must be one of %q", cfg.Runner.ToolCacheMode, ToolCacheModes)
}
if cfg.Runner.LogReportLevel != "" {
if _, err := logging.ParseLevel(cfg.Runner.LogReportLevel); err != nil {
return nil, fmt.Errorf("invalid runner.log_report_level: %w", err)
}
}
if err := validateExtraHeaders(cfg.Runner.ExtraHeaders); err != nil {
return nil, err
}
+5
View File
@@ -172,6 +172,11 @@ func TestLoadDefault_ToolCacheMode(t *testing.T) {
assert.Contains(t, err.Error(), "tool_cache_mode")
}
func TestLoadDefault_RejectsInvalidLogReportLevel(t *testing.T) {
_, err := LoadDefault(write(t, "runner:\n log_report_level: loud\n"))
require.ErrorContains(t, err, "runner.log_report_level")
}
func TestLoadDefault_LoadsExtraHeaders(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(path, []byte("runner:\n extra_headers:\n X-Proxy-Token: proxy-token\n"), 0o600))
+8 -2
View File
@@ -89,6 +89,7 @@ type Reporter struct {
logReportInterval time.Duration
logReportMaxLatency time.Duration
logBatchSize int
logReportLevel logging.Level
stateReportInterval time.Duration
// closeTimeout bounds each RPC attempt in the final flush, on a context
// detached from r.ctx so a server cancel can't abort the acknowledgement.
@@ -133,6 +134,7 @@ func NewReporter(ctx context.Context, cancel context.CancelFunc, client client.C
logReportInterval: cfg.Runner.LogReportInterval,
logReportMaxLatency: cfg.Runner.LogReportMaxLatency,
logBatchSize: cfg.Runner.LogReportBatchSize,
logReportLevel: logging.GetLevel(),
stateReportInterval: cfg.Runner.StateReportInterval,
closeTimeout: cfg.Runner.ReportCloseTimeout,
logNotify: make(chan struct{}, 1),
@@ -146,6 +148,10 @@ func NewReporter(ctx context.Context, cancel context.CancelFunc, client client.C
jobLog: openJobLog(cfg.Log.Job, task.Id, time.Now()),
}
if level, err := logging.ParseLevel(cfg.Runner.LogReportLevel); err == nil { // unset keeps log.level
rv.logReportLevel = level
}
rv.daemonWait = 6 * rv.effectiveCloseTimeout()
registerGlobalMasks(rv)
@@ -954,9 +960,9 @@ func (r *Reporter) duringSteps() bool {
// shouldAppendLogRow reports whether a non-raw_output entry should be written
// to the job log: only when we are between steps and the entry's level is
// within the globally configured log level.
// within runner.log_report_level.
func (r *Reporter) shouldAppendLogRow(entry logging.Record) bool {
return !r.duringSteps() && entry.Level >= logging.GetLevel()
return !r.duringSteps() && entry.Level >= r.logReportLevel
}
var stringToResult = map[string]runnerv1.Result{
+3
View File
@@ -309,7 +309,10 @@ func TestReporter_LogLevelFiltering(t *testing.T) {
taskCtx, err := structpb.NewStruct(map[string]any{})
require.NoError(t, err)
cfg, _ := config.LoadDefault("")
cfg.Runner.LogReportLevel = "warn"
reporter := NewReporter(ctx, cancel, client, &runnerv1.Task{Context: taskCtx}, cfg)
assert.False(t, reporter.shouldAppendLogRow(logging.Record{Level: logging.LevelInfo}))
assert.True(t, reporter.shouldAppendLogRow(logging.Record{Level: logging.LevelWarn}))
reporter.RunDaemon()
defer func() {
require.NoError(t, reporter.Close(""))