enhance: test on Windows and fix what it found (#1348)

Adds a weekly and on-demand workflow that runs the cross-compiled tests on Windows Server 2025 Core in a [dockur/windows](https://github.com/dockur/windows) QEMU VM, and fixes the Windows bugs it found in host and UNC paths, env handling, shell quoting, post-task scripts, health checks, checkout, action symlinks, step summaries, Git client certificates and docker mode from a Windows host.

Host-mode steps now emit UTF-8, `bash` falls back to Git's bash instead of the WSL launcher, the runner can run as a native Windows service, steps can keep background helpers with `RUNNER_TRACKING_ID`, and Windows arm64 binaries are released.

Fixes: https://gitea.com/gitea/runner/issues/1339
Fixes: https://gitea.com/gitea/runner/issues/637
Fixes: https://gitea.com/gitea/runner/issues/1172
Fixes: https://gitea.com/gitea/runner/issues/347
Reviewed-on: https://gitea.com/gitea/runner/pulls/1348
Reviewed-by: bircni <[email protected]>
This commit is contained in:
silverwind committed 2026-11-03 01:26:10 +00:00
1 parent 6e42621b4a
commit d6f5b68a0d
85 files changed
+2578 -620

No files matched your search

+21
View File
@@ -0,0 +1,21 @@
name: windows
on:
schedule:
- cron: '0 3 * * 1'
workflow_dispatch:
env:
DOCKER_CONFIG: /tmp/docker-noauth # see test.yml
jobs:
test:
name: test on windows
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: ./.gitea/actions/go-setup
- name: prepare anonymous docker config
run: mkdir -p "$DOCKER_CONFIG" && echo '{}' > "$DOCKER_CONFIG/config.json"
- name: test on windows
run: make test-windows
-2
View File
@@ -34,8 +34,6 @@ builds:
goarch: s390x goarch: s390x
- goos: windows - goos: windows
goarch: arm goarch: arm
- goos: windows
goarch: arm64
- goos: freebsd - goos: freebsd
goarch: s390x goarch: s390x
- goos: freebsd - goos: freebsd
+9
View File
@@ -69,3 +69,12 @@ Docker and Kubernetes.
The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated
repository runners. A run path is `<workflow>@<ref>` and a log row is `<timestamp>Z <payload>`. repository runners. A run path is `<workflow>@<ref>` and a log row is `<timestamp>Z <payload>`.
## Windows tests
`make test-windows` cross-compiles the test binaries, installs Windows Server 2025 Core in a
[dockur/windows](https://github.com/dockur/windows) QEMU VM on the docker host and runs them there.
The host must be amd64 and allow privileged containers. `WINDOWS_TEST_TIMEOUT` sets the minutes
allowed for installing Windows and running the tests, 165 by default. The VM uses KVM when the host
has `/dev/kvm`, otherwise Windows is emulated and much slower. Docker-backed tests skip in the VM
and the race detector is off, it needs cgo. CI runs this weekly and on demand.
+5
View File
@@ -8,6 +8,7 @@ E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:657378e5cfd
SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker
E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly
E2E_K3S_IMAGE ?= rancher/k3s:v1.37.1-k3s1@sha256:ca7f37d993d82ef0dcdcfecb2e0e2618ea541dbaffc620c8cedebe01a82acd0d # renovate: datasource=docker E2E_K3S_IMAGE ?= rancher/k3s:v1.37.1-k3s1@sha256:ca7f37d993d82ef0dcdcfecb2e0e2618ea541dbaffc620c8cedebe01a82acd0d # renovate: datasource=docker
WINDOWS_TEST_IMAGE ?= dockurr/windows:6.06@sha256:16fd7c39a3e48bb946d29c59dc6015942e5b900a1fa5d7e5552053bfc48ee2ea # renovate: datasource=docker
GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*") GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*")
@@ -150,6 +151,10 @@ coverage-report: ## turn coverage.txt from `make test` into .tmp/coverage.md
test-dind: ## run the daemon-facing tests against the built dind image or podman (TARGET=dind|dind-rootless|podman) test-dind: ## run the daemon-facing tests against the built dind image or podman (TARGET=dind|dind-rootless|podman)
@PODMAN_TEST_IMAGE=$(PODMAN_TEST_IMAGE) ./scripts/test-dind.sh $(TARGET) @PODMAN_TEST_IMAGE=$(PODMAN_TEST_IMAGE) ./scripts/test-dind.sh $(TARGET)
.PHONY: test-windows
test-windows: ## run the tests on Windows Server Core in a QEMU VM on the docker host
@WINDOWS_TEST_IMAGE=$(WINDOWS_TEST_IMAGE) ./scripts/test-windows.sh
E2E_CONCURRENCY ?= 8 E2E_CONCURRENCY ?= 8
.PHONY: test-e2e .PHONY: test-e2e
+22 -1
View File
@@ -2,6 +2,8 @@
## Installation ## Installation
The runner runs on Linux, macOS and Windows.
### Prerequisites ### Prerequisites
On the runner: On the runner:
@@ -89,6 +91,23 @@ If the registry succeed, it will run immediately. Next time, you could run the r
./gitea-runner daemon ./gitea-runner daemon
``` ```
### Windows service
Register the runner first, keeping `gitea-runner.exe`, `config.yaml` and the `.runner` registration file in the same directory. From an elevated Command Prompt, install and start the service with absolute paths:
```bat
sc.exe create gitea-runner binPath= "\"C:\gitea-runner\gitea-runner.exe\" daemon --config \"C:\gitea-runner\config.yaml\"" start= auto
sc.exe start gitea-runner
```
The service runs from the executable's directory and appends its logs to `gitea-runner.log` there. Its account needs write access to that directory and access to the tools used by jobs. Use `sc.exe stop gitea-runner` to stop it, allowing running jobs to finish according to `runner.shutdown_timeout`.
On shutdown or reboot, Windows waits for the service's preshutdown timeout at most, so set it above `runner.shutdown_timeout`, here six minutes for `5m`. It takes effect after the next reboot:
```bat
reg.exe add HKLM\SYSTEM\CurrentControlSet\Services\gitea-runner /v PreshutdownTimeout /t REG_DWORD /d 360000 /f
```
### Run with docker ### Run with docker
The recommended Docker-in-Docker image runs its own Docker daemon for jobs. Jobs never get the host's Docker socket, which is equivalent to root access on the host. The recommended Docker-in-Docker image runs its own Docker daemon for jobs. Jobs never get the host's Docker socket, which is equivalent to root access on the host.
@@ -233,7 +252,9 @@ The `daemon` command also accepts `--labels` (which defaults to the `GITEA_RUNNE
Whenever the resulting labels differ from the ones in the registration file, they are written back to it and re-declared to the Gitea instance on startup. Whenever the resulting labels differ from the ones in the registration file, they are written back to it and re-declared to the Gitea instance on startup.
> **Note:** A runner that only exposes `host` labels still needs access to a Docker daemon (e.g. a mounted `/var/run/docker.sock`) whenever a job uses a `docker://` action or a service container. `host` labels only change where the job's own steps run; container-based steps and actions are still executed with Docker. > **Note:** A runner that only exposes `host` labels still needs access to a Docker daemon (e.g. a mounted `/var/run/docker.sock`) whenever a job uses a `docker://` action or a service container. `host` labels only change where the job's own steps run; container-based steps and actions are still executed with Docker. Windows hosts do not support container actions and `docker://` steps in host jobs, like GitHub's Windows runners.
On Windows hosts, a step's background processes end with the job unless it opts out, see [host-mode background processes](docs/host-mode.md).
#### Service containers #### Service containers
+1 -1
View File
@@ -281,7 +281,7 @@ func TestSafeResolve(t *testing.T) {
for name, tc := range tests { for name, tc := range tests {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
require.Equal(t, tc.want, safeResolve(baseDir, tc.input)) require.Equal(t, filepath.FromSlash(tc.want), safeResolve(baseDir, tc.input))
}) })
} }
} }
+163 -3
View File
@@ -10,6 +10,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"io" "io"
"math/rand/v2" "math/rand/v2"
"net" "net"
@@ -29,6 +30,10 @@ import (
var actionFetches sync.Map var actionFetches sync.Map
var symlink = (*os.Root).Symlink
var symlinkFallback = runtime.GOOS == "windows"
const treeSizes = ".sizes" // beside the tree rather than in it, which is copied into job containers as it is const treeSizes = ".sizes" // beside the tree rather than in it, which is copied into job containers as it is
type treeFile struct { type treeFile struct {
@@ -229,6 +234,8 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
sha := "" sha := ""
sizes := map[string]int64{} sizes := map[string]int64{}
links := map[string]string{}
var deferred []string
reader := tar.NewReader(archive) reader := tar.NewReader(archive)
for { for {
header, err := reader.Next() header, err := reader.Next()
@@ -257,9 +264,8 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
case tar.TypeReg: case tar.TypeReg:
sizes[name], err = writeTreeFile(root, name, header.Mode, reader) sizes[name], err = writeTreeFile(root, name, header.Mode, reader)
case tar.TypeSymlink: case tar.TypeSymlink:
if err = root.Symlink(header.Linkname, name); err != nil && runtime.GOOS == "windows" { links[filepath.Clean(name)] = header.Linkname
sizes[name], err = writeTreeFile(root, name, 0, strings.NewReader(header.Linkname)) deferred = append(deferred, filepath.Clean(name))
}
} }
if err != nil { if err != nil {
return "", err return "", err
@@ -268,12 +274,166 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
if !isFullObjectID(sha) { if !isFullObjectID(sha) {
return "", errors.New("the archive names no commit") return "", errors.New("the archive names no commit")
} }
linksByParent := map[string][]string{}
if symlinkFallback {
for name := range links {
linksByParent[filepath.Dir(name)] = append(linksByParent[filepath.Dir(name)], filepath.Base(name))
}
}
for requireTarget := symlinkFallback; len(deferred) > 0; {
unresolved := deferred[:0]
for _, name := range deferred {
if err := ctx.Err(); err != nil {
return "", err
}
if requireTarget {
if _, err := root.Stat(filepath.Dir(name) + string(filepath.Separator) + links[name]); err != nil {
unresolved = append(unresolved, name)
continue
}
}
if err := symlink(root, links[name], name); err == nil || !symlinkFallback {
if err != nil {
return "", err
}
continue
}
if err := copyTreeLink(ctx, root, name, links, linksByParent, sizes); err != nil {
return "", fmt.Errorf("materialize symlink %q: %w", filepath.ToSlash(name), err)
}
}
if len(unresolved) == len(deferred) {
requireTarget = false
}
deferred = unresolved
}
if err := root.Close(); err != nil { if err := root.Close(); err != nil {
return "", err return "", err
} }
return sha, installTree(ctx, dir, sha, temp, sizes) return sha, installTree(ctx, dir, sha, temp, sizes)
} }
func resolveTreeLink(name string, links map[string]string) (string, bool) {
if !filepath.IsLocal(name) {
return "", false
}
parts := strings.Split(name, string(filepath.Separator))
var resolved []string
for count := 0; len(parts) > 0; {
part := parts[0]
parts = parts[1:]
switch part {
case "", ".":
continue
case "..":
if len(resolved) == 0 {
return "", false
}
resolved = resolved[:len(resolved)-1]
continue
}
target, ok := links[filepath.Join(append(resolved, part)...)]
if !ok {
resolved = append(resolved, part)
continue
}
count++
if target == "" || os.IsPathSeparator(target[0]) || filepath.VolumeName(target) != "" || count > 255 {
return "", false
}
parts = append(strings.Split(filepath.FromSlash(target), string(filepath.Separator)), parts...)
}
return filepath.Join(append([]string{"."}, resolved...)...), true
}
var (
errTreeLinkTarget = errors.New("unmaterializable symlink target")
errTreeLinkCycle = fmt.Errorf("cyclic %w", errTreeLinkTarget)
)
func copyTreeLink(ctx context.Context, root *os.Root, name string, links map[string]string, linksByParent map[string][]string, sizes map[string]int64) error {
writeLinkText := func(destination, target string) error { // as git writes links it cannot create
if err := root.RemoveAll(destination); err != nil {
return err
}
for written := range sizes {
if written == destination || strings.HasPrefix(written, destination+string(filepath.Separator)) {
delete(sizes, written)
}
}
size, err := writeTreeFile(root, destination, 0o644, strings.NewReader(target))
sizes[destination] = size
return err
}
active := map[string]bool{}
var copyTarget func(string, string) error
copyTarget = func(source, destination string) error {
if err := ctx.Err(); err != nil {
return err
}
source, ok := resolveTreeLink(source, links)
if !ok {
return errTreeLinkTarget
}
file, err := root.Open(source)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return errTreeLinkTarget
}
return err
}
defer file.Close()
info, err := file.Stat()
if err != nil {
return err
}
if info.Mode().IsRegular() {
sizes[destination], err = writeTreeFile(root, destination, int64(info.Mode().Perm()), file)
return err
}
if !info.IsDir() {
return errTreeLinkTarget
}
if active[source] {
return errTreeLinkCycle
}
active[source] = true
defer delete(active, source)
entries, err := file.ReadDir(-1)
if err != nil {
return err
}
children := map[string]bool{}
for _, entry := range entries {
children[entry.Name()] = true
}
for _, child := range linksByParent[source] {
children[child] = true
}
if err := root.MkdirAll(destination, info.Mode().Perm()); err != nil {
return err
}
for child := range children {
childSource, childDestination := filepath.Join(source, child), filepath.Join(destination, child)
err := copyTarget(childSource, childDestination)
if target, isLink := links[childSource]; isLink && errors.Is(err, errTreeLinkTarget) && !errors.Is(err, errTreeLinkCycle) {
err = writeLinkText(childDestination, target)
}
if err != nil {
return err
}
}
return nil
}
if err := copyTarget(name, name); !errors.Is(err, errTreeLinkTarget) {
return err
}
if err := ctx.Err(); err != nil {
return err
}
return writeLinkText(name, links[name])
}
var lockTree = AcquireCloneLock // replaced in tests to see installTree wait for a job holding the tree var lockTree = AcquireCloneLock // replaced in tests to see installTree wait for a job holding the tree
// installTree moves temp into place as dir/sha, replacing a tree that failed treeIntact. // installTree moves temp into place as dir/sha, replacing a tree that failed treeIntact.
+151 -1
View File
@@ -8,6 +8,7 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"net/http" "net/http"
"net/http/cgi" "net/http/cgi"
@@ -40,6 +41,7 @@ func TestFetchAction(t *testing.T) {
require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755)) require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755))
require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link"))) require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link")))
require.NoError(t, gitCmd("-C", remote, "add", ".")) require.NoError(t, gitCmd("-C", remote, "add", "."))
require.NoError(t, gitCmd("-C", remote, "update-index", "--chmod=+x", "run.sh"))
require.NoError(t, gitCmd("-C", remote, "commit", "-m", "initial")) require.NoError(t, gitCmd("-C", remote, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", remote, "tag", "v1")) require.NoError(t, gitCmd("-C", remote, "tag", "v1"))
expectedSha := gitRevParse(t, remote, "main") expectedSha := gitRevParse(t, remote, "main")
@@ -192,7 +194,9 @@ func TestFetchAction(t *testing.T) {
assert.NoDirExists(t, filepath.Join(tree, ".git")) assert.NoDirExists(t, filepath.Join(tree, ".git"))
info, err := os.Stat(filepath.Join(tree, "run.sh")) info, err := os.Stat(filepath.Join(tree, "run.sh"))
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm()) if runtime.GOOS != "windows" {
assert.EqualValues(t, 0o755, info.Mode().Perm())
}
link, err := os.Readlink(filepath.Join(tree, "link")) link, err := os.Readlink(filepath.Join(tree, "link"))
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, "action.yml", link) assert.Equal(t, "action.yml", link)
@@ -391,6 +395,79 @@ func TestExtractTreeConfinesEntries(t *testing.T) {
} }
} }
func TestExtractTreeCopiesDeferredLinks(t *testing.T) {
originalSymlink, originalFallback := symlink, symlinkFallback
symlink = func(_root *os.Root, _target, _name string) error { return errors.New("symlink privilege unavailable") }
symlinkFallback = true
t.Cleanup(func() { symlink, symlinkFallback = originalSymlink, originalFallback })
for _, test := range []struct {
name string
links [][2]string
files []string
text bool
}{
{name: "file chain", links: [][2]string{{"index.js", "entry.js"}, {"entry.js", "dist/index.js"}}, files: []string{"index.js", "entry.js"}},
{name: "directory chain", links: [][2]string{{"lib", "bundle"}, {"bundle", "dist"}, {"dist/entry.js", "index.js"}}, files: []string{"lib/index.js", "lib/entry.js", "lib/nested/index.js", "bundle/index.js", "bundle/entry.js", "bundle/nested/index.js"}},
{name: "directory in target path", links: [][2]string{{"index.js", "bundle/nested/index.js"}, {"bundle", "dist"}}, files: []string{"index.js", "bundle/nested/index.js"}},
{name: "parent after directory link", links: [][2]string{{"index.js", "bundle/../index.js"}, {"bundle", "dist/nested"}}, files: []string{"index.js", "bundle/index.js"}},
{name: "outside", links: [][2]string{{"index.js", "../outside.js"}}, text: true},
{name: "absolute", links: [][2]string{{"index.js", t.TempDir()}}, text: true},
{name: "dangling", links: [][2]string{{"index.js", "missing.js"}}, text: true},
{name: "dangling child of directory link", links: [][2]string{{"lib", "dist"}, {"dist/optional", "missing.js"}}, files: []string{"lib/index.js", "lib/nested/index.js"}},
{name: "cycle", links: [][2]string{{"index.js", "entry.js"}, {"entry.js", "index.js"}}, text: true},
{name: "directory cycle", links: [][2]string{{"dist/loop", "."}}, text: true},
} {
t.Run(test.name, func(t *testing.T) {
sha := strings.Repeat("a", 40)
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
require.NoError(t, writer.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, PAXRecords: map[string]string{"comment": sha}}))
require.NoError(t, writer.WriteHeader(&tar.Header{Name: "dist", Typeflag: tar.TypeDir, Mode: 0o755}))
require.NoError(t, writer.WriteHeader(&tar.Header{Name: "dist/nested", Typeflag: tar.TypeDir, Mode: 0o755}))
for _, link := range test.links {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: link[0], Typeflag: tar.TypeSymlink, Linkname: link[1]}))
}
content := "console.log('action');\n"
for _, name := range []string{"dist/index.js", "dist/nested/index.js"} {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: name, Typeflag: tar.TypeReg, Mode: 0o755, Size: int64(len(content))}))
_, err := writer.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, writer.Close())
dir := t.TempDir()
_, err := extractTree(t.Context(), &archive, dir, false)
require.NoError(t, err)
assert.True(t, treeIntact(dir, sha))
if test.text {
for _, link := range test.links {
data, err := os.ReadFile(filepath.Join(dir, sha, link[0]))
require.NoError(t, err)
assert.Equal(t, link[1], string(data))
info, err := os.Lstat(filepath.Join(dir, sha, link[0]))
require.NoError(t, err)
assert.True(t, info.Mode().IsRegular())
}
}
for _, name := range test.files {
data, err := os.ReadFile(filepath.Join(dir, sha, name))
require.NoError(t, err)
assert.Equal(t, content, string(data))
info, err := os.Lstat(filepath.Join(dir, sha, name))
require.NoError(t, err)
assert.True(t, info.Mode().IsRegular())
if runtime.GOOS != "windows" {
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm())
}
require.NoError(t, os.WriteFile(filepath.Join(dir, sha, name), nil, 0o644))
assert.False(t, treeIntact(dir, sha), name)
require.NoError(t, os.WriteFile(filepath.Join(dir, sha, name), []byte(content), 0o755))
}
})
}
}
func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) { func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) {
if runtime.GOOS != "linux" { if runtime.GOOS != "linux" {
t.Skip("macOS and Windows store no name that is not UTF-8") t.Skip("macOS and Windows store no name that is not UTF-8")
@@ -410,3 +487,76 @@ func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) {
require.FileExists(t, filepath.Join(dir, sha, "caf\xe9.js")) require.FileExists(t, filepath.Join(dir, sha, "caf\xe9.js"))
assert.True(t, treeIntact(dir, sha)) assert.True(t, treeIntact(dir, sha))
} }
func TestExtractTreeCreatesLinksInPlatformOrder(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("requires symlink privileges")
}
original, originalFallback := symlink, symlinkFallback
var created []string
symlink = func(root *os.Root, target, name string) error {
created = append(created, name)
return root.Symlink(target, name)
}
t.Cleanup(func() { symlink, symlinkFallback = original, originalFallback })
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
require.NoError(t, writer.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, PAXRecords: map[string]string{"comment": strings.Repeat("a", 40)}}))
for _, entry := range []*tar.Header{
{Name: "a", Typeflag: tar.TypeSymlink, Linkname: "b"},
{Name: "b", Typeflag: tar.TypeSymlink, Linkname: "dist"},
{Name: "dangling", Typeflag: tar.TypeSymlink, Linkname: "missing"},
{Name: "dist", Typeflag: tar.TypeDir, Mode: 0o755},
} {
require.NoError(t, writer.WriteHeader(entry))
}
require.NoError(t, writer.Close())
for _, test := range []struct {
name string
fallback bool
want []string
}{
{name: "native", want: []string{"a", "b", "dangling"}},
{name: "fallback", fallback: true, want: []string{"b", "a", "dangling"}},
} {
t.Run(test.name, func(t *testing.T) {
symlinkFallback = test.fallback
created = nil
_, err := extractTree(t.Context(), bytes.NewReader(archive.Bytes()), t.TempDir(), false)
require.NoError(t, err)
assert.Equal(t, test.want, created)
})
}
}
func TestCopyTreeLinkCancellationBetweenEntries(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
require.NoError(t, err)
defer root.Close()
require.NoError(t, root.Mkdir("source", 0o755))
for _, name := range []string{"first", "second"} {
_, err := writeTreeFile(root, filepath.Join("source", name), 0o644, strings.NewReader("content"))
require.NoError(t, err)
}
sizes := map[string]int64{}
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
observed := &treeCopyContext{Context: ctx, check: func() {
if len(sizes) == 1 {
cancel()
}
}}
require.ErrorIs(t, copyTreeLink(observed, root, "copy", map[string]string{"copy": "source"}, nil, sizes), context.Canceled)
assert.Len(t, sizes, 1)
}
type treeCopyContext struct {
context.Context
check func()
}
func (ctx *treeCopyContext) Err() error {
ctx.check()
return ctx.Context.Err()
}
+32 -8
View File
@@ -5,6 +5,7 @@
package git package git
import ( import (
"bytes"
"context" "context"
"crypto/sha256" "crypto/sha256"
"encoding/base64" "encoding/base64"
@@ -27,6 +28,7 @@ import (
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/internal/pkg/lock" "gitea.com/gitea/runner/internal/pkg/lock"
"gitea.com/gitea/runner/internal/pkg/process"
) )
var ( var (
@@ -387,6 +389,17 @@ func RemoteConfig(token string, insecureSkipTLS bool) []string {
return config return config
} }
func ClientCertConfig(prefix, certFile, keyFile, executionOS string) []string {
if certFile == "" {
return nil
}
config := []string{prefix + "sslCert=" + certFile, prefix + "sslKey=" + keyFile}
if strings.EqualFold(executionOS, "windows") {
config = append(config, prefix+"sslBackend=openssl")
}
return config
}
// ConfigEnv applies `-c` style entries through the environment, keeping secrets out of argv. // ConfigEnv applies `-c` style entries through the environment, keeping secrets out of argv.
func ConfigEnv(config ...string) map[string]string { func ConfigEnv(config ...string) map[string]string {
env := map[string]string{"GIT_CONFIG_COUNT": strconv.Itoa(len(config))} env := map[string]string{"GIT_CONFIG_COUNT": strconv.Itoa(len(config))}
@@ -444,9 +457,7 @@ func runGitWithStdin(ctx context.Context, dir string, input *NewGitCloneExecutor
config := []string{"core.autocrlf=false", "core.eol=lf"} // keep action files LF config := []string{"core.autocrlf=false", "core.eol=lf"} // keep action files LF
if input != nil { if input != nil {
config = append(config, RemoteConfig(input.Token, input.InsecureSkipTLS)...) config = append(config, RemoteConfig(input.Token, input.InsecureSkipTLS)...)
if input.ClientCertFile != "" { config = append(config, ClientCertConfig("http.", input.ClientCertFile, input.ClientKeyFile, runtime.GOOS)...)
config = append(config, "http.sslCert="+input.ClientCertFile, "http.sslKey="+input.ClientKeyFile)
}
} }
if dir != "" { if dir != "" {
args = append([]string{"-C", dir}, args...) args = append([]string{"-C", dir}, args...)
@@ -461,15 +472,28 @@ func runGitWithStdin(ctx context.Context, dir string, input *NewGitCloneExecutor
cmd.Env = append(cmd.Env, key+"="+value) cmd.Env = append(cmd.Env, key+"="+value)
} }
cmd.Env = append(cmd.Env, "GIT_TERMINAL_PROMPT=0") cmd.Env = append(cmd.Env, "GIT_TERMINAL_PROMPT=0")
if runtime.GOOS != "windows" { var treeKill *process.TreeKill
if runtime.GOOS == "windows" {
treeKill = process.NewTreeKill(cmd) // a cancelled git leaves its remote helper holding the repository otherwise
} else {
cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) } // lets git remove its lock files cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) } // lets git remove its lock files
} }
cmd.WaitDelay = time.Second cmd.WaitDelay = time.Second
var stdout bytes.Buffer
var stderr strings.Builder var stderr strings.Builder
cmd.Stdout = &stdout
cmd.Stderr = &stderr cmd.Stderr = &stderr
output, err := cmd.Output() err := cmd.Start()
if err != nil { if err == nil {
return output, fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(stderr.String())) if treeKill != nil {
if killer, captureErr := treeKill.Capture(cmd.Process); captureErr == nil {
defer killer.Close()
}
}
err = cmd.Wait()
} }
return output, nil if err != nil {
return stdout.Bytes(), fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(stderr.String()))
}
return stdout.Bytes(), nil
} }
+21 -4
View File
@@ -54,6 +54,22 @@ func TestMain(m *testing.M) {
os.Exit(code) os.Exit(code)
} }
func TestClientCertConfig(t *testing.T) {
for _, testcase := range []struct {
prefix, certFile, executionOS string
expected []string
}{
{"http.", "cert.pem", "windows", []string{"http.sslCert=cert.pem", "http.sslKey=key.pem", "http.sslBackend=openssl"}},
{"http.https://example.com/.", "cert.pem", "Windows", []string{"http.https://example.com/.sslCert=cert.pem", "http.https://example.com/.sslKey=key.pem", "http.https://example.com/.sslBackend=openssl"}},
{"http.", "cert.pem", "Linux", []string{"http.sslCert=cert.pem", "http.sslKey=key.pem"}},
{"http.", "", "Windows", nil},
} {
t.Run(testcase.prefix+testcase.certFile+testcase.executionOS, func(t *testing.T) {
assert.Equal(t, testcase.expected, ClientCertConfig(testcase.prefix, testcase.certFile, "key.pem", testcase.executionOS))
})
}
}
func TestFindGitSlug(t *testing.T) { func TestFindGitSlug(t *testing.T) {
for _, testcase := range []struct { for _, testcase := range []struct {
remoteURL string remoteURL string
@@ -185,7 +201,8 @@ func TestGitFindRef(t *testing.T) {
func TestGitCloneExecutor(t *testing.T) { func TestGitCloneExecutor(t *testing.T) {
remoteDir, workDir := newGitRemote(t) remoteDir, workDir := newGitRemote(t)
require.NoError(t, os.WriteFile(filepath.Join(workDir, "action.sh"), []byte("#!/bin/sh\necho hi\n"), 0o755)) require.NoError(t, os.WriteFile(filepath.Join(workDir, "action.sh"), []byte("#!/bin/sh\necho hi\n"), 0o755))
require.NoError(t, gitCmd("-C", workDir, "add", "action.sh")) require.NoError(t, os.WriteFile(filepath.Join(workDir, ".gitattributes"), []byte("*.sh text\n"), 0o644))
require.NoError(t, gitCmd("-C", workDir, "add", "action.sh", ".gitattributes"))
require.NoError(t, gitCmd("-C", workDir, "commit", "-m", "initial")) require.NoError(t, gitCmd("-C", workDir, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", workDir, "tag", "v2")) require.NoError(t, gitCmd("-C", workDir, "tag", "v2"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main")) require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
@@ -200,7 +217,7 @@ func TestGitCloneExecutor(t *testing.T) {
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:refs/heads/"+fullSha[:4])) require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:refs/heads/"+fullSha[:4]))
hostConfig := filepath.Join(t.TempDir(), "gitconfig") hostConfig := filepath.Join(t.TempDir(), "gitconfig")
require.NoError(t, os.WriteFile(hostConfig, []byte("[core]\n\tautocrlf = true\n[url \""+remoteDir+"\"]\n\tinsteadOf = https://example.invalid/action\n"), 0o644)) require.NoError(t, os.WriteFile(hostConfig, []byte("[core]\n\tautocrlf = true\n\teol = crlf\n[url "+strconv.Quote(remoteDir)+"]\n\tinsteadOf = https://example.invalid/action\n"), 0o644))
t.Setenv("GIT_CONFIG_GLOBAL", hostConfig) t.Setenv("GIT_CONFIG_GLOBAL", hostConfig)
t.Setenv("GIT_DIR", t.TempDir()) t.Setenv("GIT_DIR", t.TempDir())
@@ -676,8 +693,8 @@ func TestNewGitCloneExecutorFetchSendsTokenAndHonoursContext(t *testing.T) {
select { select {
case <-reached: case <-reached:
case <-time.After(10 * time.Second): case err := <-done:
t.Fatal("the executor never reached the remote") t.Fatalf("the executor finished before reaching the remote: %v", err)
} }
assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), authorization) assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), authorization)
cancel() cancel()
+20
View File
@@ -0,0 +1,20 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package common
import "strings"
// LookupFold finds name in values, ignoring case where the environment does, as GitHub's inputs always do.
func LookupFold[V any](values map[string]V, name string, caseInsensitive bool) (V, bool) {
if value, ok := values[name]; ok || !caseInsensitive {
return value, ok
}
for key, value := range values {
if strings.EqualFold(key, name) {
return value, true
}
}
var zero V
return zero, false
}
+3
View File
@@ -10,6 +10,7 @@ import (
"fmt" "fmt"
"io" "io"
"path" "path"
"runtime"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -20,6 +21,8 @@ import (
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
) )
var hostOS = runtime.GOOS // replaced in tests
// ExitCodeError reports a non-zero process exit code from a container command. // ExitCodeError reports a non-zero process exit code from a container command.
type ExitCodeError int type ExitCodeError int
+6 -3
View File
@@ -125,6 +125,10 @@ func createBuildContext(ctx context.Context, contextDir, relDockerfile string) (
} }
func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, includes []string, pm *patternmatcher.PatternMatcher) error { func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, includes []string, pm *patternmatcher.PatternMatcher) error {
contextDir, err := filepath.EvalSymlinks(contextDir)
if err != nil {
return err
}
tw := tar.NewWriter(w) tw := tar.NewWriter(w)
seen := map[string]bool{} seen := map[string]bool{}
hardlinks := map[uint64]string{} hardlinks := map[uint64]string{}
@@ -132,8 +136,7 @@ func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, incl
for _, include := range includes { for _, include := range includes {
include = filepath.FromSlash(include) include = filepath.FromSlash(include)
var parents []parentMatch var parents []parentMatch
root := strings.TrimSuffix(contextDir, string(filepath.Separator)) + string(filepath.Separator) + include // unlike filepath.Join, follows a symlinked context dir as docker build does err := filepath.WalkDir(filepath.Join(contextDir, include), func(file string, entry fs.DirEntry, err error) error {
err := filepath.WalkDir(root, func(file string, entry fs.DirEntry, err error) error {
if err != nil { if err != nil {
return leaveOut(ctx, file, err) return leaveOut(ctx, file, err)
} }
@@ -210,7 +213,7 @@ func addBuildContextEntry(ctx context.Context, tw *tar.Writer, file, rel string,
return leaveOut(ctx, file, err) return leaveOut(ctx, file, err)
} }
} }
header, err := tar.FileInfoHeader(noNameLookups{info}, link) header, err := tar.FileInfoHeader(noNameLookups{info}, filepath.ToSlash(link))
if err != nil { if err != nil {
return err return err
} }
+1 -1
View File
@@ -50,7 +50,7 @@ func TestCreateBuildContextFollowsDockerignoreAndSymlinkedRootButKeepsDockerfile
contexts := []string{dir} contexts := []string{dir}
if link := filepath.Join(t.TempDir(), "link"); os.Symlink(dir, link) == nil { if link := filepath.Join(t.TempDir(), "link"); os.Symlink(dir, link) == nil {
contexts = append(contexts, link+string(filepath.Separator)) contexts = append(contexts, link, link+string(filepath.Separator))
} }
for _, contextDir := range contexts { for _, contextDir := range contexts {
+8 -4
View File
@@ -351,7 +351,7 @@ type containerConfig struct {
// parse parses the args for the specified command and generates a Config, // parse parses the args for the specified command and generates a Config,
// a HostConfig and returns them with the specified command. // a HostConfig and returns them with the specified command.
// If the specified args are not valid, it will return an error. // If the specified args are not valid, it will return an error.
func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*containerConfig, error) { //nolint:gocyclo,unparam // verbatim copy from docker/cli func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*containerConfig, error) { //nolint:gocyclo // verbatim copy from docker/cli
var ( var (
attachStdin = copts.attach.Get("stdin") attachStdin = copts.attach.Get("stdin")
attachStdout = copts.attach.Get("stdout") attachStdout = copts.attach.Get("stdout")
@@ -724,6 +724,10 @@ func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*con
}, nil }, nil
} }
func isUserDefinedNetwork(mode container.NetworkMode) bool { // Docker's helper uses the host OS instead of the daemon's.
return !mode.IsDefault() && mode != network.NetworkBridge && mode != network.NetworkHost && !mode.IsNone() && !mode.IsContainer()
}
// parseNetworkOpts converts --network advanced options to endpoint-specs, and combines // parseNetworkOpts converts --network advanced options to endpoint-specs, and combines
// them with the old --network-alias and --links. If returns an error if conflicting options // them with the old --network-alias and --links. If returns an error if conflicting options
// are found. // are found.
@@ -752,7 +756,7 @@ func parseNetworkOpts(copts *containerOptions) (map[string]*network.EndpointSett
} }
for i, n := range copts.netMode.Value() { for i, n := range copts.netMode.Value() {
if container.NetworkMode(n.Target).IsUserDefined() { if isUserDefinedNetwork(container.NetworkMode(n.Target)) {
hasUserDefined = true hasUserDefined = true
} else { } else {
hasNonUserDefined = true hasNonUserDefined = true
@@ -814,7 +818,7 @@ func applyContainerOptions(n *opts.NetworkAttachmentOpts, copts *containerOption
} }
// For a user-defined network, "--link" is an endpoint option, it creates an alias. But, // For a user-defined network, "--link" is an endpoint option, it creates an alias. But,
// for the default bridge it defines a legacy-link. // for the default bridge it defines a legacy-link.
if container.NetworkMode(n.Target).IsUserDefined() && copts.links.Len() > 0 { if isUserDefinedNetwork(container.NetworkMode(n.Target)) && copts.links.Len() > 0 {
n.Links = slices.Clone(copts.links.GetSlice()) n.Links = slices.Clone(copts.links.GetSlice())
} }
if copts.ipv4Address != nil { if copts.ipv4Address != nil {
@@ -840,7 +844,7 @@ func parseNetworkAttachmentOpt(ep opts.NetworkAttachmentOpts) (*network.Endpoint
if strings.TrimSpace(ep.Target) == "" { if strings.TrimSpace(ep.Target) == "" {
return nil, errors.New("no name set for network") return nil, errors.New("no name set for network")
} }
if !container.NetworkMode(ep.Target).IsUserDefined() { if !isUserDefinedNetwork(container.NetworkMode(ep.Target)) {
if len(ep.Aliases) > 0 { if len(ep.Aliases) > 0 {
return nil, errors.New("network-scoped aliases are only supported for user-defined networks") return nil, errors.New("network-scoped aliases are only supported for user-defined networks")
} }
+7
View File
@@ -484,6 +484,10 @@ type probeClient struct {
containers []container.Summary containers []container.Summary
} }
func (c *probeClient) Info(_ctx context.Context, _opts mobyclient.InfoOptions) (mobyclient.SystemInfoResult, error) {
return mobyclient.SystemInfoResult{}, nil
}
func (c *probeClient) Close() error { func (c *probeClient) Close() error {
return nil return nil
} }
@@ -766,6 +770,9 @@ func TestDaemonSeesDir(t *testing.T) {
} }
func TestRunnerContainerWorkdir(t *testing.T) { func TestRunnerContainerWorkdir(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("a runner container is Linux")
}
workdir := t.TempDir() workdir := t.TempDir()
t.Chdir(workdir) t.Chdir(workdir)
volume := container.MountPoint{Type: mount.TypeVolume, Source: "/daemon/_data", Destination: filepath.Dir(workdir)} volume := container.MountPoint{Type: mount.TypeVolume, Source: "/daemon/_data", Destination: filepath.Dir(workdir)}
+38 -15
View File
@@ -366,6 +366,7 @@ type containerReference struct {
id string id string
input *NewContainerInput input *NewContainerInput
pullPolicy string pullPolicy string
daemonOS string
UID int UID int
GID int GID int
// attachDone is closed by the attach() streaming goroutine once it has // attachDone is closed by the attach() streaming goroutine once it has
@@ -375,13 +376,18 @@ type containerReference struct {
LinuxContainerEnvironmentExtensions LinuxContainerEnvironmentExtensions
} }
func GetDockerClient(ctx context.Context) (cli client.APIClient, err error) { func GetDockerClient(ctx context.Context) (client.APIClient, error) {
cli, _, err := newDockerClient(ctx)
return cli, err
}
func newDockerClient(ctx context.Context) (cli client.APIClient, daemonOS string, err error) {
dockerHost := os.Getenv("DOCKER_HOST") dockerHost := os.Getenv("DOCKER_HOST")
if strings.HasPrefix(dockerHost, "ssh://") { if strings.HasPrefix(dockerHost, "ssh://") {
var args []string var args []string
if args, err = sshArgs(dockerHost); err != nil { if args, err = sshArgs(dockerHost); err != nil {
return nil, err return nil, "", err
} }
cli, err = client.New( cli, err = client.New(
client.WithHost("http://docker.example.com"), // the dialer below ignores the address client.WithHost("http://docker.example.com"), // the dialer below ignores the address
@@ -391,16 +397,17 @@ func GetDockerClient(ctx context.Context) (cli client.APIClient, err error) {
cli, err = client.New(client.FromEnv) cli, err = client.New(client.FromEnv)
} }
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to connect to docker daemon: %w", err) return nil, "", fmt.Errorf("failed to connect to docker daemon: %w", err)
} }
// Best-effort API version negotiation, matching the old client.NegotiateAPIVersion // Best-effort API version negotiation, matching the old client.NegotiateAPIVersion
// behaviour. Ping failures here are non-fatal: the connection is exercised again on // behaviour. Ping failures here are non-fatal: the connection is exercised again on
// the first real call, and the client falls back to the daemon's default API version. // the first real call, and the client falls back to the daemon's default API version.
if _, err := cli.Ping(ctx, client.PingOptions{NegotiateAPIVersion: true}); err != nil { ping, err := cli.Ping(ctx, client.PingOptions{NegotiateAPIVersion: true})
if err != nil {
common.Logger(ctx).Warnf("docker daemon ping during version negotiation failed, continuing: %v", err) common.Logger(ctx).Warnf("docker daemon ping during version negotiation failed, continuing: %v", err)
} }
return cli, nil return cli, ping.OSType, nil
} }
func GetHostInfo(ctx context.Context) (system.Info, error) { func GetHostInfo(ctx context.Context) (system.Info, error) {
@@ -440,16 +447,32 @@ func RunnerArch(ctx context.Context) string {
return info.Architecture return info.Architecture
} }
func (cr *containerReference) targetOS(ctx context.Context) string {
if cr.daemonOS == "" {
cr.daemonOS = "linux"
if cr.cli != nil {
if result, err := cr.cli.Info(ctx, client.InfoOptions{}); err == nil && result.Info.OSType != "" {
cr.daemonOS = result.Info.OSType
}
}
}
return cr.daemonOS
}
func (cr *containerReference) markExecutable(ctx context.Context) bool {
return hostOS == "windows" && cr.targetOS(ctx) == "linux"
}
func (cr *containerReference) connect() common.Executor { func (cr *containerReference) connect() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
if cr.cli != nil { if cr.cli != nil {
return nil return nil
} }
cli, err := GetDockerClient(ctx) cli, daemonOS, err := newDockerClient(ctx)
if err != nil { if err != nil {
return err return err
} }
cr.cli = cli cr.cli, cr.daemonOS = cli, daemonOS
return nil return nil
} }
} }
@@ -459,6 +482,7 @@ func (cr *containerReference) Close() common.Executor {
if cr.cli != nil { if cr.cli != nil {
err := cr.cli.Close() err := cr.cli.Close()
cr.cli = nil cr.cli = nil
cr.daemonOS = ""
if err != nil { if err != nil {
return fmt.Errorf("failed to close client: %w", err) return fmt.Errorf("failed to close client: %w", err)
} }
@@ -603,7 +627,8 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
copts.privileged = false copts.privileged = false
} }
containerConfig, err := parse(flags, copts, runtime.GOOS) serverOS := cr.targetOS(ctx)
containerConfig, err := parse(flags, copts, serverOS)
if err != nil { if err != nil {
return fmt.Errorf("cannot process container options: '%s': '%w'", options, err) return fmt.Errorf("cannot process container options: '%s': '%w'", options, err)
} }
@@ -618,7 +643,7 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
// For Gitea, forcing --privileged off is not enough, other options reach the host too // For Gitea, forcing --privileged off is not enough, other options reach the host too
if !hostConfig.Privileged { if !hostConfig.Privileged {
trusted, err := parseOptionsHostConfig(cr.input.RunnerOptions) trusted, err := parseOptionsHostConfig(cr.input.RunnerOptions, serverOS)
if err != nil { if err != nil {
return err return err
} }
@@ -731,9 +756,7 @@ func (cr *containerReference) create(capAdd, capDrop []string) common.Executor {
cr.sanitizeConfig(ctx, hostConfig) cr.sanitizeConfig(ctx, hostConfig)
var networkingConfig *network.NetworkingConfig var networkingConfig *network.NetworkingConfig
n := hostConfig.NetworkMode if isUserDefinedNetwork(hostConfig.NetworkMode) && len(input.NetworkAliases) > 0 {
// IsUserDefined and IsHost are broken on windows
if n.IsUserDefined() && n != "host" && len(input.NetworkAliases) > 0 {
networkingConfig = &network.NetworkingConfig{ networkingConfig = &network.NetworkingConfig{
EndpointsConfig: map[string]*network.EndpointSettings{ EndpointsConfig: map[string]*network.EndpointSettings{
input.NetworkMode: {Aliases: input.NetworkAliases}, input.NetworkMode: {Aliases: input.NetworkAliases},
@@ -919,7 +942,7 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, ski
if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") { if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") {
extractionRoot = actPath // keeps tar entries from crossing the image's /var/run symlink extractionRoot = actPath // keeps tar entries from crossing the image's /var/run symlink
} }
if err := writeDirTar(ctx, tarFile, path.Join("/", strings.TrimPrefix(dstPath, extractionRoot)), srcPath, useGitIgnore, skipGitDir, cr.UID, cr.GID); err != nil { if err := writeDirTar(ctx, tarFile, path.Join("/", strings.TrimPrefix(dstPath, extractionRoot)), srcPath, useGitIgnore, skipGitDir, cr.UID, cr.GID, cr.markExecutable(ctx)); err != nil {
return err return err
} }
@@ -1090,12 +1113,12 @@ func resetOption[T any](logger *logging.Logger, option string, field *T, trusted
// parseOptionsHostConfig parses one options string on its own, to see what it alone asks for. // parseOptionsHostConfig parses one options string on its own, to see what it alone asks for.
// Even "" goes through the parser, or its empty slices and maps would differ from a real parse. // Even "" goes through the parser, or its empty slices and maps would differ from a real parse.
func parseOptionsHostConfig(options string) (*container.HostConfig, error) { func parseOptionsHostConfig(options, serverOS string) (*container.HostConfig, error) {
flags, copts, _, err := parseContainerOptions(options) flags, copts, _, err := parseContainerOptions(options)
if err != nil { if err != nil {
return nil, err return nil, err
} }
containerConfig, err := parse(flags, copts, runtime.GOOS) containerConfig, err := parse(flags, copts, serverOS)
if err != nil { if err != nil {
return nil, fmt.Errorf("cannot process container options: '%s': '%w'", options, err) return nil, fmt.Errorf("cannot process container options: '%s': '%w'", options, err)
} }
+135 -2
View File
@@ -27,10 +27,12 @@ import (
"gitea.com/gitea/runner/act/common/logging/logtest" "gitea.com/gitea/runner/act/common/logging/logtest"
cerrdefs "github.com/containerd/errdefs" cerrdefs "github.com/containerd/errdefs"
"github.com/kballard/go-shellquote"
"github.com/moby/moby/api/pkg/stdcopy" "github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount" "github.com/moby/moby/api/types/mount"
"github.com/moby/moby/api/types/network" "github.com/moby/moby/api/types/network"
"github.com/moby/moby/api/types/system"
mobyclient "github.com/moby/moby/client" mobyclient "github.com/moby/moby/client"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
@@ -65,6 +67,14 @@ func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
type mockDockerClient struct { type mockDockerClient struct {
mobyclient.APIClient mobyclient.APIClient
mock.Mock mock.Mock
info system.Info
infoErr error
infoCalls int
}
func (m *mockDockerClient) Info(_ctx context.Context, _opts mobyclient.InfoOptions) (mobyclient.SystemInfoResult, error) {
m.infoCalls++
return mobyclient.SystemInfoResult{Info: m.info}, m.infoErr
} }
func (m *mockDockerClient) ServerVersion(ctx context.Context, opts mobyclient.ServerVersionOptions) (mobyclient.ServerVersionResult, error) { func (m *mockDockerClient) ServerVersion(ctx context.Context, opts mobyclient.ServerVersionOptions) (mobyclient.ServerVersionResult, error) {
@@ -689,11 +699,11 @@ func TestMergeContainerConfigsDropsVolumeDriversFromWorkflows(t *testing.T) {
} }
func TestMergeContainerConfigsKeepsTheRunnersFilesAndEnvToItself(t *testing.T) { func TestMergeContainerConfigsKeepsTheRunnersFilesAndEnvToItself(t *testing.T) {
hostFile := filepath.Join(t.TempDir(), "host.env") hostFile := filepath.Join(t.TempDir(), "host file.env")
require.NoError(t, os.WriteFile(hostFile, []byte("STOLEN=from-the-host\n"), 0o600)) require.NoError(t, os.WriteFile(hostFile, []byte("STOLEN=from-the-host\n"), 0o600))
t.Setenv("RUNNER_SECRET", "s3cr3t") t.Setenv("RUNNER_SECRET", "s3cr3t")
for _, option := range []string{"--env-file " + hostFile, "--label-file " + hostFile} { for _, option := range []string{shellquote.Join("--env-file", hostFile), shellquote.Join("--label-file", hostFile)} {
logger := logging.New(logtest.New()) logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{NetworkMode: "bridge", WorkflowOptions: option}} cr := &containerReference{input: &NewContainerInput{NetworkMode: "bridge", WorkflowOptions: option}}
@@ -933,6 +943,7 @@ func TestContainerInfoFromInspect(t *testing.T) {
"/etc/hosts": "", "/etc/hosts": "",
"/etc/hostname": "", "/etc/hostname": "",
}, info.Mounts) }, info.Mounts)
assert.Equal(t, "/var/lib/docker/volumes/job/_data/subdir", info.DaemonPath("/workspace/owner/repo/subdir"))
}) })
} }
@@ -998,6 +1009,39 @@ func TestDockerCreateKeepsInputEntrypoint(t *testing.T) {
} }
} }
func TestDockerCreateNetworkAliases(t *testing.T) {
for _, test := range []struct {
networkMode string
wantAliases bool
}{
{"job-network", true},
{"host", false},
{"bridge", false},
{"default", false},
{"none", false},
{"container:x", false},
} {
t.Run(test.networkMode, func(t *testing.T) {
cr := &containerReference{
input: &NewContainerInput{NetworkMode: test.networkMode, NetworkAliases: []string{"service"}},
cli: &probeClient{create: func(_ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
assert.Equal(t, container.NetworkMode(test.networkMode), opts.HostConfig.NetworkMode)
if test.wantAliases {
assert.Equal(t, &network.NetworkingConfig{EndpointsConfig: map[string]*network.EndpointSettings{
"job-network": {Aliases: []string{"service"}},
}}, opts.NetworkingConfig)
} else {
assert.Nil(t, opts.NetworkingConfig)
}
return mobyclient.ContainerCreateResult{ID: "created"}, nil
}},
}
require.NoError(t, cr.create(nil, nil)(t.Context()))
assert.Equal(t, "created", cr.id)
})
}
}
func TestMergeContainerConfigsKeepsNetworkAliasesFromOptions(t *testing.T) { func TestMergeContainerConfigsKeepsNetworkAliasesFromOptions(t *testing.T) {
logger := logging.New(logtest.New()) logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{ cr := &containerReference{input: &NewContainerInput{
@@ -1019,3 +1063,92 @@ func TestSupportsContainerImagePlatformDaemonError(t *testing.T) {
require.ErrorContains(t, err, "cannot connect to the Docker daemon") require.ErrorContains(t, err, "cannot connect to the Docker daemon")
assert.False(t, supported) assert.False(t, supported)
} }
func TestContainerOptionsUseDaemonOS(t *testing.T) {
for _, testcase := range []struct {
name string
serverOS string
infoErr error
device string
expected container.DeviceMapping
}{
{name: "Linux daemon", serverOS: "linux", device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "unknown daemon", device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "unavailable daemon info", infoErr: errors.New("unavailable"), device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "Windows daemon", serverOS: "windows", device: "class/01234567-89ab-cdef-0123-456789abcdef", expected: container.DeviceMapping{PathOnHost: "class/01234567-89ab-cdef-0123-456789abcdef"}},
} {
t.Run(testcase.name, func(t *testing.T) {
cli := &mockDockerClient{info: system.Info{OSType: testcase.serverOS}, infoErr: testcase.infoErr}
cr := &containerReference{cli: cli, input: &NewContainerInput{RunnerOptions: "--device=" + testcase.device, WorkflowOptions: "--device=" + testcase.device, NetworkMode: "bridge"}}
for range 2 {
hostConfig := &container.HostConfig{}
require.NoError(t, cr.mergeContainerConfigs(t.Context(), &container.Config{}, hostConfig))
assert.Equal(t, []container.DeviceMapping{testcase.expected}, hostConfig.Devices)
}
assert.Equal(t, 1, cli.infoCalls)
})
}
}
func TestConnectTakesDaemonOSFromPing(t *testing.T) {
var infos atomic.Int32
daemon := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
writer.Header().Set("API-Version", "1.47")
writer.Header().Set("OSType", "windows")
if strings.HasSuffix(request.URL.Path, "/info") {
infos.Add(1)
}
}))
t.Cleanup(daemon.Close)
t.Setenv("DOCKER_HOST", daemon.URL)
cr := &containerReference{input: &NewContainerInput{}}
require.NoError(t, cr.connect()(t.Context()))
assert.Equal(t, "windows", cr.targetOS(t.Context()))
assert.Zero(t, infos.Load())
}
func TestContainerCopyModes(t *testing.T) {
for _, testcase := range []struct {
hostOS, serverOS string
expected int64
}{
{"windows", "linux", 0o755},
{"windows", "windows", 0o666},
{"linux", "linux", 0o666},
} {
t.Run(testcase.hostOS+"-"+testcase.serverOS, func(t *testing.T) {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "run.sh"), []byte("#!/bin/sh\n"), 0o666))
require.NoError(t, os.Chmod(filepath.Join(source, "run.sh"), 0o666))
file := &FileEntry{Name: "run.sh", Mode: 0o600, Body: "#!/bin/sh\n"}
original := hostOS
hostOS = testcase.hostOS
t.Cleanup(func() { hostOS = original })
cli := &mockDockerClient{info: system.Info{OSType: testcase.serverOS}}
var modes []int64
cli.On("CopyToContainer", mock.Anything, "job", mock.Anything).
Run(func(args mock.Arguments) {
reader := tar.NewReader(args.Get(2).(mobyclient.CopyToContainerOptions).Content)
header, err := reader.Next()
require.NoError(t, err)
modes = append(modes, header.Mode)
assert.Equal(t, "run.sh", filepath.Base(header.Name))
content, err := io.ReadAll(reader)
require.NoError(t, err)
assert.Equal(t, file.Body, string(content))
_, err = reader.Next()
require.ErrorIs(t, err, io.EOF)
}).Return(mobyclient.CopyToContainerResult{}, nil).Twice()
cr := &containerReference{cli: cli, id: "job", input: &NewContainerInput{}}
require.NoError(t, cr.copyContent("/workspace", file)(t.Context()))
require.NoError(t, cr.copyDir("/workspace", source+string(filepath.Separator), false, true)(t.Context()))
assert.Equal(t, []int64{0o600, testcase.expected}, modes)
if testcase.hostOS == "windows" {
assert.Equal(t, 1, cli.infoCalls)
} else {
assert.Zero(t, cli.infoCalls)
}
cli.AssertExpectations(t)
})
}
}
+12 -1
View File
@@ -4,7 +4,18 @@
package container package container
import "context" import (
"context"
"path"
"strings"
)
func JoinPath(elem ...string) string { // path.Join would collapse the leading // of a UNC act path on a Windows host
if len(elem) > 0 && strings.HasPrefix(elem[0], "//") {
return "/" + path.Join(elem...)
}
return path.Join(elem...)
}
type ExecutionsEnvironment interface { type ExecutionsEnvironment interface {
Container Container
+20 -84
View File
@@ -40,7 +40,8 @@ type HostEnvironment struct {
// when Workdir points at a caller-owned checkout (e.g. `act` local mode). // when Workdir points at a caller-owned checkout (e.g. `act` local mode).
CleanWorkdir bool CleanWorkdir bool
ActPath string ActPath string
CleanUp func() Root string
TrackingID string // RUNNER_TRACKING_ID of the job, a step with another value keeps its leftovers
StdOut io.Writer StdOut io.Writer
AllocatePTY bool // allocate a pseudo-TTY for each step's process AllocatePTY bool // allocate a pseudo-TTY for each step's process
@@ -261,6 +262,7 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
// Flush any buffered, not-yet-newline-terminated trailing line, as the docker backend // Flush any buffered, not-yet-newline-terminated trailing line, as the docker backend
// does in waitForCommand, so the final line of a command's output is not lost. // does in waitForCommand, so the final line of a command's output is not lost.
defer common.FlushWriter(e.StdOut) defer common.FlushWriter(e.StdOut)
process.UTF8Console()
f, err := lookupPathHost(command[0], env, e.StdOut) f, err := lookupPathHost(command[0], env, e.StdOut)
if err != nil { if err != nil {
@@ -270,7 +272,12 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
cmd.Path = f cmd.Path = f
cmd.Args = command cmd.Args = command
cmd.Stdout = e.StdOut cmd.Stdout = e.StdOut
cmd.Env = getEnvListFromMap(env) trackingID := e.TrackingID
if value, ok := common.LookupFold(env, "RUNNER_TRACKING_ID", e.IsEnvironmentCaseInsensitive()); ok {
trackingID = value
}
tracked := trackingID == e.TrackingID
cmd.Env = append(getEnvListFromMap(env), "RUNNER_TRACKING_ID="+trackingID)
cmd.Stderr = e.StdOut cmd.Stderr = e.StdOut
cmd.Dir = wd cmd.Dir = wd
cmd.SysProcAttr = process.SysProcAttr(cmdline, false) cmd.SysProcAttr = process.SysProcAttr(cmdline, false)
@@ -278,6 +285,7 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
// Kills the step's whole tree on cancellation and bounds the post-exit I/O // Kills the step's whole tree on cancellation and bounds the post-exit I/O
// wait, so an orphan holding cmd's stdout pipe cannot hang cmd.Wait(). // wait, so an orphan holding cmd's stdout pipe cannot hang cmd.Wait().
treeKill := process.NewTreeKill(cmd) treeKill := process.NewTreeKill(cmd)
treeKill.KeepLeftovers = !tracked
var ppty *os.File var ppty *os.File
var tty *os.File var tty *os.File
@@ -311,10 +319,10 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
if err := cmd.Start(); err != nil { if err := cmd.Start(); err != nil {
return err return err
} }
// Assign before the step's Killer so the step's job nests inside the group's; if tracked {
// cancellation still scopes to this step's tree. if err := e.processGroup(ctx).Assign(cmd.Process); err != nil {
if err := e.processGroup(ctx).Assign(cmd.Process); err != nil { common.Logger(ctx).Warnf("could not assign the step's process to the job's process group, a process it leaves behind may outlive the job: %v", err)
common.Logger(ctx).Warnf("could not assign the step's process to the job's process group; a process it leaves behind may outlive the job: %v", err) }
} }
if k, kerr := treeKill.Capture(cmd.Process); kerr != nil { if k, kerr := treeKill.Capture(cmd.Process); kerr != nil {
common.Logger(ctx).Warnf("process tree kill setup failed, falling back to single-process kill: %v", kerr) common.Logger(ctx).Warnf("process tree kill setup failed, falling back to single-process kill: %v", kerr)
@@ -365,9 +373,7 @@ func (e *HostEnvironment) UpdateFromEnv(srcPath string, env *map[string]string)
// removeAll is a var so tests can substitute a blocking stub. // removeAll is a var so tests can substitute a blocking stub.
var removeAll = os.RemoveAll var removeAll = os.RemoveAll
// removeAllWithContext returns once the delete finishes or ctx is cancelled. On // Returns on cancel while an uninterruptible delete keeps running, freeing the runner's capacity slot.
// cancellation the goroutine leaks: a delete inside a syscall cannot be
// interrupted (see runWithTimeout).
func removeAllWithContext(ctx context.Context, path string) error { func removeAllWithContext(ctx context.Context, path string) error {
done := make(chan error, 1) done := make(chan error, 1)
go func() { done <- removeAll(path) }() go func() { done <- removeAll(path) }()
@@ -409,35 +415,6 @@ func removePathWithRetry(ctx context.Context, path string) error {
return lastErr return lastErr
} }
// buildWindowsWorkspaceKillScript builds a PowerShell command that taskkills
// every process tree whose ExecutablePath or CommandLine references one of the
// given workspace dirs, releasing file handles for cleanup. Win32_Process
// exposes both fields (Get-Process doesn't, wmic is deprecated); matching is on
// the dir+separator prefix via ordinal String methods, so a name-prefix sibling
// (job1 vs job10) is spared and path metacharacters stay literal.
func buildWindowsWorkspaceKillScript(dirs []string) string {
quoted := make([]string, len(dirs))
for i, d := range dirs {
// Single-quoted PowerShell literal; escape ' by doubling it.
quoted[i] = "'" + strings.ReplaceAll(d, "'", "''") + "'"
}
return `$paths = @(` + strings.Join(quoted, ",") + `)
$selfPid = $PID
Get-CimInstance Win32_Process -ErrorAction SilentlyContinue | Where-Object {
if ($_.ProcessId -eq $selfPid) { return $false }
foreach ($p in $paths) {
$prefix = $p + '\'
if ($_.ExecutablePath -and $_.ExecutablePath.StartsWith($prefix, [System.StringComparison]::OrdinalIgnoreCase)) { return $true }
if ($_.CommandLine -and $_.CommandLine.IndexOf($prefix, [System.StringComparison]::OrdinalIgnoreCase) -ge 0) { return $true }
}
return $false
} | ForEach-Object {
& taskkill.exe /PID $_.ProcessId /T /F 2>$null | Out-Null
}
`
}
func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) { func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) {
if runtime.GOOS != "windows" { if runtime.GOOS != "windows" {
return return
@@ -471,20 +448,10 @@ func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) {
return return
} }
script := buildWindowsWorkspaceKillScript(dirs) if killed, err := process.KillProcessesWithPaths(killCtx, dirs, e.TrackingID); err != nil {
logger.Debugf("workspace process cleanup reported errors: %v", err)
cmd := exec.CommandContext(killCtx, "powershell.exe", "-NoProfile", "-NonInteractive", "-Command", script)
out, err := cmd.CombinedOutput()
if err != nil {
logger.Debugf("workspace process-tree kill via PowerShell failed: %v output=%s", err, strings.TrimSpace(string(out)))
}
// Win32_Process exposes no working directory, so the scan above misses a
// process that merely runs in a workspace dir while pinning a handle on it.
if killed, err := process.KillProcessesWithCWDUnder(killCtx, dirs); err != nil {
logger.Debugf("workspace process kill by working directory reported errors: %v", err)
} else if killed > 0 { } else if killed > 0 {
logger.Debugf("terminated %d leftover process(es) by workspace working directory", killed) logger.Debugf("terminated %d leftover workspace process(es)", killed)
} }
} }
@@ -492,26 +459,6 @@ func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) {
// wedge the runner slot. A var so tests can shrink it. // wedge the runner slot. A var so tests can shrink it.
var hostCleanupTimeout = 30 * time.Second var hostCleanupTimeout = 30 * time.Second
// runWithTimeout returns context.DeadlineExceeded once timeout elapses, leaking
// the goroutine: a delete blocked in a syscall (AV filter driver, dead network
// mount) cannot be interrupted, and leaking scratch state beats losing the
// runner's capacity slot forever. The idle stale-dir sweep reclaims it later.
func runWithTimeout(fn func(), timeout time.Duration) error {
done := make(chan struct{})
go func() {
defer close(done)
fn()
}()
timer := time.NewTimer(timeout)
defer timer.Stop()
select {
case <-done:
return nil
case <-timer.C:
return context.DeadlineExceeded
}
}
func (e *HostEnvironment) Remove() common.Executor { func (e *HostEnvironment) Remove() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
logger := common.Logger(ctx) logger := common.Logger(ctx)
@@ -523,25 +470,14 @@ func (e *HostEnvironment) Remove() common.Executor {
} }
e.terminateRunningProcesses(ctx) e.terminateRunningProcesses(ctx)
// Removes per-job misc state only, never the toolcache root. Bounded because
// CleanUp is a caller-supplied, typically unbounded os.RemoveAll.
if e.CleanUp != nil {
logger.Debugf("running host environment cleanup callback")
if err := runWithTimeout(e.CleanUp, hostCleanupTimeout); err != nil {
logger.Warnf("host environment cleanup did not finish within %s; continuing job completion, scratch state may be leaked and is reclaimed by the idle stale-dir sweep", hostCleanupTimeout)
} else {
logger.Debugf("host environment cleanup callback finished")
}
}
// Detach: a cancelled ctx would skip removePathWithRetry's retries, // Detach: a cancelled ctx would skip removePathWithRetry's retries,
// which absorb Windows file-handle release lag after the kill above. // which absorb Windows file-handle release lag after the kill above.
rmCtx, rmCancel := context.WithTimeout(context.Background(), hostCleanupTimeout) rmCtx, rmCancel := context.WithTimeout(context.Background(), hostCleanupTimeout)
defer rmCancel() defer rmCancel()
var errs []error var errs []error
if err := removePathWithRetry(rmCtx, e.Path); err != nil { if err := removePathWithRetry(rmCtx, cmp.Or(e.Root, e.Path)); err != nil {
logger.Warnf("failed to remove host misc state %s: %v", e.Path, err) logger.Warnf("failed to remove host misc state %s: %v", cmp.Or(e.Root, e.Path), err)
errs = append(errs, err) errs = append(errs, err)
} }
if e.CleanWorkdir { if e.CleanWorkdir {
+35 -82
View File
@@ -161,10 +161,12 @@ func TestHostEnvironmentRemoveWorkdir(t *testing.T) {
Path: filepath.Join(miscRoot, "hostexecutor"), Path: filepath.Join(miscRoot, "hostexecutor"),
Workdir: workdir, Workdir: workdir,
CleanWorkdir: owned, CleanWorkdir: owned,
CleanUp: func() { _ = os.RemoveAll(miscRoot) }, Root: miscRoot,
} }
require.NoError(t, os.MkdirAll(e.Path, 0o700)) require.NoError(t, os.MkdirAll(e.Path, 0o700))
require.NoError(t, os.MkdirAll(filepath.Join(miscRoot, "tmp"), 0o700))
require.NoError(t, e.Remove()(t.Context())) require.NoError(t, e.Remove()(t.Context()))
assert.NoDirExists(t, miscRoot)
_, err := os.Stat(workdir) _, err := os.Stat(workdir)
if owned { if owned {
assert.ErrorIs(t, err, os.ErrNotExist) assert.ErrorIs(t, err, os.ErrNotExist)
@@ -201,87 +203,38 @@ func TestRemoveAllWithContextDoesNotHangOnStuckDelete(t *testing.T) {
} }
func TestHostEnvironmentRemoveDoesNotHang(t *testing.T) { func TestHostEnvironmentRemoveDoesNotHang(t *testing.T) {
for _, phase := range []string{"CleanUp callback", "path removal"} { synctest.Test(t, func(t *testing.T) {
t.Run(phase, func(t *testing.T) { origTimeout, origRemoveAll := hostCleanupTimeout, removeAll
synctest.Test(t, func(t *testing.T) { hostCleanupTimeout = 100 * time.Millisecond
origTimeout := hostCleanupTimeout release, stubDone := make(chan struct{}), make(chan struct{})
hostCleanupTimeout = 100 * time.Millisecond removeAll = func(string) error {
t.Cleanup(func() { hostCleanupTimeout = origTimeout }) defer close(stubDone)
release, stubDone := make(chan struct{}), make(chan struct{}) <-release
block := func() { return nil
defer close(stubDone) }
<-release t.Cleanup(func() {
} close(release)
e := &HostEnvironment{Path: filepath.Join(t.TempDir(), "misc", "hostexecutor"), StdOut: os.Stdout} <-stubDone
if phase == "CleanUp callback" { hostCleanupTimeout, removeAll = origTimeout, origRemoveAll
e.CleanUp = block
} else {
origRemoveAll := removeAll
removeAll = func(string) error {
block()
return nil
}
t.Cleanup(func() { removeAll = origRemoveAll })
}
t.Cleanup(func() {
close(release)
<-stubDone
})
require.NoError(t, os.MkdirAll(e.Path, 0o700))
done := make(chan error, 1)
go func() { done <- e.Remove()(common.WithLogger(t.Context(), logging.Default())) }()
select {
case err := <-done:
require.NoError(t, err)
case <-time.After(10 * time.Second):
t.Fatalf("Remove() hung on a stuck %s", phase)
}
})
}) })
e := &HostEnvironment{Path: filepath.Join(t.TempDir(), "misc", "hostexecutor"), StdOut: os.Stdout}
require.NoError(t, os.MkdirAll(e.Path, 0o700))
done := make(chan error, 1)
go func() { done <- e.Remove()(common.WithLogger(t.Context(), logging.Default())) }()
select {
case err := <-done:
require.NoError(t, err)
case <-time.After(10 * time.Second):
t.Fatal("Remove() hung on a stuck path removal")
}
})
}
func TestJoinPath(t *testing.T) {
assert.Equal(t, "//server/share/act/workflow/envs.txt", JoinPath("//server/share/act", "workflow/envs.txt"))
assert.Equal(t, "/var/run/act/workflow/envs.txt", JoinPath("/var/run/act/", "workflow", "envs.txt"))
for _, root := range []string{"//server/share/act", "/var/run/act", "C:/act"} {
assert.Equal(t, root+"/actions/key/main.go.out", JoinPath(root+"/actions/key", "main.go.out"))
assert.Equal(t, root+"/workflow", JoinPath(root+"/workflow/outputcmd.txt", ".."))
} }
} }
func TestBuildWindowsWorkspaceKillScript(t *testing.T) {
t.Run("single dir", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript([]string{`C:\workspace\job1`})
assert.Contains(t, s, `$paths = @('C:\workspace\job1')`)
assert.Contains(t, s, "$selfPid = $PID")
assert.Contains(t, s, "$_.ProcessId -eq $selfPid")
assert.Contains(t, s, `$prefix = $p + '\'`)
assert.Contains(t, s, "$_.ExecutablePath.StartsWith($prefix")
assert.Contains(t, s, "$_.CommandLine.IndexOf($prefix")
assert.Contains(t, s, "taskkill.exe /PID $_.ProcessId /T /F")
})
t.Run("multiple dirs comma-separated", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript([]string{
`C:\work\path`,
`C:\work\workdir`,
`C:\Users\runner\AppData\Local\Temp\job-42`,
})
assert.Contains(t, s, `'C:\work\path'`)
assert.Contains(t, s, `'C:\work\workdir'`)
assert.Contains(t, s, `'C:\Users\runner\AppData\Local\Temp\job-42'`)
assert.Contains(t, s, `'C:\work\path','C:\work\workdir',`)
})
t.Run("path with single quote is escaped", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript([]string{`C:\work\it's\path`})
assert.Contains(t, s, `'C:\work\it''s\path'`)
assert.NotContains(t, s, `'C:\work\it's\path'`)
})
t.Run("path with wildcard metacharacters is matched literally", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript([]string{`C:\work\[job]?1`})
assert.Contains(t, s, `'C:\work\[job]?1'`)
assert.NotContains(t, s, "-like")
assert.Contains(t, s, "StartsWith")
assert.Contains(t, s, "IndexOf")
})
t.Run("empty dir list still produces a valid script", func(t *testing.T) {
s := buildWindowsWorkspaceKillScript(nil)
assert.Contains(t, s, "$paths = @()")
assert.Contains(t, s, "Get-CimInstance Win32_Process")
})
}
@@ -0,0 +1,138 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"bufio"
"context"
"fmt"
"io"
"net"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"unsafe"
"github.com/stretchr/testify/require"
"golang.org/x/sys/windows"
)
func TestHostEnvironmentTrackingCleanup(t *testing.T) {
for _, tracking := range []string{"default", "", "shared-helper"} {
t.Run("tracking="+tracking, func(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), 30*time.Second)
defer cancel()
listener, err := net.Listen("tcp4", "127.0.0.1:0")
require.NoError(t, err)
defer listener.Close()
stop := context.AfterFunc(ctx, func() { _ = listener.Close() })
defer stop()
workspace := t.TempDir()
environment := &HostEnvironment{Path: t.TempDir(), TmpDir: workspace, Workdir: workspace, TrackingID: "gitea_test", StdOut: io.Discard}
env := map[string]string{}
for _, entry := range os.Environ() {
if name, value, ok := strings.Cut(entry, "="); ok && !strings.EqualFold(name, "RUNNER_TRACKING_ID") {
env[name] = value
}
}
env["GITEA_TRACKING_HELPER"] = "step"
env["GITEA_TRACKING_WORKSPACE"] = workspace
env["GITEA_TRACKING_ADDRESS"] = listener.Addr().String()
if tracking == "default" {
env["RUNNER_TRACKING_ID"] = environment.TrackingID
env["runner_tracking_id"] = "shared-helper"
} else {
env["runner_tracking_id"] = tracking
}
result := make(chan error, 1)
go func() {
result <- environment.Exec([]string{os.Args[0], "-test.run=^TestHostTrackingHelper$"}, env, "", workspace)(ctx)
}()
step, err := listener.Accept()
require.NoError(t, err)
defer step.Close()
require.NoError(t, step.SetDeadline(time.Now().Add(20*time.Second)))
var stepPID int
_, err = fmt.Fscanln(step, &stepPID)
require.NoError(t, err)
stepHandle, err := windows.OpenProcess(windows.PROCESS_QUERY_LIMITED_INFORMATION, false, uint32(stepPID))
require.NoError(t, err)
defer func() { _ = windows.CloseHandle(stepHandle) }()
require.Eventually(t, func() bool {
var assigned int32
result, _, _ := windows.NewLazySystemDLL("kernel32.dll").NewProc("IsProcessInJob").Call(uintptr(stepHandle), 0, uintptr(unsafe.Pointer(&assigned)))
return result != 0 && assigned != 0
}, 5*time.Second, 10*time.Millisecond)
_, err = step.Write([]byte{1})
require.NoError(t, err)
var helpers []windows.Handle
for range 2 {
connection, err := listener.Accept()
require.NoError(t, err)
t.Cleanup(func() { _ = connection.Close() })
require.NoError(t, connection.SetReadDeadline(time.Now().Add(20*time.Second)))
var pid int
_, err = fmt.Fscanln(connection, &pid)
require.NoError(t, err)
handle, err := windows.OpenProcess(windows.SYNCHRONIZE|windows.PROCESS_TERMINATE, false, uint32(pid))
require.NoError(t, err)
t.Cleanup(func() {
_ = windows.TerminateProcess(handle, 1)
_, _ = windows.WaitForSingleObject(handle, 5000)
_ = windows.CloseHandle(handle)
})
helpers = append(helpers, handle)
}
select {
case err := <-result:
require.NoError(t, err)
case <-ctx.Done():
t.Fatal(ctx.Err())
}
require.NoError(t, environment.Remove()(ctx))
for _, handle := range helpers {
if tracking == "default" {
result, err := windows.WaitForSingleObject(handle, 5000)
require.NoError(t, err)
require.EqualValues(t, windows.WAIT_OBJECT_0, result)
} else {
result, err := windows.WaitForSingleObject(handle, 0)
require.NoError(t, err)
require.EqualValues(t, windows.WAIT_TIMEOUT, result)
}
}
})
}
}
func TestHostTrackingHelper(t *testing.T) {
mode := os.Getenv("GITEA_TRACKING_HELPER")
if mode == "" {
return
}
connection, err := net.Dial("tcp4", os.Getenv("GITEA_TRACKING_ADDRESS"))
require.NoError(t, err)
_, err = fmt.Fprintln(connection, os.Getpid())
require.NoError(t, err)
if mode == "step" {
_, err := bufio.NewReader(connection).ReadByte()
require.NoError(t, err)
}
if mode != "descendant" {
child := exec.Command(os.Args[0], "-test.run=^TestHostTrackingHelper$", filepath.Join(os.Getenv("GITEA_TRACKING_WORKSPACE"), "tracking-reference"))
child.Env = append(os.Environ(), "GITEA_TRACKING_HELPER="+map[string]string{"step": "helper", "helper": "descendant"}[mode])
require.NoError(t, child.Start())
if mode == "step" {
require.NoError(t, connection.Close())
os.Exit(0)
}
}
_, err = io.Copy(io.Discard, connection)
require.NoError(t, err)
require.NoError(t, connection.Close())
os.Exit(0)
}
+1 -1
View File
@@ -557,7 +557,7 @@ func (c *kubernetesContainer) Copy(destPath string, files ...*FileEntry) common.
func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor { func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
return c.extract(ctx, destPath, true, func(writer io.Writer) error { // directories are actions and workspaces, megabytes of compressible sources return c.extract(ctx, destPath, true, func(writer io.Writer) error { // directories are actions and workspaces, megabytes of compressible sources
return writeDirTar(ctx, writer, "/", srcPath, useGitIgnore, skipGitDir, 0, 0) return writeDirTar(ctx, writer, "/", srcPath, useGitIgnore, skipGitDir, 0, 0, hostOS == "windows")
}) })
} }
} }
+8 -3
View File
@@ -21,7 +21,12 @@ import (
// parseEnvFile reads a GITHUB_ENV style file, splitting lines like GitHub's runner on Windows or elsewhere. // parseEnvFile reads a GITHUB_ENV style file, splitting lines like GitHub's runner on Windows or elsewhere.
func parseEnvFile(e Container, srcPath string, env *map[string]string, windows bool) common.Executor { func parseEnvFile(e Container, srcPath string, env *map[string]string, windows bool) common.Executor {
localEnv := *env return ParseEnvFile(e, srcPath, func(name, value string) {
(*env)[name] = value
}, windows)
}
func ParseEnvFile(e Container, srcPath string, setEnv func(name, value string), windows bool) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
envTar, err := e.GetContainerArchive(ctx, srcPath) envTar, err := e.GetContainerArchive(ctx, srcPath)
if err != nil { if err != nil {
@@ -64,7 +69,7 @@ func parseEnvFile(e Container, srcPath string, env *map[string]string, windows b
multiLineEnv := strings.Index(line, "<<") multiLineEnv := strings.Index(line, "<<")
switch { switch {
case singleLineEnv != -1 && (multiLineEnv == -1 || singleLineEnv < multiLineEnv): case singleLineEnv != -1 && (multiLineEnv == -1 || singleLineEnv < multiLineEnv):
localEnv[line[:singleLineEnv]] = line[singleLineEnv+1:] setEnv(line[:singleLineEnv], line[singleLineEnv+1:])
case multiLineEnv != -1: case multiLineEnv != -1:
var multiLineEnvContent []string var multiLineEnvContent []string
multiLineEnvDelimiter := line[multiLineEnv+2:] multiLineEnvDelimiter := line[multiLineEnv+2:]
@@ -86,7 +91,7 @@ func parseEnvFile(e Container, srcPath string, env *map[string]string, windows b
if !delimiterFound { if !delimiterFound {
return fmt.Errorf("invalid format delimiter '%v' not found before end of file", multiLineEnvDelimiter) return fmt.Errorf("invalid format delimiter '%v' not found before end of file", multiLineEnvDelimiter)
} }
localEnv[line[:multiLineEnv]] = trimEnvLineEnding(strings.Join(multiLineEnvContent, ""), windows) setEnv(line[:multiLineEnv], trimEnvLineEnding(strings.Join(multiLineEnvContent, ""), windows))
default: default:
return fmt.Errorf("invalid format '%v', expected a line with '=' or '<<'", line) return fmt.Errorf("invalid format '%v', expected a line with '=' or '<<'", line)
} }
+14 -2
View File
@@ -34,6 +34,7 @@ func TestParseEnvFile(t *testing.T) {
want map[string]string want map[string]string
}{ }{
{"single line", "FOO=bar\nBAZ=qux\n", map[string]string{"FOO": "bar", "BAZ": "qux"}}, {"single line", "FOO=bar\nBAZ=qux\n", map[string]string{"FOO": "bar", "BAZ": "qux"}},
{"repeated names", "FOO=first\nfoo=middle\nFOO=last\n", map[string]string{"FOO": "last", "foo": "middle"}},
{"single line CRLF and final bare CR", "FOO=bar\r\nBAZ=qux\r", map[string]string{"FOO": "bar\r", "BAZ": "qux\r"}}, {"single line CRLF and final bare CR", "FOO=bar\r\nBAZ=qux\r", map[string]string{"FOO": "bar\r", "BAZ": "qux\r"}},
{"single line without final newline", "FOO=bar", map[string]string{"FOO": "bar"}}, {"single line without final newline", "FOO=bar", map[string]string{"FOO": "bar"}},
{"blank lines", "\nFOO=bar\n\n \nBAZ=qux\n\n", map[string]string{"FOO": "bar", "BAZ": "qux"}}, {"blank lines", "\nFOO=bar\n\n \nBAZ=qux\n\n", map[string]string{"FOO": "bar", "BAZ": "qux"}},
@@ -57,6 +58,17 @@ func TestParseEnvFile(t *testing.T) {
assert.Equal(t, test.want, env) assert.Equal(t, test.want, env)
}) })
} }
t.Run("ordered entries preserve repeated names", func(t *testing.T) {
hostEnv, envPath := newTestHostEnv(t)
require.NoError(t, os.WriteFile(envPath, []byte("FOO=first\r\nfoo<<EOF\r\nmiddle\r\nEOF\r\nFOO=last\r\n"), 0o600))
var entries [][2]string
require.NoError(t, ParseEnvFile(hostEnv, envPath, func(name, value string) {
entries = append(entries, [2]string{name, value})
}, true)(t.Context()))
assert.Equal(t, [][2]string{
{"FOO", "first"}, {"foo", "middle"}, {"FOO", "last"},
}, entries)
})
t.Run("Windows line endings", func(t *testing.T) { t.Run("Windows line endings", func(t *testing.T) {
hostEnv, envPath := newTestHostEnv(t) hostEnv, envPath := newTestHostEnv(t)
require.NoError(t, os.WriteFile(envPath, []byte("FOO=bar\r\nMULTI<<EOF\r\nline1\r\nline2\nEOF\r\n"), 0o600)) require.NoError(t, os.WriteFile(envPath, []byte("FOO=bar\r\nMULTI<<EOF\r\nline1\r\nline2\nEOF\r\n"), 0o600))
@@ -86,13 +98,13 @@ func TestParseEnvFileInvalidHeredoc(t *testing.T) {
} { } {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
e, envPath := newTestHostEnv(t) e, envPath := newTestHostEnv(t)
require.NoError(t, os.WriteFile(envPath, []byte(content), 0o600)) require.NoError(t, os.WriteFile(envPath, []byte("APPLIED=before-error\n"+content), 0o600))
env := map[string]string{} env := map[string]string{}
err := parseEnvFile(e, envPath, &env, false)(t.Context()) err := parseEnvFile(e, envPath, &env, false)(t.Context())
require.Error(t, err) require.Error(t, err)
assert.Contains(t, err.Error(), "delimiter") assert.Contains(t, err.Error(), "delimiter")
assert.Empty(t, env) assert.Equal(t, map[string]string{"APPLIED": "before-error"}, env)
}) })
} }
} }
+2 -2
View File
@@ -36,7 +36,7 @@ func writeFilesTar(ctx context.Context, w io.Writer, uid, gid int, files ...*Fil
} }
// writeDirTar prefixes entries with dstPath for extraction at root. // writeDirTar prefixes entries with dstPath for extraction at root.
func writeDirTar(ctx context.Context, w io.Writer, dstPath, srcPath string, useGitIgnore, skipGitDir bool, uid, gid int) error { func writeDirTar(ctx context.Context, w io.Writer, dstPath, srcPath string, useGitIgnore, skipGitDir bool, uid, gid int, executable bool) error {
logger := common.Logger(ctx) logger := common.Logger(ctx)
tw := tar.NewWriter(w) tw := tar.NewWriter(w)
srcPrefix := strings.TrimSuffix(filepath.Dir(srcPath), string(filepath.Separator)) + string(filepath.Separator) srcPrefix := strings.TrimSuffix(filepath.Dir(srcPath), string(filepath.Separator)) + string(filepath.Separator)
@@ -47,7 +47,7 @@ func writeDirTar(ctx context.Context, w io.Writer, dstPath, srcPath string, useG
SrcPrefix: srcPrefix, SrcPrefix: srcPrefix,
SkipGitDir: skipGitDir, SkipGitDir: skipGitDir,
UseGitIgnore: useGitIgnore, UseGitIgnore: useGitIgnore,
Handler: &filecollector.TarCollector{TarWriter: tw, UID: uid, GID: gid, DstDir: dstPath[1:]}, Handler: &filecollector.TarCollector{TarWriter: tw, UID: uid, GID: gid, DstDir: dstPath[1:], Executable: executable},
}).CollectFiles(ctx)); err != nil { }).CollectFiles(ctx)); err != nil {
return err return err
} }
+9 -6
View File
@@ -27,22 +27,25 @@ type Handler interface {
} }
type TarCollector struct { type TarCollector struct {
TarWriter *tar.Writer TarWriter *tar.Writer
UID int UID int
GID int GID int
DstDir string DstDir string
Executable bool // Windows has no exec bit, so mark everything executable like docker build
} }
func (tc TarCollector) WriteFile(fpath string, fi fs.FileInfo, linkName string, f io.Reader) error { func (tc TarCollector) WriteFile(fpath string, fi fs.FileInfo, linkName string, f io.Reader) error {
// create a new dir/file header // create a new dir/file header
header, err := tar.FileInfoHeader(fi, linkName) header, err := tar.FileInfoHeader(fi, filepath.ToSlash(linkName))
if err != nil { if err != nil {
return err return err
} }
// update the name to correctly reflect the desired destination when untaring // update the name to correctly reflect the desired destination when untaring
header.Name = path.Join(tc.DstDir, fpath) header.Name = path.Join(tc.DstDir, fpath)
header.Mode = int64(fi.Mode()) if tc.Executable {
header.Mode = header.Mode&0o755 | 0o111
}
header.Uid = tc.UID header.Uid = tc.UID
header.Gid = tc.GID header.Gid = tc.GID
+4
View File
@@ -54,6 +54,7 @@ func TestCollectFilesKeepsWhatGitTracksOrDoesNotIgnore(t *testing.T) {
} }
expected := map[string]string{".gitignore": "", "ign/tracked": "", "nested/kept": "", "renamed": "", "sub/.gitignore": "", "sub/kept": ""} expected := map[string]string{".gitignore": "", "ign/tracked": "", "nested/kept": "", "renamed": "", "sub/.gitignore": "", "sub/kept": ""}
if runtime.GOOS != "windows" { if runtime.GOOS != "windows" {
require.NoError(t, os.Chmod(filepath.Join(repoDir, "renamed"), 0o777|os.ModeSticky))
require.NoError(t, os.Symlink(".env", filepath.Join(repoDir, "test.env"))) require.NoError(t, os.Symlink(".env", filepath.Join(repoDir, "test.env")))
expected["test.env"] = ".env" expected["test.env"] = ".env"
} }
@@ -69,6 +70,9 @@ func TestCollectFilesKeepsWhatGitTracksOrDoesNotIgnore(t *testing.T) {
for header, err := tr.Next(); err != io.EOF; header, err = tr.Next() { for header, err := tr.Next(); err != io.EOF; header, err = tr.Next() {
require.NoError(t, err) require.NoError(t, err)
files[header.Name] = header.Linkname files[header.Name] = header.Linkname
if header.Name == "renamed" && runtime.GOOS != "windows" {
assert.Equal(t, int64(0o1777), header.Mode)
}
} }
assert.Contains(t, files, ".git/HEAD") assert.Contains(t, files, ".git/HEAD")
maps.DeleteFunc(files, func(name, _ string) bool { return strings.Contains("/"+name, "/.git/") }) maps.DeleteFunc(files, func(name, _ string) bool { return strings.Contains("/"+name, "/.git/") })
+25
View File
@@ -0,0 +1,25 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package lookpath
import (
"path/filepath"
"strings"
)
func lookPathWindows(file string, env map[string]string, lookup func(string, map[string]string) (string, error)) (string, error) {
found, err := lookup(file, env)
isBash := strings.EqualFold(file, "bash") || strings.EqualFold(file, "bash.exe")
if !isBash || err == nil && !strings.EqualFold(filepath.Base(filepath.Dir(found)), "System32") { // System32's bash is the WSL launcher, Git's bash replaces it
return found, err
}
if git, gitErr := lookup("git.exe", env); gitErr == nil {
for _, relative := range []string{"../bin/bash.exe", "../usr/bin/bash.exe", "bash.exe"} {
if bash, bashErr := lookup(filepath.Join(filepath.Dir(git), relative), env); bashErr == nil {
return bash, nil
}
}
}
return found, err
}
+56
View File
@@ -0,0 +1,56 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package lookpath
import (
"io/fs"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLookPathWindowsFallsBackToGitBash(t *testing.T) {
root := "C:" + string(filepath.Separator)
system32 := filepath.Join(root, "Windows", "System32")
gitCmd, gitBin := filepath.Join(root, "Git", "cmd"), filepath.Join(root, "Git", "bin")
msys := filepath.Join(root, "msys64", "usr", "bin")
for _, test := range []struct {
name, command, want string
path []string
}{
{"wsl launcher first", "bash", filepath.Join(gitBin, "bash.exe"), []string{system32, gitCmd}},
{"no bash on path", "BASH.EXE", filepath.Join(gitBin, "bash.exe"), []string{gitCmd}},
{"own bash first", "bash", filepath.Join(msys, "bash.exe"), []string{msys, system32, gitCmd}},
{"no git", "bash", filepath.Join(system32, "bash.exe"), []string{system32}},
{"other command", "sh", filepath.Join(system32, "sh.exe"), []string{system32, gitCmd}},
} {
t.Run(test.name, func(t *testing.T) {
files := map[string]bool{
filepath.Join(system32, "bash.exe"): true, filepath.Join(system32, "sh.exe"): true,
filepath.Join(gitCmd, "git.exe"): true, filepath.Join(gitBin, "bash.exe"): true, filepath.Join(msys, "bash.exe"): true,
}
lookup := func(file string, env map[string]string) (string, error) {
candidates := []string{file}
if !strings.ContainsAny(file, `:\/`) {
candidates = nil
for directory := range strings.SplitSeq(env["PATH"], ";") {
candidates = append(candidates, filepath.Join(directory, strings.TrimSuffix(strings.ToLower(file), ".exe")+".exe"))
}
}
for _, candidate := range candidates {
if files[filepath.Clean(candidate)] {
return filepath.Clean(candidate), nil
}
}
return "", fs.ErrNotExist
}
got, err := lookPathWindows(test.command, map[string]string{"PATH": strings.Join(test.path, ";")}, lookup)
require.NoError(t, err)
assert.Equal(t, test.want, got)
})
}
}
+4
View File
@@ -53,6 +53,10 @@ func findExecutable(file string, exts []string) (string, error) {
// a suitable candidate. // a suitable candidate.
// The result may be an absolute path or a path relative to the current directory. // The result may be an absolute path or a path relative to the current directory.
func LookPath2(file string, env map[string]string) (string, error) { func LookPath2(file string, env map[string]string) (string, error) {
return lookPathWindows(file, env, lookPath)
}
func lookPath(file string, env map[string]string) (string, error) {
var exts []string var exts []string
x := getenv(env, `PATHEXT`) x := getenv(env, `PATHEXT`)
if x != "" { if x != "" {
+11 -22
View File
@@ -198,7 +198,7 @@ func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction
if err := maybeCopyToActionDir(ctx, step, actionDir, actionPath, containerActionDir); err != nil { if err := maybeCopyToActionDir(ctx, step, actionDir, actionPath, containerActionDir); err != nil {
return err return err
} }
containerArgs := nodeActionCommand(path.Join(containerActionDir, action.Runs.Main)) containerArgs := nodeActionCommand(container.JoinPath(containerActionDir, action.Runs.Main))
logger.Debugf("executing remote job container: %s", containerArgs) logger.Debugf("executing remote job container: %s", containerArgs)
rc.ApplyExtraPath(ctx, step.getEnv()) rc.ApplyExtraPath(ctx, step.getEnv())
@@ -226,7 +226,7 @@ func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction
execFileName := action.Runs.Main + ".out" execFileName := action.Runs.Main + ".out"
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Main}, *step.getEnv(), "", containerActionDir), rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Main}, *step.getEnv(), "", containerActionDir),
rc.JobContainer.Exec([]string{filepath.Join(containerActionDir, execFileName)}, *step.getEnv(), "", ""), rc.JobContainer.Exec([]string{container.JoinPath(containerActionDir, execFileName)}, *step.getEnv(), "", ""),
)(ctx) )(ctx)
default: default:
return fmt.Errorf("the runs.using key must be one of: %v, got %s", []string{ return fmt.Errorf("the runs.using key must be one of: %v, got %s", []string{
@@ -295,6 +295,9 @@ func (rc *RunContext) dockerActionImage(ctx context.Context, step actionStep, ac
func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, basedir string, localAction bool, stage stepStage) error { func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, basedir string, localAction bool, stage stepStage) error {
logger := common.Logger(ctx) logger := common.Logger(ctx)
rc := step.getRunContext() rc := step.getRunContext()
if rc.isWindowsHost() {
return errWindowsHostContainer
}
action := step.getActionModel() action := step.getActionModel()
if rc.kubernetes { if rc.kubernetes {
return errKubernetesDocker return errKubernetesDocker
@@ -525,22 +528,8 @@ func populateEnvsFromInput(ctx context.Context, step step, action *model.Action)
return nil return nil
} }
// lookupFold finds name in values, ignoring case where the environment does, as GitHub's inputs always do.
func lookupFold[V any](values map[string]V, name string, caseInsensitive bool) (V, bool) {
if value, ok := values[name]; ok || !caseInsensitive {
return value, ok
}
for key, value := range values {
if strings.EqualFold(key, name) {
return value, true
}
}
var zero V
return zero, false
}
func stepInput(step *model.Step, name string) string { func stepInput(step *model.Step, name string) string {
value, _ := lookupFold(step.With, name, true) value, _ := common.LookupFold(step.With, name, true)
return value return value
} }
@@ -548,7 +537,7 @@ func warnUnexpectedInputs(ctx context.Context, action *model.Action, inputs map[
docker := action.Runs.Using.IsDocker() docker := action.Runs.Using.IsDocker()
var unexpectedInputs []string var unexpectedInputs []string
for input := range inputs { for input := range inputs {
_, declared := lookupFold(action.Inputs, input, true) _, declared := common.LookupFold(action.Inputs, input, true)
containerInput := docker && (strings.EqualFold(input, "entryPoint") || strings.EqualFold(input, "args")) containerInput := docker && (strings.EqualFold(input, "entryPoint") || strings.EqualFold(input, "args"))
if !declared && !containerInput { if !declared && !containerInput {
unexpectedInputs = append(unexpectedInputs, input) unexpectedInputs = append(unexpectedInputs, input)
@@ -665,7 +654,7 @@ func runPreStep(step actionStep) common.Executor {
return err return err
} }
containerArgs := nodeActionCommand(path.Join(containerActionDir, action.Runs.Pre)) containerArgs := nodeActionCommand(container.JoinPath(containerActionDir, action.Runs.Pre))
logger.Debugf("executing remote job container: %s", containerArgs) logger.Debugf("executing remote job container: %s", containerArgs)
rc.ApplyExtraPath(ctx, step.getEnv()) rc.ApplyExtraPath(ctx, step.getEnv())
@@ -697,7 +686,7 @@ func runPreStep(step actionStep) common.Executor {
execFileName := action.Runs.Pre + ".out" execFileName := action.Runs.Pre + ".out"
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Pre}, *step.getEnv(), "", containerActionDir), rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Pre}, *step.getEnv(), "", containerActionDir),
rc.JobContainer.Exec([]string{filepath.Join(containerActionDir, execFileName)}, *step.getEnv(), "", ""), rc.JobContainer.Exec([]string{container.JoinPath(containerActionDir, execFileName)}, *step.getEnv(), "", ""),
)(ctx) )(ctx)
default: default:
return nil return nil
@@ -765,7 +754,7 @@ func runPostStep(step actionStep) common.Executor {
populateEnvsFromSavedState(step.getEnv(), step, rc) populateEnvsFromSavedState(step.getEnv(), step, rc)
containerArgs := nodeActionCommand(path.Join(containerActionDir, action.Runs.Post)) containerArgs := nodeActionCommand(container.JoinPath(containerActionDir, action.Runs.Post))
logger.Debugf("executing remote job container: %s", containerArgs) logger.Debugf("executing remote job container: %s", containerArgs)
rc.ApplyExtraPath(ctx, step.getEnv()) rc.ApplyExtraPath(ctx, step.getEnv())
@@ -794,7 +783,7 @@ func runPostStep(step actionStep) common.Executor {
execFileName := action.Runs.Post + ".out" execFileName := action.Runs.Post + ".out"
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Post}, *step.getEnv(), "", containerActionDir), rc.JobContainer.Exec([]string{"go", "build", "-o", execFileName, action.Runs.Post}, *step.getEnv(), "", containerActionDir),
rc.JobContainer.Exec([]string{filepath.Join(containerActionDir, execFileName)}, *step.getEnv(), "", ""), rc.JobContainer.Exec([]string{container.JoinPath(containerActionDir, execFileName)}, *step.getEnv(), "", ""),
)(ctx) )(ctx)
default: default:
+35 -2
View File
@@ -115,7 +115,7 @@ runs:
"cwd": { "cwd": {
Description: "(Actual working directory)", Description: "(Actual working directory)",
Required: false, Required: false,
Default: "actionDir/actionPath", Default: filepath.Join("actionDir", "actionPath"),
}, },
"command": { "command": {
Description: "(Actual program)", Description: "(Actual program)",
@@ -144,7 +144,7 @@ runs:
} }
writeFile := func(filename string, _data []byte, perm fs.FileMode) error { writeFile := func(filename string, _data []byte, perm fs.FileMode) error {
assert.Equal(t, "actionDir/actionPath/trampoline.js", filename) assert.Equal(t, filepath.Join("actionDir", "actionPath", "trampoline.js"), filename)
assert.Equal(t, fs.FileMode(0o644), perm) assert.Equal(t, fs.FileMode(0o644), perm)
return nil return nil
} }
@@ -248,6 +248,39 @@ func TestActionRunner(t *testing.T) {
} }
} }
func TestGoActionUsesLinuxExecutablePaths(t *testing.T) {
for _, stage := range []string{"main", "pre", "post"} {
t.Run(stage, func(t *testing.T) {
cm := &containerMock{}
containerActionDir := "/var/run/act/actions/" + model.UsesHash("/org/repo@ref")
cm.On("CopyDir", containerActionDir+"/", mock.Anything, false, true).Return(noopExecutor).Maybe()
cm.On("Exec", []string{"go", "build", "-o", stage + ".go.out", stage + ".go"}, mock.Anything, "", containerActionDir).Return(noopExecutor).Once()
cm.On("Exec", []string{containerActionDir + "/" + stage + ".go.out"}, mock.Anything, "", "").Return(noopExecutor).Once()
step := &stepActionRemote{
Step: &model.Step{ID: "step", Uses: "org/repo@ref"},
RunContext: &RunContext{
Config: &Config{NoActionPatch: true, PresetGitHubContext: &model.GithubContext{}},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},
JobContainer: cm,
},
action: &model.Action{Runs: model.ActionRuns{Using: model.ActionRunsUsingGo, Main: "main.go", Pre: "pre.go", Post: "post.go"}},
actionDir: t.TempDir(),
env: map[string]string{},
remoteAction: mustNewRemoteAction(t, "org/repo@ref"),
}
switch stage {
case "main":
require.NoError(t, runActionImpl(step, step.actionDir, step.remoteAction)(t.Context()))
case "pre":
require.NoError(t, runPreStep(step)(t.Context()))
case "post":
require.NoError(t, runPostStep(step)(t.Context()))
}
cm.AssertExpectations(t)
})
}
}
func TestNodeActionCommandPreservesSymlinkedEntrypoint(t *testing.T) { func TestNodeActionCommandPreservesSymlinkedEntrypoint(t *testing.T) {
if runtime.GOOS == "windows" { if runtime.GOOS == "windows" {
t.Skip("symlink creation requires privileges on Windows") t.Skip("symlink creation requires privileges on Windows")
+1 -1
View File
@@ -165,7 +165,7 @@ func (rc *RunContext) refuseUnsecureCommand(ctx context.Context, command string)
} }
caseInsensitive := rc.JobContainer != nil && rc.JobContainer.IsEnvironmentCaseInsensitive() caseInsensitive := rc.JobContainer != nil && rc.JobContainer.IsEnvironmentCaseInsensitive()
for _, env := range []map[string]string{rc.currentStepEnv(), rc.Env, rc.GlobalEnv} { for _, env := range []map[string]string{rc.currentStepEnv(), rc.Env, rc.GlobalEnv} {
if value, ok := lookupFold(env, allowUnsecureCommandsVar, caseInsensitive); ok { if value, ok := common.LookupFold(env, allowUnsecureCommandsVar, caseInsensitive); ok {
if isTruthyEnv(value) { if isTruthyEnv(value) {
return false return false
} }
+1 -2
View File
@@ -10,7 +10,6 @@ import (
"errors" "errors"
"fmt" "fmt"
"maps" "maps"
"path"
"reflect" "reflect"
"regexp" "regexp"
"strings" "strings"
@@ -160,7 +159,7 @@ func getHashFilesFunction(ctx context.Context, rc *RunContext) func(v []reflect.
Name: name, Name: name,
Mode: 0o644, Mode: 0o644,
Body: hashfiles, Body: hashfiles,
}).Then(rc.JobContainer.Exec([]string{"node", path.Join(rc.JobContainer.GetActPath(), name)}, env, "", ""))(timeoutCtx) }).Then(rc.JobContainer.Exec([]string{"node", container.JoinPath(rc.JobContainer.GetActPath(), name)}, env, "", ""))(timeoutCtx)
quotedPatterns := expressionStringEscape(strings.Join(patterns, ", ")) quotedPatterns := expressionStringEscape(strings.Join(patterns, ", "))
switch { switch {
case err == nil: case err == nil:
+41 -18
View File
@@ -23,6 +23,7 @@ import (
"sync" "sync"
"time" "time"
"unicode" "unicode"
"unicode/utf8"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
@@ -30,6 +31,8 @@ import (
"gitea.dev/actionslib/pkg/exprparser" "gitea.dev/actionslib/pkg/exprparser"
"gitea.dev/actionslib/pkg/model" "gitea.dev/actionslib/pkg/model"
unicodeencoding "golang.org/x/text/encoding/unicode"
"golang.org/x/text/transform"
) )
const ( const (
@@ -461,7 +464,7 @@ func evaluateJobEnv(ctx context.Context, rc *RunContext) (map[string]string, err
} }
} }
if workflowEnv != nil { if workflowEnv != nil {
evaluated = mergeMaps(workflowEnv, env) evaluated = rc.mergeEnv(workflowEnv, env)
evaluator = rc.NewExpressionEvaluatorWithEnv(ctx, evaluated) evaluator = rc.NewExpressionEvaluatorWithEnv(ctx, evaluated)
} }
var err error var err error
@@ -569,14 +572,34 @@ func tryUploadJobSummary(ctx context.Context, rc *RunContext) {
actPath := rc.JobContainer.GetActPath() actPath := rc.JobContainer.GetActPath()
// Reuse a single client across all step uploads so connections can be pooled. // Reuse a single client across all step uploads so connections can be pooled.
client := &http.Client{Timeout: jobSummaryUploadRequestTimeout, Transport: rc.Config.instanceTransportFor(base)} client := &http.Client{Timeout: jobSummaryUploadRequestTimeout, Transport: rc.Config.instanceTransportFor(base)}
maskers := rc.secretMaskers()
longestMask := 0
for i := 0; i < len(maskers); i += 2 {
longestMask = max(longestMask, len(maskers[i]))
}
masker := NewSecretReplacer(maskers)
limit := maxJobSummaryBytes + longestMask
for i := range rc.Run.Job().Steps { for i := range rc.Run.Job().Steps {
summaryPath := path.Join(actPath, "workflow", "step-summary-"+strconv.Itoa(i)+".md") summaryPath := container.JoinPath(actPath, "workflow", "step-summary-"+strconv.Itoa(i)+".md")
body, ok := readSingleFileFromContainerArchive(ctx, rc.JobContainer, summaryPath, maxJobSummaryBytes) body, ok := readSingleFileFromContainerArchive(ctx, rc.JobContainer, summaryPath, int64(limit+1))
if !ok || len(body) == 0 { if !ok || len(body) == 0 {
continue continue
} }
// Gitea renders summaries on the run page, so mask before the upload. masked := masker.Replace(string(body))
uploadJobSummary(ctx, client, base+strconv.Itoa(i)+"/summary", runtimeToken, []byte(rc.maskSecrets(string(body)))) split := len(body) > limit
if split { // the read can end inside a secret, so keep only what masks alike without the lookahead
unsplit := masker.Replace(string(body[:maxJobSummaryBytes]))
kept := 0
for kept < len(unsplit) && kept < len(masked) && unsplit[kept] == masked[kept] {
kept++
}
for kept > 0 && kept < len(masked) && !utf8.RuneStart(masked[kept]) {
kept--
}
masked = masked[:kept]
}
body = truncateJobSummary(ctx, []byte(masked), summaryPath, maxJobSummaryBytes, split)
uploadJobSummary(ctx, client, base+strconv.Itoa(i)+"/summary", runtimeToken, body)
} }
} }
@@ -690,26 +713,26 @@ func readSingleFileFromContainerArchive(ctx context.Context, env container.Execu
if entryName != requestedPath && entryName != path.Base(requestedPath) { if entryName != requestedPath && entryName != path.Base(requestedPath) {
continue continue
} }
// Summaries larger than the limit are truncated rather than dropped, so the b, err := io.ReadAll(io.LimitReader(transform.NewReader(tr, unicodeencoding.BOMOverride(unicodeencoding.UTF8.NewDecoder())), maxBytes))
// user still gets the leading content (mirroring how GitHub caps oversized
// step summaries instead of discarding them). Read one extra byte so an
// over-limit file is detected from the actual stream rather than trusting
// header.Size, then cap the returned content at maxBytes.
b, err := io.ReadAll(io.LimitReader(tr, maxBytes+1))
if err != nil { if err != nil {
return nil, false return nil, false
} }
if int64(len(b)) > maxBytes {
// Reserve room for the marker so the marked-up result still fits in maxBytes.
marker := []byte(jobSummaryTruncationMarker)
keep := max(maxBytes-int64(len(marker)), 0)
b = append(b[:keep], marker...)
common.Logger(ctx).Warnf("job summary truncated: path=%s max=%d", p, maxBytes)
}
return b, true return b, true
} }
} }
func truncateJobSummary(ctx context.Context, body []byte, summaryPath string, maxBytes int, split bool) []byte {
if !split && len(body) <= maxBytes {
return body
}
keep := max(min(len(body), maxBytes-len(jobSummaryTruncationMarker)), 0)
for keep > 0 && keep < len(body) && !utf8.RuneStart(body[keep]) {
keep--
}
common.Logger(ctx).Warnf("job summary truncated: path=%s max=%d", summaryPath, maxBytes)
return append(body[:keep], jobSummaryTruncationMarker...)
}
// useJobStageLogger tags the log entries of a job level executor with the stage they belong to. // useJobStageLogger tags the log entries of a job level executor with the stage they belong to.
func useJobStageLogger(stage string, executor common.Executor) common.Executor { func useJobStageLogger(stage string, executor common.Executor) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
+87 -52
View File
@@ -34,6 +34,8 @@ import (
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4" yaml "go.yaml.in/yaml/v4"
"golang.org/x/text/encoding"
"golang.org/x/text/encoding/unicode"
) )
func TestJobExecutor(t *testing.T) { func TestJobExecutor(t *testing.T) {
@@ -932,49 +934,42 @@ func TestExtractJobIDFromRuntimeToken(t *testing.T) {
assert.Equal(t, int64(0), extractJobIDFromRuntimeToken("")) assert.Equal(t, int64(0), extractJobIDFromRuntimeToken(""))
} }
func TestReadSingleFileFromContainerArchiveFindsMatchingRegularFile(t *testing.T) { func TestReadSingleFileFromContainerArchive(t *testing.T) {
ctx := t.Context() retainedBytes := 200 - len(jobSummaryTruncationMarker)
cm := &containerMock{}
cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return(
io.NopCloser(bytes.NewReader(tarArchive(t,
tarEntry{name: "workflow", typeflag: tar.TypeDir},
tarEntry{name: "other.md", body: "wrong"},
tarEntry{name: "SUMMARY.md", body: "right"},
))),
nil,
).Once()
body, ok := readSingleFileFromContainerArchive(ctx, cm, "/var/run/act/workflow/SUMMARY.md", 1024)
assert.True(t, ok)
assert.Equal(t, []byte("right"), body)
cm.AssertExpectations(t)
}
func TestReadSingleFileFromContainerArchiveSizeLimit(t *testing.T) {
for name, tt := range map[string]struct { for name, tt := range map[string]struct {
content string content string
maxBytes int64 maxBytes int64
want string want string
}{ }{
"truncates oversized summary": {strings.Repeat("a", 300), 200, strings.Repeat("a", 200-len(jobSummaryTruncationMarker)) + jobSummaryTruncationMarker}, "truncates oversized summary without reading entire file": {strings.Repeat("a", 100000), 200, strings.Repeat("a", 200-len(jobSummaryTruncationMarker)) + jobSummaryTruncationMarker},
"keeps exact limit": {"abc", 3, "abc"}, "keeps exact limit": {"abc", 3, "abc"},
"strips UTF-8 BOM before limit": {"\xef\xbb\xbfabc", 3, "abc"},
"decodes UTF-16 before limit": {"\xff\xfea\x00b\x00c\x00", 3, "abc"},
"limits expanded UTF-16": {"\xff\xfe" + strings.Repeat("\x2d\x4e", 150), 200, strings.Repeat("中", retainedBytes/3) + jobSummaryTruncationMarker},
"keeps complete UTF-8 characters": {strings.Repeat("a", retainedBytes-1) + "€" + strings.Repeat("a", 200), 200, strings.Repeat("a", retainedBytes-1) + jobSummaryTruncationMarker},
} { } {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
logger, hook := logtest.NewLogger() logger, hook := logtest.NewLogger()
ctx := common.WithLogger(t.Context(), logger) ctx := common.WithLogger(t.Context(), logger)
cm := &containerMock{} cm := &containerMock{}
archive := bytes.NewReader(tarArchive(t,
tarEntry{name: "workflow", typeflag: tar.TypeDir},
tarEntry{name: "other.md", body: "wrong"},
tarEntry{name: "SUMMARY.md", body: tt.content},
))
cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return( cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return(
io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{name: "SUMMARY.md", body: tt.content}))), io.NopCloser(archive),
nil, nil,
).Once() ).Once()
body, ok := readSingleFileFromContainerArchive(ctx, cm, "/var/run/act/workflow/SUMMARY.md", tt.maxBytes) body, ok := readSingleFileFromContainerArchive(ctx, cm, "/var/run/act/workflow/SUMMARY.md", tt.maxBytes+1)
assert.LessOrEqual(t, int64(len(body)), tt.maxBytes+1)
body = truncateJobSummary(ctx, body, "/var/run/act/workflow/SUMMARY.md", int(tt.maxBytes), false)
assert.True(t, ok) assert.True(t, ok)
assert.Equal(t, []byte(tt.want), body) assert.Equal(t, []byte(tt.want), body)
assert.LessOrEqual(t, int64(len(body)), tt.maxBytes) assert.LessOrEqual(t, int64(len(body)), tt.maxBytes)
if int64(len(tt.content)) > tt.maxBytes { if strings.HasSuffix(tt.want, jobSummaryTruncationMarker) {
if assert.Len(t, hook.AllRecords(), 1) { if assert.Len(t, hook.AllRecords(), 1) {
assert.Contains(t, hook.LastRecord().Message, "job summary truncated") assert.Contains(t, hook.LastRecord().Message, "job summary truncated")
} }
@@ -1113,35 +1108,75 @@ func TestJobSetContinueOnError(t *testing.T) {
} }
func TestTryUploadJobSummaryMasksSecrets(t *testing.T) { func TestTryUploadJobSummaryMasksSecrets(t *testing.T) {
var got string const prefix = "deployed true with s3cr3t and runtime-added via pr0xypw 😀"
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { const maskedPrefix = "deployed true with *** and *** via *** 😀"
body, err := io.ReadAll(r.Body) padding := strings.Repeat("a", maxJobSummaryBytes-len(jobSummaryTruncationMarker)-len(prefix)-2)
assert.NoError(t, err) content := prefix + padding + "s3cr3t" + strings.Repeat("中", 100)
got = string(body) want := maskedPrefix + padding + "***" + strings.Repeat("中", (len(prefix)-len(maskedPrefix)-1)/3) + jobSummaryTruncationMarker
w.WriteHeader(http.StatusNoContent) longSecret := "PREFIX" + strings.Repeat("xy", 100) + "Z"
})) readLimit := maxJobSummaryBytes + base64.StdEncoding.EncodedLen(len(longSecret)) + 1
defer server.Close() shrinkingPrefix := strings.Repeat(longSecret, (readLimit-len(longSecret)+1)/len(longSecret))
for name, testcase := range map[string]struct {
encoding encoding.Encoding
content, want string
masks []string
}{
"UTF-8": {unicode.UTF8, content, want, nil},
"UTF-8 BOM": {unicode.UTF8BOM, content, want, nil},
"UTF-16LE BOM": {unicode.UTF16(unicode.LittleEndian, unicode.UseBOM), content, want, nil},
"UTF-16BE BOM": {unicode.UTF16(unicode.BigEndian, unicode.UseBOM), content, want, nil},
"secret split by the read": {unicode.UTF8, strings.Repeat("s3cr3t", maxJobSummaryBytes/3), "", nil},
"split read keeps every masked secret before the limit": {
unicode.UTF8,
"# public\n" + strings.Repeat(longSecret, readLimit/len(longSecret)+1),
"# public\n" + strings.Repeat("***", (maxJobSummaryBytes-len("# public\n"))/len(longSecret)) + jobSummaryTruncationMarker,
[]string{longSecret},
},
"nested secret expands the split tail": {
unicode.UTF8,
shrinkingPrefix + strings.Repeat("*", readLimit-len(shrinkingPrefix)-len(longSecret)+1) + longSecret,
"",
[]string{"xy", longSecret},
},
} {
t.Run(name, func(t *testing.T) {
var got string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
assert.NoError(t, err)
got = string(body)
w.WriteHeader(http.StatusNoContent)
}))
defer server.Close()
cm := &containerMock{} content, err := testcase.encoding.NewEncoder().Bytes([]byte(testcase.content))
cm.On("GetContainerArchive", mock.Anything, "/var/run/act/workflow/step-summary-0.md").Return( require.NoError(t, err)
io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{ cm := &containerMock{}
name: "step-summary-0.md", body: "deployed true with s3cr3t and runtime-added via pr0xypw", cm.On("GetContainerArchive", mock.Anything, "/var/run/act/workflow/step-summary-0.md").Return(
}))), io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{
nil, name: "step-summary-0.md", body: string(content),
).Once() }))),
nil,
).Once()
rc := newJobSummaryRC(map[string]string{ rc := newJobSummaryRC(map[string]string{
"GITEA_ACTIONS_CAPABILITIES": "job-summary", "GITEA_ACTIONS_CAPABILITIES": "job-summary",
"ACTIONS_RUNTIME_URL": server.URL, "ACTIONS_RUNTIME_URL": server.URL,
"ACTIONS_RUNTIME_TOKEN": fakeRuntimeToken(34), "ACTIONS_RUNTIME_TOKEN": fakeRuntimeToken(34),
"GITEA_RUN_ID": "12", "GITEA_RUN_ID": "12",
}, cm, 1) }, cm, 1)
rc.Config.Secrets = map[string]string{"TOK": "s3cr3t", "ACTIONS_STEP_DEBUG": "true"} rc.Config.Secrets = map[string]string{"TOK": "s3cr3t", "ACTIONS_STEP_DEBUG": "true"}
rc.Config.ExtraMasks = []string{"pr0xypw"} rc.Config.ExtraMasks = append([]string{"pr0xypw"}, testcase.masks...)
rc.Masks = []string{"runtime-added"} rc.Masks = []string{"runtime-added"}
tryUploadJobSummary(t.Context(), rc) tryUploadJobSummary(t.Context(), rc)
assert.Equal(t, "deployed true with *** and *** via ***", got) if testcase.want == "" {
cm.AssertExpectations(t) assert.Equal(t, jobSummaryTruncationMarker, strings.TrimLeft(got, "*"))
} else {
assert.Equal(t, testcase.want, got)
}
cm.AssertExpectations(t)
})
}
} }
+4 -4
View File
@@ -82,8 +82,8 @@ func (rc *RunContext) runJobHook(ctx context.Context, hookPath, name string) err
// the first one wrote. // the first one wrote.
func (rc *RunContext) setupHookFileCommands(ctx context.Context, env map[string]string) error { func (rc *RunContext) setupHookFileCommands(ctx context.Context, env map[string]string) error {
actPath := rc.JobContainer.GetActPath() actPath := rc.JobContainer.GetActPath()
env["GITHUB_ENV"] = path.Join(actPath, hookEnvFileCommand) env["GITHUB_ENV"] = container.JoinPath(actPath, hookEnvFileCommand)
env["GITHUB_PATH"] = path.Join(actPath, hookPathFileCommand) env["GITHUB_PATH"] = container.JoinPath(actPath, hookPathFileCommand)
aliasGiteaEnv(env) aliasGiteaEnv(env)
return rc.JobContainer.Copy(actPath, return rc.JobContainer.Copy(actPath,
@@ -94,7 +94,7 @@ func (rc *RunContext) setupHookFileCommands(ctx context.Context, env map[string]
func (rc *RunContext) processHookFileCommands(ctx context.Context) error { func (rc *RunContext) processHookFileCommands(ctx context.Context) error {
err := processRunnerEnvFileCommand(ctx, hookEnvFileCommand, rc, rc.setEnvFile) err := processRunnerEnvFileCommand(ctx, hookEnvFileCommand, rc, rc.setEnvFile)
if pathErr := rc.UpdateExtraPath(ctx, path.Join(rc.JobContainer.GetActPath(), hookPathFileCommand)); pathErr != nil && err == nil { if pathErr := rc.UpdateExtraPath(ctx, container.JoinPath(rc.JobContainer.GetActPath(), hookPathFileCommand)); pathErr != nil && err == nil {
err = pathErr err = pathErr
} }
return err return err
@@ -109,7 +109,7 @@ func hookCommand(hookPath string) (cmd []string, shell string) {
case ".sh": case ".sh":
return []string{"bash", "-e", hookPath}, "bash -e {0}" return []string{"bash", "-e", hookPath}, "bash -e {0}"
case ".ps1": case ".ps1":
return []string{"pwsh", "-command", ". '" + hookPath + "'"}, `pwsh -command ". '{0}'"` return []string{"pwsh", "-command", ". '" + expressionStringEscape(hookPath) + "'"}, `pwsh -command ". '{0}'"`
default: default:
return []string{hookPath}, "" return []string{hookPath}, ""
} }
+4 -3
View File
@@ -246,9 +246,10 @@ func TestHookCommand(t *testing.T) {
cmd []string cmd []string
shell string shell string
}{ }{
"/hooks/started.sh": {[]string{"bash", "-e", "/hooks/started.sh"}, "bash -e {0}"}, "/hooks/started.sh": {[]string{"bash", "-e", "/hooks/started.sh"}, "bash -e {0}"},
"/hooks/started.PS1": {[]string{"pwsh", "-command", ". '/hooks/started.PS1'"}, `pwsh -command ". '{0}'"`}, "/hooks/started.PS1": {[]string{"pwsh", "-command", ". '/hooks/started.PS1'"}, `pwsh -command ". '{0}'"`},
"/hooks/started": {[]string{"/hooks/started"}, ""}, "/hooks/started": {[]string{"/hooks/started"}, ""},
`C:\hooks\O'Brien\started.ps1`: {[]string{"pwsh", "-command", `. 'C:\hooks\O''Brien\started.ps1'`}, `pwsh -command ". '{0}'"`},
} { } {
cmd, shell := hookCommand(hookPath) cmd, shell := hookCommand(hookPath)
assert.Equal(t, want.cmd, cmd, hookPath) assert.Equal(t, want.cmd, cmd, hookPath)
+5 -1
View File
@@ -479,11 +479,15 @@ func (f textFormatter) isColored() bool {
// such as a container name or a job summary. Masks added at runtime count, so a summary written // such as a container name or a job summary. Masks added at runtime count, so a summary written
// after ::add-mask:: is covered too. // after ::add-mask:: is covered too.
func (rc *RunContext) maskSecrets(value string) string { func (rc *RunContext) maskSecrets(value string) string {
return NewSecretReplacer(rc.secretMaskers()).Replace(value)
}
func (rc *RunContext) secretMaskers() []string {
oldnew := rc.Config.maskers() oldnew := rc.Config.maskers()
for _, mask := range rc.Masks { for _, mask := range rc.Masks {
oldnew = AppendSecretMasker(oldnew, mask) oldnew = AppendSecretMasker(oldnew, mask)
} }
return NewSecretReplacer(oldnew).Replace(value) return oldnew
} }
// maskers is every value this job's configuration says to hide, whatever the sink. // maskers is every value this job's configuration says to hide, whatever the sink.
+7 -5
View File
@@ -256,7 +256,7 @@ func TestActionScriptPaths(t *testing.T) {
} }
func TestPatchActionsAtTheContainerCopy(t *testing.T) { func TestPatchActionsAtTheContainerCopy(t *testing.T) {
copiedBundles := func(t *testing.T, noPatch bool) ([]string, []*container.FileEntry) { copiedBundles := func(t *testing.T, noPatch bool) ([]string, []*container.FileEntry, int64) {
t.Helper() t.Helper()
cm := &containerMock{} cm := &containerMock{}
@@ -274,6 +274,8 @@ func TestPatchActionsAtTheContainerCopy(t *testing.T) {
require.NoError(t, os.MkdirAll(filepath.Dir(script), 0o755)) require.NoError(t, os.MkdirAll(filepath.Dir(script), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(script), "index.js"), []byte("require('./cache.js')"), 0o600)) require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(script), "index.js"), []byte("require('./cache.js')"), 0o600))
require.NoError(t, os.WriteFile(script, []byte(gateTSC), 0o600)) require.NoError(t, os.WriteFile(script, []byte(gateTSC), 0o600))
info, err := os.Stat(script)
require.NoError(t, err)
var calls []string var calls []string
var copied []*container.FileEntry var copied []*container.FileEntry
@@ -291,22 +293,22 @@ func TestPatchActionsAtTheContainerCopy(t *testing.T) {
source, err := os.ReadFile(script) source, err := os.ReadFile(script)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, gateTSC, string(source), "the shared copy is never written") assert.Equal(t, gateTSC, string(source), "the shared copy is never written")
return calls, copied return calls, copied, int64(info.Mode().Perm())
} }
t.Run("patched over the container's copy", func(t *testing.T) { t.Run("patched over the container's copy", func(t *testing.T) {
calls, copied := copiedBundles(t, false) calls, copied, mode := copiedBundles(t, false)
assert.Equal(t, []string{"CopyDir", "Copy"}, calls) assert.Equal(t, []string{"CopyDir", "Copy"}, calls)
require.Len(t, copied, 1) require.Len(t, copied, 1)
assert.Equal(t, "dist/cache.js", copied[0].Name) assert.Equal(t, "dist/cache.js", copied[0].Name)
assert.Equal(t, int64(0o600), copied[0].Mode) assert.Equal(t, mode, copied[0].Mode)
assert.True(t, gateOpened(copied[0].Body)) assert.True(t, gateOpened(copied[0].Body))
}) })
// The escape hatch, for an action the edit breaks: the artifact actions refuse again, and the // The escape hatch, for an action the edit breaks: the artifact actions refuse again, and the
// cache client keeps to v1. // cache client keeps to v1.
t.Run("as shipped when the runner is told not to patch", func(t *testing.T) { t.Run("as shipped when the runner is told not to patch", func(t *testing.T) {
calls, copied := copiedBundles(t, true) calls, copied, _ := copiedBundles(t, true)
assert.Equal(t, []string{"CopyDir"}, calls) assert.Equal(t, []string{"CopyDir"}, calls)
assert.Empty(t, copied) assert.Empty(t, copied)
}) })
+3 -2
View File
@@ -160,8 +160,9 @@ func TestCloneRemoteReusableWorkflowSendsInstanceTokenAndClientCert(t *testing.T
Run: &model.Run{JobID: "j1", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"j1": {}}}}, Run: &model.Run{JobID: "j1", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"j1": {}}}},
} }
require.Error(t, cloneRemoteReusableWorkflow(rc, server.URL+"/owner/repo", "main", t.TempDir())(t.Context())) err = cloneRemoteReusableWorkflow(rc, server.URL+"/owner/repo", "main", t.TempDir())(t.Context())
require.True(t, gotCert) require.Error(t, err)
require.True(t, gotCert, "clone failed before receiving a client certificate: %v", err)
require.Equal(t, "token", gotToken) require.Equal(t, "token", gotToken)
} }
+64 -16
View File
@@ -81,6 +81,7 @@ type RunContext struct {
jobRunDefaults model.RunDefaults // defaults.run, resolved once at job setup as GitHub does jobRunDefaults model.RunDefaults // defaults.run, resolved once at job setup as GitHub does
platformImage string // container.image or the runs-on pick, resolved once by isEnabled platformImage string // container.image or the runs-on pick, resolved once by isEnabled
jobIndex, jobTotal int // strategy.job-index and job-total, total 0 when unknown jobIndex, jobTotal int // strategy.job-index and job-total, total 0 when unknown
trackingID string // RUNNER_TRACKING_ID of the job, generated on first use
runtimeToken string // ACTIONS_RUNTIME_TOKEN of the workflow run, shared with its called workflows and composite actions runtimeToken string // ACTIONS_RUNTIME_TOKEN of the workflow run, shared with its called workflows and composite actions
jobEnv map[string]string // workflow and job env as evaluated for the container and service specs jobEnv map[string]string // workflow and job env as evaluated for the container and service specs
@@ -191,7 +192,7 @@ func (rc *RunContext) GetEnv() map[string]string {
if rc.Run != nil && rc.Run.Workflow != nil && rc.Config != nil { if rc.Run != nil && rc.Run.Workflow != nil && rc.Config != nil {
job := rc.Run.Job() job := rc.Run.Job()
if job != nil { if job != nil {
rc.Env = mergeMaps(rc.Run.Workflow.Env, job.Environment(), rc.Config.Env) rc.Env = rc.mergeEnv(rc.Run.Workflow.Env, job.Environment(), rc.Config.Env)
} }
} }
if rc.runtimeToken != "" { if rc.runtimeToken != "" {
@@ -429,27 +430,58 @@ func (rc *RunContext) startHostEnvironment() common.Executor {
Workdir: rc.Config.Workdir, Workdir: rc.Config.Workdir,
CleanWorkdir: rc.Config.CleanWorkdir, CleanWorkdir: rc.Config.CleanWorkdir,
ActPath: actPath, ActPath: actPath,
CleanUp: func() { Root: miscpath,
os.RemoveAll(miscpath) TrackingID: rc.runnerTrackingID(),
}, StdOut: logWriter,
StdOut: logWriter, AllocatePTY: rc.Config.AllocatePTY,
AllocatePTY: rc.Config.AllocatePTY,
} }
rc.cleanUpJobContainer = rc.JobContainer.Remove() rc.cleanUpJobContainer = rc.JobContainer.Remove()
rc.setRunnerContextEnv(ctx) rc.setRunnerContextEnv(ctx)
for _, env := range os.Environ() { rc.importHostEnv(os.Environ())
if k, v, ok := strings.Cut(env, "="); ok {
// don't override
if _, ok := rc.Env[k]; !ok {
rc.Env[k] = v
}
}
}
return rc.copyWorkflowFiles()(ctx) return rc.copyWorkflowFiles()(ctx)
} }
} }
func (rc *RunContext) runnerTrackingID() string {
job := rc.topLevelRunContext()
if job.trackingID == "" {
job.trackingID = "gitea_" + rand.Text()
}
return job.trackingID
}
func (rc *RunContext) isEnvCaseInsensitive() bool {
if rc.JobContainer != nil {
return rc.JobContainer.IsEnvironmentCaseInsensitive()
}
return runtime.GOOS == "windows" && rc.IsHostEnv()
}
func (rc *RunContext) importHostEnv(environ []string) {
for _, env := range environ {
if key, value, ok := strings.Cut(env, "="); ok {
if key == "RUNNER_TRACKING_ID" || rc.isEnvCaseInsensitive() && strings.EqualFold(key, "RUNNER_TRACKING_ID") {
continue
}
_, exists := common.LookupFold(rc.Env, key, rc.isEnvCaseInsensitive())
if !exists {
rc.Env[key] = value
}
}
}
}
func (rc *RunContext) mergeEnv(envs ...map[string]string) map[string]string {
if rc.isEnvCaseInsensitive() {
env := map[string]string{}
mergeIntoMapCaseInsensitive(env, envs...)
return env
} else {
return mergeMaps(envs...)
}
}
func (rc *RunContext) setRunnerContextEnv(ctx context.Context) { func (rc *RunContext) setRunnerContextEnv(ctx context.Context) {
for k, v := range rc.getRunnerContext(ctx) { for k, v := range rc.getRunnerContext(ctx) {
if v, ok := v.(string); ok { if v, ok := v.(string); ok {
@@ -460,6 +492,15 @@ func (rc *RunContext) setRunnerContextEnv(ctx context.Context) {
var errKubernetesDocker = errors.New("docker actions and docker:// steps are not supported on Kubernetes") var errKubernetesDocker = errors.New("docker actions and docker:// steps are not supported on Kubernetes")
var errWindowsHostContainer = errors.New("container actions and docker:// steps are not supported in Windows host jobs")
var hostOS = runtime.GOOS // replaced in tests
func (rc *RunContext) isWindowsHost() bool {
_, host := rc.JobContainer.(*container.HostEnvironment)
return host && hostOS == "windows"
}
func (rc *RunContext) startKubernetesPod() common.Executor { func (rc *RunContext) startKubernetesPod() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
if rc.containerSpec.Options != "" || len(rc.containerSpec.Volumes) > 0 || rc.containerSpec.Credentials != nil { if rc.containerSpec.Options != "" || len(rc.containerSpec.Volumes) > 0 || rc.containerSpec.Credentials != nil {
@@ -1311,6 +1352,10 @@ func (rc *RunContext) resolvePlatformImage(ctx context.Context) error {
} }
image, rc.kubernetes = strings.CutPrefix(image, "kubernetes://") image, rc.kubernetes = strings.CutPrefix(image, "kubernetes://")
rc.platformImage = cmp.Or(rc.containerSpec.Image, image) rc.platformImage = cmp.Or(rc.containerSpec.Image, image)
if rc.isEnvCaseInsensitive() {
rc.Env = nil // Rebuild scopes after the host platform is known.
rc.GetEnv()
}
return nil return nil
} }
@@ -1509,7 +1554,6 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
RepositoryOwner: rc.Config.Env["GITHUB_REPOSITORY_OWNER"], RepositoryOwner: rc.Config.Env["GITHUB_REPOSITORY_OWNER"],
RetentionDays: cmp.Or(rc.Config.Env["GITHUB_RETENTION_DAYS"], "0"), RetentionDays: cmp.Or(rc.Config.Env["GITHUB_RETENTION_DAYS"], "0"),
RunnerPerflog: cmp.Or(rc.Config.Env["RUNNER_PERFLOG"], "/dev/null"), RunnerPerflog: cmp.Or(rc.Config.Env["RUNNER_PERFLOG"], "/dev/null"),
RunnerTrackingID: rc.Config.Env["RUNNER_TRACKING_ID"],
Repository: rc.Config.Env["GITHUB_REPOSITORY"], Repository: rc.Config.Env["GITHUB_REPOSITORY"],
Ref: rc.Config.Env["GITHUB_REF"], Ref: rc.Config.Env["GITHUB_REF"],
Sha: rc.Config.Env["SHA_REF"], Sha: rc.Config.Env["SHA_REF"],
@@ -1523,6 +1567,8 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
ghc.EventPath = rc.JobContainer.GetActPath() + "/workflow/event.json" ghc.EventPath = rc.JobContainer.GetActPath() + "/workflow/event.json"
} }
ghc.RunnerTrackingID = rc.runnerTrackingID()
if preset := rc.Config.PresetGitHubContext; preset != nil { if preset := rc.Config.PresetGitHubContext; preset != nil {
merged := *preset merged := *preset
merged.Workflow, merged.Job, merged.Workspace, merged.EventPath = ghc.Workflow, ghc.Job, ghc.Workspace, ghc.EventPath merged.Workflow, merged.Job, merged.Workspace, merged.EventPath = ghc.Workflow, ghc.Job, ghc.Workspace, ghc.EventPath
@@ -1645,7 +1691,9 @@ func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubCon
env["GITHUB_REPOSITORY_OWNER"] = github.RepositoryOwner env["GITHUB_REPOSITORY_OWNER"] = github.RepositoryOwner
env["GITHUB_RETENTION_DAYS"] = github.RetentionDays env["GITHUB_RETENTION_DAYS"] = github.RetentionDays
env["RUNNER_PERFLOG"] = github.RunnerPerflog env["RUNNER_PERFLOG"] = github.RunnerPerflog
env["RUNNER_TRACKING_ID"] = github.RunnerTrackingID if _, ok := common.LookupFold(env, "RUNNER_TRACKING_ID", rc.isEnvCaseInsensitive()); !ok {
env["RUNNER_TRACKING_ID"] = github.RunnerTrackingID
}
env["GITHUB_BASE_REF"] = github.BaseRef env["GITHUB_BASE_REF"] = github.BaseRef
env["GITHUB_HEAD_REF"] = github.HeadRef env["GITHUB_HEAD_REF"] = github.HeadRef
env["GITHUB_SERVER_URL"] = github.ServerURL env["GITHUB_SERVER_URL"] = github.ServerURL
+50 -3
View File
@@ -19,6 +19,7 @@ import (
"path/filepath" "path/filepath"
"runtime" "runtime"
"slices" "slices"
"strconv"
"strings" "strings"
"sync" "sync"
"testing" "testing"
@@ -1082,6 +1083,51 @@ func TestRunContextGetEnv(t *testing.T) {
} }
} }
func TestRunContextEnvCaseSensitivity(t *testing.T) {
for _, caseInsensitive := range []bool{false, true} {
t.Run(strconv.FormatBool(caseInsensitive), func(t *testing.T) {
rc := createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, "runs-on: self-hosted\nenv:\n build_mode: job\n config_value: job", ""),
})
rc.Run.Workflow.Env = map[string]string{"BUILD_MODE": "workflow", "CONFIG_VALUE": "workflow", "PATH": "workflow-path", "runner_tool_cache": "workflow-cache"}
rc.Config.Env = map[string]string{"CONFIG_VALUE": "config", "RUNNER_TOOL_CACHE": "configured-cache"}
rc.Config.DisableActEnv, rc.Config.NoSkipCheckout = true, true
rc.Env = nil
rc.GetEnv()
rc.JobContainer = fakeContainer{ExecutionsEnvironment: &container.HostEnvironment{}, caseInsensitive: caseInsensitive}
require.NoError(t, rc.resolvePlatformImage(t.Context()))
if caseInsensitive {
assert.Equal(t, map[string]string{"build_mode": "job", "CONFIG_VALUE": "config", "PATH": "workflow-path", "RUNNER_TOOL_CACHE": "configured-cache"}, rc.GetEnv())
} else {
assert.Equal(t, map[string]string{
"BUILD_MODE": "workflow", "build_mode": "job", "CONFIG_VALUE": "config",
"config_value": "job", "PATH": "workflow-path", "runner_tool_cache": "workflow-cache", "RUNNER_TOOL_CACHE": "configured-cache",
}, rc.GetEnv())
}
want := maps.Clone(rc.GetEnv())
rc.Run.Workflow.Env = nil
require.NoError(t, rc.Run.Workflow.RawEnv.Encode(`${{ fromJSON('{"BUILD_MODE":"workflow","CONFIG_VALUE":"workflow","PATH":"workflow-path","runner_tool_cache":"workflow-cache"}') }}`))
rc.Env = nil
evaluated, err := evaluateJobEnv(t.Context(), rc)
require.NoError(t, err)
assert.Equal(t, want, evaluated)
rc.Env = evaluated
rc.importHostEnv([]string{"Path=host-path", "HOST_ONLY=value", "RUNNER_TRACKING_ID=parent", "runner_tracking_id=parent-lower"})
assert.NotContains(t, rc.Env, "RUNNER_TRACKING_ID")
assert.Equal(t, "workflow-path", rc.Env["PATH"])
assert.Equal(t, "value", rc.Env["HOST_ONLY"])
if caseInsensitive {
assert.NotContains(t, rc.Env, "Path")
assert.NotContains(t, rc.Env, "runner_tracking_id")
} else {
assert.Equal(t, "host-path", rc.Env["Path"])
assert.Equal(t, "parent-lower", rc.Env["runner_tracking_id"])
}
})
}
}
func TestSetActionEnvRefreshesInputs(t *testing.T) { func TestSetActionEnvRefreshesInputs(t *testing.T) {
rc := &RunContext{} rc := &RunContext{}
rc.setActionEnv(map[string]string{"INPUT_MSG": "pre"}) rc.setActionEnv(map[string]string{"INPUT_MSG": "pre"})
@@ -1405,13 +1451,14 @@ func TestGetJobContextReportsContainers(t *testing.T) {
} }
func TestCaptureJobContainerInfoExportsDockerWorkspace(t *testing.T) { func TestCaptureJobContainerInfoExportsDockerWorkspace(t *testing.T) {
workdir := t.TempDir()
job := &containerMock{} job := &containerMock{}
job.On("Inspect", mock.Anything).Return(&container.Info{ job.On("Inspect", mock.Anything).Return(&container.Info{
ID: "job-container-id", ID: "job-container-id",
Mounts: map[string]string{"/workspace": "/var/lib/docker/volumes/job/_data"}, Mounts: map[string]string{job.ToContainerPath(workdir): "/var/lib/docker/volumes/job/_data"},
}, nil) }, nil)
rc := &RunContext{ rc := &RunContext{
Config: &Config{Workdir: "/workspace/owner/repo/"}, Config: &Config{Workdir: filepath.Join(workdir, "owner", "repo") + string(filepath.Separator)},
Env: map[string]string{}, Env: map[string]string{},
JobContainer: job, JobContainer: job,
} }
@@ -1421,7 +1468,7 @@ func TestCaptureJobContainerInfoExportsDockerWorkspace(t *testing.T) {
assert.Equal(t, "job-container-id", rc.jobContainerID) assert.Equal(t, "job-container-id", rc.jobContainerID)
assert.Equal(t, "/var/lib/docker/volumes/job/_data/owner/repo", rc.Env["GITEA_DOCKER_WORKSPACE"]) assert.Equal(t, "/var/lib/docker/volumes/job/_data/owner/repo", rc.Env["GITEA_DOCKER_WORKSPACE"])
rc.Config.Workdir = "/elsewhere" rc.Config.Workdir = filepath.Dir(workdir)
rc.Env = map[string]string{} rc.Env = map[string]string{}
require.NoError(t, rc.captureJobContainerInfo()(t.Context())) require.NoError(t, rc.captureJobContainerInfo()(t.Context()))
assert.NotContains(t, rc.Env, "GITEA_DOCKER_WORKSPACE") assert.NotContains(t, rc.Env, "GITEA_DOCKER_WORKSPACE")
+43
View File
@@ -0,0 +1,43 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package runner
import (
"strings"
"testing"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRunEventHostEnvironmentUTF8(t *testing.T) {
handler := logtest.New()
t.Cleanup(func() {
if t.Failed() {
for _, record := range handler.AllRecords() {
stepIDs, _ := record.Attr("stepID")
t.Logf("stepID=%v: %s", stepIDs, record.Message)
}
}
})
table := TestJobFileInfo{
workdir: workdir,
workflowPath: "windows-utf8",
eventName: "push",
platforms: map[string]string{"windows-latest": "-self-hosted"},
}
table.runTest(WithJobLoggerFactory(t.Context(), factoryFunc(func() logging.Handler { return handler })), t, &Config{Env: testCheckoutEnv()})
var outputSteps []string
for _, record := range handler.AllRecords() {
if strings.TrimRight(record.Message, "\r\n") == "你好,世界" {
stepIDs, ok := record.Attr("stepID")
require.True(t, ok)
outputSteps = append(outputSteps, stepIDs.([]string)...)
}
}
assert.Equal(t, []string{"cmd", "pwsh", "powershell"}, outputSteps)
}
+21 -19
View File
@@ -69,10 +69,15 @@ func (s stepStage) String() string {
} }
func processRunnerEnvFileCommand(ctx context.Context, fileName string, rc *RunContext, setter func(context.Context, map[string]string, string)) error { func processRunnerEnvFileCommand(ctx context.Context, fileName string, rc *RunContext, setter func(context.Context, map[string]string, string)) error {
if rc.JobContainer.IsEnvironmentCaseInsensitive() {
return container.ParseEnvFile(rc.JobContainer, container.JoinPath(rc.JobContainer.GetActPath(), fileName), func(name, value string) {
setter(ctx, map[string]string{"name": name}, value)
}, true)(ctx)
}
env := map[string]string{} env := map[string]string{}
err := rc.JobContainer.UpdateFromEnv(path.Join(rc.JobContainer.GetActPath(), fileName), &env)(ctx) err := rc.JobContainer.UpdateFromEnv(container.JoinPath(rc.JobContainer.GetActPath(), fileName), &env)(ctx)
for k, v := range env { for key, value := range env {
setter(ctx, map[string]string{"name": k}, v) setter(ctx, map[string]string{"name": key}, value)
} }
return err return err
} }
@@ -139,16 +144,16 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
actPath := rc.JobContainer.GetActPath() actPath := rc.JobContainer.GetActPath()
outputFileCommand := path.Join("workflow", "outputcmd.txt") outputFileCommand := path.Join("workflow", "outputcmd.txt")
(*step.getEnv())["GITHUB_OUTPUT"] = path.Join(actPath, outputFileCommand) (*step.getEnv())["GITHUB_OUTPUT"] = container.JoinPath(actPath, outputFileCommand)
stateFileCommand := path.Join("workflow", "statecmd.txt") stateFileCommand := path.Join("workflow", "statecmd.txt")
(*step.getEnv())["GITHUB_STATE"] = path.Join(actPath, stateFileCommand) (*step.getEnv())["GITHUB_STATE"] = container.JoinPath(actPath, stateFileCommand)
pathFileCommand := path.Join("workflow", "pathcmd.txt") pathFileCommand := path.Join("workflow", "pathcmd.txt")
(*step.getEnv())["GITHUB_PATH"] = path.Join(actPath, pathFileCommand) (*step.getEnv())["GITHUB_PATH"] = container.JoinPath(actPath, pathFileCommand)
envFileCommand := path.Join("workflow", "envs.txt") envFileCommand := path.Join("workflow", "envs.txt")
(*step.getEnv())["GITHUB_ENV"] = path.Join(actPath, envFileCommand) (*step.getEnv())["GITHUB_ENV"] = container.JoinPath(actPath, envFileCommand)
// Per-step summary file. Composite sub-steps share the outer job step's index // Per-step summary file. Composite sub-steps share the outer job step's index
// via the Parent chain so all writes from within a composite action accumulate // via the Parent chain so all writes from within a composite action accumulate
@@ -156,7 +161,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
topRC := rc.topLevelRunContext() topRC := rc.topLevelRunContext()
stepSummaryIndex := topRC.CurrentStepIndex stepSummaryIndex := topRC.CurrentStepIndex
summaryFileCommand := path.Join("workflow", "step-summary-"+strconv.Itoa(stepSummaryIndex)+".md") summaryFileCommand := path.Join("workflow", "step-summary-"+strconv.Itoa(stepSummaryIndex)+".md")
(*step.getEnv())["GITHUB_STEP_SUMMARY"] = path.Join(actPath, summaryFileCommand) (*step.getEnv())["GITHUB_STEP_SUMMARY"] = container.JoinPath(actPath, summaryFileCommand)
aliasGiteaEnv(*step.getEnv()) aliasGiteaEnv(*step.getEnv())
// Reset the per-phase file-command files. GITHUB_STEP_SUMMARY is intentionally // Reset the per-phase file-command files. GITHUB_STEP_SUMMARY is intentionally
@@ -203,7 +208,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
if fileErr := processRunnerEnvFileCommand(fileCommandCtx, outputFileCommand, rc, rc.setOutput); fileErr != nil && err == nil { if fileErr := processRunnerEnvFileCommand(fileCommandCtx, outputFileCommand, rc, rc.setOutput); fileErr != nil && err == nil {
err = fileErr err = fileErr
} }
if fileErr := rc.UpdateExtraPath(fileCommandCtx, path.Join(actPath, pathFileCommand)); fileErr != nil && err == nil { if fileErr := rc.UpdateExtraPath(fileCommandCtx, container.JoinPath(actPath, pathFileCommand)); fileErr != nil && err == nil {
err = fileErr err = fileErr
} }
} }
@@ -351,17 +356,14 @@ func mergeIntoMapCaseInsensitive(target map[string]string, maps ...map[string]st
for k := range target { for k := range target {
foldKeys[strings.ToLower(k)] = k foldKeys[strings.ToLower(k)] = k
} }
toKey := func(s string) string {
foldKey := strings.ToLower(s)
if k, ok := foldKeys[foldKey]; ok {
return k
}
foldKeys[foldKey] = s
return s
}
for _, m := range maps { for _, m := range maps {
for k, v := range m { for key, value := range m {
target[toKey(k)] = v foldKey := strings.ToLower(key)
if previous, ok := foldKeys[foldKey]; ok {
delete(target, previous)
}
foldKeys[foldKey] = key
target[key] = value
} }
} }
} }
+3
View File
@@ -52,6 +52,9 @@ func (sd *stepDocker) runUsesContainer() common.Executor {
step := sd.Step step := sd.Step
return func(ctx context.Context) error { return func(ctx context.Context) error {
if rc.isWindowsHost() {
return errWindowsHostContainer
}
if rc.kubernetes { if rc.kubernetes {
return errKubernetesDocker return errKubernetesDocker
} }
+30
View File
@@ -159,3 +159,33 @@ func TestStepDockerNewStepContainer(t *testing.T) {
}) })
} }
} }
func TestWindowsHostContainerActions(t *testing.T) {
for _, test := range []struct {
name, hostOS string
jobContainer container.ExecutionsEnvironment
wantError bool
}{
{name: "Windows host", hostOS: "windows", jobContainer: &container.HostEnvironment{}, wantError: true},
{name: "Linux host", hostOS: "linux", jobContainer: &container.HostEnvironment{}},
{name: "macOS host", hostOS: "darwin", jobContainer: &container.HostEnvironment{}},
{name: "Windows with Linux job container", hostOS: "windows", jobContainer: &containerMock{}},
} {
t.Run(test.name, func(t *testing.T) {
original := hostOS
hostOS = test.hostOS
t.Cleanup(func() { hostOS = original })
rc := &RunContext{JobContainer: test.jobContainer, kubernetes: true}
wantError := errKubernetesDocker.Error()
if test.wantError {
wantError = errWindowsHostContainer.Error()
}
t.Run("docker step", func(t *testing.T) {
require.EqualError(t, (&stepDocker{RunContext: rc}).runUsesContainer()(t.Context()), wantError)
})
t.Run("container action", func(t *testing.T) {
require.EqualError(t, execAsDocker(t.Context(), &stepActionRemote{RunContext: rc}, "action", "", "", false, stepStageMain), wantError)
})
})
}
}
+31 -2
View File
@@ -100,7 +100,7 @@ func printRunActionHeader(ctx context.Context, step *model.Step, action *model.A
maps.Copy(inputs, step.With) maps.Copy(inputs, step.With)
if action != nil { if action != nil {
for name := range action.Inputs { for name := range action.Inputs {
if _, provided := lookupFold(step.With, name, true); !provided { if _, provided := common.LookupFold(step.With, name, true); !provided {
inputs[name] = env[inputEnvKey(name)] inputs[name] = env[inputEnvKey(name)]
} }
} }
@@ -292,7 +292,7 @@ func (sr *stepRun) setupShellCommand(ctx context.Context) (name, script string,
name += ".sh" name += ".sh"
case "pwsh", "powershell": case "pwsh", "powershell":
name += ".ps1" name += ".ps1"
runPrepend = "$ErrorActionPreference = 'stop'" runPrepend = "$ErrorActionPreference = 'stop'\n$OutputEncoding = [System.Text.UTF8Encoding]::new($false)\ntry { [Console]::OutputEncoding = $OutputEncoding } catch {}"
runAppend = "if ((Test-Path -LiteralPath variable:/LASTEXITCODE)) { exit $LASTEXITCODE }" runAppend = "if ((Test-Path -LiteralPath variable:/LASTEXITCODE)) { exit $LASTEXITCODE }"
case "cmd": case "cmd":
name += ".cmd" name += ".cmd"
@@ -302,6 +302,12 @@ func (sr *stepRun) setupShellCommand(ctx context.Context) (name, script string,
} }
script = fmt.Sprintf("%s\n%s\n%s", runPrepend, script, runAppend) script = fmt.Sprintf("%s\n%s\n%s", runPrepend, script, runAppend)
switch shellCommand {
case "powershell":
script = "\ufeff" + script
case "cmd":
script = strings.ReplaceAll(strings.ReplaceAll(script, "\r\n", "\n"), "\n", "\r\n")
}
if !strings.Contains(script, "::add-mask::") && !sr.RunContext.Config.InsecureSecrets { if !strings.Contains(script, "::add-mask::") && !sr.RunContext.Config.InsecureSecrets {
logger.Debugf("Wrote command \n%s\n to '%s'", script, name) logger.Debugf("Wrote command \n%s\n to '%s'", script, name)
@@ -311,7 +317,30 @@ func (sr *stepRun) setupShellCommand(ctx context.Context) (name, script string,
rc := sr.getRunContext() rc := sr.getRunContext()
scriptPath := fmt.Sprintf("%s/%s", rc.JobContainer.GetActPath(), name) scriptPath := fmt.Sprintf("%s/%s", rc.JobContainer.GetActPath(), name)
if step.Shell == "pwsh" || step.Shell == "powershell" {
scriptPath = expressionStringEscape(scriptPath)
}
if step.Shell == "bash" || step.Shell == "sh" || step.Shell == "python" {
if rc.isWindowsHost() {
scriptPath = `"` + scriptPath + `"`
} else {
scriptPath = shellquote.Join(scriptPath)
}
}
sr.cmdline = strings.ReplaceAll(scCmd, `{0}`, scriptPath) sr.cmdline = strings.ReplaceAll(scCmd, `{0}`, scriptPath)
if rc.isWindowsHost() {
command := strings.TrimSpace(sr.cmdline)
if strings.HasPrefix(command, `"`) {
executable, _, closed := strings.Cut(command[1:], `"`)
if !closed || executable == "" {
return "", "", fmt.Errorf("invalid shell option %q: expected a double-quoted executable", step.Shell)
}
sr.cmd = []string{executable}
} else {
sr.cmd = strings.Fields(command)[:1]
}
return name, script, nil
}
sr.cmd, err = shellquote.Split(sr.cmdline) sr.cmd, err = shellquote.Split(sr.cmdline)
return name, script, err return name, script, err
+86 -23
View File
@@ -11,6 +11,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strings"
"testing" "testing"
"gitea.com/gitea/runner/act/container" "gitea.com/gitea/runner/act/container"
@@ -80,14 +81,17 @@ func TestStepRun(t *testing.T) {
} }
func TestStepRunShellParity(t *testing.T) { func TestStepRunShellParity(t *testing.T) {
tests := []struct { const powershellScript = "$ErrorActionPreference = 'stop'\n$OutputEncoding = [System.Text.UTF8Encoding]::new($false)\ntry { [Console]::OutputEncoding = $OutputEncoding } catch {}\necho 你好,世界\nif ((Test-Path -LiteralPath variable:/LASTEXITCODE)) { exit $LASTEXITCODE }"
for _, test := range []struct {
name, run, shell, workingDir string name, run, shell, workingDir string
hostOS, wantCmdline string
actPath string actPath string
env map[string]string env map[string]string
host bool host bool
probeErr error probeErr error
wantExt string wantExt string
wantCmd []string wantCmd []string
wantScript string
wantErr string wantErr string
}{ }{
{ {
@@ -105,24 +109,79 @@ func TestStepRunShellParity(t *testing.T) {
wantCmd: []string{"sh", "-e", "/var/run/act/workflow/1.sh"}, wantCmd: []string{"sh", "-e", "/var/run/act/workflow/1.sh"},
}, },
{ {
name: "builtin pwsh", name: "builtin pwsh",
shell: "pwsh", run: "echo 你好,世界",
actPath: "/var/run/act with spaces", shell: "pwsh",
wantExt: ".ps1", actPath: "/var/run/act with spaces/O'Brien",
wantCmd: []string{"pwsh", "-command", ". '/var/run/act with spaces/workflow/1.ps1'"}, wantExt: ".ps1",
wantCmd: []string{"pwsh", "-command", ". '/var/run/act with spaces/O''Brien/workflow/1.ps1'"},
wantScript: powershellScript,
}, },
{ {
name: "builtin powershell", name: "builtin powershell",
shell: "powershell", actPath: "/var/run/act/O'Brien",
wantExt: ".ps1", run: "echo 你好,世界",
wantCmd: []string{"powershell", "-command", ". '/var/run/act/workflow/1.ps1'"}, shell: "powershell",
wantExt: ".ps1",
wantCmd: []string{"powershell", "-command", ". '/var/run/act/O''Brien/workflow/1.ps1'"},
wantScript: "\ufeff" + powershellScript,
}, },
{ {
name: "custom pwsh template", name: "custom pwsh template",
shell: "pwsh -NoProfile -File {0}", run: "echo 你好,世界",
wantExt: ".ps1", shell: "pwsh -NoProfile -File {0}",
wantCmd: []string{"pwsh", "-NoProfile", "-File", "/var/run/act/workflow/1.ps1"}, wantExt: ".ps1",
wantCmd: []string{"pwsh", "-NoProfile", "-File", "/var/run/act/workflow/1.ps1"},
wantScript: powershellScript,
}, },
{
name: "builtin cmd reads and emits UTF-8 without a BOM",
run: "echo 你好,世界",
shell: "cmd",
wantExt: ".cmd",
wantCmd: []string{"cmd", "/D", "/E:ON", "/V:OFF", "/S", "/C", "CALL /var/run/act/workflow/1.cmd"},
wantScript: "@echo off\r\necho 你好,世界\r\n",
},
{
name: "Windows custom executable preserves backslashes and raw arguments",
shell: `C:\Python313\python.exe -X 'utf8' "{0}"`,
actPath: `C:\runner\O'Brien`,
hostOS: "windows",
wantCmd: []string{`C:\Python313\python.exe`},
wantCmdline: `C:\Python313\python.exe -X 'utf8' "{0}"`,
},
{
name: "Windows quoted custom executable preserves spaces and backslashes",
shell: `"C:\Program Files\Python313\python.exe" "{0}"`,
actPath: `C:\runner`,
hostOS: "windows",
wantCmd: []string{`C:\Program Files\Python313\python.exe`},
wantCmdline: `"C:\Program Files\Python313\python.exe" "{0}"`,
},
{
name: "Windows PowerShell escapes apostrophes in the raw command line",
run: "echo 你好,世界",
shell: "pwsh",
actPath: `C:\runner\O'Brien`,
hostOS: "windows",
wantExt: ".ps1",
wantCmd: []string{"pwsh"},
wantCmdline: `pwsh -command ". '{0}'"`,
wantScript: powershellScript,
},
{
name: "Windows unclosed executable quote",
shell: `"C:\Program Files\python.exe {0}`,
actPath: `C:\runner`,
hostOS: "windows",
wantErr: "expected a double-quoted executable",
},
{name: "linux builtin bash quotes script path", shell: "bash", actPath: `C:/Users/Jane Doe/O'Brien`, wantCmd: []string{"bash", "--noprofile", "--norc", "-e", "-o", "pipefail", `C:/Users/Jane Doe/O'Brien/workflow/1.sh`}},
{name: "linux builtin sh quotes script path", shell: "sh", actPath: `C:/Users/Jane Doe/O'Brien`, wantCmd: []string{"sh", "-e", `C:/Users/Jane Doe/O'Brien/workflow/1.sh`}},
{name: "linux builtin python quotes script path", shell: "python", actPath: `C:/Users/Jane Doe/O'Brien`, wantExt: ".py", wantCmd: []string{"python", `C:/Users/Jane Doe/O'Brien/workflow/1.py`}},
{name: "windows builtin bash quotes script path", shell: "bash", hostOS: "windows", actPath: `C:\Users\Jane Doe\O'Brien`, wantExt: ".sh", wantCmd: []string{"bash"}, wantCmdline: `bash --noprofile --norc -e -o pipefail "{0}"`},
{name: "windows builtin sh quotes script path", shell: "sh", hostOS: "windows", actPath: `C:\Users\Jane Doe\O'Brien`, wantExt: ".sh", wantCmd: []string{"sh"}, wantCmdline: `sh -e "{0}"`},
{name: "windows builtin python quotes script path", shell: "python", hostOS: "windows", actPath: `C:\Users\Jane Doe\O'Brien`, wantExt: ".py", wantCmd: []string{"python"}, wantCmdline: `python "{0}"`},
{ {
name: "missing placeholder", name: "missing placeholder",
shell: "bash -e", shell: "bash -e",
@@ -156,9 +215,7 @@ func TestStepRunShellParity(t *testing.T) {
workingDir: "${{ DIR }}", workingDir: "${{ DIR }}",
wantErr: "unable to interpolate the working directory:", wantErr: "unable to interpolate the working directory:",
}, },
} } {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) { t.Run(test.name, func(t *testing.T) {
cm := &containerMock{} cm := &containerMock{}
var jobContainer container.ExecutionsEnvironment = &shellContainerMock{cm} var jobContainer container.ExecutionsEnvironment = &shellContainerMock{cm}
@@ -189,22 +246,28 @@ func TestStepRunShellParity(t *testing.T) {
env: test.env, env: test.env,
} }
original := hostOS
hostOS = cmp.Or(test.hostOS, "linux")
t.Cleanup(func() { hostOS = original })
name, script, err := sr.setupShellCommand(t.Context()) name, script, err := sr.setupShellCommand(t.Context())
if test.wantErr != "" { if test.wantErr != "" {
require.ErrorContains(t, err, test.wantErr) require.ErrorContains(t, err, test.wantErr)
return return
} }
require.NoError(t, err) require.NoError(t, err)
if test.wantExt == "" { if test.wantExt == "" && test.hostOS != "windows" {
test.wantExt = ".sh" test.wantExt = ".sh"
} }
wantScript := "\necho hi\n"
if test.wantExt == ".ps1" {
wantScript = "$ErrorActionPreference = 'stop'\necho hi\nif ((Test-Path -LiteralPath variable:/LASTEXITCODE)) { exit $LASTEXITCODE }"
}
assert.Equal(t, "workflow/1"+test.wantExt, name) assert.Equal(t, "workflow/1"+test.wantExt, name)
assert.Equal(t, wantScript, script) assert.Equal(t, cmp.Or(test.wantScript, "\necho hi\n"), script)
assert.Equal(t, test.wantCmd, sr.cmd) assert.Equal(t, test.wantCmd, sr.cmd)
if test.wantCmdline != "" {
scriptPath := jobContainer.GetActPath() + "/workflow/1" + test.wantExt
if test.shell == "pwsh" || test.shell == "powershell" {
scriptPath = strings.ReplaceAll(scriptPath, "'", "''")
}
assert.Equal(t, strings.ReplaceAll(test.wantCmdline, "{0}", scriptPath), sr.cmdline)
}
assert.Equal(t, test.env["DIR"], sr.WorkingDirectory) assert.Equal(t, test.env["DIR"], sr.WorkingDirectory)
cm.AssertExpectations(t) cm.AssertExpectations(t)
}) })
Loaded 50 of 85 files, more files were not shown because too many files have changed in this diff. Show more