Compare commits

..
38 Commits
Author SHA1 Message Date
silverwindandbircni 0e4c8b6b78 fix: misc improvements (#1415)
- Stop overwriting the image's `ImageOS`. `ubuntu-latest` got `ubuntu20` on the 24.04 default image.
- Derive `ImageOS` from `os-release` when the image sets none. Fall back to `ubuntu24` for `ubuntu-latest`.
- Keep an `ImageOS` set in the job, step or container env.
- Honor the image's `ImageOS` on Kubernetes.
- Treat steps after a job timeout as cancelled. `cancelled()` and `always()` cleanup now runs. The job still fails.
- Record nested reusable workflow results. Jobs that `needs:` a nested call now run.
- Forward outputs through nested reusable workflows. A bad output fails the call.
- Pass the job's `GITHUB_TOKEN` and `GITEA_TOKEN` to called workflows by default. An explicit mapping overrides them.
- Probe a started container's uid and gid concurrently.
- Cache the Docker container env from its first inspect.
- Check `ImageOS` in the Kubernetes e2e test.

---------

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1415
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-16 20:51:26 +00:00
silverwind 1f2ea1b83a fix: honor zero and empty values for fetch backoff and idle cleanup (#1418)
`fetch_interval_max: 0` is documented to disable polling backoff but became 5s. Zeros for `workdir_cleanup_age` and `idle_cleanup_interval` were ignored when they came from a YAML merge key or alias, so cleanup still ran, while an empty or null value turned cleanup off. These defaults are now seeded before the file is read, as `cache` already does, so a written 0 means off and an empty value keeps the default.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1418
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-16 20:43:37 +00:00
Renovate Bot 891f3fde38 fix(deps): update dependencies (#1414)
This PR contains the following updates:

| Package | Type | Update | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|---|---|
| docker.gitea.com/runner-images |  | digest | `b96ba46` → `139605d` |  |  |
| [docker/login-action](https://github.com/docker/login-action) ([changelog](https://github.com/docker/login-action/compare/81c6f93d4bc991e510b342a00b419b3bb4efc5c6..8aa5831fb8d48d8f76594c7ddde61ae58019f550)) | action | digest | `81c6f93` → `8aa5831` |  |  |
| [github.com/spf13/pflag](https://github.com/spf13/pflag) | require | minor | `v1.0.10` → `v1.1.0` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fspf13%2fpflag/v1.1.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fspf13%2fpflag/v1.0.10/v1.1.0?slim=true) |
| [go](https://go.dev/) ([source](https://github.com/golang/go)) | toolchain | patch | `1.27.3` → `1.27.4` | ![age](https://developer.mend.io/api/mc/badges/age/golang-version/go/1.27.4?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/golang-version/go/1.27.3/1.27.4?slim=true) |
| [golang.org/x/sync](https://pkg.go.dev/golang.org/x/sync) | require | minor | [`v0.24.0` → `v0.25.0`](https://cs.opensource.google/go/x/sync/+/refs/tags/v0.24.0...refs/tags/v0.25.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fsync/v0.25.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fsync/v0.24.0/v0.25.0?slim=true) |
| [golang.org/x/sys](https://pkg.go.dev/golang.org/x/sys) | require | minor | [`v0.49.0` → `v0.50.0`](https://cs.opensource.google/go/x/sys/+/refs/tags/v0.49.0...refs/tags/v0.50.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fsys/v0.50.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fsys/v0.49.0/v0.50.0?slim=true) |
| [golang.org/x/term](https://pkg.go.dev/golang.org/x/term) | require | minor | [`v0.47.0` → `v0.48.0`](https://cs.opensource.google/go/x/term/+/refs/tags/v0.47.0...refs/tags/v0.48.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fterm/v0.48.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fterm/v0.47.0/v0.48.0?slim=true) |
| [golang.org/x/text](https://pkg.go.dev/golang.org/x/text) | require | minor | [`v0.43.0` → `v0.44.0`](https://cs.opensource.google/go/x/text/+/refs/tags/v0.43.0...refs/tags/v0.44.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2ftext/v0.44.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2ftext/v0.43.0/v0.44.0?slim=true) |
| [quay.io/containers/podman](https://quay.io/containers/podman) ([source](https://github.com/podman-container-tools/podman)) |  | major | `v5.8.9-immutable` → `v6.1.3-immutable` | ![age](https://developer.mend.io/api/mc/badges/age/docker/quay.io%2fcontainers%2fpodman/v6.1.3?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/docker/quay.io%2fcontainers%2fpodman/v5.8.9/v6.1.3?slim=true) |

---

### HTTP/1 server crash due to MaxBytesReader header race in net/http
[CVE-2026-97054](https://nvd.nist.gov/vuln/detail/CVE-2026-97054) / [GO-2026-6859](https://pkg.go.dev/vuln/GO-2026-6859)

<details>
<summary>More information</summary>

#### Details
For HTTP/1 servers, when MaxBytesReader's limit is exceeded, "Connection: close" header is automatically set without synchronization, which might race with another header being set explicitly in the server handler. As a result, an HTTP/1 server that utilizes MaxBytesReader or MaxBytesHandler can crash due to runtime panic if its handler reads the request body concurrently with writing the response header.

#### Severity
Unknown

#### References
- [https://go.dev/cl/862185](https://go.dev/cl/862185)
- [https://go.dev/issue/82563](https://go.dev/issue/82563)
- [https://groups.google.com/g/golang-announce/c/iY8TY0wWgbM](https://groups.google.com/g/golang-announce/c/iY8TY0wWgbM)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6859) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Release Notes

<details>
<summary>spf13/pflag (github.com/spf13/pflag)</summary>

### [`v1.1.0`](https://github.com/spf13/pflag/releases/tag/v1.1.0)

[Compare Source](https://github.com/spf13/pflag/compare/v1.0.10...v1.1.0)

It's been long enough, and enough things have been fixed, that it might be time to release a new version, even if there isn't any particular big new issue to market as the reason for this release.

If you notice anything breaking, it is entirely un-intentional, but you might also have been relying on [undocumented](https://www.hyrumslaw.com/) or [buggy](https://xkcd.com/1172/) behavior. Please file an issue describing what you're doing, how it worked before this release and why it doesn't work anymore, and pin your projects to `v1.0.10` (the latest release before this one) while we decide how to address your concerns.

#### What's Changed

- README: update badges and minor linting fixes by [@&#8203;thaJeztah](https://github.com/thaJeztah) in [#&#8203;452](https://github.com/spf13/pflag/pull/452)
- Show 'correct' explicit bool value syntax in help by [@&#8203;alimpfard](https://github.com/alimpfard) in [#&#8203;453](https://github.com/spf13/pflag/pull/453)
- Fix for Go 1.24 by [@&#8203;pterjan](https://github.com/pterjan) in [#&#8203;460](https://github.com/spf13/pflag/pull/460)
- fix(map): ensure string-to-string flags maintain stable pointers by [@&#8203;brandon1024](https://github.com/brandon1024) in [#&#8203;457](https://github.com/spf13/pflag/pull/457)
- Fix getFlagType error message by [@&#8203;wellitonscheer](https://github.com/wellitonscheer) in [#&#8203;462](https://github.com/spf13/pflag/pull/462)
- update actions, and fix linting issues by [@&#8203;thaJeztah](https://github.com/thaJeztah) in [#&#8203;450](https://github.com/spf13/pflag/pull/450)
- fix: README.md indicating the use of godoc(deprecated) by [@&#8203;malklera](https://github.com/malklera) in [#&#8203;467](https://github.com/spf13/pflag/pull/467)
- Fix broken link to POSIX argument conventions by [@&#8203;tomasaschan](https://github.com/tomasaschan) in [#&#8203;464](https://github.com/spf13/pflag/pull/464)
- Replace raw stdlib parse errors in typed flag Set methods ([#&#8203;269](https://github.com/spf13/pflag/issues/269)) by [@&#8203;reuvenharrison](https://github.com/reuvenharrison) in [#&#8203;469](https://github.com/spf13/pflag/pull/469)
- Extend `UnknownFlags` to `UnknownFlagsHandling` and introduce `PassUnknownFlagToArgs` ([#&#8203;337](https://github.com/spf13/pflag/issues/337)) by [@&#8203;ikedam](https://github.com/ikedam) in [#&#8203;440](https://github.com/spf13/pflag/pull/440)
- fix: allow nil default IP flags in GetIP by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;483](https://github.com/spf13/pflag/pull/483)
- fix: allow hex input in UintSlice by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;484](https://github.com/spf13/pflag/pull/484)
- fix: honor custom IsBoolFlag compatibility by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;487](https://github.com/spf13/pflag/pull/487)
- docs: clarify SortFlags example with complete runnable program by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;488](https://github.com/spf13/pflag/pull/488)
- docs: add release process documentation by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;490](https://github.com/spf13/pflag/pull/490)
- fix: print to set destination, not hardcoded os.Stderr by [@&#8203;quite](https://github.com/quite) in [#&#8203;502](https://github.com/spf13/pflag/pull/502)
- ci: bump golangci-lint to v2.13 for Go 1.27 by [@&#8203;hdimer](https://github.com/hdimer) in [#&#8203;508](https://github.com/spf13/pflag/pull/508)
- Initialize the target map when a StringToString flag has a nil default by [@&#8203;nikolauspschuetz](https://github.com/nikolauspschuetz) in [#&#8203;506](https://github.com/spf13/pflag/pull/506)
- fix: honour deprecated ParseErrorsWhitelist.UnknownFlagsHandling by [@&#8203;lenamonj](https://github.com/lenamonj) in [#&#8203;507](https://github.com/spf13/pflag/pull/507)
- test: add StringSlice Append coverage by [@&#8203;LY-19418](https://github.com/LY-19418) in [#&#8203;504](https://github.com/spf13/pflag/pull/504)
- Keep wrapping after an unbreakable usage word by [@&#8203;AzazelSensei](https://github.com/AzazelSensei) in [#&#8203;503](https://github.com/spf13/pflag/pull/503)
- Validate IP slice values before appending or replacing by [@&#8203;agammann](https://github.com/agammann) in [#&#8203;513](https://github.com/spf13/pflag/pull/513)
- fix(ci): Use the latest golangci-lint release by [@&#8203;tomasaschan](https://github.com/tomasaschan) in [#&#8203;525](https://github.com/spf13/pflag/pull/525)
- Reject IPv6 addresses as IPv4 masks by [@&#8203;yuefdev](https://github.com/yuefdev) in [#&#8203;524](https://github.com/spf13/pflag/pull/524)
- Report the flag name as typed in deprecation warnings by [@&#8203;hdimer](https://github.com/hdimer) in [#&#8203;500](https://github.com/spf13/pflag/pull/500)
- Document non-interspersed argument parsing by [@&#8203;yuefdev](https://github.com/yuefdev) in [#&#8203;523](https://github.com/spf13/pflag/pull/523)
- Preserve trailing whitespace in StringToString formatting by [@&#8203;r4venrudy](https://github.com/r4venrudy) in [#&#8203;522](https://github.com/spf13/pflag/pull/522)
- Reject extra CSV records in StringToString arguments by [@&#8203;Harshil-1603](https://github.com/Harshil-1603) in [#&#8203;521](https://github.com/spf13/pflag/pull/521)
- Preserve current string values in CopyToGoFlagSet by [@&#8203;adenzhou1350](https://github.com/adenzhou1350) in [#&#8203;520](https://github.com/spf13/pflag/pull/520)
- Fix type identification and mismatch checks for text flags by [@&#8203;dewanshshekhar](https://github.com/dewanshshekhar) in [#&#8203;526](https://github.com/spf13/pflag/pull/526)
- fix: enforce native integer flag bounds on 32-bit platforms by [@&#8203;gdpu11](https://github.com/gdpu11) in [#&#8203;534](https://github.com/spf13/pflag/pull/534)
- fix: preserve count flag values when parsing fails by [@&#8203;gdpu11](https://github.com/gdpu11) in [#&#8203;533](https://github.com/spf13/pflag/pull/533)

#### New Contributors

- [@&#8203;alimpfard](https://github.com/alimpfard) made their first contribution in [#&#8203;453](https://github.com/spf13/pflag/pull/453)
- [@&#8203;pterjan](https://github.com/pterjan) made their first contribution in [#&#8203;460](https://github.com/spf13/pflag/pull/460)
- [@&#8203;brandon1024](https://github.com/brandon1024) made their first contribution in [#&#8203;457](https://github.com/spf13/pflag/pull/457)
- [@&#8203;wellitonscheer](https://github.com/wellitonscheer) made their first contribution in [#&#8203;462](https://github.com/spf13/pflag/pull/462)
- [@&#8203;malklera](https://github.com/malklera) made their first contribution in [#&#8203;467](https://github.com/spf13/pflag/pull/467)
- [@&#8203;reuvenharrison](https://github.com/reuvenharrison) made their first contribution in [#&#8203;469](https://github.com/spf13/pflag/pull/469)
- [@&#8203;ikedam](https://github.com/ikedam) made their first contribution in [#&#8203;440](https://github.com/spf13/pflag/pull/440)
- [@&#8203;happysnaker](https://github.com/happysnaker) made their first contribution in [#&#8203;483](https://github.com/spf13/pflag/pull/483)
- [@&#8203;quite](https://github.com/quite) made their first contribution in [#&#8203;502](https://github.com/spf13/pflag/pull/502)
- [@&#8203;hdimer](https://github.com/hdimer) made their first contribution in [#&#8203;508](https://github.com/spf13/pflag/pull/508)
- [@&#8203;nikolauspschuetz](https://github.com/nikolauspschuetz) made their first contribution in [#&#8203;506](https://github.com/spf13/pflag/pull/506)
- [@&#8203;lenamonj](https://github.com/lenamonj) made their first contribution in [#&#8203;507](https://github.com/spf13/pflag/pull/507)
- [@&#8203;LY-19418](https://github.com/LY-19418) made their first contribution in [#&#8203;504](https://github.com/spf13/pflag/pull/504)
- [@&#8203;AzazelSensei](https://github.com/AzazelSensei) made their first contribution in [#&#8203;503](https://github.com/spf13/pflag/pull/503)
- [@&#8203;agammann](https://github.com/agammann) made their first contribution in [#&#8203;513](https://github.com/spf13/pflag/pull/513)
- [@&#8203;yuefdev](https://github.com/yuefdev) made their first contribution in [#&#8203;524](https://github.com/spf13/pflag/pull/524)
- [@&#8203;r4venrudy](https://github.com/r4venrudy) made their first contribution in [#&#8203;522](https://github.com/spf13/pflag/pull/522)
- [@&#8203;Harshil-1603](https://github.com/Harshil-1603) made their first contribution in [#&#8203;521](https://github.com/spf13/pflag/pull/521)
- [@&#8203;adenzhou1350](https://github.com/adenzhou1350) made their first contribution in [#&#8203;520](https://github.com/spf13/pflag/pull/520)
- [@&#8203;dewanshshekhar](https://github.com/dewanshshekhar) made their first contribution in [#&#8203;526](https://github.com/spf13/pflag/pull/526)
- [@&#8203;gdpu11](https://github.com/gdpu11) made their first contribution in [#&#8203;534](https://github.com/spf13/pflag/pull/534)

**Full Changelog**: <https://github.com/spf13/pflag/compare/v1.0.10...v1.1.0>

</details>

<details>
<summary>podman-container-tools/podman (quay.io/containers/podman)</summary>

### [`v6.1.3`](https://github.com/podman-container-tools/podman/releases/tag/v6.1.3)

[Compare Source](https://github.com/podman-container-tools/podman/compare/v5.8.9...v6.1.3)

##### Security

- This release addresses [CVE-2026-94603](https://github.com/podman-container-tools/podman/security/advisories/GHSA-2cvf-wqm6-wr9g), where a `podman run` on a checkpoint image (any image with the `io.podman.annotations.checkpoint.runtime.name` annotation) could disable all sandboxing, including sandboxing specified by the user, when the container was created.

##### Breaking Changes

- Removed support for checkpoint images in `podman run` due to serious security concerns with the different security models of running images and running checkpoints. Checkpoints ignore user-specified security configuration and are very difficult to run safely.

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4yMjIuNSIsInVwZGF0ZWRJblZlciI6IjQ0LjIzOS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1414
Reviewed-by: silverwind <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
2026-11-16 15:15:09 +00:00
silverwind 4f59e3e418 fix: support cache-mode and make low-trust runs cache read-only (#1412)
Low-trust runs like `pull_request_target` on the default branch could poison caches that `push` runs restore. As on GitHub, they now only restore unless `cache-mode` allows saving.

This and the Gitea change can be deployed in any order.

Refs:
https://gitea.com/gitea/actionslib/pulls/33
https://github.com/go-gitea/gitea/pull/40343

_Written by Claude (Opus 5.5)._
Reviewed-on: https://gitea.com/gitea/runner/pulls/1412
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-13 09:33:32 +00:00
clintlmartinandsilverwind c2046d597c fix: stop host teardown from deleting a workdir it never created (#1409)
Host jobs run in a per-job directory under `host.workdir_parent`, which teardown removes. Teardown also deleted `<container.workdir_parent>/<owner>/<repo>`, a path host jobs never create. With an empty parent and no repository that was the filesystem root, and any existing directory at that path was deleted although the job never used it.

This removes that delete instead of guarding it, so teardown only removes what the job created.

---------

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1409
Reviewed-by: bircni <[email protected]>
Co-authored-by: clintlmartin <[email protected]>
2026-11-13 06:50:44 +00:00
silverwind 645b17de72 fix: align common behaviour with GitHub's runner (#1413)
Closes differences from GitHub's runner that common workflows hit:

- `ACTIONS_STEP_DEBUG` set as a repository variable enables debug output as on GitHub, where a secret of the same name still takes precedence. `runner.debug` and `RUNNER_DEBUG` are now also set inside composite actions, which lost the secret.
- `github.env` and `github.path` hold the file-command paths instead of being empty, so `echo "A=1" >> "${{ github.env }}"` works.
- Remote composite actions only get a pre stage when they nest actions. That stage evaluated `with:` before earlier steps ran, so an input like `${{ fromJSON(steps.meta.outputs.json).v }}` failed the job.
- `ports` on a job container are published, they were ignored while service ports worked.
- Dockerfile action build output reaches the job log. It was logged at debug and dropped, so a failing `RUN` showed only the final error.
- Local Docker actions honor their `.dockerignore`, as `docker build` does.
- Cancelling or timing out a host-mode step sends SIGINT, then SIGTERM after 7.5 seconds and SIGKILL after 2.5 more, as GitHub's runner does, so `trap` handlers and shutdown hooks run.
- `.sh` job hooks fall back to `sh` where bash is missing, and `.ps1` hooks to `powershell` where pwsh is missing, as on GitHub.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1413
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-12 15:28:47 +00:00
silverwind 0dfec377d2 fix: keep composite post steps in their own step scope (#1410)
Each composite step's post ran its earlier siblings' posts inside its own step-ID scope, so their log records listed siblings as ancestors, e.g. `outer.results.nodepost` instead of `outer.nodepost`. Every post now gets its own scope. Execution order and run-all-posts behavior are unchanged.

Prerequisite for https://gitea.com/gitea/runner/issues/809.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1410
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-11 19:31:21 +00:00
silverwind b5305c6785 docs: add a launchd example for macOS (#1408)
Adds `examples/launchd` with a plist and the steps to run the runner on macOS, next to the existing systemd example.

The runner runs in the login session of the user that jobs should run as, which gives jobs the login keychain and the GUI. The README also covers the two problems that setup runs into: macOS blocks the runner from hosts on the Mac's own subnet, and Git's keychain helper can stop a job at a dialog.

_Written by Claude (Fable 5.2)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1408
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-10 17:17:35 +00:00
bircni a021d3139d enhance: make shared tool cache safe at any capacity (#1401)
Co-authored-by: bircni <[email protected]>
2026-11-10 14:25:50 +00:00
bircniandsilverwind 84c0fcd1fa fix!: remove the dbfs log workaround and require Gitea 1.26.2 (#1395)
Gitea 1.26.2 finalizes logs on an empty `UpdateLog` with `NoMore` (https://github.com/go-gitea/gitea/pull/37687), so the sentinel row is no longer needed. On older servers, the final log of most jobs would never be archived and stays in `dbfs_data`, so this raises the minimum Gitea version. It lives in `ver.MinGiteaVersion`, which the error for servers without `Declare` now names.

Fixes: https://gitea.com/gitea/runner/issues/953
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1395
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-10 11:35:16 +00:00
bircniandsilverwind c59caa9689 refactor!: remove the builtin actions (#1407)
Removes `builtin:checkout`, `builtin:upload-artifact` and `builtin:download-artifact`, along with the container helpers only they used.

Builtin actions are tied to the runner version, so users cannot pin them, and they diverge from the shareable actions ecosystem that tools like zizmor build on. Workflows should use `actions/checkout`, `actions/upload-artifact` and `actions/download-artifact`, or lightweight third-party actions such as https://github.com/taiki-e/checkout-action.

**Breaking:** steps with `uses: builtin:checkout`, released in v4.0.0, now fail and need to switch to one of the actions above. The artifact actions were never released.

Docs: https://gitea.com/gitea/docs/pulls/643
Fixes: https://gitea.com/gitea/runner/issues/1405
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1407
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-10 11:05:38 +00:00
Renovate Bot 85d007c47f chore(deps): update dependencies (#1406)
This PR contains the following updates:

| Package | Type | Update | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|---|---|
| docker.gitea.com/runner-images |  | digest | `657378e` → `b96ba46` |  |  |
| [github.com/docker/cli](https://github.com/docker/cli) | require | patch | `v29.9.2+incompatible` → `v29.9.3+incompatible` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fdocker%2fcli/v29.9.3+incompatible?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fdocker%2fcli/v29.9.2+incompatible/v29.9.3+incompatible?slim=true) |
| [rancher/k3s](https://github.com/k3s-io/k3s) |  | patch | `v1.37.1-k3s1` → `v1.37.2-k3s1` | ![age](https://developer.mend.io/api/mc/badges/age/docker/rancher%2fk3s/v1.37.2?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/docker/rancher%2fk3s/v1.37.1/v1.37.2?slim=true) |

---

### Release Notes

<details>
<summary>docker/cli (github.com/docker/cli)</summary>

### [`v29.9.3+incompatible`](https://github.com/docker/cli/compare/v29.9.2...v29.9.3)

[Compare Source](https://github.com/docker/cli/compare/v29.9.2...v29.9.3)

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xOTkuMiIsInVwZGF0ZWRJblZlciI6IjQ0LjE5OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1406
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
2026-11-09 04:31:50 +00:00
silverwindandbircni d34e4cf11a fix: stop a stalled docker daemon from hanging jobs and the e2e suite (#1404)
When the docker daemon stopped answering on https://gitea.com/gitea/runner/actions/runs/1204971, unbounded exec runs hung the e2e suite until the 20m `go test` panic, which skipped teardown. Exec runs and teardown are now bounded, and attached exec streams, including the runner's user ID probe, close with their context.

CLI daemons now stop with SIGTERM, so job volumes no longer leak.

_Written by Claude (Opus 5.5)._

---------

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1404
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-07 08:07:41 +00:00
8e999cd039 fix: re-detect the cache address per job and prefer IPv4 (#1398)
The cache address was detected once at startup, and the interface fallback ranked ethernet names above IPv4. A runner that started before DHCPv4 finished could advertise a global IPv6 address to IPv4-only job networks until it restarted, failing every artifact and cache call.

IPv4 now ranks first, and the address is detected again for each job unless `cache.host` sets it. Detecting an IPv6 address logs a warning.

Fixes: https://gitea.com/gitea/runner/issues/1379
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1398
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-07 06:41:25 +00:00
silverwind cd748053b0 docs: trim comments and docs from workspace emptying change (#1403)
Follow-up to https://gitea.com/gitea/runner/commit/79faf5e55df7a3b312fb4888b7405a58e01f567c. Reverts the post-task-script doc change, which is too much detail for that page. Also shortens the `DrainGracePeriod` comment and the other comments that commit added, since the code and log messages already say most of it.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1403
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-07 06:14:41 +00:00
bircni 9bb72bbe5c enhance: add ubuntu-26.04 to the default labels (#1396)
Newly registered runners get an `ubuntu-26.04` label alongside the existing defaults.

Ref: https://gitea.com/gitea/runner/issues/1334
Reviewed-on: https://gitea.com/gitea/runner/pulls/1396
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-07 05:12:04 +00:00
79faf5e55d fix: empty root-owned bind workspaces for non-root runners (#1399)
A job running as root leaves root-owned files in the bind-mounted workspace. A non-root runner on Linux cannot remove them, so workspaces pile up with only a warning. Such a runner now empties the workspace as root from inside the job container, right before removing the container. Root runners and Docker Desktop already remove these files themselves and skip this.

The workspace is only emptied when it is exactly the runner's own mountpoint with no mount below it. Otherwise, or when the command fails, the existing removal and warning apply. Jobs of a reusable workflow share the workspace, so they are not covered and still end in the existing warning.

Fixes: https://gitea.com/gitea/runner/issues/879
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1399
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-07 04:33:19 +00:00
bircni 993389d428 docs: trust a private registry CA in the dind images (#1397)
The bundled Docker daemon pulls job images, so it needs to trust a private registry's CA itself, which `runner.ca_file` doesn't cover. Documents the `certs.d` mount for both dind images.

Ref: https://gitea.com/gitea/runner/issues/1390
Reviewed-on: https://gitea.com/gitea/runner/pulls/1397
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-06 21:36:06 +00:00
silverwind db76fd5159 enhance: improve hashFiles coercion and log grouping (#1394)
- Fix `hashFiles()` argument conversion for numbers, booleans, and null to match GitHub expression semantics.
- Fix rendering of mixed-case, indented, and prefixed legacy group commands while preserving recognized prefixes and escaping.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1394
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-06 20:03:39 +00:00
bircni c8b16f4792 enhance: add builtin:download-artifact (#1344)
Co-authored-by: bircni <[email protected]>
2026-11-06 03:32:59 +00:00
bircni 02446620df enhance: add builtin:upload-artifact (#1343)
Co-authored-by: bircni <[email protected]>
2026-11-06 03:17:49 +00:00
8c8950d9ef refactor(container): stream copies into the job and archives out of the host (#1340)
Adds `CopyTarStream` to the job environments. It extracts directories and files into the job as they are written, kept below the destination and owned by the job user along with any directories it creates, so built-in actions can put large content in place without buffering it. The host's `GetContainerArchive` now streams instead of building the archive in memory.

---------

Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Co-authored-by: Lunny Xiao <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1340
Reviewed-by: Lunny Xiao <[email protected]>
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-06 02:22:40 +00:00
bircniandsilverwind b88d111cc5 feat: port the path patterns of @actions/glob (#1342)
Built-in actions that select files need to match what the JavaScript actions match. This ports @actions/glob 0.6.1 and the minimatch 3 subset it uses, as bundled in `actions/upload-artifact` v7. It only answers questions about paths; walking the files stays with the caller. Only malformed bracket expressions and some non-ASCII case folding on Windows can match differently.

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1342
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-06 01:13:07 +00:00
silverwind bb6d793cb9 fix: make container.sweep opt-in (#1389)
Sweeping removed every container a job started through its Docker proxy, which silently tore down services that jobs deploy to the runner's own Docker daemon. Deploying from a runner on the target host is common, so `container.sweep` now defaults to `false` and CI-only runners can opt in with `container.sweep: true`.

Fixes https://gitea.com/gitea/runner/issues/1385

Reviewed-on: https://gitea.com/gitea/runner/pulls/1389
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-05 13:50:46 +00:00
silverwind 95ef17d022 fix: prevent masking gaps in workflow command logs (#1384)
Mask partially escaped workflow command output and complete one-character values while preserving literal output and stream line endings.

Fixes: https://gitea.com/gitea/runner/issues/300
Reviewed-on: https://gitea.com/gitea/runner/pulls/1384
Reviewed-by: bircni <[email protected]>
2026-11-05 08:01:44 +00:00
Lunny Xiaoandsilverwind ee03687688 fix: upgrade Alpine packages in the runner images (#1388)
All images run `apk upgrade` before installing packages, so they pick up Alpine fixes their base images lag on. `docker:29.9.2-dind` shipped libexpat 2.9.0-r0 while Alpine 3.24 provides 2.10.0-r0 with the security fixes, and both `docker:29.9.3-dind` and `alpine:3.24` still ship apk-tools 3.0.8-r0 where 3.0.9-r0 is available. The dind bases are also bumped to `docker:29.9.3`, which drops several vulnerable bundled binaries' findings.

Verified by building the targets locally: libexpat is 2.10.0-r0 (was 2.9.0-r0), apk-tools is 3.0.9-r0 (was 3.0.8-r0), and Trivy reports no Alpine package findings on the dind images.

Assisted-by: Codet:claude
Assisted-by: Claude Code:claude-opus-5-5
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1388
Reviewed-by: silverwind <[email protected]>
Co-authored-by: Lunny Xiao <[email protected]>
2026-11-05 07:45:27 +00:00
silverwind 30e600d8f6 test: speed up (#1386)
Reduce redundant fixture setup, Git subprocesses, and large data copies while preserving test coverage.

Local macOS measurements against main:

| Measurement | Before | After |
| --- | ---: | ---: |
| Git package time, median of 3 alternating runs | 4.09 s | 3.48 s (-15%) |
| Allocations across 18 action-bundle cases | 489 MiB | 251 MiB (-49%) |
| Traced Git starts per checkout table run | 234 | 175 |

Reviewed-on: https://gitea.com/gitea/runner/pulls/1386
Reviewed-by: bircni <[email protected]>
2026-11-05 07:33:17 +00:00
silverwind 664978c819 test: fix windows path test (#1387)
`TestCopyWorkflowFilesHandsNodeTheRunnerCAUnlessTheJobOrImageSetsIt` from https://gitea.com/gitea/runner/pulls/1366 fails on Windows. `HostEnvironment.GetActPath` turns backslashes into slashes there, so `NODE_EXTRA_CA_CERTS` gets the slash form while the test expects the raw temp dir path. The runner behaviour is correct, only the expectation is wrong.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1387
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-05 07:14:56 +00:00
aa56f46c89 enhance: trust a private CA with runner.ca_file and add container.volumes (#1366)
Adds `container.volumes`, mounted into every Docker job, service and action container. Sources are paths on the Docker daemon's host and need no `valid_volumes` entry, and workflow volumes may use them only with the same access.

Adds `runner.ca_file`, a private CA trusted for the Gitea instance besides the system's by the runner's connections and its artifact proxy. Runner-side git and `builtin:checkout` use it as their CA bundle for the instance, persisted with the checkout's credentials. Jobs get it at `/var/run/act/ca.crt` with `NODE_EXTRA_CA_CERTS` pointing there unless the job or its image sets it. Git in jobs, as run by `actions/checkout`, needs the CA in the image's bundle, for example through `container.volumes`.

Fixes: https://gitea.com/gitea/runner/issues/568
Fixes: https://gitea.com/gitea/runner/issues/671
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1366
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-05 05:28:40 +00:00
silverwindandLunny Xiao 7a9fccb842 ci: sign release checksums with the imported GPG key (#1382)
The release job has imported the Teabot GPG key since https://gitea.com/gitea/runner/pulls/116, but `.goreleaser.yaml` never had a `signs:` block, so releases were never signed. Add it so `checksums.txt` gets a detached `checksums.txt.sig`, the same way https://gitea.com/gitea/cli signs its releases.

The nightly job imports the key as well since the `signs:` block applies to nightly builds, and the R2 publisher mirrors the signature to dl.gitea.com.

_Written by Claude (Opus 5.5)._

---------

Co-authored-by: Lunny Xiao <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1382
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-05 04:28:22 +00:00
silverwindandLunny Xiao 7d6e1ff021 test: fix TestRunExecListUsesJobEventAndAllPlans hang on macOS (#1378)
`TestRunExecListUsesJobEventAndAllPlans` hangs until the test timeout on macOS.

Regression from https://gitea.com/gitea/runner/pulls/1338, which added a fifth `runExecList` call to the test to cover `exec --list --detect-event` without events. That took the captured output from 498 to 560 bytes. `captureStdout` swapped `os.Stdout` for a pipe it only read after the callback returned, and on macOS 27.2 an unread pipe holds 512 bytes, so the write blocked forever. Linux pipes hold 64 KiB, so was unaffected.

`printList` and `runExecList` now write to a passed writer, `exec --list` hands them `cmd.OutOrStdout()`, and the tests pass a buffer, so no test swaps `os.Stdout` anymore.

---------

Co-authored-by: Lunny Xiao <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1378
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-05 04:22:01 +00:00
silverwind 319567213d docs: use latest image tags in examples (#1383)
The README's Docker example and the Docker and Kubernetes examples pointed at `nightly` images, while the README recommends `latest-dind`. Deployments copied from them should run the latest release, not a build of `main`.

None of the examples changed since v5.3.0, so they work as written with the current `latest` tags.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1383
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 23:00:45 +00:00
silverwind 0093731295 chore(deps): update go.yaml.in/yaml/v4 to v4.0.0 (#1381)
Update go-yaml from the rc.6 pseudo-version to the stable v4.0.0 release, and actionslib to v1.5.2, which does the same in https://gitea.com/gitea/actionslib/pulls/32.

v4 accepts `:` as a mapping with an empty key, so the reusable workflow tests now use an unclosed `[` as their invalid workflow. Gitea counterpart: https://github.com/go-gitea/gitea/pull/40222.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1381
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 20:05:17 +00:00
silverwind 3e1b4b42c2 fix(cache): stop the cache server from running its volume out of space (#1377)
Committing a cache upload copied every part into a new archive before deleting the parts, so a 5 GB save needed 10 GB free. Failed writes and assemblies left their partial bytes until the next stale sweep, up to an hour later, and every other save on the runner failed meanwhile. Nothing made room for an upload, so once the cache had filled its volume down to the free space floor, larger saves kept failing.

- Assemble in place, deleting each part once appended.
- Drop partial parts and archives on failure.
- Before a part would cross the floor, evict the least recently used entries.
- Reject uploads to reservations the sweep already deleted, which left files nothing removed.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1377
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 16:38:21 +00:00
silverwind dcba546cb3 fix: align process handling and fix Windows crashes (#1376)
The intermittent Windows test crashes came from step errors formatting a nil `rc.Run`. `fmt` recovers the nil dereference, but on CPUs with AMX the recovered fault overflows into the heap (https://github.com/golang/go/issues/81238). The errors now name the job, and the Windows test VM hides AMX and runs on UTC, so cached boots no longer jump back 8 hours mid-test.

Windows commands now start suspended and resume once they are in their job objects, so no child escapes them. That drops the process scan reading other processes' memory and the last `unsafe` code, which `depguard` now denies. Step processes outlive a crashed runner, and commands are no longer looked up in the runner's working directory, both as with GitHub's runner.

Like GitHub's runner, host jobs on Linux and macOS now end the processes carrying the job's `RUNNER_TRACKING_ID`, and image pulls and Dockerfile action builds are retried.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1376
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 16:32:22 +00:00
silverwind 142d9f8c88 fix: give each Dockerfile action image its own ID (#1372)
Identical Dockerfile builds in concurrent jobs share one image ID under their per-job tags. When a job ends and removes its tag, the daemon deletes the image while another job's build is about to tag it, failing that job with `No such image`. This also hits `TestRunEvent` in CI, where two parallel subtests build the same action.

Label each build with its tag, so every job gets its own image on top of the shared cached layers and removing it never touches another job's.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1372
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 07:08:45 +00:00
silverwindandbircni 54b2cb944c fix: give git-lfs the checkout token once (#1371)
`builtin:checkout` set the token both in its persisted credentials file and in the step's environment. git-lfs before 3.8.0 then sent `Authorization` twice on the LFS batch request, which nginx rejects with 400.

Write the credentials file once the checkout succeeded, since it only serves later steps. A failed checkout no longer persists credentials.

Users can also upgrade git-lfs to fix the root bug, but this fix is worth to keep regardless.

Fixes: https://gitea.com/gitea/runner/issues/1370
Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1371
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 06:42:00 +00:00
Zettat123andsilverwind b65fd0f8f8 ci: run the Windows tests on a runner with KVM (#1369)
Run the Windows tests on `ubuntu-latest-kvm` as a PR job, with the installed Windows cached, minutes instead of ~1h40m. Aligned with https://gitea.com/gitea/gitea-mcp/pulls/321 and https://gitea.com/gitea/cli/pulls/79.

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1369
Reviewed-by: silverwind <[email protected]>
Co-authored-by: Zettat123 <[email protected]>
2026-11-04 02:08:38 +00:00
151 changed files with 2820 additions and 2794 deletions

No files matched your search

+9 -1
View File
@@ -34,6 +34,13 @@ jobs:
with:
go-version-file: "go.mod"
check-latest: true
- name: Import GPG key
id: import_gpg
uses: crazy-max/ghaction-import-gpg@2dc316deee8e90f13e1a351ab510b4d5bc0c82cd # v7
with:
gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
passphrase: ${{ secrets.PASSPHRASE }}
fingerprint: CC64B1DB67ABBEECAB24B6455FC346329753F4B0
- name: goreleaser
uses: goreleaser/goreleaser-action@a4f02ab5355a589de60d917a659ce617ed19b3de # v7
with:
@@ -47,6 +54,7 @@ jobs:
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
GORELEASER_FORCE_TOKEN: "gitea"
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GPG_FINGERPRINT: ${{ steps.import_gpg.outputs.fingerprint }}
release-image:
runs-on: ubuntu-latest
@@ -79,7 +87,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
- name: Login to DockerHub
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
+1 -1
View File
@@ -82,7 +82,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
- name: Login to DockerHub
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
-21
View File
@@ -1,21 +0,0 @@
name: windows
on:
schedule:
- cron: '0 3 * * 1'
workflow_dispatch:
env:
DOCKER_CONFIG: /tmp/docker-noauth # see test.yml
jobs:
test:
name: test on windows
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: ./.gitea/actions/go-setup
- name: prepare anonymous docker config
run: mkdir -p "$DOCKER_CONFIG" && echo '{}' > "$DOCKER_CONFIG/config.json"
- name: test on windows
run: make test-windows
+26
View File
@@ -57,6 +57,32 @@ jobs:
make coverage-report
cat .tmp/coverage.md >> "$GITHUB_STEP_SUMMARY"
windows:
name: test on windows
runs-on: ubuntu-latest-kvm
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: ./.gitea/actions/go-setup
- name: prepare anonymous docker config
run: mkdir -p "$DOCKER_CONFIG" && echo '{}' > "$DOCKER_CONFIG/config.json"
- name: windows vm cache key
id: windows-vm
run: echo "key=windows-vm-$(grep '^WINDOWS_TEST_IMAGE' Makefile | cat - scripts/test-windows.sh scripts/test-windows-setup.ps1 | sha256sum | cut -c1-16)" >> "$GITHUB_OUTPUT"
- uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/windows-vm
key: ${{ steps.windows-vm.outputs.key }}
- name: test on windows
id: test-windows
run: WINDOWS_TEST_CACHE="$HOME/.cache/windows-vm" make test-windows
env:
WINDOWS_TEST_TIMEOUT: 45
- if: always() && steps.test-windows.outputs.save-cache == 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/windows-vm
key: ${{ steps.windows-vm.outputs.key }}
e2e:
name: gitea compatibility (${{ matrix.gitea_image }})
runs-on: ubuntu-latest
+2
View File
@@ -39,6 +39,8 @@ linters:
deny:
- pkg: golang.org/x/exp
desc: it's experimental and unreliable
- pkg: unsafe
desc: use typed APIs instead
logging:
files:
- $all
+5
View File
@@ -80,6 +80,7 @@ builds:
publishers:
- name: cloudflare-r2
checksum: true
signature: true
extra_files:
- glob: ./**.xz
- glob: ./**.sha256
@@ -100,6 +101,10 @@ checksum:
extra_files:
- glob: ./**.xz
signs:
- artifacts: checksum
args: ["--batch", "-u", "{{ .Env.GPG_FINGERPRINT }}", "--output", "${signature}", "--detach-sign", "${artifact}"]
snapshot:
version_template: "{{ .Branch }}-devel"
+9 -10
View File
@@ -23,11 +23,14 @@ depending on the prefix:
covers both forms: it escapes `%` first, so a literal `%3B` becomes `%253B` that the extra
`##[…]` rules cannot match, and a raw `;` or `]` is never decoded. It is also what makes
multi-line work, `\n` becomes `%0A` and the UI turns it back into a line break.
- **Forwarding a command from step output?** Leave the payload alone, it arrived escaped and is
decoded once. Decoding here double-decodes and destroys multi-line.
- **Forwarding a command from step output?** Preserve its escaping unless masking changes it.
Decoding for display happens in the UI.
- **No prefix?** Do not escape, and split multi-line values into one row each.
- **Interpolating a secret?** Masking runs after escaping, so `AppendSecretMasker` registers the
encoded forms too.
- **Interpolating a secret?** Masking runs after escaping. Recognised command payloads are
normalised before masking to cover partially escaped values. Their original escaping is
preserved when no replacement occurs. `AppendSecretMasker` also registers encoded forms for
other output. Complete nonblank one-character values are masked, while one-byte fragments
of multiline values are skipped.
- Command *properties* also escape `%3A` and `%2C`, which the UI never decodes, so the command
handler folds an annotation's location into its decoded message and escapes the result again.
@@ -48,11 +51,6 @@ fields.
- Tests read what was logged with `logtest.NewLogger()`, or `logtest.SetDefault(t)` for the
package functions.
## Built-in actions
Each lives in `internal/pkg/action/<name>` and is registered in `builtinActions` in
`act/runner/step_action_builtin.go`.
## End-to-end compatibility tests
`make test-e2e` runs the runner against `E2E_GITEA_IMAGE`. It defaults to the nightly image.
@@ -77,4 +75,5 @@ repository runners. A run path is `<workflow>@<ref>` and a log row is `<timestam
The host must be amd64 and allow privileged containers. `WINDOWS_TEST_TIMEOUT` sets the minutes
allowed for installing Windows and running the tests, 165 by default. The VM uses KVM when the host
has `/dev/kvm`, otherwise Windows is emulated and much slower. Docker-backed tests skip in the VM
and the race detector is off, it needs cgo. CI runs this weekly and on demand.
and the race detector is off, it needs cgo. `WINDOWS_TEST_CACHE` names a directory to keep the
installed Windows in, so later runs skip the install. CI runs this on every pull request.
+7 -5
View File
@@ -19,14 +19,15 @@ RUN --mount=type=cache,target=/go/pkg/mod \
### DIND VARIANT
#
#
FROM docker:29.9.2-dind AS dind
FROM docker:29.9.3-dind AS dind
ARG VERSION=dev
LABEL org.opencontainers.image.source="https://gitea.com/gitea/runner"
LABEL org.opencontainers.image.version="${VERSION}"
RUN apk add --no-cache s6 bash git tzdata nftables
# upgrade first: the docker base image can lag behind Alpine security fixes
RUN apk upgrade --no-cache && apk add --no-cache s6 bash git tzdata nftables
COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner
COPY scripts/run.sh /usr/local/bin/run.sh
@@ -41,7 +42,7 @@ ENTRYPOINT ["s6-svscan","/etc/s6"]
### DIND-ROOTLESS VARIANT
#
#
FROM docker:29.9.2-dind-rootless AS dind-rootless
FROM docker:29.9.3-dind-rootless AS dind-rootless
ARG VERSION=dev
@@ -49,7 +50,8 @@ LABEL org.opencontainers.image.source="https://gitea.com/gitea/runner"
LABEL org.opencontainers.image.version="${VERSION}"
USER root
RUN apk add --no-cache s6 bash git tzdata nftables
# upgrade first: the docker base image can lag behind Alpine security fixes
RUN apk upgrade --no-cache && apk add --no-cache s6 bash git tzdata nftables
COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner
COPY scripts/run.sh /usr/local/bin/run.sh
@@ -75,7 +77,7 @@ ARG VERSION=dev
LABEL org.opencontainers.image.source="https://gitea.com/gitea/runner"
LABEL org.opencontainers.image.version="${VERSION}"
RUN apk add --no-cache tini bash git tzdata
RUN apk upgrade --no-cache && apk add --no-cache tini bash git tzdata
COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner
COPY scripts/run.sh /usr/local/bin/run.sh
+4 -4
View File
@@ -3,11 +3,11 @@ GO ?= go
EXECUTABLE := gitea-runner$(shell $(GO) env GOEXE)
HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" )
PODMAN_TEST_IMAGE ?= quay.io/containers/podman:v5.8.9-immutable@sha256:87f42f37fa85b421664c30b8f5f9cfc23920057f5e5fcadf0b6f68d9edc6d415 # renovate: datasource=docker
E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:657378e5cfd898e36f820459c60ef8e69be8fcda9fd354e4381a8b654d091bbc # renovate: datasource=docker
PODMAN_TEST_IMAGE ?= quay.io/containers/podman:v6.1.3-immutable@sha256:1d844fd1fc6ce68c1e72ae19550cc3caef039306219ee007d88f7e67c5df4dd1 # renovate: datasource=docker
E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:139605d4a6ab61e48dfd7687b11184881c2d7fcf06bf739e9cadb34f73deb51d # renovate: datasource=docker
SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker
E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly
E2E_K3S_IMAGE ?= rancher/k3s:v1.37.1-k3s1@sha256:ca7f37d993d82ef0dcdcfecb2e0e2618ea541dbaffc620c8cedebe01a82acd0d # renovate: datasource=docker
E2E_K3S_IMAGE ?= rancher/k3s:v1.37.2-k3s1@sha256:21ea6e0fac4e3930c2ce8aa40ddd41a8b18cb3e9a0222c030dc005e7e88c4332 # renovate: datasource=docker
WINDOWS_TEST_IMAGE ?= dockurr/windows:6.06@sha256:16fd7c39a3e48bb946d29c59dc6015942e5b900a1fa5d7e5552053bfc48ee2ea # renovate: datasource=docker
GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*")
@@ -153,7 +153,7 @@ test-dind: ## run the daemon-facing tests against the built dind image or podman
.PHONY: test-windows
test-windows: ## run the tests on Windows Server Core in a QEMU VM on the docker host
@WINDOWS_TEST_IMAGE=$(WINDOWS_TEST_IMAGE) ./scripts/test-windows.sh
@GO=$(GO) WINDOWS_TEST_IMAGE=$(WINDOWS_TEST_IMAGE) ./scripts/test-windows.sh
E2E_CONCURRENCY ?= 8
+6 -12
View File
@@ -7,6 +7,7 @@ Gitea Runner runs the jobs of Gitea Actions workflows. Register it with a Gitea
### Prerequisites
- Linux, macOS or Windows, or FreeBSD.
- Gitea 1.26.2 or newer.
- `git` 2.34.1 or newer.
- `docker`, or `podman` with `podman.socket` enabled, for `docker` labels and Docker actions. See Docker's [install instructions](https://docs.docker.com/engine/install/).
@@ -14,9 +15,6 @@ In the job image, or on the host for host jobs:
- `bash` or `sh`, or `pwsh` or `powershell` on Windows.
- `node` for JavaScript actions and `hashFiles()`.
- `git` 2.34.1 or newer for [`builtin:checkout`](docs/builtin-actions.md).
- `git-lfs` for `builtin:checkout` with `lfs: true`.
- `ssh` for `builtin:checkout` with `ssh-key`.
Kubernetes mode needs [a few more tools](docs/kubernetes.md) in job images.
@@ -95,7 +93,7 @@ The recommended Docker-in-Docker image runs its own Docker daemon for jobs. Jobs
```bash
docker run --privileged -e GITEA_INSTANCE_URL=https://your_gitea.com -e GITEA_RUNNER_REGISTRATION_TOKEN=<your_token> \
-v "$PWD/data:/data" -v runner-docker:/var/lib/docker --name my_runner gitea/runner:nightly-dind
-v "$PWD/data:/data" -v runner-docker:/var/lib/docker --name my_runner gitea/runner:latest-dind
```
The volumes keep the registration and the image cache across container recreation. The image reads more variables, such as `GITEA_RUNNER_LABELS` and `CONFIG_FILE`, see [scripts/run.sh](scripts/run.sh) and the [Docker example](examples/docker/README.md).
@@ -112,7 +110,7 @@ Versioned and `nightly` tags take the same suffixes, like `nightly-dind`. The im
### Example deployments
See the [examples](examples) directory for more deployments, among them systemd, Kubernetes and VMs.
See the [examples](examples) directory for more deployments, among them systemd, launchd on macOS, Kubernetes and VMs.
## Configuration
@@ -164,7 +162,7 @@ ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest,macos:host
[gitea/runner-images](https://gitea.com/gitea/runner-images) has suitable images. Jobs matching no label run in `runner.default_image`, or on the host when docker is unavailable.
Host jobs still need a Docker daemon for Docker actions and `docker://` steps. Windows hosts do not support those, as on GitHub. On Windows, a step's background processes end with the job unless it opts out, see [host mode](docs/host-mode.md).
Host jobs still need a Docker daemon for Docker actions and `docker://` steps. Windows hosts do not support those, as on GitHub. A step's background processes end with the job unless it opts out, see [host mode](docs/host-mode.md).
#### Registration vs config labels
@@ -183,7 +181,7 @@ Setup actions like `setup-go` install tools into `RUNNER_TOOL_CACHE`. `runner.to
| Mode | Tool cache |
| --- | --- |
| `none` (default) | Provided by the job image. Versions it lacks are downloaded in every job. |
| `shared` | One volume for all jobs. Only safe with `runner.capacity: 1`. |
| `shared` | One volume per `runner.capacity` slot, reused by later jobs. It hides the tools later job images ship, so use images without them. |
To ship tools in a job image, install them to `/opt/hostedtoolcache/<tool>/<version>/<arch>` with an empty `<arch>.complete` file next to it, as the `-full` [runner images](https://gitea.com/gitea/runner-images) do:
@@ -243,11 +241,7 @@ volumes:
- ${GITEA_DOCKER_WORKSPACE:-.}/data:/app/data
```
In most Linux container jobs, the containers, networks and volumes a job creates are removed when it ends, `docker compose` services included. Set `container.sweep: false` if jobs deploy services to the runner's Docker daemon.
### Built-in actions (`uses: builtin:<name>`)
`uses: builtin:checkout` checks out a repository without downloading `actions/checkout` or needing Node in the job image, see [docs/builtin-actions.md](docs/builtin-actions.md).
Set `container.sweep: true` to remove the containers, networks and volumes most Linux container jobs create when they end, `docker compose` services included. Leave it off if jobs deploy services to the runner's Docker daemon.
### Caching (`actions/cache`)
+88 -32
View File
@@ -62,6 +62,7 @@ type JobCredential struct {
Ref string `json:"ref"`
BaseRef string `json:"base_ref"`
DefaultRef string `json:"default_ref"`
CacheMode string `json:"cache_mode"` // empty allows everything
// Results is the instance whose artifact service this server forwards for the job, and
// InsecureTLS how the runner reaches it; see results.go.
@@ -104,7 +105,10 @@ type Handler struct {
stopSync context.CancelFunc
syncing sync.WaitGroup
outboundIP string
outboundIPMu sync.RWMutex
outboundIP string
detectOutboundIP func() net.IP // nil when the address is configured
instanceTransport http.RoundTripper
// internalSecret guards /_internal/{register,revoke}. When set, a remote
// runner can use these endpoints to pre-register per-job
@@ -124,7 +128,7 @@ type Handler struct {
policy Policy
// freeDisk is a field so tests can drive evictForFreeSpace without a full volume.
// freeDisk is a field so tests can drive makeRoom without a full volume.
freeDisk func(string) (uint64, error)
}
@@ -143,19 +147,22 @@ type Options struct {
// in-process RegisterJob method directly.
InternalSecret string
Policy Policy
Logger *logging.Logger
Policy Policy
Logger *logging.Logger
InstanceTransport http.RoundTripper // reaches the instance artifact requests are forwarded to, nil for http.DefaultTransport
}
// StartHandler starts an embedded cache or forwards cache requests to Upstream.
func StartHandler(opts Options) (*Handler, error) {
dir, logger := opts.Dir, opts.Logger
h := &Handler{
creds: make(map[string]*credEntry),
internalSecret: opts.InternalSecret,
policy: opts.Policy.withDefaults(),
freeDisk: disk.FreeBytes,
dbLock: make(chan struct{}, 1),
creds: make(map[string]*credEntry),
internalSecret: opts.InternalSecret,
policy: opts.Policy.withDefaults(),
freeDisk: disk.FreeBytes,
dbLock: make(chan struct{}, 1),
outboundIP: opts.OutboundIP,
instanceTransport: opts.InstanceTransport,
}
if logger == nil {
@@ -164,12 +171,11 @@ func StartHandler(opts Options) (*Handler, error) {
logger = logger.With("module", "artifactcache")
h.logger = logger
if opts.OutboundIP != "" {
h.outboundIP = opts.OutboundIP
} else if ip := common.GetOutboundIP(); ip == nil {
return nil, errors.New("unable to determine outbound IP address")
} else {
h.outboundIP = ip.String()
if opts.OutboundIP == "" {
h.detectOutboundIP = common.GetOutboundIP
if h.RefreshExternalURL() == "" {
logger.Warn("unable to determine an outbound IP address for the cache server yet, set cache.host to configure one")
}
}
if opts.Upstream != "" {
@@ -213,10 +219,10 @@ func StartHandler(opts Options) (*Handler, error) {
}
router := http.NewServeMux()
router.HandleFunc("GET "+apiPath+"/cache", h.bearerAuth(h.find))
router.HandleFunc("POST "+apiPath+"/caches", h.bearerAuth(h.reserve))
router.HandleFunc("PATCH "+apiPath+"/caches/{id}", h.bearerAuth(h.upload))
router.HandleFunc("POST "+apiPath+"/caches/{id}", h.bearerAuth(h.commit))
router.HandleFunc("GET "+apiPath+"/cache", h.cacheModeAuth(false, h.find))
router.HandleFunc("POST "+apiPath+"/caches", h.cacheModeAuth(true, h.reserve))
router.HandleFunc("PATCH "+apiPath+"/caches/{id}", h.cacheModeAuth(true, h.upload))
router.HandleFunc("POST "+apiPath+"/caches/{id}", h.cacheModeAuth(true, h.commit))
router.HandleFunc("POST "+apiPath+"/clean", h.bearerAuth(h.clean))
// Artifact GET is signed via query-string HMAC because @actions/cache
// does not attach Authorization when downloading archiveLocation.
@@ -301,11 +307,36 @@ func (h *Handler) cacheForwarder(upstream string) (http.Handler, error) {
}), nil
}
// ExternalURL is "" while no address is known.
func (h *Handler) ExternalURL() string {
// TODO: make the external url configurable if necessary
if h == nil {
return ""
}
h.outboundIPMu.RLock()
defer h.outboundIPMu.RUnlock()
if h.outboundIP == "" {
return ""
}
return "http://" + net.JoinHostPort(h.outboundIP, strconv.Itoa(h.port))
}
// RefreshExternalURL re-detects an unconfigured address, which can change after startup.
func (h *Handler) RefreshExternalURL() string {
if h != nil && h.detectOutboundIP != nil {
if ip := h.detectOutboundIP(); ip != nil {
h.outboundIPMu.Lock()
if address := ip.String(); address != h.outboundIP {
h.outboundIP = address
if ip.To4() == nil {
h.logger.Warnf("detected IPv6 address %s for the cache server, jobs on IPv4-only networks cannot reach it, set cache.host if they fail to", address)
}
}
h.outboundIPMu.Unlock()
}
}
return h.ExternalURL()
}
func (h *Handler) baseURL(cred JobCredential) string {
if base := strings.TrimRight(cred.PublicURL, "/"); base != "" {
return base
@@ -773,6 +804,7 @@ func (h *Handler) upload(w http.ResponseWriter, r *http.Request) {
h.responseJSON(w, r, 400, err)
return
}
h.makeRoom(r.ContentLength)
if err := h.storage.Write(cache.ID, start, r.Body); err != nil {
h.responseJSON(w, r, 500, err)
return
@@ -875,11 +907,8 @@ func (h *Handler) commitCache(cache *Cache) error {
if h.s3 != nil {
return nil // lifecycle rules on the bucket bound its size
}
// A commit is the only thing that grows the store, so the only thing that can push the
// volume under the floor.
h.evictRepo(db, cache.Repo)
h.evictTotal(db)
h.evictForFreeSpace(db)
return nil
}
@@ -928,6 +957,30 @@ func (h *Handler) bearerAuth(handler http.HandlerFunc) http.HandlerFunc {
}
}
// cacheModeAuth is bearerAuth refusing what the job's cache-mode forbids, in the shapes the toolkit reports as a warning.
func (h *Handler) cacheModeAuth(write bool, handler http.HandlerFunc) http.HandlerFunc {
return h.bearerAuth(func(w http.ResponseWriter, r *http.Request) {
mode := credFromContext(r.Context()).CacheMode
access, forbidden := "read", "write-only"
if write {
access, forbidden = "write", "read"
}
if mode != "none" && mode != forbidden {
handler(w, r)
return
}
message := fmt.Sprintf("cache %s denied: the job's cache-mode is %s", access, mode)
switch {
case !strings.HasPrefix(r.URL.Path, cacheServiceV2Path):
h.responseJSON(w, r, http.StatusForbidden, map[string]any{"message": message})
case write:
h.responseJSON(w, r, http.StatusOK, map[string]any{"ok": false, "message": message})
default:
h.twirpError(w, r, twirpDenied, errors.New(message))
}
})
}
func (h *Handler) unauthorized(w http.ResponseWriter, r *http.Request, err error) {
if strings.HasPrefix(r.URL.Path, cacheServiceV2Path) {
h.twirpError(w, r, twirpUnauthenticated, err)
@@ -1173,9 +1226,6 @@ func (h *Handler) touchCache(id uint64, requireIncomplete bool) error {
defer db.Close()
cache, err := db.get(id)
if errors.Is(err, errCacheNotFound) {
return nil
}
if err != nil {
return err
}
@@ -1260,23 +1310,29 @@ func (h *Handler) gcCache() {
h.evictExpired(db)
h.evictSuperseded(db)
h.evictOversized(db)
h.evictForFreeSpace(db)
_ = db.Close()
h.makeRoom(0)
}
// evictForFreeSpace bounds the volume itself, so it also covers bytes the cache never
// accounted for.
func (h *Handler) evictForFreeSpace(db *cacheDB) {
// makeRoom evicts the least recently used entries to keep need bytes above the floor, as GitHub saves new over old.
func (h *Handler) makeRoom(need int64) {
free, err := h.freeDisk(h.dir)
if err != nil {
h.logger.Debugf("free disk check: %v", err) // unsupported platform, treat as unavailable rather than full
return
}
if free >= uint64(h.policy.MinFreeDisk) {
shortfall := h.policy.MinFreeDisk + max(need, 0) - int64(free)
if h.s3 != nil || shortfall <= 0 {
return
}
db, err := h.openDB()
if err != nil {
return
}
defer db.Close()
caches := h.completedByUse(db)
total, shortfall := totalSize(caches), h.policy.MinFreeDisk-int64(free)
total := totalSize(caches)
if total <= shortfall {
// Say so, or shedding everything and still being short reads as the backstop working.
h.logger.Warnf("cache volume is %d MiB short of the free space floor with only %d MiB of cache on it; something else is filling it", shortfall/miB, total/miB)
+72 -23
View File
@@ -11,6 +11,7 @@ import (
"errors"
"fmt"
"io"
"net"
"net/http"
"net/http/httptest"
"os"
@@ -18,9 +19,12 @@ import (
"strconv"
"strings"
"testing"
"testing/iotest"
"testing/synctest"
"time"
"gitea.com/gitea/runner/act/common/logging/logtest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.etcd.io/bbolt"
@@ -49,7 +53,23 @@ func signArtifactURL(h *Handler, id int64) string {
func TestHandler_ExternalURL(t *testing.T) {
t.Parallel()
assert.Equal(t, "http://[2001:db8::1]:8080", (&Handler{outboundIP: "2001:db8::1", port: 8080}).ExternalURL())
assert.Empty(t, (*Handler)(nil).RefreshExternalURL())
assert.Equal(t, "http://cache.example:8080", (&Handler{outboundIP: "cache.example", port: 8080}).RefreshExternalURL())
logger, logs := logtest.NewLogger()
var ip net.IP
handler := &Handler{port: 8080, logger: logger, detectOutboundIP: func() net.IP { return ip }}
assert.Empty(t, handler.RefreshExternalURL())
ip = net.ParseIP("2001:db8::1")
assert.Equal(t, "http://[2001:db8::1]:8080", handler.RefreshExternalURL())
assert.Equal(t, "http://[2001:db8::1]:8080", handler.RefreshExternalURL())
ip = net.ParseIP("192.0.2.1")
assert.Equal(t, "http://[2001:db8::1]:8080", handler.ExternalURL())
assert.Equal(t, "http://192.0.2.1:8080", handler.RefreshExternalURL())
ip = nil
assert.Equal(t, "http://192.0.2.1:8080", handler.RefreshExternalURL())
require.Len(t, logs.AllRecords(), 1)
assert.Contains(t, logs.LastRecord().Message, "IPv6")
}
func TestHandler_Upstream(t *testing.T) {
@@ -673,7 +693,7 @@ func TestHandler_evictPolicy(t *testing.T) {
}
}
func TestHandler_evictForFreeSpace(t *testing.T) {
func TestHandler_makeRoom(t *testing.T) {
t.Parallel()
free := func(n int64) func(string) (uint64, error) {
return func(string) (uint64, error) { return uint64(n), nil }
@@ -682,14 +702,16 @@ func TestHandler_evictForFreeSpace(t *testing.T) {
for _, tc := range []struct {
name string
freeDisk func(string) (uint64, error)
upload int64
kept []string
}{
{"ample free space evicts nothing", free(defaultMinFreeDisk), []string{"oldest", "middle", "newest"}},
{"a small shortfall sheds one entry", free(defaultMinFreeDisk - 4*miB), []string{"middle", "newest"}},
{"a shortfall the cache cannot cover sheds all of it", free(0), nil},
{"ample free space evicts nothing", free(defaultMinFreeDisk), 0, []string{"oldest", "middle", "newest"}},
{"room for an upload sheds the oldest entry", free(defaultMinFreeDisk), 4 * miB, []string{"middle", "newest"}},
{"a shortfall the cache cannot cover sheds all of it", free(0), 0, nil},
{
"an unreadable volume is treated as unavailable, not as full",
func(string) (uint64, error) { return 0, errors.New("unsupported") },
0,
[]string{"oldest", "middle", "newest"},
},
} {
@@ -702,11 +724,7 @@ func TestHandler_evictForFreeSpace(t *testing.T) {
}
handler := newTestHandler(t, Policy{}, entries...)
handler.freeDisk = tc.freeDisk
db, err := handler.openDB()
require.NoError(t, err)
handler.evictForFreeSpace(db)
require.NoError(t, db.Close())
handler.makeRoom(tc.upload)
assert.ElementsMatch(t, tc.kept, keptKeys(t, handler, entries))
})
@@ -852,6 +870,9 @@ func TestHandler_touchCacheSkipsFreshCompletedEntries(t *testing.T) {
func newTestHandler(t *testing.T, policy Policy, entries ...*Cache) *Handler {
t.Helper()
handler := startTestHandler(t, Options{Policy: policy})
if len(entries) == 0 {
return handler
}
db, err := handler.openDB()
require.NoError(t, err)
@@ -888,6 +909,13 @@ func keptKeys(t *testing.T, handler *Handler, entries []*Cache) []string {
return kept
}
func TestStorageDropsFailedWrites(t *testing.T) {
storage, err := NewStorage(t.TempDir())
require.NoError(t, err)
require.Error(t, storage.Write(1, 0, io.MultiReader(strings.NewReader("partial"), iotest.ErrReader(io.ErrUnexpectedEOF))))
assert.NoFileExists(t, storage.tempName(1, 0))
}
func writeBlob(t *testing.T, handler *Handler, id uint64) {
t.Helper()
require.NoError(t, handler.storage.Write(id, 0, strings.NewReader("a")))
@@ -934,6 +962,10 @@ func TestHandler_RejectsBearer(t *testing.T) {
func TestHandler_UnregisterRevokes(t *testing.T) {
t.Parallel()
handler, err := StartHandler(Options{Dir: filepath.Join(t.TempDir(), "artifactcache")})
require.NoError(t, err)
defer handler.Close()
for _, tc := range []struct {
name string
token string
@@ -944,9 +976,6 @@ func TestHandler_UnregisterRevokes(t *testing.T) {
{"reference counted", "shared", "v", 2},
} {
t.Run(tc.name, func(t *testing.T) {
handler, err := StartHandler(Options{Dir: filepath.Join(t.TempDir(), "artifactcache")})
require.NoError(t, err)
defer handler.Close()
revokers := make([]func(), tc.registrations)
for i := range revokers {
revokers[i] = handler.RegisterJob(tc.token, JobCredential{Repo: testRepo})
@@ -974,7 +1003,7 @@ func TestHandler_UnregisterRevokes(t *testing.T) {
}
}
func TestHandler_CrossRepoIsolation(t *testing.T) {
func TestHandler_CrossRepoAndCacheModeIsolation(t *testing.T) {
t.Parallel()
dir := filepath.Join(t.TempDir(), "artifactcache")
handler, err := StartHandler(Options{Dir: dir})
@@ -983,6 +1012,8 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"})
handler.RegisterJob("token-a-pull", JobCredential{Repo: "owner/repoA", Ref: "refs/pull/1/head"})
handler.RegisterJob("token-a-read", JobCredential{Repo: "owner/repoA", CacheMode: "read"})
handler.RegisterJob("token-a-write-only", JobCredential{Repo: "owner/repoA", CacheMode: "write-only"})
base := handler.ExternalURL() + apiPath
key := "shared-key"
@@ -1012,12 +1043,31 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
resp.Body.Close()
assert.Equal(t, http.StatusNoContent, resp.StatusCode)
resp, err = clientA.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version))
require.NoError(t, err)
resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
clientRead := &http.Client{Transport: &bearerTransport{token: "token-a-read"}}
for _, client := range []*http.Client{clientA, clientRead} {
resp, err = client.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version))
require.NoError(t, err)
resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
}
for _, client := range []*http.Client{clientB, {Transport: &bearerTransport{token: "token-a-pull"}}} {
requireDenied := func(prefix string, do func() (*http.Response, error)) {
resp, err := do()
require.NoError(t, err)
defer resp.Body.Close()
denied := map[string]string{}
require.NoError(t, json.UnmarshalRead(resp.Body, &denied))
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
assert.True(t, strings.HasPrefix(denied["message"], prefix), denied["message"])
}
requireDenied("cache write denied:", func() (*http.Response, error) {
return clientRead.Post(base+"/caches", "application/json", strings.NewReader(`{"key":"new-key","version":"`+version+`"}`))
})
requireDenied("cache read denied:", func() (*http.Response, error) {
return (&http.Client{Transport: &bearerTransport{token: "token-a-write-only"}}).Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version))
})
for _, client := range []*http.Client{clientB, {Transport: &bearerTransport{token: "token-a-pull"}}, clientRead} {
req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison")))
require.NoError(t, err)
req.Header.Set("Content-Range", "bytes 0-5/*")
@@ -1043,9 +1093,8 @@ func TestHandler_ArtifactSignature(t *testing.T) {
base := handler.ExternalURL() + apiPath
good := signArtifactURL(handler, 1)
other, err := StartHandler(Options{Dir: filepath.Join(t.TempDir(), "artifactcache2")})
require.NoError(t, err)
defer other.Close()
secret := bytes.Clone(handler.secret)
secret[0] ^= 1
for _, tc := range []struct {
name string
@@ -1054,7 +1103,7 @@ func TestHandler_ArtifactSignature(t *testing.T) {
{"missing signature", fmt.Sprintf("%s/artifacts/%d", base, 1)},
{"tampered signature", good[:len(good)-4] + "dead"},
{"expired signature", handler.signedArtifactURL(JobCredential{}, 1, time.Now().Add(-time.Second))},
{"signature from a different server", base + strings.SplitN(signArtifactURL(other, 1), apiPath, 2)[1]},
{"signature from a different server", base + strings.SplitN(signArtifactURL(&Handler{secret: secret}, 1), apiPath, 2)[1]},
} {
t.Run(tc.name, func(t *testing.T) {
resp, err := testClient.Get(tc.url)
+7 -3
View File
@@ -40,12 +40,13 @@ const (
twirpInternal = "internal"
twirpUnavailable = "unavailable"
twirpUnauthenticated = "unauthenticated"
twirpDenied = "permission_denied"
)
func (h *Handler) registerV2Routes(router *http.ServeMux) {
router.HandleFunc("POST "+cacheServiceV2Path+"/CreateCacheEntry", h.bearerAuth(h.v2CreateCacheEntry))
router.HandleFunc("POST "+cacheServiceV2Path+"/FinalizeCacheEntryUpload", h.bearerAuth(h.v2FinalizeCacheEntryUpload))
router.HandleFunc("POST "+cacheServiceV2Path+"/GetCacheEntryDownloadURL", h.bearerAuth(h.v2GetCacheEntryDownloadURL))
router.HandleFunc("POST "+cacheServiceV2Path+"/CreateCacheEntry", h.cacheModeAuth(true, h.v2CreateCacheEntry))
router.HandleFunc("POST "+cacheServiceV2Path+"/FinalizeCacheEntryUpload", h.cacheModeAuth(true, h.v2FinalizeCacheEntryUpload))
router.HandleFunc("POST "+cacheServiceV2Path+"/GetCacheEntryDownloadURL", h.cacheModeAuth(false, h.v2GetCacheEntryDownloadURL))
router.HandleFunc("PUT "+blobPath+"/{id}", h.signedAuth(blobUploadPurpose, h.v2UploadBlob))
}
@@ -218,6 +219,7 @@ func (h *Handler) v2UploadBlob(w http.ResponseWriter, r *http.Request) {
h.responseJSON(w, r, http.StatusBadRequest, err)
return
}
h.makeRoom(r.ContentLength)
query := r.URL.Query()
switch strings.ToLower(query.Get("comp")) {
@@ -266,6 +268,8 @@ func (h *Handler) twirpError(w http.ResponseWriter, r *http.Request, code string
status = http.StatusServiceUnavailable
case twirpUnauthenticated:
status = http.StatusUnauthorized
case twirpDenied:
status = http.StatusForbidden
}
h.responseJSON(w, r, status, map[string]any{"code": code, "msg": err.Error()})
}
+27
View File
@@ -14,6 +14,7 @@ import (
"strconv"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -84,6 +85,8 @@ func TestCacheServiceV2RoundTrip(t *testing.T) {
unsigned := fmt.Sprintf("%s%s/1", handler.ExternalURL(), blobPath)
assert.Equal(t, http.StatusUnauthorized, putBlob(t, unsigned, content))
unreserved := handler.signedURL(JobCredential{Repo: testRepo}, blobPath, blobUploadPurpose, 1, time.Now().Add(time.Minute))
assert.Equal(t, http.StatusBadRequest, putBlob(t, unreserved, content))
finalized, uploadURL := saveV2(t, handler, "deps-v1", "abc123", content)
require.Equal(t, true, finalized["ok"])
@@ -224,6 +227,30 @@ func TestCacheServiceV2Lookups(t *testing.T) {
assert.NotEmpty(t, got["archiveLocation"])
})
t.Run("a read-only job only restores and a write-only one only saves", func(t *testing.T) {
handler.RegisterJob("read-only", JobCredential{Repo: testRepo, CacheMode: "read"})
readOnlyClient := &http.Client{Transport: &bearerTransport{token: "read-only"}}
assert.Equal(t, true, v2Call(t, handler, readOnlyClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})["ok"])
for _, method := range []string{"CreateCacheEntry", "FinalizeCacheEntryUpload"} {
got := v2Call(t, handler, readOnlyClient, method, map[string]any{"key": "deps-new", "version": "v1", "size_bytes": 1})
assert.Equal(t, false, got["ok"], method)
assert.True(t, strings.HasPrefix(got["message"].(string), "cache write denied:"), method)
}
handler.RegisterJob("write-only", JobCredential{Repo: testRepo, CacheMode: "write-only"})
writeOnlyClient := &http.Client{Transport: &bearerTransport{token: "write-only"}}
assert.Equal(t, true, v2Call(t, handler, writeOnlyClient, "CreateCacheEntry", map[string]any{"key": "deps-write-only", "version": "v1"})["ok"])
resp, err := writeOnlyClient.Post(handler.ExternalURL()+cacheServiceV2Path+"/GetCacheEntryDownloadURL", "application/json", strings.NewReader(`{"key":"deps-abc","version":"v1"}`))
require.NoError(t, err)
defer resp.Body.Close()
got := map[string]string{}
require.NoError(t, json.UnmarshalRead(resp.Body, &got))
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
assert.Equal(t, twirpDenied, got["code"])
assert.True(t, strings.HasPrefix(got["msg"], "cache read denied:"), got["msg"])
})
t.Run("another repository sees nothing", func(t *testing.T) {
handler.RegisterJob("other-runtime-token", JobCredential{Repo: "other/repo"})
otherClient := &http.Client{Transport: &bearerTransport{token: "other-runtime-token"}}
+1 -1
View File
@@ -67,7 +67,7 @@ func (h *Handler) forwardOrNotFound(w http.ResponseWriter, r *http.Request) {
h.twirpError(w, r, twirpInternal, fmt.Errorf("cache server cannot reach the artifact service at %s: %w", target, err))
},
}
if cred.InsecureTLS {
if proxy.Transport = h.instanceTransport; proxy.Transport == nil && cred.InsecureTLS { // the instance transport carries runner.insecure and the client cert
proxy.Transport = insecureTransport
}
proxy.ServeHTTP(w, r)
+17
View File
@@ -133,3 +133,20 @@ func TestFrontResultsService(t *testing.T) {
})
}
}
func TestFrontResultsServiceForwardsThroughTheInstanceTransport(t *testing.T) {
t.Parallel()
gitea := httptest.NewTLSServer(http.HandlerFunc(func(writer http.ResponseWriter, _request *http.Request) { _, _ = io.WriteString(writer, "{}") }))
defer gitea.Close()
handler, err := StartHandler(Options{Dir: t.TempDir(), OutboundIP: "127.0.0.1", InstanceTransport: gitea.Client().Transport})
require.NoError(t, err)
defer handler.Close()
defer handler.RegisterJob("token", JobCredential{Results: gitea.URL})()
req, err := http.NewRequestWithContext(t.Context(), http.MethodPost, handler.ExternalURL()+artifactServicePath+"CreateArtifact", nil)
require.NoError(t, err)
resp, err := (&http.Client{Transport: &bearerTransport{token: "token"}}).Do(req)
require.NoError(t, err)
defer resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
}
+25 -27
View File
@@ -9,6 +9,7 @@ import (
"crypto/sha256"
"encoding/hex"
"encoding/json/v2"
"errors"
"fmt"
"io"
"net/http"
@@ -51,9 +52,10 @@ func (s *Storage) writeFile(name string, reader io.Reader) error {
if err != nil {
return err
}
defer file.Close()
_, err = io.Copy(file, reader)
if err = errors.Join(err, file.Close()); err != nil {
_ = os.Remove(name)
}
return err
}
@@ -102,50 +104,46 @@ func (s *Storage) Commit(id uint64, size int64) (int64, error) {
return 0, err
}
written, err := assemble(name, tempNames)
if err != nil {
return 0, err
if err == nil && size >= 0 && written != size { // old clients like actions/cache@v2 send no size
err = fmt.Errorf("broken file: %v != %v", written, size)
}
// If size is less than 0, it means the size is unknown.
// We can't check the size of the file, just skip the check.
// It happens when the request comes from old versions of actions, like `actions/cache@v2`.
if size >= 0 && written != size {
if err != nil {
_ = os.Remove(name)
return 0, fmt.Errorf("broken file: %v != %v", written, size)
return 0, err
}
return written, nil
}
// assemble concatenates the uploaded parts into name. A single part, which is what the v2 API
// produces below the client's block threshold, is already the whole archive and is moved.
// assemble appends the parts into the first, deleting each once copied, so it never needs twice the space.
func assemble(name string, tempNames []string) (int64, error) {
if len(tempNames) == 1 {
info, err := os.Stat(tempNames[0])
if err != nil {
return 0, err
}
return info.Size(), os.Rename(tempNames[0], name)
if len(tempNames) == 0 {
return 0, errors.New("nothing was uploaded")
}
file, err := os.Create(name)
if err := os.Rename(tempNames[0], name); err != nil {
return 0, err
}
file, err := os.OpenFile(name, os.O_WRONLY|os.O_APPEND, 0)
if err != nil {
return 0, err
}
defer file.Close()
var written int64
for _, v := range tempNames {
f, err := os.Open(v)
for _, v := range tempNames[1:] {
part, err := os.Open(v)
if err != nil {
return 0, err
}
n, err := io.Copy(file, f)
_ = f.Close()
_, err = io.Copy(file, part)
_ = part.Close()
if err != nil {
return 0, err
}
written += n
_ = os.Remove(v)
}
return written, nil
info, err := file.Stat()
if err != nil {
return 0, err
}
return info.Size(), file.Close()
}
func (s *Storage) Serve(w http.ResponseWriter, r *http.Request, id uint64) {
+24
View File
@@ -7,7 +7,9 @@ package common
import (
"context"
"fmt"
"math/rand/v2"
"runtime/debug"
"time"
"gitea.com/gitea/runner/act/common/logging"
)
@@ -158,6 +160,28 @@ func (e Executor) IfBool(conditional bool) Executor {
})
}
// RetryBackoff is the wait between Retry's attempts, random between 1 and 10 seconds like GitHub's runner.
var RetryBackoff = func() time.Duration { return time.Second + rand.N(9*time.Second) }
// Retry runs the executor up to attempts times until it succeeds.
func (e Executor) Retry(attempts int) Executor {
return func(ctx context.Context) error {
for attempt := 1; ; attempt++ {
err := e(ctx)
if err == nil || attempt == attempts || ctx.Err() != nil {
return err
}
backoff := RetryBackoff()
Logger(ctx).Warnf("%v, retrying in %s", err, backoff.Round(time.Second))
select {
case <-ctx.Done():
return err
case <-time.After(backoff):
}
}
}
}
// Finally adds an executor to run after other executor
func (e Executor) Finally(finally Executor) Executor {
return func(ctx context.Context) error {
+26
View File
@@ -11,6 +11,7 @@ import (
"sync"
"sync/atomic"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -134,6 +135,31 @@ func TestExecutorConditionalsAndFinally(t *testing.T) {
assert.Equal(t, []string{"if-true", "if-not", "if-bool", "main", "finally"}, calls)
}
func TestExecutorRetryStopsOnSuccessOrAfterAllAttempts(t *testing.T) {
backoff := RetryBackoff
RetryBackoff = func() time.Duration { return 0 }
t.Cleanup(func() { RetryBackoff = backoff })
transient := errors.New("transient")
for _, tc := range []struct {
failures, calls int
err error
}{
{failures: 2, calls: 3},
{failures: 5, calls: 3, err: transient},
} {
var calls int
err := Executor(func(context.Context) error {
calls++
if calls <= tc.failures {
return transient
}
return nil
}).Retry(3)(t.Context())
require.Equal(t, tc.err, err)
require.Equal(t, tc.calls, calls)
}
}
func TestExecutorFinallyReturnsFinallyErrorWithOriginal(t *testing.T) {
finalErr := errors.New("cleanup failed")
for _, mainErr := range []error{context.Canceled, nil} {
+14 -3
View File
@@ -36,7 +36,7 @@ func TestFetchAction(t *testing.T) {
root := t.TempDir()
remote := filepath.Join(root, "org", "action")
require.NoError(t, gitCmd("init", "--initial-branch=main", remote))
require.NoError(t, os.CopyFS(remote, os.DirFS(filepath.Join(gitFixtureDir, "work"))))
require.NoError(t, os.WriteFile(filepath.Join(remote, "action.yml"), []byte("name: action\n"), 0o644))
require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755))
require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link")))
@@ -53,6 +53,7 @@ func TestFetchAction(t *testing.T) {
var archiveRequests []http.Header
var mirror *httptest.Server
archiveCache := map[string][]byte{}
revisions := map[string]string{}
requests, archives, failures := 0, "", 0
handler := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
mu.Lock()
@@ -79,7 +80,14 @@ func TestFetchAction(t *testing.T) {
http.Redirect(writer, request, mirror.URL+request.URL.Path, http.StatusFound)
return
}
sha, _ := gitOutput(request.Context(), remote, "rev-parse", strings.TrimSuffix(ref, ".tar.gz"))
sha, ok := revisions[ref]
if !ok {
var err error
sha, err = gitOutput(request.Context(), remote, "rev-parse", strings.TrimSuffix(ref, ".tar.gz"))
if err == nil {
revisions[ref] = sha
}
}
etag := fmt.Sprintf(`W/"%s-tar.gz-%t"`, sha, archives != "unprefixed")
if request.Header.Get("If-None-Match") == etag {
writer.WriteHeader(http.StatusNotModified)
@@ -208,6 +216,9 @@ func TestFetchAction(t *testing.T) {
require.NoError(t, gitCmd("-C", remote, "commit", "--allow-empty", "-m", "moved"))
require.NoError(t, gitCmd("-C", remote, "tag", "--force", "v1"))
expectedSha = gitRevParse(t, remote, "v1")
mu.Lock()
clear(revisions)
mu.Unlock()
moved, movedSha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, expectedSha, movedSha)
@@ -253,7 +264,7 @@ func TestFetchAction(t *testing.T) {
})
t.Run("downloads again over a tree whose files changed size or that has no record, not under a job copying it", func(t *testing.T) {
sha := gitRevParse(t, remote, "main")
sha := expectedSha
for _, mode := range []string{"prefixed", "none"} {
serve(mode, 0)
input := newInput(sha)
+11 -6
View File
@@ -185,6 +185,7 @@ type NewGitCloneExecutorInput struct {
// For Gitea
InsecureSkipTLS bool
ClientCertFile, ClientKeyFile string // PEM, absolute as git runs in other directories
CAFile string // PEM, absolute, replaces git's CA bundle
Transport http.RoundTripper // for FetchAction's archive downloads, nil for http.DefaultTransport
}
@@ -389,12 +390,16 @@ func RemoteConfig(token string, insecureSkipTLS bool) []string {
return config
}
func ClientCertConfig(prefix, certFile, keyFile, executionOS string) []string {
if certFile == "" {
return nil
// SSLConfig uses OpenSSL on Windows, as Schannel takes no PEM key and ignores sslCAInfo.
func SSLConfig(prefix, certFile, keyFile, caFile, executionOS string) []string {
var config []string
if certFile != "" {
config = append(config, prefix+"sslCert="+certFile, prefix+"sslKey="+keyFile)
}
config := []string{prefix + "sslCert=" + certFile, prefix + "sslKey=" + keyFile}
if strings.EqualFold(executionOS, "windows") {
if caFile != "" {
config = append(config, prefix+"sslCAInfo="+caFile)
}
if config != nil && strings.EqualFold(executionOS, "windows") {
config = append(config, prefix+"sslBackend=openssl")
}
return config
@@ -457,7 +462,7 @@ func runGitWithStdin(ctx context.Context, dir string, input *NewGitCloneExecutor
config := []string{"core.autocrlf=false", "core.eol=lf"} // keep action files LF
if input != nil {
config = append(config, RemoteConfig(input.Token, input.InsecureSkipTLS)...)
config = append(config, ClientCertConfig("http.", input.ClientCertFile, input.ClientKeyFile, runtime.GOOS)...)
config = append(config, SSLConfig("http.", input.ClientCertFile, input.ClientKeyFile, input.CAFile, runtime.GOOS)...)
}
if dir != "" {
args = append([]string{"-C", dir}, args...)
+34 -40
View File
@@ -54,18 +54,18 @@ func TestMain(m *testing.M) {
os.Exit(code)
}
func TestClientCertConfig(t *testing.T) {
func TestSSLConfig(t *testing.T) {
for _, testcase := range []struct {
prefix, certFile, executionOS string
expected []string
prefix, certFile, caFile, executionOS string
expected []string
}{
{"http.", "cert.pem", "windows", []string{"http.sslCert=cert.pem", "http.sslKey=key.pem", "http.sslBackend=openssl"}},
{"http.https://example.com/.", "cert.pem", "Windows", []string{"http.https://example.com/.sslCert=cert.pem", "http.https://example.com/.sslKey=key.pem", "http.https://example.com/.sslBackend=openssl"}},
{"http.", "cert.pem", "Linux", []string{"http.sslCert=cert.pem", "http.sslKey=key.pem"}},
{"http.", "", "Windows", nil},
{"http.https://example.com/.", "cert.pem", "ca.pem", "Windows", []string{"http.https://example.com/.sslCert=cert.pem", "http.https://example.com/.sslKey=key.pem", "http.https://example.com/.sslCAInfo=ca.pem", "http.https://example.com/.sslBackend=openssl"}},
{"http.", "", "ca.pem", "windows", []string{"http.sslCAInfo=ca.pem", "http.sslBackend=openssl"}},
{"http.", "cert.pem", "ca.pem", "Linux", []string{"http.sslCert=cert.pem", "http.sslKey=key.pem", "http.sslCAInfo=ca.pem"}},
{"http.", "", "", "Windows", nil},
} {
t.Run(testcase.prefix+testcase.certFile+testcase.executionOS, func(t *testing.T) {
assert.Equal(t, testcase.expected, ClientCertConfig(testcase.prefix, testcase.certFile, "key.pem", testcase.executionOS))
t.Run(testcase.prefix+testcase.certFile+testcase.caFile+testcase.executionOS, func(t *testing.T) {
assert.Equal(t, testcase.expected, SSLConfig(testcase.prefix, testcase.certFile, "key.pem", testcase.caFile, testcase.executionOS))
})
}
}
@@ -112,8 +112,7 @@ func TestFindGitMetadataOfSHA256RepositoryUsesOrigin(t *testing.T) {
t.Parallel()
basedir := t.TempDir()
const remoteURL = "https://github.com/owner/repo.git"
require.NoError(t, gitCmd("init", "--object-format=sha256", "--initial-branch=main", basedir))
require.NoError(t, os.RemoveAll(filepath.Join(basedir, ".git", "hooks")))
require.NoError(t, gitCmd("init", "--template=", "--object-format=sha256", "--initial-branch=main", basedir))
require.NoError(t, gitCmd("-C", basedir, "commit", "--allow-empty", "-m", "init"))
require.NoError(t, gitCmd("-C", basedir, "remote", "add", "origin", remoteURL))
require.NoError(t, gitCmd("-C", basedir, "config", "--add", "remote.origin.url", "https://github.com/other/repo.git"))
@@ -205,16 +204,13 @@ func TestGitCloneExecutor(t *testing.T) {
require.NoError(t, gitCmd("-C", workDir, "add", "action.sh", ".gitattributes"))
require.NoError(t, gitCmd("-C", workDir, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", workDir, "tag", "v2"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "v2"))
require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "act-fails"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "branch-commit"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "act-fails"))
fullSha := gitRevParse(t, workDir, "main")
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:refs/heads/"+fullSha[:4]))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main", "v2", "act-fails", "main:refs/heads/"+fullSha[:4]))
hostConfig := filepath.Join(t.TempDir(), "gitconfig")
require.NoError(t, os.WriteFile(hostConfig, []byte("[core]\n\tautocrlf = true\n\teol = crlf\n[url "+strconv.Quote(remoteDir)+"]\n\tinsteadOf = https://example.invalid/action\n"), 0o644))
@@ -282,10 +278,10 @@ func TestGitCloneExecutor(t *testing.T) {
func TestGitCloneExecutorReclonesWhenOriginURLChanges(t *testing.T) {
t.Parallel()
createRemote := func(message string) string {
remoteDir, workDir := newGitRemote(t, message)
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
return remoteDir
dir := t.TempDir()
require.NoError(t, os.CopyFS(dir, os.DirFS(filepath.Join(gitFixtureDir, "work"))))
require.NoError(t, gitCmd("-C", dir, "commit", "--allow-empty", "-m", message))
return dir
}
oldRemoteDir := createRemote("old-action")
@@ -405,31 +401,30 @@ func TestGitCloneExecutorShallow(t *testing.T) {
remoteDir, workDir := newGitRemote(t, "c1", "c2", "c3")
require.NoError(t, gitCmd("-C", workDir, "tag", "v1"))
unadvertisedSHA := gitRevParse(t, workDir, "HEAD~1")
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "v1"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main", "v1"))
shallowMarker := func(dir string) string { return filepath.Join(dir, ".git", "shallow") }
for name, ref := range map[string]string{"branch is cloned shallowly": "main", "tag is cloned shallowly": "v1"} {
t.Run(name, func(t *testing.T) {
t.Parallel()
dir := t.TempDir()
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir, Ref: ref, Dir: dir, Depth: 1,
})(t.Context()))
assert.FileExists(t, shallowMarker(dir), "clone should be shallow")
assert.Equal(t, 1, gitRevCount(t, dir), "only the tip commit should be present")
assert.Equal(t, "c3", gitHeadSubject(t, dir))
assert.Equal(t, []string{"c3"}, gitSubjects(t, dir), "only the tip commit should be present")
})
}
unadvertisedRemote := func(t *testing.T, allowUnadvertised bool) string {
remote := filepath.Join(t.TempDir(), "remote.git")
require.NoError(t, gitCmd("clone", "--bare", remoteDir, remote))
require.NoError(t, gitCmd("-C", remote, "config", "uploadpack.allowAnySHA1InWant", strconv.FormatBool(allowUnadvertised)))
require.NoError(t, gitCmd("clone", "--bare", "--template=", "--config", "uploadpack.allowAnySHA1InWant="+strconv.FormatBool(allowUnadvertised), remoteDir, remote))
return remote
}
t.Run("commit hash resolves when the remote accepts a direct shallow fetch", func(t *testing.T) {
t.Parallel()
dir := t.TempDir()
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: unadvertisedRemote(t, false), Ref: unadvertisedSHA, Dir: dir, Depth: 1,
@@ -438,6 +433,7 @@ func TestGitCloneExecutorShallow(t *testing.T) {
})
t.Run("commit hash is fetched shallowly, moved to another hash, then reused without reaching the remote", func(t *testing.T) {
t.Parallel()
remote := unadvertisedRemote(t, true)
dir := t.TempDir()
cloneAt := func(ref string) error {
@@ -448,7 +444,7 @@ func TestGitCloneExecutorShallow(t *testing.T) {
require.NoError(t, cloneAt(unadvertisedSHA))
assert.FileExists(t, shallowMarker(dir))
assert.Equal(t, 1, gitRevCount(t, dir))
assert.Len(t, gitSubjects(t, dir), 1)
assert.Equal(t, unadvertisedSHA, gitRevParse(t, dir, "HEAD"))
olderSHA := gitRevParse(t, workDir, "HEAD~2")
@@ -461,6 +457,8 @@ func TestGitCloneExecutorShallow(t *testing.T) {
})
t.Run("moving hexadecimal-named branch updates while staying shallow", func(t *testing.T) {
t.Parallel()
remoteDir, workDir := newGitRemote(t, "c2", "c3")
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:deadbeef"))
dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{
@@ -473,19 +471,17 @@ func TestGitCloneExecutorShallow(t *testing.T) {
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:deadbeef"))
require.NoError(t, clone(t.Context()))
assert.Equal(t, "c4", gitHeadSubject(t, dir), "reused shallow clone should update to the new tip")
assert.Equal(t, []string{"c4"}, gitSubjects(t, dir), "reused shallow clone should update to the new tip")
assert.FileExists(t, shallowMarker(dir), "repo should remain shallow after update")
assert.Equal(t, 1, gitRevCount(t, dir))
})
t.Run("sha256 full hash is cloned shallowly, reused without the remote, and its 40-character prefix is short", func(t *testing.T) {
t.Parallel()
remote := filepath.Join(t.TempDir(), "remote.git")
require.NoError(t, gitCmd("init", "--bare", "--object-format=sha256", "--initial-branch=main", remote))
work := t.TempDir()
require.NoError(t, gitCmd("init", "--object-format=sha256", "--initial-branch=main", work))
require.NoError(t, gitCmd("-C", work, "remote", "add", "origin", remote))
require.NoError(t, gitCmd("init", "--template=", "--object-format=sha256", "--initial-branch=main", work))
require.NoError(t, gitCmd("-C", work, "commit", "--allow-empty", "-m", "pinned"))
require.NoError(t, gitCmd("-C", work, "push", "origin", "main"))
require.NoError(t, gitCmd("clone", "--bare", "--template=", work, remote))
sha := gitRevParse(t, work, "HEAD")
dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remote, Ref: sha, Dir: dir, Depth: 1})
@@ -504,11 +500,11 @@ func TestGitCloneExecutorPinnedHashIgnoresShadowingRef(t *testing.T) {
pinned := gitRevParse(t, workDir, "HEAD")
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "decoy"))
require.NoError(t, gitCmd("-C", workDir, "tag", pinned))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", pinned))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main", pinned))
for name, depth := range map[string]int{"shallow": 1, "full clone": 0} {
t.Run(name, func(t *testing.T) {
t.Parallel()
dir := t.TempDir()
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir, Ref: pinned, Dir: dir, Depth: depth,
@@ -540,13 +536,11 @@ func gitRevParse(t *testing.T, dir, rev string) string {
return strings.TrimSpace(string(out))
}
func gitRevCount(t *testing.T, dir string) int {
func gitSubjects(t *testing.T, dir string) []string {
t.Helper()
out, err := exec.Command("git", "-C", dir, "rev-list", "--count", "HEAD").Output()
out, err := exec.Command("git", "-C", dir, "log", "--format=%s").Output()
require.NoError(t, err)
n, err := strconv.Atoi(strings.TrimSpace(string(out)))
require.NoError(t, err)
return n
return strings.Split(strings.TrimSpace(string(out)), "\n")
}
func gitHeadSubject(t *testing.T, dir string) string {
@@ -680,7 +674,7 @@ func TestNewGitCloneExecutorFetchSendsTokenAndHonoursContext(t *testing.T) {
})
dir := filepath.Join(t.TempDir(), "cached-action")
require.NoError(t, gitCmd("init", dir))
require.NoError(t, os.CopyFS(dir, os.DirFS(filepath.Join(gitFixtureDir, "work"))))
require.NoError(t, gitCmd("-C", dir, "remote", "add", "origin", server.URL))
ctx, cancel := context.WithCancel(t.Context())
+8 -4
View File
@@ -6,6 +6,7 @@ package common
import (
"net"
"slices"
"sort"
"strings"
)
@@ -55,19 +56,22 @@ func GetOutboundIP() net.IP {
}
func preferredIP(ips []interfaceIP) net.IP {
ips = slices.DeleteFunc(ips, func(ip interfaceIP) bool {
return len(ip.Name) == 15 && strings.HasPrefix(ip.Name, "br-") && strings.Trim(ip.Name[3:], "0123456789abcdef") == "" // a Docker network's bridge, removed with the network
})
if len(ips) == 0 {
return nil
}
sort.Slice(ips, func(i, j int) bool {
first, second := ips[i], ips[j]
// ethernet is preferred
if vi, vj := strings.HasPrefix(first.Name, "e"), strings.HasPrefix(second.Name, "e"); vi != vj {
// IPv4 is preferred, as job networks are IPv4-only by default
if vi, vj := first.To4() != nil, second.To4() != nil; vi != vj {
return vi
}
// IPv4 is preferred
if vi, vj := first.To4() != nil, second.To4() != nil; vi != vj {
// ethernet is preferred
if vi, vj := strings.HasPrefix(first.Name, "e"), strings.HasPrefix(second.Name, "e"); vi != vj {
return vi
}
+2 -1
View File
@@ -13,8 +13,9 @@ import (
func TestPreferredIP(t *testing.T) {
assert.Nil(t, preferredIP(nil))
assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{{IP: net.ParseIP("2001:db8::1")}}))
assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{
assert.Equal(t, net.ParseIP("192.0.2.1"), preferredIP([]interfaceIP{
{IP: net.ParseIP("192.0.2.1"), Name: "wlan0"},
{IP: net.ParseIP("2001:db8::1"), Name: "eth0"},
{IP: net.ParseIP("172.18.0.1"), Name: "br-0123456789ab"},
}))
}
+6
View File
@@ -14,6 +14,7 @@ import (
"strings"
"sync"
"sync/atomic"
"time"
"gitea.com/gitea/runner/act/common"
@@ -23,6 +24,9 @@ import (
var hostOS = runtime.GOOS // replaced in tests
// DrainGracePeriod caps the wait for container output to drain so the log tail is not truncated
const DrainGracePeriod = 2 * time.Second
// ExitCodeError reports a non-zero process exit code from a container command.
type ExitCodeError int
@@ -148,6 +152,8 @@ type NewDockerBuildExecutorInput struct {
ContextDir string
Dockerfile string
BuildContext io.Reader
Dockerignore []byte // applied to BuildContext as the docker CLI applies it
Stdout io.Writer
ImageTag string
Platform string
BuildArgs map[string]*string
+93 -2
View File
@@ -8,10 +8,13 @@ package container
import (
"archive/tar"
"bytes"
"context"
"fmt"
"io"
"io/fs"
"os"
"path"
"path/filepath"
"runtime"
"strings"
@@ -48,6 +51,7 @@ func NewDockerBuildExecutor(input NewDockerBuildExecutorInput) common.Executor {
options := client.ImageBuildOptions{
Tags: []string{input.ImageTag},
Labels: map[string]string{"com.gitea.runner.image": input.ImageTag}, // own image ID per tag, so removing one never deletes another job's identical build
Remove: true,
AuthConfigs: LoadDockerAuthConfigs(ctx),
Dockerfile: input.Dockerfile,
@@ -62,7 +66,7 @@ func NewDockerBuildExecutor(input NewDockerBuildExecutorInput) common.Executor {
}
var buildContext io.ReadCloser
if input.BuildContext != nil {
buildContext = io.NopCloser(input.BuildContext)
buildContext, err = filterBuildContext(input.BuildContext, input.Dockerignore, input.Dockerfile)
} else {
buildContext, err = createBuildContext(ctx, input.ContextDir, input.Dockerfile)
}
@@ -77,10 +81,97 @@ func NewDockerBuildExecutor(input NewDockerBuildExecutorInput) common.Executor {
if err != nil {
return err
}
return logDockerResponse(logger, resp.Body, false)
err = logDockerResponse(logger, resp.Body, func(format string, args ...any) { fmt.Fprintf(input.Stdout, format, args...) })
common.FlushWriter(input.Stdout)
return err
}
}
func filterBuildContext(buildContext io.Reader, dockerignore []byte, relDockerfile string) (io.ReadCloser, error) {
excludes, err := ignorefile.ReadAll(bytes.NewReader(dockerignore))
if err != nil || len(excludes) == 0 {
return io.NopCloser(buildContext), err
}
pm, err := patternmatcher.New(excludes)
if err != nil {
return nil, err
}
reader, writer := io.Pipe()
go func() {
writer.CloseWithError(writeFilteredBuildContext(writer, buildContext, pm, path.Clean(filepath.ToSlash(relDockerfile))))
}()
return reader, nil
}
func writeFilteredBuildContext(w io.Writer, buildContext io.Reader, pm *patternmatcher.PatternMatcher, relDockerfile string) error {
spool, err := os.CreateTemp("", "gitea-runner-build-") // excluded content, for a kept hardlink to it
if err != nil {
return err
}
defer os.Remove(spool.Name())
defer spool.Close()
reader := tar.NewReader(buildContext)
writer := tar.NewWriter(w)
buffer := make([]byte, 32*1024)
excluded := map[string]*excludedBuildContextEntry{}
for {
header, err := reader.Next()
if err == io.EOF {
return writer.Close()
}
if err != nil {
return err
}
name := path.Clean(header.Name)
var target *excludedBuildContextEntry
if header.Typeflag == tar.TypeLink {
target = excluded[path.Clean(header.Linkname)]
}
if name != ".dockerignore" && name != relDockerfile {
skip, err := pm.MatchesOrParentMatches(name)
if err != nil {
return err
}
if skip {
if target != nil {
excluded[name] = target
} else if header.Typeflag != tar.TypeDir {
offset, err := spool.Seek(0, io.SeekCurrent)
if err != nil {
return err
}
excluded[name] = &excludedBuildContextEntry{header: header, offset: offset}
if _, err := io.CopyBuffer(spool, reader, buffer); err != nil {
return err
}
}
continue
}
}
var content io.Reader = reader
if target != nil && target.keptName != "" {
header.Linkname = target.keptName
} else if target != nil {
header.Typeflag, header.Linkname, header.Size = target.header.Typeflag, target.header.Linkname, target.header.Size
content = io.NewSectionReader(spool, target.offset, header.Size)
target.keptName = header.Name
}
if err := writer.WriteHeader(header); err != nil {
return err
}
if _, err := io.CopyBuffer(writer, content, buffer); err != nil {
return err
}
}
}
type excludedBuildContextEntry struct {
header *tar.Header
offset int64
keptName string
}
func createBuildContext(ctx context.Context, contextDir, relDockerfile string) (io.ReadCloser, error) {
common.Logger(ctx).Debugf("Creating archive for build context dir '%s' with relative dockerfile '%s'", contextDir, relDockerfile)
+61 -3
View File
@@ -7,6 +7,8 @@ package container
import (
"archive/tar"
"bytes"
"fmt"
"io"
"net/http"
"net/http/httptest"
@@ -18,10 +20,12 @@ import (
"github.com/stretchr/testify/require"
)
func TestDockerBuildReportsDaemonError(t *testing.T) {
func TestDockerBuildLabelsImageWithItsTagAndReportsDaemonError(t *testing.T) {
var labels string
daemon := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
writer.Header().Set("API-Version", "1.47")
if strings.HasSuffix(request.URL.Path, "/build") {
labels = request.URL.Query().Get("labels")
writer.Header().Set("Content-Type", "application/json")
writer.WriteHeader(http.StatusInternalServerError)
_, _ = io.WriteString(writer, `{"message":"build rejected"}`)
@@ -33,17 +37,18 @@ func TestDockerBuildReportsDaemonError(t *testing.T) {
t.Setenv("DOCKER_AUTH_CONFIG", "")
require.ErrorContains(t, NewDockerBuildExecutor(NewDockerBuildExecutorInput{
ImageTag: "action:test", Dockerfile: "Dockerfile", BuildContext: strings.NewReader(""),
ImageTag: "action:test", Dockerfile: "Dockerfile", BuildContext: strings.NewReader(""), Stdout: io.Discard,
})(t.Context()), "build rejected")
require.JSONEq(t, `{"com.gitea.runner.image":"action:test"}`, labels)
}
func TestCreateBuildContextFollowsDockerignoreAndSymlinkedRootButKeepsDockerfile(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.Mkdir(filepath.Join(dir, "build"), 0o755))
for name, content := range map[string]string{
".dockerignore": "*.log\nbuild\n!build/keep.txt\nDockerfile", "Dockerfile": "FROM scratch", "a.txt": "a",
"a.log": "log", "build/drop.txt": "drop", "build/keep.txt": "keep",
} {
require.NoError(t, os.MkdirAll(filepath.Join(dir, filepath.Dir(name)), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644))
}
require.NoError(t, os.Mkdir(filepath.Join(dir, "empty"), 0o755))
@@ -69,3 +74,56 @@ func TestCreateBuildContextFollowsDockerignoreAndSymlinkedRootButKeepsDockerfile
require.Equal(t, []string{".dockerignore", "a.txt", "build/keep.txt", "empty/", "Dockerfile"}, names, contextDir)
}
}
func TestFilterBuildContextAppliesDockerignoreAndMaterializesHardlinksToExcludedFiles(t *testing.T) {
const dockerignore = "*.log\nbuild\n!build/keep.txt\nActionfile\n.dockerignore\n"
for _, prefix := range []string{"", "./"} {
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
for _, entry := range []struct {
name, body, link string
typeflag byte
}{
{"Actionfile", "FROM scratch", "", tar.TypeReg},
{"output.log", "log", "", tar.TypeReg},
{"build/", "", "", tar.TypeDir},
{"build/drop.txt", "drop", "", tar.TypeReg},
{"build/keep.txt", "keep", "", tar.TypeReg},
{"entrypoint", "", prefix + "output.log", tar.TypeLink},
{"other-entrypoint", "", prefix + "output.log", tar.TypeLink},
{".dockerignore", dockerignore, "", tar.TypeReg},
} {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: prefix + entry.name, Linkname: entry.link, Typeflag: entry.typeflag, Size: int64(len(entry.body)), Mode: 0o644}))
_, err := io.WriteString(writer, entry.body)
require.NoError(t, err)
}
require.NoError(t, writer.Close())
unfiltered, err := filterBuildContext(bytes.NewReader(archive.Bytes()), nil, "Actionfile")
require.NoError(t, err)
passedThrough, err := io.ReadAll(unfiltered)
require.NoError(t, err)
require.Equal(t, archive.Bytes(), passedThrough)
filtered, err := filterBuildContext(bytes.NewReader(archive.Bytes()), []byte(dockerignore), "Actionfile")
require.NoError(t, err)
var entries []string
for reader := tar.NewReader(filtered); ; {
header, err := reader.Next()
if err == io.EOF {
break
}
require.NoError(t, err)
body, err := io.ReadAll(reader)
require.NoError(t, err)
entries = append(entries, fmt.Sprintf("%s %c %s %s", strings.TrimPrefix(header.Name, prefix), header.Typeflag, strings.TrimPrefix(header.Linkname, prefix), body))
}
require.Equal(t, []string{
"Actionfile 0 FROM scratch",
"build/keep.txt 0 keep",
"entrypoint 0 log",
"other-entrypoint 1 entrypoint ",
".dockerignore 0 " + dockerignore,
}, entries, prefix)
}
}
+1 -1
View File
@@ -38,7 +38,7 @@ func loadScratchImage(t *testing.T, platform string) string {
response, err := cli.ImageLoad(t.Context(), &archive)
require.NoError(t, err)
defer response.Close()
require.NoError(t, logDockerResponse(common.Logger(t.Context()), response, false))
require.NoError(t, logDockerResponse(common.Logger(t.Context()), response, common.Logger(t.Context()).Debugf))
t.Cleanup(func() {
_, _ = cli.ImageRemove(context.WithoutCancel(t.Context()), tag, mobyclient.ImageRemoveOptions{Force: true})
})
+10 -25
View File
@@ -8,6 +8,7 @@ package container
import (
"bufio"
"cmp"
"encoding/json/v2"
"errors"
"io"
@@ -26,10 +27,7 @@ type dockerMessage struct {
Progress string `json:"progress"`
}
func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, isError bool) error {
if dockerResponse == nil {
return nil
}
func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, log func(string, ...any)) error {
defer dockerResponse.Close()
scanner := bufio.NewScanner(dockerResponse)
@@ -38,41 +36,28 @@ func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, isE
line := scanner.Bytes()
var msg dockerMessage
if err := json.Unmarshal(line, &msg); err != nil {
writeLog(logger, false, "Unable to unmarshal line [%s] ==> %v", string(line), err)
logger.Debugf("Unable to unmarshal line [%s] ==> %v", string(line), err)
continue
}
if msg.Error != "" {
writeLog(logger, isError, "%s", msg.Error)
return errors.New(msg.Error)
}
if msg.ErrorDetail.Message != "" {
writeLog(logger, isError, "%s", msg.ErrorDetail.Message)
return errors.New(msg.ErrorDetail.Message)
if message := cmp.Or(msg.Error, msg.ErrorDetail.Message); message != "" {
log("%s", message)
return errors.New(message)
}
switch {
case msg.Status != "":
if msg.Progress != "" {
writeLog(logger, isError, "%s :: %s :: %s\n", msg.Status, msg.ID, msg.Progress)
log("%s :: %s :: %s\n", msg.Status, msg.ID, msg.Progress)
} else {
writeLog(logger, isError, "%s :: %s\n", msg.Status, msg.ID)
log("%s :: %s\n", msg.Status, msg.ID)
}
case msg.Stream != "":
writeLog(logger, isError, "%s", msg.Stream)
log("%s", msg.Stream)
default:
writeLog(logger, false, "Unable to handle line: %s", string(line))
logger.Debugf("Unable to handle line: %s", string(line))
}
}
return scanner.Err()
}
func writeLog(logger *logging.Logger, isError bool, format string, args ...any) {
if isError {
logger.Errorf(format, args...)
} else {
logger.Debugf(format, args...)
}
}
+3 -4
View File
@@ -22,7 +22,7 @@ import (
// NewDockerPullExecutor function to create a run executor for the container
func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
return func(ctx context.Context) error {
return common.Executor(func(ctx context.Context) error {
logger := common.Logger(ctx)
logger.Debugf("docker pull %v", input.Image)
@@ -61,11 +61,10 @@ func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
// stream rather than on the call itself, so both have to be checked
pullOnce := func(opts client.ImagePullOptions) error {
reader, err := cli.ImagePull(ctx, imageRef, opts)
streamErr := logDockerResponse(logger, reader, err != nil)
if err != nil {
return err
}
return streamErr
return logDockerResponse(logger, reader, logger.Debugf)
}
err = pullOnce(imagePullOptions)
@@ -83,7 +82,7 @@ func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
return nil
}
return fmt.Errorf("failed to pull image '%s' (%s): %w", imageRef, input.Platform, err)
}
}).Retry(3)
}
func getImagePullOptions(ctx context.Context, input NewDockerPullExecutorInput) (client.ImagePullOptions, error) {
+4 -3
View File
@@ -95,15 +95,16 @@ func TestIsPinnedImage(t *testing.T) {
// The pull path reports a failure the daemon sent mid-stream, so it must carry the reason
// whichever of the two shapes the daemon used.
func TestLogDockerResponseError(t *testing.T) {
logger := logging.New(logtest.New())
logger, hook := logtest.NewLogger()
streamErr := func(line string) error {
return logDockerResponse(logger, io.NopCloser(strings.NewReader(line)), false)
return logDockerResponse(logger, io.NopCloser(strings.NewReader(line)), logger.Infof)
}
require.EqualError(t, streamErr(`{"error":"toomanyrequests: rate limit exceeded"}`), "toomanyrequests: rate limit exceeded")
require.EqualError(t, streamErr(`{"errorDetail":{"message":"unexpected EOF"}}`), "unexpected EOF")
require.NoError(t, streamErr(`{"status":"Downloading"}`))
require.NoError(t, streamErr(`{"stream":"`+strings.Repeat("x", 70000)+`"}`))
assert.Equal(t, logging.LevelInfo, hook.LastRecord().Level)
require.ErrorIs(t, logDockerResponse(logger, io.NopCloser(io.MultiReader(
strings.NewReader(`{"status":"Downloading"}`+"\n"), iotest.ErrReader(io.ErrUnexpectedEOF),
)), false), io.ErrUnexpectedEOF)
)), logger.Infof), io.ErrUnexpectedEOF)
}
+31 -24
View File
@@ -23,6 +23,7 @@ import (
"slices"
"strconv"
"strings"
"sync"
"time"
"gitea.com/gitea/runner/act/common"
@@ -41,13 +42,6 @@ import (
specs "github.com/opencontainers/image-spec/specs-go/v1"
)
// drainGracePeriod bounds how long we wait for an output-copy goroutine to
// finish draining a container's output before returning, so that neither a
// cancellation (waitForCommand) nor a normal container exit (wait) truncates
// the tail of the log. It is a safety bound: in the common case the stream
// reaches EOF and the goroutine returns well before this elapses.
const drainGracePeriod = 2 * time.Second
// NewContainer creates a reference to a container
func NewContainer(input *NewContainerInput) ExecutionsEnvironment {
// Resolved up front because the image pull runs before the container is created.
@@ -90,8 +84,13 @@ func (cr *containerReference) Start(attach bool) common.Executor {
cr.attach().IfBool(attach),
cr.start(),
cr.wait().IfBool(attach),
cr.tryReadID("-u", func(id int) { cr.UID = id }),
cr.tryReadID("-g", func(id int) { cr.GID = id }),
func(ctx context.Context) error {
var wg sync.WaitGroup
wg.Go(func() { _ = cr.tryReadID("-u", func(id int) { cr.UID = id })(ctx) })
_ = cr.tryReadID("-g", func(id int) { cr.GID = id })(ctx)
wg.Wait()
return nil
},
func(ctx context.Context) error {
// If this fails, then folders have wrong permissions on non root container
if cr.UID != 0 || cr.GID != 0 {
@@ -191,6 +190,9 @@ func (cr *containerReference) Inspect(ctx context.Context) (*Info, error) {
} else if err != nil {
return nil, err
}
if cr.containerEnvID != cr.id && result.Container.Config != nil {
cr.containerEnv, cr.containerEnvID = opts.ConvertKVStringsToMap(result.Container.Config.Env), cr.id
}
return containerInfoFromInspect(result.Container), nil
}
@@ -314,15 +316,13 @@ func (cr *containerReference) UpdateFromEnv(srcPath string, env *map[string]stri
func (cr *containerReference) UpdateFromImageEnv(env *map[string]string) common.Executor {
envMap := *env
return common.Executor(func(ctx context.Context) error {
inspect, err := cr.cli.ContainerInspect(ctx, cr.id, client.ContainerInspectOptions{})
if err != nil {
common.Logger(ctx).Error(err)
return fmt.Errorf("inspect container: %w", err)
if cr.containerEnvID != cr.id {
if _, err := cr.Inspect(ctx); err != nil {
common.Logger(ctx).Error(err)
return fmt.Errorf("inspect container: %w", err)
}
}
if inspect.Container.Config == nil {
return nil
}
for key, value := range opts.ConvertKVStringsToMap(inspect.Container.Config.Env) {
for key, value := range cr.containerEnv {
if envMap[key] == "" {
envMap[key] = value
} else if key == "PATH" {
@@ -372,7 +372,9 @@ type containerReference struct {
// attachDone is closed by the attach() streaming goroutine once it has
// drained and flushed the container's output. wait() blocks on it so the
// tail of the log lands before the step proceeds.
attachDone chan struct{}
attachDone chan struct{}
containerEnv map[string]string // fixed at creation, cached by inspect for containerEnvID
containerEnvID string
LinuxContainerEnvironmentExtensions
}
@@ -861,6 +863,7 @@ func (cr *containerReference) tryReadID(opt string, cbk func(id int)) common.Exe
return nil
}
defer resp.Close()
defer context.AfterFunc(ctx, resp.Close)()
sid, err := resp.Reader.ReadString('\n')
if err != nil {
@@ -906,7 +909,7 @@ func (cr *containerReference) waitForCommand(ctx context.Context, resp client.Hi
if err != nil {
logger.Debugf("Failed to drain command output after cancellation: %v", err)
}
case <-time.After(drainGracePeriod):
case <-time.After(DrainGracePeriod):
logger.Warn("Timed out draining command output after cancellation")
}
@@ -938,10 +941,7 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, ski
logger.Error(err)
}
}()
extractionRoot := "/"
if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") {
extractionRoot = actPath // keeps tar entries from crossing the image's /var/run symlink
}
extractionRoot := cr.extractionRoot(dstPath)
if err := writeDirTar(ctx, tarFile, path.Join("/", strings.TrimPrefix(dstPath, extractionRoot)), srcPath, useGitIgnore, skipGitDir, cr.UID, cr.GID, cr.markExecutable(ctx)); err != nil {
return err
}
@@ -954,6 +954,13 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, ski
}
}
func (cr *containerReference) extractionRoot(dstPath string) string {
if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") {
return actPath // keeps tar entries from crossing the image's /var/run symlink
}
return "/"
}
func (cr *containerReference) copyContent(dstPath string, files ...*FileEntry) common.Executor {
return func(ctx context.Context) error {
if cr.id == "" {
@@ -1046,7 +1053,7 @@ func (cr *containerReference) wait() common.Executor {
if cr.attachDone != nil {
select {
case <-cr.attachDone:
case <-time.After(drainGracePeriod):
case <-time.After(DrainGracePeriod):
logger.Warn("Timed out draining container output")
}
cr.attachDone = nil
+34 -7
View File
@@ -45,14 +45,15 @@ func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
Return(mobyclient.ContainerInspectResult{Container: container.InspectResponse{Config: &container.Config{Env: []string{
"PATH=/opt/node/x64/bin:/usr/bin:/bin", "CONFLICT_VAR=container-value", "SOME_RANDOM_VAR=",
"ANOTHER_ONE=old-value", "ANOTHER_ONE=BUT_I_HAVE_VALUE", "LITERAL='quoted'=$HOME\nsecond line",
}}}}, nil)
}}}}, nil).Once()
env := map[string]string{
"PATH": "/custom/bin",
"RANDOM_VAR": "WITH_VALUE",
"ANOTHER_VAR": "",
"CONFLICT_VAR": "step-value",
}
require.NoError(t, (&containerReference{cli: client, id: "amd64-job"}).UpdateFromImageEnv(&env)(t.Context()))
cr := &containerReference{cli: client, id: "amd64-job"}
require.NoError(t, cr.UpdateFromImageEnv(&env)(t.Context()))
assert.Equal(t, map[string]string{
"PATH": "/custom/bin:/opt/node/x64/bin:/usr/bin:/bin",
"RANDOM_VAR": "WITH_VALUE",
@@ -62,6 +63,10 @@ func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
"CONFLICT_VAR": "step-value",
"LITERAL": "'quoted'=$HOME\nsecond line",
}, env)
pathEnv := map[string]string{}
require.NoError(t, cr.UpdateFromImageEnv(&pathEnv)(t.Context()))
assert.Equal(t, "/opt/node/x64/bin:/usr/bin:/bin", pathEnv["PATH"])
client.AssertExpectations(t)
}
type mockDockerClient struct {
@@ -122,6 +127,11 @@ func (m *mockDockerClient) CopyToContainer(ctx context.Context, id string, optio
return args.Get(0).(mobyclient.CopyToContainerResult), args.Error(1)
}
func (m *mockDockerClient) ContainerStatPath(ctx context.Context, id string, options mobyclient.ContainerStatPathOptions) (mobyclient.ContainerStatPathResult, error) {
args := m.Called(ctx, id, options)
return args.Get(0).(mobyclient.ContainerStatPathResult), args.Error(1)
}
func (m *mockDockerClient) ContainerInspect(ctx context.Context, id string, opts mobyclient.ContainerInspectOptions) (mobyclient.ContainerInspectResult, error) {
args := m.Called(ctx, id, opts)
return args.Get(0).(mobyclient.ContainerInspectResult), args.Error(1)
@@ -351,6 +361,21 @@ func TestDockerAttachStreamClosesWithItsContext(t *testing.T) {
client.AssertExpectations(t)
}
func TestDockerReadIDStreamClosesWithItsContext(t *testing.T) {
conn, peer := net.Pipe()
t.Cleanup(func() { _ = peer.Close() })
client := &mockDockerClient{}
client.On("ExecCreate", mock.Anything, "123", mock.AnythingOfType("client.ExecCreateOptions")).
Return(mobyclient.ExecCreateResult{ID: "id"}, nil)
client.On("ExecAttach", mock.Anything, "id", mobyclient.ExecAttachOptions{}).
Return(mobyclient.ExecAttachResult{HijackedResponse: mobyclient.NewHijackedResponse(conn, "")}, nil)
ctx, cancel := context.WithCancel(t.Context())
cancel()
require.NoError(t, (&containerReference{id: "123", cli: client}).tryReadID("-u", func(int) { t.Error("read an ID after cancellation") })(ctx))
client.AssertExpectations(t)
}
func TestDockerWaitFailure(t *testing.T) {
ctx := t.Context()
statusCh := make(chan container.WaitResponse, 1)
@@ -540,6 +565,9 @@ func TestRejectsMissingContainer(t *testing.T) {
}
func TestPublicCopyPipelineHandlesStaleID(t *testing.T) {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "x"), nil, 0o644))
for _, testcase := range []struct {
name, destination, extractionRoot, entry string
directory bool
@@ -567,8 +595,6 @@ func TestPublicCopyPipelineHandlesStaleID(t *testing.T) {
cr := &containerReference{id: "stale", cli: client, input: &NewContainerInput{Name: "job-1"}}
if testcase.directory {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "x"), nil, 0o644))
require.NoError(t, cr.CopyDir(testcase.destination, source+string(filepath.Separator), false, true)(ctx))
} else {
require.NoError(t, cr.Copy(testcase.destination, &FileEntry{Name: "x", Mode: 0o644})(ctx))
@@ -1108,6 +1134,10 @@ func TestConnectTakesDaemonOSFromPing(t *testing.T) {
}
func TestContainerCopyModes(t *testing.T) {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "run.sh"), []byte("#!/bin/sh\n"), 0o666))
require.NoError(t, os.Chmod(filepath.Join(source, "run.sh"), 0o666))
for _, testcase := range []struct {
hostOS, serverOS string
expected int64
@@ -1117,9 +1147,6 @@ func TestContainerCopyModes(t *testing.T) {
{"linux", "linux", 0o666},
} {
t.Run(testcase.hostOS+"-"+testcase.serverOS, func(t *testing.T) {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "run.sh"), []byte("#!/bin/sh\n"), 0o666))
require.NoError(t, os.Chmod(filepath.Join(source, "run.sh"), 0o666))
file := &FileEntry{Name: "run.sh", Mode: 0o600, Body: "#!/bin/sh\n"}
original := hostOS
hostOS = testcase.hostOS
+47 -99
View File
@@ -6,7 +6,6 @@ package container
import (
"archive/tar"
"bytes"
"cmp"
"context"
"errors"
@@ -32,18 +31,15 @@ import (
)
type HostEnvironment struct {
Path string
TmpDir string
ToolCache string
Workdir string
// CleanWorkdir means teardown owns Workdir and may delete it. Leave false
// when Workdir points at a caller-owned checkout (e.g. `act` local mode).
CleanWorkdir bool
ActPath string
Root string
TrackingID string // RUNNER_TRACKING_ID of the job, a step with another value keeps its leftovers
StdOut io.Writer
AllocatePTY bool // allocate a pseudo-TTY for each step's process
Path string
TmpDir string
ToolCache string
Workdir string
ActPath string
Root string
TrackingID string // RUNNER_TRACKING_ID of the job, a step with another value keeps its leftovers
StdOut io.Writer
AllocatePTY bool // allocate a pseudo-TTY for each step's process
// procGroup owns every process the job's steps start. Atomic: Remove may read
// it while a step is still starting.
@@ -57,7 +53,7 @@ func (e *HostEnvironment) processGroup(ctx context.Context) *process.Group {
e.procGroupOnce.Do(func() {
group, err := process.NewGroup()
if err != nil {
common.Logger(ctx).Warnf("could not create the job's process group; processes a step leaves behind can only be reclaimed by the workspace scan: %v", err)
common.Logger(ctx).Warnf("could not create the job's process group, processes a step leaves behind may outlive the job: %v", err)
return
}
e.procGroup.Store(group)
@@ -117,17 +113,24 @@ func (e *HostEnvironment) Inspect(_ context.Context) (*Info, error) {
}
func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
buf := &bytes.Buffer{}
tw := tar.NewWriter(buf)
defer tw.Close()
srcPath = filepath.Clean(srcPath)
fi, err := os.Lstat(srcPath)
if err != nil {
return nil, err
}
tc := &filecollector.TarCollector{
TarWriter: tw,
}
reader, writer := io.Pipe()
go func() {
tw := tar.NewWriter(writer)
err := writeHostArchive(ctx, &filecollector.TarCollector{TarWriter: tw}, srcPath, fi)
if err == nil {
err = tw.Close()
}
writer.CloseWithError(err)
}()
return startedArchive(reader)
}
func writeHostArchive(ctx context.Context, tc *filecollector.TarCollector, srcPath string, fi fs.FileInfo) error {
if fi.IsDir() {
srcPrefix := srcPath
if !strings.HasSuffix(srcPrefix, string(filepath.Separator)) {
@@ -138,31 +141,22 @@ func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath strin
SrcPrefix: srcPrefix,
Handler: tc,
}
err = filepath.Walk(srcPath, fc.CollectFiles(ctx))
if err != nil {
return nil, err
return filepath.Walk(srcPath, fc.CollectFiles(ctx))
}
var f io.ReadCloser
var linkname string
var err error
if fi.Mode()&fs.ModeSymlink != 0 {
if linkname, err = os.Readlink(srcPath); err != nil {
return err
}
} else {
var f io.ReadCloser
var linkname string
if fi.Mode()&fs.ModeSymlink != 0 {
linkname, err = os.Readlink(srcPath)
if err != nil {
return nil, err
}
} else {
f, err = os.Open(srcPath)
if err != nil {
return nil, err
}
defer f.Close()
}
err := tc.WriteFile(fi.Name(), fi, linkname, f)
if err != nil {
return nil, err
if f, err = os.Open(srcPath); err != nil {
return err
}
defer f.Close()
}
return io.NopCloser(buf), nil
return tc.WriteFile(fi.Name(), fi, linkname, f)
}
func (e *HostEnvironment) Pull(_ bool) common.Executor {
@@ -286,6 +280,7 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
// wait, so an orphan holding cmd's stdout pipe cannot hang cmd.Wait().
treeKill := process.NewTreeKill(cmd)
treeKill.KeepLeftovers = !tracked
treeKill.GracefulCancel = true
var ppty *os.File
var tty *os.File
@@ -415,46 +410,6 @@ func removePathWithRetry(ctx context.Context, path string) error {
return lastErr
}
func (e *HostEnvironment) terminateRunningProcesses(ctx context.Context) {
if runtime.GOOS != "windows" {
return
}
// Detached: exec.CommandContext won't start on a cancelled ctx, and a
// server cancel has already cancelled the parent ctx.
killCtx, killCancel := context.WithTimeout(context.Background(), 30*time.Second)
defer killCancel()
logger := common.Logger(ctx)
// Dirs we own; a process referencing one is a leftover. ToolCache is shared
// across jobs, and Workdir may be a caller-owned checkout.
owned := []string{e.Path, e.TmpDir}
if e.CleanWorkdir {
owned = append(owned, e.Workdir)
}
dirs := make([]string, 0, len(owned))
for _, d := range owned {
if d == "" {
continue
}
abs, err := filepath.Abs(d)
if err != nil {
continue
}
dirs = append(dirs, abs)
}
if len(dirs) == 0 {
return
}
if killed, err := process.KillProcessesWithPaths(killCtx, dirs, e.TrackingID); err != nil {
logger.Debugf("workspace process cleanup reported errors: %v", err)
} else if killed > 0 {
logger.Debugf("terminated %d leftover workspace process(es)", killed)
}
}
// hostCleanupTimeout bounds each teardown phase so one stalled delete cannot
// wedge the runner slot. A var so tests can shrink it.
var hostCleanupTimeout = 30 * time.Second
@@ -463,36 +418,29 @@ func (e *HostEnvironment) Remove() common.Executor {
return func(ctx context.Context) error {
logger := common.Logger(ctx)
// End lingering processes before removing the workspace; on Windows their
// file locks block cleanup. Closing the group is deterministic, the scan a net.
// End lingering processes before removing the workspace, on Windows their file locks block cleanup.
if err := e.procGroup.Load().Close(); err != nil {
logger.Debugf("closing the job's process group failed: %v", err)
}
e.terminateRunningProcesses(ctx)
if killed, err := process.KillTrackedProcesses(e.TrackingID); err != nil {
logger.Debugf("killing the job's leftover processes reported errors: %v", err)
} else if killed > 0 {
logger.Debugf("killed %d leftover process(es) of the job", killed)
}
// Detach: a cancelled ctx would skip removePathWithRetry's retries,
// which absorb Windows file-handle release lag after the kill above.
rmCtx, rmCancel := context.WithTimeout(context.Background(), hostCleanupTimeout)
defer rmCancel()
var errs []error
if err := removePathWithRetry(rmCtx, cmp.Or(e.Root, e.Path)); err != nil {
err := removePathWithRetry(rmCtx, cmp.Or(e.Root, e.Path))
if err != nil {
logger.Warnf("failed to remove host misc state %s: %v", cmp.Or(e.Root, e.Path), err)
errs = append(errs, err)
}
if e.CleanWorkdir {
if err := removePathWithRetry(rmCtx, e.Workdir); err != nil {
logger.Warnf("failed to remove host workspace %s: %v", e.Workdir, err)
errs = append(errs, err)
}
if errors.Is(err, context.DeadlineExceeded) {
return nil // a stalled delete must not fail job completion
}
for _, err := range errs {
if !errors.Is(err, context.DeadlineExceeded) {
return errors.Join(errs...)
}
}
// Teardown timed out; warned above. Do not fail job completion over it.
return nil
return err
}
}
+16 -27
View File
@@ -32,6 +32,11 @@ func TestActionPlatformNames(t *testing.T) {
}
func TestCopyDir(t *testing.T) {
src := filepath.Join(t.TempDir(), "action")
require.NoError(t, os.MkdirAll(filepath.Join(src, ".git", "objects"), 0o700))
require.NoError(t, os.WriteFile(filepath.Join(src, ".git", "objects", "pack"), []byte("x"), 0o600))
require.NoError(t, os.WriteFile(filepath.Join(src, "action.yml"), []byte("name: x\n"), 0o600))
for _, test := range []struct {
name string
useGitIgnore, skipGitDir bool
@@ -41,11 +46,6 @@ func TestCopyDir(t *testing.T) {
{name: "unfiltered outside a git repository", useGitIgnore: true},
} {
t.Run(test.name, func(t *testing.T) {
src := filepath.Join(t.TempDir(), "action")
require.NoError(t, os.MkdirAll(filepath.Join(src, ".git", "objects"), 0o700))
require.NoError(t, os.WriteFile(filepath.Join(src, ".git", "objects", "pack"), []byte("x"), 0o600))
require.NoError(t, os.WriteFile(filepath.Join(src, "action.yml"), []byte("name: x\n"), 0o600))
dest := filepath.Join(t.TempDir(), "dest")
require.NoError(t, (&HostEnvironment{}).CopyDir(dest, src+string(filepath.Separator), test.useGitIgnore, test.skipGitDir)(t.Context()))
@@ -76,7 +76,11 @@ func TestGetContainerArchive(t *testing.T) {
require.NoError(t, err)
assert.Equal(t, expectedContent, content)
_, err = reader.Next()
assert.ErrorIs(t, err, io.EOF)
require.ErrorIs(t, err, io.EOF)
require.NoError(t, archive.Close())
_, err = e.GetContainerArchive(ctx, filepath.Join(e.Path, "missing"))
assert.ErrorIs(t, err, os.ErrNotExist)
}
func TestHostEnvironmentExecExitCode(t *testing.T) {
@@ -154,27 +158,12 @@ func TestHostEnvironmentAllocatePTY(t *testing.T) {
}
func TestHostEnvironmentRemoveWorkdir(t *testing.T) {
for name, owned := range map[string]bool{"preserves caller workdir": false, "cleans owned workdir": true} {
t.Run(name, func(t *testing.T) {
miscRoot, workdir := t.TempDir(), t.TempDir()
e := &HostEnvironment{
Path: filepath.Join(miscRoot, "hostexecutor"),
Workdir: workdir,
CleanWorkdir: owned,
Root: miscRoot,
}
require.NoError(t, os.MkdirAll(e.Path, 0o700))
require.NoError(t, os.MkdirAll(filepath.Join(miscRoot, "tmp"), 0o700))
require.NoError(t, e.Remove()(t.Context()))
assert.NoDirExists(t, miscRoot)
_, err := os.Stat(workdir)
if owned {
assert.ErrorIs(t, err, os.ErrNotExist)
} else {
require.NoError(t, err)
}
})
}
miscRoot, workdir := t.TempDir(), t.TempDir()
e := &HostEnvironment{Path: filepath.Join(miscRoot, "hostexecutor"), Workdir: workdir, Root: miscRoot}
require.NoError(t, os.MkdirAll(e.Path, 0o700))
require.NoError(t, e.Remove()(t.Context()))
assert.NoDirExists(t, miscRoot)
assert.DirExists(t, workdir)
}
func TestRemoveAllWithContextDoesNotHangOnStuckDelete(t *testing.T) {
+8 -34
View File
@@ -4,18 +4,15 @@
package container
import (
"bufio"
"context"
"fmt"
"io"
"net"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"unsafe"
"github.com/stretchr/testify/require"
"golang.org/x/sys/windows"
@@ -40,7 +37,6 @@ func TestHostEnvironmentTrackingCleanup(t *testing.T) {
}
}
env["GITEA_TRACKING_HELPER"] = "step"
env["GITEA_TRACKING_WORKSPACE"] = workspace
env["GITEA_TRACKING_ADDRESS"] = listener.Addr().String()
if tracking == "default" {
env["RUNNER_TRACKING_ID"] = environment.TrackingID
@@ -52,23 +48,6 @@ func TestHostEnvironmentTrackingCleanup(t *testing.T) {
go func() {
result <- environment.Exec([]string{os.Args[0], "-test.run=^TestHostTrackingHelper$"}, env, "", workspace)(ctx)
}()
step, err := listener.Accept()
require.NoError(t, err)
defer step.Close()
require.NoError(t, step.SetDeadline(time.Now().Add(20*time.Second)))
var stepPID int
_, err = fmt.Fscanln(step, &stepPID)
require.NoError(t, err)
stepHandle, err := windows.OpenProcess(windows.PROCESS_QUERY_LIMITED_INFORMATION, false, uint32(stepPID))
require.NoError(t, err)
defer func() { _ = windows.CloseHandle(stepHandle) }()
require.Eventually(t, func() bool {
var assigned int32
result, _, _ := windows.NewLazySystemDLL("kernel32.dll").NewProc("IsProcessInJob").Call(uintptr(stepHandle), 0, uintptr(unsafe.Pointer(&assigned)))
return result != 0 && assigned != 0
}, 5*time.Second, 10*time.Millisecond)
_, err = step.Write([]byte{1})
require.NoError(t, err)
var helpers []windows.Handle
for range 2 {
connection, err := listener.Accept()
@@ -114,23 +93,18 @@ func TestHostTrackingHelper(t *testing.T) {
if mode == "" {
return
}
if mode != "descendant" {
child := exec.Command(os.Args[0], "-test.run=^TestHostTrackingHelper$")
child.Env = append(os.Environ(), "GITEA_TRACKING_HELPER="+map[string]string{"step": "helper", "helper": "descendant"}[mode])
require.NoError(t, child.Start())
if mode == "step" {
os.Exit(0)
}
}
connection, err := net.Dial("tcp4", os.Getenv("GITEA_TRACKING_ADDRESS"))
require.NoError(t, err)
_, err = fmt.Fprintln(connection, os.Getpid())
require.NoError(t, err)
if mode == "step" {
_, err := bufio.NewReader(connection).ReadByte()
require.NoError(t, err)
}
if mode != "descendant" {
child := exec.Command(os.Args[0], "-test.run=^TestHostTrackingHelper$", filepath.Join(os.Getenv("GITEA_TRACKING_WORKSPACE"), "tracking-reference"))
child.Env = append(os.Environ(), "GITEA_TRACKING_HELPER="+map[string]string{"step": "helper", "helper": "descendant"}[mode])
require.NoError(t, child.Start())
if mode == "step" {
require.NoError(t, connection.Close())
os.Exit(0)
}
}
_, err = io.Copy(io.Discard, connection)
require.NoError(t, err)
require.NoError(t, connection.Close())
+9 -13
View File
@@ -4,7 +4,6 @@
package container
import (
"bufio"
"bytes"
"cmp"
"compress/gzip"
@@ -51,6 +50,7 @@ type KubernetesPod struct {
job *kubernetesContainer
services []*kubernetesContainer
arch, path string
imageOS string
ips []net.IP
transfer *transferServer
transferImage string
@@ -433,11 +433,12 @@ func (c *kubernetesContainer) Start(_ bool) common.Executor {
if err != nil {
return err
}
probed := strings.SplitN(output, "\n", 3)
for len(probed) < 3 {
probed := strings.SplitN(output, "\x00", 5)
for len(probed) < 5 {
probed = append(probed, "")
}
c.pod.arch, c.gzip, c.pod.path = goArchToActionArch(probed[0]), probed[1] == "gzip", probed[2]
c.pod.imageOS = cmp.Or(probed[3], ImageOSFromOSRelease(probed[4]))
return nil
}
}
@@ -448,7 +449,7 @@ func (p *KubernetesPod) probe(ctx context.Context, container string) (string, er
deadline := time.Now().Add(time.Minute)
for delay := 100 * time.Millisecond; ; delay = min(2*delay, 2*time.Second) {
var stdout bytes.Buffer
err := p.client.exec(ctx, p.name, container, []string{"sh", "-c", `printf '%s\n%s\n%s' "$(uname -m)" "$(command -v gzip >/dev/null && echo gzip)" "$PATH"`}, nil, &stdout, io.Discard)
err := p.client.exec(ctx, p.name, container, []string{"sh", "-c", `printf '%s\0%s\0%s\0%s\0%s' "$(uname -m)" "$(command -v gzip >/dev/null && echo gzip)" "$PATH" "${ImageOS-}" "$(cat /etc/os-release 2>/dev/null || cat /usr/lib/os-release 2>/dev/null)"`}, nil, &stdout, io.Discard)
var exitErr ExitCodeError
var apiErr *apiError
if err == nil || ctx.Err() != nil || errors.As(err, &exitErr) || time.Now().After(deadline) ||
@@ -573,15 +574,7 @@ func (c *kubernetesContainer) GetContainerArchive(ctx context.Context, srcPath s
}
writer.CloseWithError(err)
}()
buffered := bufio.NewReader(reader)
if _, err := buffered.Peek(1); err != nil {
reader.Close()
return nil, err
}
return struct {
io.Reader
io.Closer
}{buffered, reader}, nil
return startedArchive(reader)
}
func (c *kubernetesContainer) UpdateFromEnv(srcPath string, env *map[string]string) common.Executor {
@@ -591,6 +584,9 @@ func (c *kubernetesContainer) UpdateFromEnv(srcPath string, env *map[string]stri
func (c *kubernetesContainer) UpdateFromImageEnv(env *map[string]string) common.Executor {
return func(context.Context) error {
(*env)["PATH"] = cmp.Or((*env)["PATH"], c.pod.path)
if c.pod.imageOS != "" {
(*env)["ImageOS"] = cmp.Or((*env)["ImageOS"], c.pod.imageOS)
}
return nil
}
}
+4 -4
View File
@@ -193,7 +193,7 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
path := workdir + ":" + os.Getenv("PATH")
kubeconfig, cluster := fakeKubernetes(t, `{"spec":{"containers":[{},{}]},"status":{"phase":"Running","containerStatuses":[
{"name":"job","ready":true,"state":{"running":{}}},
{"name":"my-db","ready":true,"state":{"running":{}}}]}}`, "PATH="+path)
{"name":"my-db","ready":true,"state":{"running":{}}}]}}`, "PATH="+path, "ImageOS=ubuntu24-arm64")
options := KubernetesOptions{Kubeconfig: kubeconfig, RunnerUUID: "runner-uuid", PodTemplates: []map[string]any{
{"spec": map[string]any{
"automountServiceAccountToken": true,
@@ -309,9 +309,9 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
assert.Contains(t, lastExec(), "command="+url.QueryEscape(workdir+"/act/actions/sha/"))
assert.Contains(t, lastExec(), url.QueryEscape("gzip -dc |"))
pathEnv := map[string]string{}
require.NoError(t, job.UpdateFromImageEnv(&pathEnv)(ctx))
assert.Equal(t, path, pathEnv["PATH"])
imageEnv := map[string]string{}
require.NoError(t, job.UpdateFromImageEnv(&imageEnv)(ctx))
assert.Equal(t, map[string]string{"PATH": path, "ImageOS": "ubuntu24-arm64"}, imageEnv)
info, err := service.Inspect(ctx)
require.NoError(t, err)
@@ -108,3 +108,22 @@ func (*LinuxContainerEnvironmentExtensions) GetRunnerContext(ctx context.Context
func (*LinuxContainerEnvironmentExtensions) IsEnvironmentCaseInsensitive() bool {
return false
}
// ImageOSFromOSRelease derives ImageOS from an os-release file, empty for anything but Ubuntu, the only Linux GitHub names one.
func ImageOSFromOSRelease(osRelease string) string {
var id, major string
for line := range strings.Lines(osRelease) {
key, value, _ := strings.Cut(strings.TrimSpace(line), "=")
value = strings.Trim(value, `"'`)
switch key {
case "ID":
id = value
case "VERSION_ID":
major, _, _ = strings.Cut(value, ".")
}
}
if id != "ubuntu" || major == "" {
return ""
}
return id + major
}
+17 -15
View File
@@ -5,6 +5,7 @@ package container
import (
"bufio"
"bytes"
"os"
"path/filepath"
"strings"
@@ -15,14 +16,9 @@ import (
"golang.org/x/text/encoding/unicode"
)
func newTestHostEnv(t *testing.T) (*HostEnvironment, string) {
t.Helper()
e := &HostEnvironment{Path: t.TempDir()}
return e, filepath.Join(e.Path, "envfile")
}
func TestParseEnvFile(t *testing.T) {
t.Parallel()
hostEnv := &HostEnvironment{Path: t.TempDir()}
largeValue := strings.Repeat("x", 2*1024*1024)
encodeUTF16 := func(endianness unicode.Endianness) string {
content, err := unicode.UTF16(endianness, unicode.UseBOM).NewEncoder().Bytes([]byte("FOO=bar\r\nMULTI<<EOF\r\nline1\r\nEOF\r\n"))
@@ -51,15 +47,15 @@ func TestParseEnvFile(t *testing.T) {
{"UTF16 big endian", encodeUTF16(unicode.BigEndian), map[string]string{"FOO": "bar\r", "MULTI": "line1\r"}},
} {
t.Run(test.name, func(t *testing.T) {
e, envPath := newTestHostEnv(t)
envPath := filepath.Join(hostEnv.Path, test.name)
require.NoError(t, os.WriteFile(envPath, []byte(test.content), 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env, false)(t.Context()))
require.NoError(t, parseEnvFile(hostEnv, envPath, &env, false)(t.Context()))
assert.Equal(t, test.want, env)
})
}
t.Run("ordered entries preserve repeated names", func(t *testing.T) {
hostEnv, envPath := newTestHostEnv(t)
envPath := filepath.Join(hostEnv.Path, "ordered")
require.NoError(t, os.WriteFile(envPath, []byte("FOO=first\r\nfoo<<EOF\r\nmiddle\r\nEOF\r\nFOO=last\r\n"), 0o600))
var entries [][2]string
require.NoError(t, ParseEnvFile(hostEnv, envPath, func(name, value string) {
@@ -70,7 +66,7 @@ func TestParseEnvFile(t *testing.T) {
}, entries)
})
t.Run("Windows line endings", func(t *testing.T) {
hostEnv, envPath := newTestHostEnv(t)
envPath := filepath.Join(hostEnv.Path, "windows")
require.NoError(t, os.WriteFile(envPath, []byte("FOO=bar\r\nMULTI<<EOF\r\nline1\r\nline2\nEOF\r\n"), 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(hostEnv, envPath, &env, true)(t.Context()))
@@ -80,16 +76,22 @@ func TestParseEnvFile(t *testing.T) {
func TestParseEnvFileLineExceedsBufferReportsScannerError(t *testing.T) {
t.Parallel()
e, envPath := newTestHostEnv(t)
require.NoError(t, os.WriteFile(envPath, []byte("FOO<<EOF\n"+strings.Repeat("x", 17*1024*1024)+"\nEOF\n"), 0o600))
hostEnv := &HostEnvironment{Path: t.TempDir()}
envPath := filepath.Join(hostEnv.Path, "envfile")
const header, footer = "FOO<<EOF\n", "\nEOF\n"
content := bytes.Repeat([]byte("x"), len(header)+17*1024*1024+len(footer))
copy(content, header)
copy(content[len(content)-len(footer):], footer)
require.NoError(t, os.WriteFile(envPath, content, 0o600))
env := map[string]string{}
err := parseEnvFile(e, envPath, &env, false)(t.Context())
err := parseEnvFile(hostEnv, envPath, &env, false)(t.Context())
require.ErrorIs(t, err, bufio.ErrTooLong)
assert.Contains(t, err.Error(), "reading env file")
}
func TestParseEnvFileInvalidHeredoc(t *testing.T) {
hostEnv := &HostEnvironment{Path: t.TempDir()}
for name, content := range map[string]string{
"missing delimiter": "FOO<<EOF\nline1\nline2\n",
"empty name": "<<EOF\nline1\nEOF\n",
@@ -97,11 +99,11 @@ func TestParseEnvFileInvalidHeredoc(t *testing.T) {
"delimiter with carriage return": "FOO<<EOF\nline1\nEOF\r\n",
} {
t.Run(name, func(t *testing.T) {
e, envPath := newTestHostEnv(t)
envPath := filepath.Join(hostEnv.Path, name)
require.NoError(t, os.WriteFile(envPath, []byte("APPLIED=before-error\n"+content), 0o600))
env := map[string]string{}
err := parseEnvFile(e, envPath, &env, false)(t.Context())
err := parseEnvFile(hostEnv, envPath, &env, false)(t.Context())
require.Error(t, err)
assert.Contains(t, err.Error(), "delimiter")
assert.Equal(t, map[string]string{"APPLIED": "before-error"}, env)
+14
View File
@@ -6,6 +6,7 @@ package container
import (
"archive/tar"
"bufio"
"context"
"io"
"path/filepath"
@@ -15,6 +16,19 @@ import (
"gitea.com/gitea/runner/act/filecollector"
)
// startedArchive returns reader once its first byte arrives, so failing to start archiving errors here, not on read.
func startedArchive(reader *io.PipeReader) (io.ReadCloser, error) {
buffered := bufio.NewReader(reader)
if _, err := buffered.Peek(1); err != nil {
reader.Close()
return nil, err
}
return struct {
io.Reader
io.Closer
}{buffered, reader}, nil
}
func writeFilesTar(ctx context.Context, w io.Writer, uid, gid int, files ...*FileEntry) error {
tw := tar.NewWriter(w)
for _, file := range files {
+9 -7
View File
@@ -28,10 +28,12 @@ func gitCmd(t *testing.T, args ...string) {
}
func TestCollectFilesKeepsWhatGitTracksOrDoesNotIgnore(t *testing.T) {
repoDir := filepath.Join(t.TempDir(), "mygitrepo")
for _, dir := range []string{repoDir, filepath.Join(repoDir, "sub"), filepath.Join(repoDir, "nested")} {
gitCmd(t, "init", "-q", dir)
repoDir := t.TempDir()
gitCmd(t, "init", "-q", "--template=", repoDir)
for _, dir := range []string{"sub", "nested"} {
require.NoError(t, os.CopyFS(filepath.Join(repoDir, dir, ".git"), os.DirFS(filepath.Join(repoDir, ".git"))))
}
require.NoError(t, os.Mkdir(filepath.Join(repoDir, "ign"), 0o755))
for name, body := range map[string]string{
".gitignore": ".*\nign/\n",
".env": "",
@@ -43,15 +45,15 @@ func TestCollectFilesKeepsWhatGitTracksOrDoesNotIgnore(t *testing.T) {
"sub/kept": "",
"nested/kept": "",
} {
require.NoError(t, os.MkdirAll(filepath.Dir(filepath.Join(repoDir, name)), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(repoDir, name), []byte(body), 0o644))
}
require.NoError(t, os.Mkdir(filepath.Join(repoDir, "uninitialized"), 0o755))
gitCmd(t, "-C", repoDir, "add", "-f", ".gitignore", "Renamed", "ign/tracked")
require.NoError(t, os.Rename(filepath.Join(repoDir, "Renamed"), filepath.Join(repoDir, "renamed")))
for _, gitlink := range []string{"sub", "uninitialized"} {
gitCmd(t, "-C", repoDir, "update-index", "--add", "--cacheinfo", "160000,4b825dc642cb6eb9a060e54bf8d69288fbee4904,"+gitlink)
}
gitCmd(t, "-C", repoDir, "update-index", "--add",
"--cacheinfo", "160000,4b825dc642cb6eb9a060e54bf8d69288fbee4904,sub",
"--cacheinfo", "160000,4b825dc642cb6eb9a060e54bf8d69288fbee4904,uninitialized",
)
expected := map[string]string{".gitignore": "", "ign/tracked": "", "nested/kept": "", "renamed": "", "sub/.gitignore": "", "sub/kept": ""}
if runtime.GOOS != "windows" {
require.NoError(t, os.Chmod(filepath.Join(repoDir, "renamed"), 0o777|os.ModeSticky))
-3
View File
@@ -80,9 +80,6 @@ func lookPath(file string, env map[string]string) (string, error) {
return "", &Error{file, err}
}
}
if f, err := findExecutable(filepath.Join(".", file), exts); err == nil {
return f, nil
}
path := getenv(env, "path")
for _, dir := range filepath.SplitList(path) {
if f, err := findExecutable(filepath.Join(dir, file), exts); err == nil {
+20
View File
@@ -0,0 +1,20 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package lookpath
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
)
func TestLookPath2IgnoresTheCurrentDirectory(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "tool.exe"), nil, 0o755))
t.Chdir(dir)
_, err := LookPath2("tool", map[string]string{"PATH": t.TempDir()})
require.EqualError(t, err, ErrNotFound.Error())
}
+23 -24
View File
@@ -339,34 +339,32 @@ func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, b
if !correctArchExists || rc.Config.ForceRebuild {
logger.Debugf("image '%s' for architecture '%s' will be built from context '%s", image, rc.Config.ContainerArchitecture, contextDir)
var buildContext io.ReadCloser
if localAction {
buildContext, err = rc.JobContainer.GetContainerArchive(ctx, contextDir+"/.")
if err != nil {
return err
prepImage = common.Executor(func(ctx context.Context) error {
input := container.NewDockerBuildExecutorInput{
ContextDir: contextDir,
Dockerfile: fileName,
ImageTag: image,
Platform: rc.Config.ContainerArchitecture,
BuildArgs: rc.proxyBuildArgs(),
Stdout: common.NewLineWriter(rawLogLine(ctx)),
}
defer buildContext.Close()
}
prepImage = ContainerNewDockerBuildExecutor(container.NewDockerBuildExecutorInput{
ContextDir: contextDir,
Dockerfile: fileName,
ImageTag: image,
BuildContext: buildContext,
Platform: rc.Config.ContainerArchitecture,
BuildArgs: rc.proxyBuildArgs(),
})
if buildContext == nil {
// Held across the whole build: the daemon drains contextDir lazily.
inner := prepImage
prepImage = func(ctx context.Context) error {
unlock, err := git.AcquireCloneLock(ctx, actionDir)
if localAction {
input.Dockerignore, _ = readSingleFileFromContainerArchive(ctx, rc.JobContainer, contextDir+"/.dockerignore", 1<<20)
buildContext, err := rc.JobContainer.GetContainerArchive(ctx, contextDir+"/.")
if err != nil {
return err
}
defer buildContext.Close()
input.BuildContext = buildContext
} else {
unlock, err := git.AcquireCloneLock(ctx, actionDir) // held across the whole build, the daemon drains contextDir lazily
if err != nil {
return err
}
defer unlock()
return inner(ctx)
}
}
return ContainerNewDockerBuildExecutor(input)(ctx)
}).Retry(3)
} else {
logger.Debugf("image '%s' for architecture '%s' already exists", image, rc.Config.ContainerArchitecture)
}
@@ -476,7 +474,7 @@ func newStepContainer(ctx context.Context, step step, image string, cmd, entrypo
envList = append(envList, rc.runnerEnv(ctx)...)
binds, mounts := rc.GetBindsAndMounts(ctx)
binds, mounts, _ := rc.GetBindsAndMounts(ctx)
networkMode := "container:" + rc.jobContainerName()
if rc.IsHostEnv() {
networkMode = "default"
@@ -591,7 +589,8 @@ func shouldRunPreStep(step actionStep) common.Conditional {
func hasPreStep(step actionStep) common.Conditional {
return func(ctx context.Context) bool {
action := step.getActionModel()
return action.Runs.Using.IsComposite() ||
// a pre stage evaluates `with` before earlier steps run, so only a composite nesting actions gets one
return (action.Runs.Using.IsComposite() && slices.ContainsFunc(action.Runs.Steps, func(compositeStep model.Step) bool { return compositeStep.Uses != "" })) ||
(action.Runs.Using.IsNode() &&
action.Runs.Pre != "") ||
(action.Runs.Using.IsDocker() &&
+2 -2
View File
@@ -217,11 +217,11 @@ func (rc *RunContext) compositeExecutor(action *model.Action) *compositeSteps {
steps = append(steps, newCompositeStepLogExecutor(rc.newCompositeCommandExecutor(step.main()), stepID))
// run the post executor in reverse order
stepPost := rc.newCompositeCommandExecutor(step.post())
stepPost := newCompositeStepLogExecutor(rc.newCompositeCommandExecutor(step.post()), stepID)
if postExecutor != nil {
stepPost = stepPost.Finally(postExecutor)
}
postExecutor = newCompositeStepLogExecutor(stepPost, stepID)
postExecutor = stepPost
}
steps = append(steps, common.JobError)
+14 -1
View File
@@ -92,13 +92,26 @@ func TestCompositeActionParity(t *testing.T) {
return "", "", assert.AnError
})
rc := &RunContext{
Config: &Config{GitHubInstance: "github.com", ActionCacheDir: t.TempDir(), PresetGitHubContext: &model.GithubContext{}},
Config: &Config{GitHubInstance: "github.com", PresetGitHubContext: &model.GithubContext{}},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},
JobContainer: &jobContainerMock{},
}
require.ErrorIs(t, rc.compositeExecutor(&model.Action{Runs: model.ActionRuns{Using: "composite", Steps: []model.Step{{ID: "nested", Uses: "org/action@v1"}}}}).pre(t.Context()), assert.AnError)
})
t.Run("post steps log under their own step ids", func(t *testing.T) {
hook := logtest.NewWithLevel(logging.LevelDebug)
rc := &RunContext{Config: &Config{}, JobContainer: &jobContainerMock{}}
steps := rc.compositeExecutor(&model.Action{Runs: model.ActionRuns{Using: "composite", Steps: []model.Step{{ID: "first", Uses: "./first"}, {ID: "second", Uses: "./second"}}}})
require.NoError(t, steps.post(common.WithLogger(t.Context(), logging.New(hook))))
var stepIDs []any
for _, record := range hook.AllRecords() {
ids, _ := record.Attr("stepID")
stepIDs = append(stepIDs, ids)
}
assert.Equal(t, []any{[]string{"second"}, []string{"first"}}, stepIDs)
})
}
func TestAppendUniqueMasks(t *testing.T) {
+21 -14
View File
@@ -249,6 +249,7 @@ func TestActionRunner(t *testing.T) {
}
func TestGoActionUsesLinuxExecutablePaths(t *testing.T) {
actionRoot := t.TempDir()
for _, stage := range []string{"main", "pre", "post"} {
t.Run(stage, func(t *testing.T) {
cm := &containerMock{}
@@ -264,7 +265,7 @@ func TestGoActionUsesLinuxExecutablePaths(t *testing.T) {
JobContainer: cm,
},
action: &model.Action{Runs: model.ActionRuns{Using: model.ActionRunsUsingGo, Main: "main.go", Pre: "pre.go", Post: "post.go"}},
actionDir: t.TempDir(),
actionDir: filepath.Join(actionRoot, stage),
env: map[string]string{},
remoteAction: mustNewRemoteAction(t, "org/repo@ref"),
}
@@ -504,11 +505,21 @@ func TestExecAsDockerHoldsCloneLockForRemoteUncached(t *testing.T) {
}
func TestExecAsDockerImageChecks(t *testing.T) {
actionRoot := t.TempDir()
originalExists, originalBuild, originalContainer := ContainerImageExistsLocally, ContainerNewDockerBuildExecutor, ContainerNewContainer
t.Cleanup(func() {
ContainerImageExistsLocally, ContainerNewDockerBuildExecutor, ContainerNewContainer = originalExists, originalBuild, originalContainer
})
ContainerNewContainer = func(*container.NewContainerInput) container.ExecutionsEnvironment { return fakeContainer{} }
var removed bool
fakeDockerDaemon(t, func(writer http.ResponseWriter, request *http.Request) {
if request.Method == http.MethodDelete {
removed = true
_, _ = io.WriteString(writer, `[]`)
} else {
_, _ = io.WriteString(writer, `{"Architecture":"aarch64"}`)
}
})
for _, testcase := range []struct {
name, platform string
@@ -529,18 +540,11 @@ func TestExecAsDockerImageChecks(t *testing.T) {
checks = append(checks, platform)
return testcase.exists && (platform == "any" || testcase.matches), nil
}
var built, removed bool
var built bool
removed = false
ContainerNewDockerBuildExecutor = func(container.NewDockerBuildExecutorInput) common.Executor {
return func(context.Context) error { built = true; return nil }
}
fakeDockerDaemon(t, func(writer http.ResponseWriter, request *http.Request) {
if request.Method == http.MethodDelete {
removed = true
_, _ = io.WriteString(writer, `[]`)
} else {
_, _ = io.WriteString(writer, `{"Architecture":"aarch64"}`)
}
})
step := &stepActionRemote{
Step: &model.Step{ID: "action"},
RunContext: &RunContext{
@@ -555,7 +559,7 @@ func TestExecAsDockerImageChecks(t *testing.T) {
action: &model.Action{Runs: model.ActionRuns{Using: "docker", Image: "Dockerfile"}},
env: map[string]string{},
}
dir := t.TempDir()
dir := filepath.Join(actionRoot, testcase.name)
require.NoError(t, execAsDocker(t.Context(), step, "action", dir, dir, false, stepStageMain))
assert.Equal(t, testcase.wantChecks, checks)
assert.Equal(t, testcase.wantBuild, built)
@@ -727,9 +731,9 @@ func TestExecAsDockerStageEntrypoint(t *testing.T) {
cm.On("Close").Return(func(context.Context) error { return nil })
if tc.stage == stepStageMain {
require.NoError(t, runActionImpl(step, t.TempDir(), step.remoteAction)(ctx))
require.NoError(t, runActionImpl(step, "", step.remoteAction)(ctx))
} else {
require.NoError(t, execAsDocker(ctx, step, "action", t.TempDir(), t.TempDir(), false, tc.stage))
require.NoError(t, execAsDocker(ctx, step, "action", "", "", false, tc.stage))
}
if tc.warning == "" {
assert.Empty(t, hook.AllRecords())
@@ -747,12 +751,15 @@ func TestExecAsDockerStageEntrypoint(t *testing.T) {
}
}
func TestDockerActionHasPreAndPostStep(t *testing.T) {
func TestActionHasPreAndPostStep(t *testing.T) {
newStep := func(runs model.ActionRuns) actionStep {
return &stepActionRemote{action: &model.Action{Runs: runs}}
}
ctx := t.Context()
assert.False(t, hasPreStep(newStep(model.ActionRuns{Using: "composite", Steps: []model.Step{{Run: "true"}}}))(ctx))
assert.True(t, hasPreStep(newStep(model.ActionRuns{Using: "composite", Steps: []model.Step{{Run: "true"}, {Uses: "actions/cache@v4"}}}))(ctx))
assert.False(t, hasPreStep(newStep(model.ActionRuns{Using: "docker", Image: "Dockerfile"}))(ctx))
assert.False(t, hasPostStep(newStep(model.ActionRuns{Using: "docker", Image: "Dockerfile"}))(ctx))
+8 -5
View File
@@ -101,10 +101,10 @@ func TestMainStepsExecutorRunsAlwaysStepsAfterInterruption(t *testing.T) {
assert.Equal(t, []string{"step1", "always-step"}, ran)
require.NoError(t, laterStepCtxErr)
assert.Equal(t, wantError == context.Canceled, rc.jobCancelled)
assert.Equal(t, "cancelled", rc.getJobContext().Status)
setJobResult(t.Context(), rc, rc, true)
if wantError == context.DeadlineExceeded {
assert.True(t, rc.jobFailed)
assert.Equal(t, "failure", rc.getJobContext().Status)
assert.Equal(t, "failure", rc.Run.Job().Result)
} else {
assert.Equal(t, "cancelled", rc.Run.Job().Result)
@@ -259,8 +259,11 @@ func TestReportStepErrorTreatsCancelAsInterruption(t *testing.T) {
assert.True(t, rc.jobCancelled)
// a real error still fails the job
failed := common.WithJobErrorContainer(t.Context())
reportStepError(failed, rc, assert.AnError)
require.ErrorIs(t, common.JobError(failed), assert.AnError)
assert.True(t, rc.jobFailed)
for _, stepErr := range []error{assert.AnError, context.DeadlineExceeded} {
rc := &RunContext{}
failed := common.WithJobErrorContainer(t.Context())
reportStepError(failed, rc, stepErr)
require.ErrorIs(t, common.JobError(failed), stepErr)
assert.Equal(t, "failure", rc.getJobContext().Status)
}
}
+5 -1
View File
@@ -118,7 +118,11 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
rawLogger.Infof("%s", formatAnnotation(command, kvPairs, arg))
return false
case "group", "endgroup":
return true
if strings.HasPrefix(line, "::"+command+"::") || strings.HasPrefix(line, "##["+command+"]") {
return true
}
rawLogger.Infof("::%s::%s", command, EscapeCommandData(arg))
return false
}
return echoed
}
Loaded 50 of 151 files, more files were not shown because too many files have changed in this diff. Show more