Compare commits

...
19 Commits
Author SHA1 Message Date
silverwindandbircni 0e4c8b6b78 fix: misc improvements (#1415)
- Stop overwriting the image's `ImageOS`. `ubuntu-latest` got `ubuntu20` on the 24.04 default image.
- Derive `ImageOS` from `os-release` when the image sets none. Fall back to `ubuntu24` for `ubuntu-latest`.
- Keep an `ImageOS` set in the job, step or container env.
- Honor the image's `ImageOS` on Kubernetes.
- Treat steps after a job timeout as cancelled. `cancelled()` and `always()` cleanup now runs. The job still fails.
- Record nested reusable workflow results. Jobs that `needs:` a nested call now run.
- Forward outputs through nested reusable workflows. A bad output fails the call.
- Pass the job's `GITHUB_TOKEN` and `GITEA_TOKEN` to called workflows by default. An explicit mapping overrides them.
- Probe a started container's uid and gid concurrently.
- Cache the Docker container env from its first inspect.
- Check `ImageOS` in the Kubernetes e2e test.

---------

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1415
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-16 20:51:26 +00:00
silverwind 1f2ea1b83a fix: honor zero and empty values for fetch backoff and idle cleanup (#1418)
`fetch_interval_max: 0` is documented to disable polling backoff but became 5s. Zeros for `workdir_cleanup_age` and `idle_cleanup_interval` were ignored when they came from a YAML merge key or alias, so cleanup still ran, while an empty or null value turned cleanup off. These defaults are now seeded before the file is read, as `cache` already does, so a written 0 means off and an empty value keeps the default.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1418
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-16 20:43:37 +00:00
Renovate Bot 891f3fde38 fix(deps): update dependencies (#1414)
This PR contains the following updates:

| Package | Type | Update | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|---|---|
| docker.gitea.com/runner-images |  | digest | `b96ba46` → `139605d` |  |  |
| [docker/login-action](https://github.com/docker/login-action) ([changelog](https://github.com/docker/login-action/compare/81c6f93d4bc991e510b342a00b419b3bb4efc5c6..8aa5831fb8d48d8f76594c7ddde61ae58019f550)) | action | digest | `81c6f93` → `8aa5831` |  |  |
| [github.com/spf13/pflag](https://github.com/spf13/pflag) | require | minor | `v1.0.10` → `v1.1.0` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fspf13%2fpflag/v1.1.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fspf13%2fpflag/v1.0.10/v1.1.0?slim=true) |
| [go](https://go.dev/) ([source](https://github.com/golang/go)) | toolchain | patch | `1.27.3` → `1.27.4` | ![age](https://developer.mend.io/api/mc/badges/age/golang-version/go/1.27.4?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/golang-version/go/1.27.3/1.27.4?slim=true) |
| [golang.org/x/sync](https://pkg.go.dev/golang.org/x/sync) | require | minor | [`v0.24.0` → `v0.25.0`](https://cs.opensource.google/go/x/sync/+/refs/tags/v0.24.0...refs/tags/v0.25.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fsync/v0.25.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fsync/v0.24.0/v0.25.0?slim=true) |
| [golang.org/x/sys](https://pkg.go.dev/golang.org/x/sys) | require | minor | [`v0.49.0` → `v0.50.0`](https://cs.opensource.google/go/x/sys/+/refs/tags/v0.49.0...refs/tags/v0.50.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fsys/v0.50.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fsys/v0.49.0/v0.50.0?slim=true) |
| [golang.org/x/term](https://pkg.go.dev/golang.org/x/term) | require | minor | [`v0.47.0` → `v0.48.0`](https://cs.opensource.google/go/x/term/+/refs/tags/v0.47.0...refs/tags/v0.48.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fterm/v0.48.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fterm/v0.47.0/v0.48.0?slim=true) |
| [golang.org/x/text](https://pkg.go.dev/golang.org/x/text) | require | minor | [`v0.43.0` → `v0.44.0`](https://cs.opensource.google/go/x/text/+/refs/tags/v0.43.0...refs/tags/v0.44.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2ftext/v0.44.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2ftext/v0.43.0/v0.44.0?slim=true) |
| [quay.io/containers/podman](https://quay.io/containers/podman) ([source](https://github.com/podman-container-tools/podman)) |  | major | `v5.8.9-immutable` → `v6.1.3-immutable` | ![age](https://developer.mend.io/api/mc/badges/age/docker/quay.io%2fcontainers%2fpodman/v6.1.3?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/docker/quay.io%2fcontainers%2fpodman/v5.8.9/v6.1.3?slim=true) |

---

### HTTP/1 server crash due to MaxBytesReader header race in net/http
[CVE-2026-97054](https://nvd.nist.gov/vuln/detail/CVE-2026-97054) / [GO-2026-6859](https://pkg.go.dev/vuln/GO-2026-6859)

<details>
<summary>More information</summary>

#### Details
For HTTP/1 servers, when MaxBytesReader's limit is exceeded, "Connection: close" header is automatically set without synchronization, which might race with another header being set explicitly in the server handler. As a result, an HTTP/1 server that utilizes MaxBytesReader or MaxBytesHandler can crash due to runtime panic if its handler reads the request body concurrently with writing the response header.

#### Severity
Unknown

#### References
- [https://go.dev/cl/862185](https://go.dev/cl/862185)
- [https://go.dev/issue/82563](https://go.dev/issue/82563)
- [https://groups.google.com/g/golang-announce/c/iY8TY0wWgbM](https://groups.google.com/g/golang-announce/c/iY8TY0wWgbM)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6859) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Release Notes

<details>
<summary>spf13/pflag (github.com/spf13/pflag)</summary>

### [`v1.1.0`](https://github.com/spf13/pflag/releases/tag/v1.1.0)

[Compare Source](https://github.com/spf13/pflag/compare/v1.0.10...v1.1.0)

It's been long enough, and enough things have been fixed, that it might be time to release a new version, even if there isn't any particular big new issue to market as the reason for this release.

If you notice anything breaking, it is entirely un-intentional, but you might also have been relying on [undocumented](https://www.hyrumslaw.com/) or [buggy](https://xkcd.com/1172/) behavior. Please file an issue describing what you're doing, how it worked before this release and why it doesn't work anymore, and pin your projects to `v1.0.10` (the latest release before this one) while we decide how to address your concerns.

#### What's Changed

- README: update badges and minor linting fixes by [@&#8203;thaJeztah](https://github.com/thaJeztah) in [#&#8203;452](https://github.com/spf13/pflag/pull/452)
- Show 'correct' explicit bool value syntax in help by [@&#8203;alimpfard](https://github.com/alimpfard) in [#&#8203;453](https://github.com/spf13/pflag/pull/453)
- Fix for Go 1.24 by [@&#8203;pterjan](https://github.com/pterjan) in [#&#8203;460](https://github.com/spf13/pflag/pull/460)
- fix(map): ensure string-to-string flags maintain stable pointers by [@&#8203;brandon1024](https://github.com/brandon1024) in [#&#8203;457](https://github.com/spf13/pflag/pull/457)
- Fix getFlagType error message by [@&#8203;wellitonscheer](https://github.com/wellitonscheer) in [#&#8203;462](https://github.com/spf13/pflag/pull/462)
- update actions, and fix linting issues by [@&#8203;thaJeztah](https://github.com/thaJeztah) in [#&#8203;450](https://github.com/spf13/pflag/pull/450)
- fix: README.md indicating the use of godoc(deprecated) by [@&#8203;malklera](https://github.com/malklera) in [#&#8203;467](https://github.com/spf13/pflag/pull/467)
- Fix broken link to POSIX argument conventions by [@&#8203;tomasaschan](https://github.com/tomasaschan) in [#&#8203;464](https://github.com/spf13/pflag/pull/464)
- Replace raw stdlib parse errors in typed flag Set methods ([#&#8203;269](https://github.com/spf13/pflag/issues/269)) by [@&#8203;reuvenharrison](https://github.com/reuvenharrison) in [#&#8203;469](https://github.com/spf13/pflag/pull/469)
- Extend `UnknownFlags` to `UnknownFlagsHandling` and introduce `PassUnknownFlagToArgs` ([#&#8203;337](https://github.com/spf13/pflag/issues/337)) by [@&#8203;ikedam](https://github.com/ikedam) in [#&#8203;440](https://github.com/spf13/pflag/pull/440)
- fix: allow nil default IP flags in GetIP by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;483](https://github.com/spf13/pflag/pull/483)
- fix: allow hex input in UintSlice by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;484](https://github.com/spf13/pflag/pull/484)
- fix: honor custom IsBoolFlag compatibility by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;487](https://github.com/spf13/pflag/pull/487)
- docs: clarify SortFlags example with complete runnable program by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;488](https://github.com/spf13/pflag/pull/488)
- docs: add release process documentation by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;490](https://github.com/spf13/pflag/pull/490)
- fix: print to set destination, not hardcoded os.Stderr by [@&#8203;quite](https://github.com/quite) in [#&#8203;502](https://github.com/spf13/pflag/pull/502)
- ci: bump golangci-lint to v2.13 for Go 1.27 by [@&#8203;hdimer](https://github.com/hdimer) in [#&#8203;508](https://github.com/spf13/pflag/pull/508)
- Initialize the target map when a StringToString flag has a nil default by [@&#8203;nikolauspschuetz](https://github.com/nikolauspschuetz) in [#&#8203;506](https://github.com/spf13/pflag/pull/506)
- fix: honour deprecated ParseErrorsWhitelist.UnknownFlagsHandling by [@&#8203;lenamonj](https://github.com/lenamonj) in [#&#8203;507](https://github.com/spf13/pflag/pull/507)
- test: add StringSlice Append coverage by [@&#8203;LY-19418](https://github.com/LY-19418) in [#&#8203;504](https://github.com/spf13/pflag/pull/504)
- Keep wrapping after an unbreakable usage word by [@&#8203;AzazelSensei](https://github.com/AzazelSensei) in [#&#8203;503](https://github.com/spf13/pflag/pull/503)
- Validate IP slice values before appending or replacing by [@&#8203;agammann](https://github.com/agammann) in [#&#8203;513](https://github.com/spf13/pflag/pull/513)
- fix(ci): Use the latest golangci-lint release by [@&#8203;tomasaschan](https://github.com/tomasaschan) in [#&#8203;525](https://github.com/spf13/pflag/pull/525)
- Reject IPv6 addresses as IPv4 masks by [@&#8203;yuefdev](https://github.com/yuefdev) in [#&#8203;524](https://github.com/spf13/pflag/pull/524)
- Report the flag name as typed in deprecation warnings by [@&#8203;hdimer](https://github.com/hdimer) in [#&#8203;500](https://github.com/spf13/pflag/pull/500)
- Document non-interspersed argument parsing by [@&#8203;yuefdev](https://github.com/yuefdev) in [#&#8203;523](https://github.com/spf13/pflag/pull/523)
- Preserve trailing whitespace in StringToString formatting by [@&#8203;r4venrudy](https://github.com/r4venrudy) in [#&#8203;522](https://github.com/spf13/pflag/pull/522)
- Reject extra CSV records in StringToString arguments by [@&#8203;Harshil-1603](https://github.com/Harshil-1603) in [#&#8203;521](https://github.com/spf13/pflag/pull/521)
- Preserve current string values in CopyToGoFlagSet by [@&#8203;adenzhou1350](https://github.com/adenzhou1350) in [#&#8203;520](https://github.com/spf13/pflag/pull/520)
- Fix type identification and mismatch checks for text flags by [@&#8203;dewanshshekhar](https://github.com/dewanshshekhar) in [#&#8203;526](https://github.com/spf13/pflag/pull/526)
- fix: enforce native integer flag bounds on 32-bit platforms by [@&#8203;gdpu11](https://github.com/gdpu11) in [#&#8203;534](https://github.com/spf13/pflag/pull/534)
- fix: preserve count flag values when parsing fails by [@&#8203;gdpu11](https://github.com/gdpu11) in [#&#8203;533](https://github.com/spf13/pflag/pull/533)

#### New Contributors

- [@&#8203;alimpfard](https://github.com/alimpfard) made their first contribution in [#&#8203;453](https://github.com/spf13/pflag/pull/453)
- [@&#8203;pterjan](https://github.com/pterjan) made their first contribution in [#&#8203;460](https://github.com/spf13/pflag/pull/460)
- [@&#8203;brandon1024](https://github.com/brandon1024) made their first contribution in [#&#8203;457](https://github.com/spf13/pflag/pull/457)
- [@&#8203;wellitonscheer](https://github.com/wellitonscheer) made their first contribution in [#&#8203;462](https://github.com/spf13/pflag/pull/462)
- [@&#8203;malklera](https://github.com/malklera) made their first contribution in [#&#8203;467](https://github.com/spf13/pflag/pull/467)
- [@&#8203;reuvenharrison](https://github.com/reuvenharrison) made their first contribution in [#&#8203;469](https://github.com/spf13/pflag/pull/469)
- [@&#8203;ikedam](https://github.com/ikedam) made their first contribution in [#&#8203;440](https://github.com/spf13/pflag/pull/440)
- [@&#8203;happysnaker](https://github.com/happysnaker) made their first contribution in [#&#8203;483](https://github.com/spf13/pflag/pull/483)
- [@&#8203;quite](https://github.com/quite) made their first contribution in [#&#8203;502](https://github.com/spf13/pflag/pull/502)
- [@&#8203;hdimer](https://github.com/hdimer) made their first contribution in [#&#8203;508](https://github.com/spf13/pflag/pull/508)
- [@&#8203;nikolauspschuetz](https://github.com/nikolauspschuetz) made their first contribution in [#&#8203;506](https://github.com/spf13/pflag/pull/506)
- [@&#8203;lenamonj](https://github.com/lenamonj) made their first contribution in [#&#8203;507](https://github.com/spf13/pflag/pull/507)
- [@&#8203;LY-19418](https://github.com/LY-19418) made their first contribution in [#&#8203;504](https://github.com/spf13/pflag/pull/504)
- [@&#8203;AzazelSensei](https://github.com/AzazelSensei) made their first contribution in [#&#8203;503](https://github.com/spf13/pflag/pull/503)
- [@&#8203;agammann](https://github.com/agammann) made their first contribution in [#&#8203;513](https://github.com/spf13/pflag/pull/513)
- [@&#8203;yuefdev](https://github.com/yuefdev) made their first contribution in [#&#8203;524](https://github.com/spf13/pflag/pull/524)
- [@&#8203;r4venrudy](https://github.com/r4venrudy) made their first contribution in [#&#8203;522](https://github.com/spf13/pflag/pull/522)
- [@&#8203;Harshil-1603](https://github.com/Harshil-1603) made their first contribution in [#&#8203;521](https://github.com/spf13/pflag/pull/521)
- [@&#8203;adenzhou1350](https://github.com/adenzhou1350) made their first contribution in [#&#8203;520](https://github.com/spf13/pflag/pull/520)
- [@&#8203;dewanshshekhar](https://github.com/dewanshshekhar) made their first contribution in [#&#8203;526](https://github.com/spf13/pflag/pull/526)
- [@&#8203;gdpu11](https://github.com/gdpu11) made their first contribution in [#&#8203;534](https://github.com/spf13/pflag/pull/534)

**Full Changelog**: <https://github.com/spf13/pflag/compare/v1.0.10...v1.1.0>

</details>

<details>
<summary>podman-container-tools/podman (quay.io/containers/podman)</summary>

### [`v6.1.3`](https://github.com/podman-container-tools/podman/releases/tag/v6.1.3)

[Compare Source](https://github.com/podman-container-tools/podman/compare/v5.8.9...v6.1.3)

##### Security

- This release addresses [CVE-2026-94603](https://github.com/podman-container-tools/podman/security/advisories/GHSA-2cvf-wqm6-wr9g), where a `podman run` on a checkpoint image (any image with the `io.podman.annotations.checkpoint.runtime.name` annotation) could disable all sandboxing, including sandboxing specified by the user, when the container was created.

##### Breaking Changes

- Removed support for checkpoint images in `podman run` due to serious security concerns with the different security models of running images and running checkpoints. Checkpoints ignore user-specified security configuration and are very difficult to run safely.

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4yMjIuNSIsInVwZGF0ZWRJblZlciI6IjQ0LjIzOS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1414
Reviewed-by: silverwind <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
2026-11-16 15:15:09 +00:00
silverwind 4f59e3e418 fix: support cache-mode and make low-trust runs cache read-only (#1412)
Low-trust runs like `pull_request_target` on the default branch could poison caches that `push` runs restore. As on GitHub, they now only restore unless `cache-mode` allows saving.

This and the Gitea change can be deployed in any order.

Refs:
https://gitea.com/gitea/actionslib/pulls/33
https://github.com/go-gitea/gitea/pull/40343

_Written by Claude (Opus 5.5)._
Reviewed-on: https://gitea.com/gitea/runner/pulls/1412
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-13 09:33:32 +00:00
clintlmartinandsilverwind c2046d597c fix: stop host teardown from deleting a workdir it never created (#1409)
Host jobs run in a per-job directory under `host.workdir_parent`, which teardown removes. Teardown also deleted `<container.workdir_parent>/<owner>/<repo>`, a path host jobs never create. With an empty parent and no repository that was the filesystem root, and any existing directory at that path was deleted although the job never used it.

This removes that delete instead of guarding it, so teardown only removes what the job created.

---------

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1409
Reviewed-by: bircni <[email protected]>
Co-authored-by: clintlmartin <[email protected]>
2026-11-13 06:50:44 +00:00
silverwind 645b17de72 fix: align common behaviour with GitHub's runner (#1413)
Closes differences from GitHub's runner that common workflows hit:

- `ACTIONS_STEP_DEBUG` set as a repository variable enables debug output as on GitHub, where a secret of the same name still takes precedence. `runner.debug` and `RUNNER_DEBUG` are now also set inside composite actions, which lost the secret.
- `github.env` and `github.path` hold the file-command paths instead of being empty, so `echo "A=1" >> "${{ github.env }}"` works.
- Remote composite actions only get a pre stage when they nest actions. That stage evaluated `with:` before earlier steps ran, so an input like `${{ fromJSON(steps.meta.outputs.json).v }}` failed the job.
- `ports` on a job container are published, they were ignored while service ports worked.
- Dockerfile action build output reaches the job log. It was logged at debug and dropped, so a failing `RUN` showed only the final error.
- Local Docker actions honor their `.dockerignore`, as `docker build` does.
- Cancelling or timing out a host-mode step sends SIGINT, then SIGTERM after 7.5 seconds and SIGKILL after 2.5 more, as GitHub's runner does, so `trap` handlers and shutdown hooks run.
- `.sh` job hooks fall back to `sh` where bash is missing, and `.ps1` hooks to `powershell` where pwsh is missing, as on GitHub.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1413
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-12 15:28:47 +00:00
silverwind 0dfec377d2 fix: keep composite post steps in their own step scope (#1410)
Each composite step's post ran its earlier siblings' posts inside its own step-ID scope, so their log records listed siblings as ancestors, e.g. `outer.results.nodepost` instead of `outer.nodepost`. Every post now gets its own scope. Execution order and run-all-posts behavior are unchanged.

Prerequisite for https://gitea.com/gitea/runner/issues/809.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1410
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-11 19:31:21 +00:00
silverwind b5305c6785 docs: add a launchd example for macOS (#1408)
Adds `examples/launchd` with a plist and the steps to run the runner on macOS, next to the existing systemd example.

The runner runs in the login session of the user that jobs should run as, which gives jobs the login keychain and the GUI. The README also covers the two problems that setup runs into: macOS blocks the runner from hosts on the Mac's own subnet, and Git's keychain helper can stop a job at a dialog.

_Written by Claude (Fable 5.2)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1408
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-10 17:17:35 +00:00
bircni a021d3139d enhance: make shared tool cache safe at any capacity (#1401)
Co-authored-by: bircni <[email protected]>
2026-11-10 14:25:50 +00:00
bircniandsilverwind 84c0fcd1fa fix!: remove the dbfs log workaround and require Gitea 1.26.2 (#1395)
Gitea 1.26.2 finalizes logs on an empty `UpdateLog` with `NoMore` (https://github.com/go-gitea/gitea/pull/37687), so the sentinel row is no longer needed. On older servers, the final log of most jobs would never be archived and stays in `dbfs_data`, so this raises the minimum Gitea version. It lives in `ver.MinGiteaVersion`, which the error for servers without `Declare` now names.

Fixes: https://gitea.com/gitea/runner/issues/953
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1395
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-10 11:35:16 +00:00
bircniandsilverwind c59caa9689 refactor!: remove the builtin actions (#1407)
Removes `builtin:checkout`, `builtin:upload-artifact` and `builtin:download-artifact`, along with the container helpers only they used.

Builtin actions are tied to the runner version, so users cannot pin them, and they diverge from the shareable actions ecosystem that tools like zizmor build on. Workflows should use `actions/checkout`, `actions/upload-artifact` and `actions/download-artifact`, or lightweight third-party actions such as https://github.com/taiki-e/checkout-action.

**Breaking:** steps with `uses: builtin:checkout`, released in v4.0.0, now fail and need to switch to one of the actions above. The artifact actions were never released.

Docs: https://gitea.com/gitea/docs/pulls/643
Fixes: https://gitea.com/gitea/runner/issues/1405
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1407
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-10 11:05:38 +00:00
Renovate Bot 85d007c47f chore(deps): update dependencies (#1406)
This PR contains the following updates:

| Package | Type | Update | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|---|---|
| docker.gitea.com/runner-images |  | digest | `657378e` → `b96ba46` |  |  |
| [github.com/docker/cli](https://github.com/docker/cli) | require | patch | `v29.9.2+incompatible` → `v29.9.3+incompatible` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fdocker%2fcli/v29.9.3+incompatible?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fdocker%2fcli/v29.9.2+incompatible/v29.9.3+incompatible?slim=true) |
| [rancher/k3s](https://github.com/k3s-io/k3s) |  | patch | `v1.37.1-k3s1` → `v1.37.2-k3s1` | ![age](https://developer.mend.io/api/mc/badges/age/docker/rancher%2fk3s/v1.37.2?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/docker/rancher%2fk3s/v1.37.1/v1.37.2?slim=true) |

---

### Release Notes

<details>
<summary>docker/cli (github.com/docker/cli)</summary>

### [`v29.9.3+incompatible`](https://github.com/docker/cli/compare/v29.9.2...v29.9.3)

[Compare Source](https://github.com/docker/cli/compare/v29.9.2...v29.9.3)

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xOTkuMiIsInVwZGF0ZWRJblZlciI6IjQ0LjE5OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1406
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
2026-11-09 04:31:50 +00:00
silverwindandbircni d34e4cf11a fix: stop a stalled docker daemon from hanging jobs and the e2e suite (#1404)
When the docker daemon stopped answering on https://gitea.com/gitea/runner/actions/runs/1204971, unbounded exec runs hung the e2e suite until the 20m `go test` panic, which skipped teardown. Exec runs and teardown are now bounded, and attached exec streams, including the runner's user ID probe, close with their context.

CLI daemons now stop with SIGTERM, so job volumes no longer leak.

_Written by Claude (Opus 5.5)._

---------

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1404
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-07 08:07:41 +00:00
8e999cd039 fix: re-detect the cache address per job and prefer IPv4 (#1398)
The cache address was detected once at startup, and the interface fallback ranked ethernet names above IPv4. A runner that started before DHCPv4 finished could advertise a global IPv6 address to IPv4-only job networks until it restarted, failing every artifact and cache call.

IPv4 now ranks first, and the address is detected again for each job unless `cache.host` sets it. Detecting an IPv6 address logs a warning.

Fixes: https://gitea.com/gitea/runner/issues/1379
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1398
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-07 06:41:25 +00:00
silverwind cd748053b0 docs: trim comments and docs from workspace emptying change (#1403)
Follow-up to https://gitea.com/gitea/runner/commit/79faf5e55df7a3b312fb4888b7405a58e01f567c. Reverts the post-task-script doc change, which is too much detail for that page. Also shortens the `DrainGracePeriod` comment and the other comments that commit added, since the code and log messages already say most of it.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1403
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-07 06:14:41 +00:00
bircni 9bb72bbe5c enhance: add ubuntu-26.04 to the default labels (#1396)
Newly registered runners get an `ubuntu-26.04` label alongside the existing defaults.

Ref: https://gitea.com/gitea/runner/issues/1334
Reviewed-on: https://gitea.com/gitea/runner/pulls/1396
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-07 05:12:04 +00:00
79faf5e55d fix: empty root-owned bind workspaces for non-root runners (#1399)
A job running as root leaves root-owned files in the bind-mounted workspace. A non-root runner on Linux cannot remove them, so workspaces pile up with only a warning. Such a runner now empties the workspace as root from inside the job container, right before removing the container. Root runners and Docker Desktop already remove these files themselves and skip this.

The workspace is only emptied when it is exactly the runner's own mountpoint with no mount below it. Otherwise, or when the command fails, the existing removal and warning apply. Jobs of a reusable workflow share the workspace, so they are not covered and still end in the existing warning.

Fixes: https://gitea.com/gitea/runner/issues/879
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1399
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-07 04:33:19 +00:00
bircni 993389d428 docs: trust a private registry CA in the dind images (#1397)
The bundled Docker daemon pulls job images, so it needs to trust a private registry's CA itself, which `runner.ca_file` doesn't cover. Documents the `certs.d` mount for both dind images.

Ref: https://gitea.com/gitea/runner/issues/1390
Reviewed-on: https://gitea.com/gitea/runner/pulls/1397
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-06 21:36:06 +00:00
silverwind db76fd5159 enhance: improve hashFiles coercion and log grouping (#1394)
- Fix `hashFiles()` argument conversion for numbers, booleans, and null to match GitHub expression semantics.
- Fix rendering of mixed-case, indented, and prefixed legacy group commands while preserving recognized prefixes and escaping.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1394
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-06 20:03:39 +00:00
121 changed files with 1499 additions and 6253 deletions

No files matched your search

+1 -1
View File
@@ -87,7 +87,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
- name: Login to DockerHub - name: Login to DockerHub
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4
with: with:
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}
+1 -1
View File
@@ -82,7 +82,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
- name: Login to DockerHub - name: Login to DockerHub
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4
with: with:
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}
-5
View File
@@ -51,11 +51,6 @@ fields.
- Tests read what was logged with `logtest.NewLogger()`, or `logtest.SetDefault(t)` for the - Tests read what was logged with `logtest.NewLogger()`, or `logtest.SetDefault(t)` for the
package functions. package functions.
## Built-in actions
Each lives in `internal/pkg/action/<name>` and is registered in `builtinActions` in
`act/runner/step_action_builtin.go`.
## End-to-end compatibility tests ## End-to-end compatibility tests
`make test-e2e` runs the runner against `E2E_GITEA_IMAGE`. It defaults to the nightly image. `make test-e2e` runs the runner against `E2E_GITEA_IMAGE`. It defaults to the nightly image.
+3 -3
View File
@@ -3,11 +3,11 @@ GO ?= go
EXECUTABLE := gitea-runner$(shell $(GO) env GOEXE) EXECUTABLE := gitea-runner$(shell $(GO) env GOEXE)
HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" ) HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" )
PODMAN_TEST_IMAGE ?= quay.io/containers/podman:v5.8.9-immutable@sha256:87f42f37fa85b421664c30b8f5f9cfc23920057f5e5fcadf0b6f68d9edc6d415 # renovate: datasource=docker PODMAN_TEST_IMAGE ?= quay.io/containers/podman:v6.1.3-immutable@sha256:1d844fd1fc6ce68c1e72ae19550cc3caef039306219ee007d88f7e67c5df4dd1 # renovate: datasource=docker
E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:657378e5cfd898e36f820459c60ef8e69be8fcda9fd354e4381a8b654d091bbc # renovate: datasource=docker E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:139605d4a6ab61e48dfd7687b11184881c2d7fcf06bf739e9cadb34f73deb51d # renovate: datasource=docker
SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker
E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly
E2E_K3S_IMAGE ?= rancher/k3s:v1.37.1-k3s1@sha256:ca7f37d993d82ef0dcdcfecb2e0e2618ea541dbaffc620c8cedebe01a82acd0d # renovate: datasource=docker E2E_K3S_IMAGE ?= rancher/k3s:v1.37.2-k3s1@sha256:21ea6e0fac4e3930c2ce8aa40ddd41a8b18cb3e9a0222c030dc005e7e88c4332 # renovate: datasource=docker
WINDOWS_TEST_IMAGE ?= dockurr/windows:6.06@sha256:16fd7c39a3e48bb946d29c59dc6015942e5b900a1fa5d7e5552053bfc48ee2ea # renovate: datasource=docker WINDOWS_TEST_IMAGE ?= dockurr/windows:6.06@sha256:16fd7c39a3e48bb946d29c59dc6015942e5b900a1fa5d7e5552053bfc48ee2ea # renovate: datasource=docker
GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*") GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*")
+3 -9
View File
@@ -7,6 +7,7 @@ Gitea Runner runs the jobs of Gitea Actions workflows. Register it with a Gitea
### Prerequisites ### Prerequisites
- Linux, macOS or Windows, or FreeBSD. - Linux, macOS or Windows, or FreeBSD.
- Gitea 1.26.2 or newer.
- `git` 2.34.1 or newer. - `git` 2.34.1 or newer.
- `docker`, or `podman` with `podman.socket` enabled, for `docker` labels and Docker actions. See Docker's [install instructions](https://docs.docker.com/engine/install/). - `docker`, or `podman` with `podman.socket` enabled, for `docker` labels and Docker actions. See Docker's [install instructions](https://docs.docker.com/engine/install/).
@@ -14,9 +15,6 @@ In the job image, or on the host for host jobs:
- `bash` or `sh`, or `pwsh` or `powershell` on Windows. - `bash` or `sh`, or `pwsh` or `powershell` on Windows.
- `node` for JavaScript actions and `hashFiles()`. - `node` for JavaScript actions and `hashFiles()`.
- `git` 2.34.1 or newer for [`builtin:checkout`](docs/builtin-actions.md).
- `git-lfs` for `builtin:checkout` with `lfs: true`.
- `ssh` for `builtin:checkout` with `ssh-key`.
Kubernetes mode needs [a few more tools](docs/kubernetes.md) in job images. Kubernetes mode needs [a few more tools](docs/kubernetes.md) in job images.
@@ -112,7 +110,7 @@ Versioned and `nightly` tags take the same suffixes, like `nightly-dind`. The im
### Example deployments ### Example deployments
See the [examples](examples) directory for more deployments, among them systemd, Kubernetes and VMs. See the [examples](examples) directory for more deployments, among them systemd, launchd on macOS, Kubernetes and VMs.
## Configuration ## Configuration
@@ -183,7 +181,7 @@ Setup actions like `setup-go` install tools into `RUNNER_TOOL_CACHE`. `runner.to
| Mode | Tool cache | | Mode | Tool cache |
| --- | --- | | --- | --- |
| `none` (default) | Provided by the job image. Versions it lacks are downloaded in every job. | | `none` (default) | Provided by the job image. Versions it lacks are downloaded in every job. |
| `shared` | One volume for all jobs. Only safe with `runner.capacity: 1`. | | `shared` | One volume per `runner.capacity` slot, reused by later jobs. It hides the tools later job images ship, so use images without them. |
To ship tools in a job image, install them to `/opt/hostedtoolcache/<tool>/<version>/<arch>` with an empty `<arch>.complete` file next to it, as the `-full` [runner images](https://gitea.com/gitea/runner-images) do: To ship tools in a job image, install them to `/opt/hostedtoolcache/<tool>/<version>/<arch>` with an empty `<arch>.complete` file next to it, as the `-full` [runner images](https://gitea.com/gitea/runner-images) do:
@@ -245,10 +243,6 @@ volumes:
Set `container.sweep: true` to remove the containers, networks and volumes most Linux container jobs create when they end, `docker compose` services included. Leave it off if jobs deploy services to the runner's Docker daemon. Set `container.sweep: true` to remove the containers, networks and volumes most Linux container jobs create when they end, `docker compose` services included. Leave it off if jobs deploy services to the runner's Docker daemon.
### Built-in actions (`uses: builtin:<name>`)
`uses: builtin:checkout`, `builtin:upload-artifact` and `builtin:download-artifact` replace their `actions/` counterparts without an action download or Node in the job image, see [docs/builtin-actions.md](docs/builtin-actions.md).
### Caching (`actions/cache`) ### Caching (`actions/cache`)
Each runner runs its own cache server. As on GitHub, a job restores caches of its own ref, its pull request's base branch and the default branch. Each runner runs its own cache server. As on GitHub, a job restores caches of its own ref, its pull request's base branch and the default branch.
+63 -11
View File
@@ -62,6 +62,7 @@ type JobCredential struct {
Ref string `json:"ref"` Ref string `json:"ref"`
BaseRef string `json:"base_ref"` BaseRef string `json:"base_ref"`
DefaultRef string `json:"default_ref"` DefaultRef string `json:"default_ref"`
CacheMode string `json:"cache_mode"` // empty allows everything
// Results is the instance whose artifact service this server forwards for the job, and // Results is the instance whose artifact service this server forwards for the job, and
// InsecureTLS how the runner reaches it; see results.go. // InsecureTLS how the runner reaches it; see results.go.
@@ -104,7 +105,9 @@ type Handler struct {
stopSync context.CancelFunc stopSync context.CancelFunc
syncing sync.WaitGroup syncing sync.WaitGroup
outboundIPMu sync.RWMutex
outboundIP string outboundIP string
detectOutboundIP func() net.IP // nil when the address is configured
instanceTransport http.RoundTripper instanceTransport http.RoundTripper
// internalSecret guards /_internal/{register,revoke}. When set, a remote // internalSecret guards /_internal/{register,revoke}. When set, a remote
@@ -158,6 +161,7 @@ func StartHandler(opts Options) (*Handler, error) {
policy: opts.Policy.withDefaults(), policy: opts.Policy.withDefaults(),
freeDisk: disk.FreeBytes, freeDisk: disk.FreeBytes,
dbLock: make(chan struct{}, 1), dbLock: make(chan struct{}, 1),
outboundIP: opts.OutboundIP,
instanceTransport: opts.InstanceTransport, instanceTransport: opts.InstanceTransport,
} }
@@ -167,12 +171,11 @@ func StartHandler(opts Options) (*Handler, error) {
logger = logger.With("module", "artifactcache") logger = logger.With("module", "artifactcache")
h.logger = logger h.logger = logger
if opts.OutboundIP != "" { if opts.OutboundIP == "" {
h.outboundIP = opts.OutboundIP h.detectOutboundIP = common.GetOutboundIP
} else if ip := common.GetOutboundIP(); ip == nil { if h.RefreshExternalURL() == "" {
return nil, errors.New("unable to determine outbound IP address") logger.Warn("unable to determine an outbound IP address for the cache server yet, set cache.host to configure one")
} else { }
h.outboundIP = ip.String()
} }
if opts.Upstream != "" { if opts.Upstream != "" {
@@ -216,10 +219,10 @@ func StartHandler(opts Options) (*Handler, error) {
} }
router := http.NewServeMux() router := http.NewServeMux()
router.HandleFunc("GET "+apiPath+"/cache", h.bearerAuth(h.find)) router.HandleFunc("GET "+apiPath+"/cache", h.cacheModeAuth(false, h.find))
router.HandleFunc("POST "+apiPath+"/caches", h.bearerAuth(h.reserve)) router.HandleFunc("POST "+apiPath+"/caches", h.cacheModeAuth(true, h.reserve))
router.HandleFunc("PATCH "+apiPath+"/caches/{id}", h.bearerAuth(h.upload)) router.HandleFunc("PATCH "+apiPath+"/caches/{id}", h.cacheModeAuth(true, h.upload))
router.HandleFunc("POST "+apiPath+"/caches/{id}", h.bearerAuth(h.commit)) router.HandleFunc("POST "+apiPath+"/caches/{id}", h.cacheModeAuth(true, h.commit))
router.HandleFunc("POST "+apiPath+"/clean", h.bearerAuth(h.clean)) router.HandleFunc("POST "+apiPath+"/clean", h.bearerAuth(h.clean))
// Artifact GET is signed via query-string HMAC because @actions/cache // Artifact GET is signed via query-string HMAC because @actions/cache
// does not attach Authorization when downloading archiveLocation. // does not attach Authorization when downloading archiveLocation.
@@ -304,11 +307,36 @@ func (h *Handler) cacheForwarder(upstream string) (http.Handler, error) {
}), nil }), nil
} }
// ExternalURL is "" while no address is known.
func (h *Handler) ExternalURL() string { func (h *Handler) ExternalURL() string {
// TODO: make the external url configurable if necessary if h == nil {
return ""
}
h.outboundIPMu.RLock()
defer h.outboundIPMu.RUnlock()
if h.outboundIP == "" {
return ""
}
return "http://" + net.JoinHostPort(h.outboundIP, strconv.Itoa(h.port)) return "http://" + net.JoinHostPort(h.outboundIP, strconv.Itoa(h.port))
} }
// RefreshExternalURL re-detects an unconfigured address, which can change after startup.
func (h *Handler) RefreshExternalURL() string {
if h != nil && h.detectOutboundIP != nil {
if ip := h.detectOutboundIP(); ip != nil {
h.outboundIPMu.Lock()
if address := ip.String(); address != h.outboundIP {
h.outboundIP = address
if ip.To4() == nil {
h.logger.Warnf("detected IPv6 address %s for the cache server, jobs on IPv4-only networks cannot reach it, set cache.host if they fail to", address)
}
}
h.outboundIPMu.Unlock()
}
}
return h.ExternalURL()
}
func (h *Handler) baseURL(cred JobCredential) string { func (h *Handler) baseURL(cred JobCredential) string {
if base := strings.TrimRight(cred.PublicURL, "/"); base != "" { if base := strings.TrimRight(cred.PublicURL, "/"); base != "" {
return base return base
@@ -929,6 +957,30 @@ func (h *Handler) bearerAuth(handler http.HandlerFunc) http.HandlerFunc {
} }
} }
// cacheModeAuth is bearerAuth refusing what the job's cache-mode forbids, in the shapes the toolkit reports as a warning.
func (h *Handler) cacheModeAuth(write bool, handler http.HandlerFunc) http.HandlerFunc {
return h.bearerAuth(func(w http.ResponseWriter, r *http.Request) {
mode := credFromContext(r.Context()).CacheMode
access, forbidden := "read", "write-only"
if write {
access, forbidden = "write", "read"
}
if mode != "none" && mode != forbidden {
handler(w, r)
return
}
message := fmt.Sprintf("cache %s denied: the job's cache-mode is %s", access, mode)
switch {
case !strings.HasPrefix(r.URL.Path, cacheServiceV2Path):
h.responseJSON(w, r, http.StatusForbidden, map[string]any{"message": message})
case write:
h.responseJSON(w, r, http.StatusOK, map[string]any{"ok": false, "message": message})
default:
h.twirpError(w, r, twirpDenied, errors.New(message))
}
})
}
func (h *Handler) unauthorized(w http.ResponseWriter, r *http.Request, err error) { func (h *Handler) unauthorized(w http.ResponseWriter, r *http.Request, err error) {
if strings.HasPrefix(r.URL.Path, cacheServiceV2Path) { if strings.HasPrefix(r.URL.Path, cacheServiceV2Path) {
h.twirpError(w, r, twirpUnauthenticated, err) h.twirpError(w, r, twirpUnauthenticated, err)
+47 -7
View File
@@ -11,6 +11,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"net"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
@@ -22,6 +23,8 @@ import (
"testing/synctest" "testing/synctest"
"time" "time"
"gitea.com/gitea/runner/act/common/logging/logtest"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.etcd.io/bbolt" "go.etcd.io/bbolt"
@@ -50,7 +53,23 @@ func signArtifactURL(h *Handler, id int64) string {
func TestHandler_ExternalURL(t *testing.T) { func TestHandler_ExternalURL(t *testing.T) {
t.Parallel() t.Parallel()
assert.Equal(t, "http://[2001:db8::1]:8080", (&Handler{outboundIP: "2001:db8::1", port: 8080}).ExternalURL()) assert.Empty(t, (*Handler)(nil).RefreshExternalURL())
assert.Equal(t, "http://cache.example:8080", (&Handler{outboundIP: "cache.example", port: 8080}).RefreshExternalURL())
logger, logs := logtest.NewLogger()
var ip net.IP
handler := &Handler{port: 8080, logger: logger, detectOutboundIP: func() net.IP { return ip }}
assert.Empty(t, handler.RefreshExternalURL())
ip = net.ParseIP("2001:db8::1")
assert.Equal(t, "http://[2001:db8::1]:8080", handler.RefreshExternalURL())
assert.Equal(t, "http://[2001:db8::1]:8080", handler.RefreshExternalURL())
ip = net.ParseIP("192.0.2.1")
assert.Equal(t, "http://[2001:db8::1]:8080", handler.ExternalURL())
assert.Equal(t, "http://192.0.2.1:8080", handler.RefreshExternalURL())
ip = nil
assert.Equal(t, "http://192.0.2.1:8080", handler.RefreshExternalURL())
require.Len(t, logs.AllRecords(), 1)
assert.Contains(t, logs.LastRecord().Message, "IPv6")
} }
func TestHandler_Upstream(t *testing.T) { func TestHandler_Upstream(t *testing.T) {
@@ -984,7 +1003,7 @@ func TestHandler_UnregisterRevokes(t *testing.T) {
} }
} }
func TestHandler_CrossRepoIsolation(t *testing.T) { func TestHandler_CrossRepoAndCacheModeIsolation(t *testing.T) {
t.Parallel() t.Parallel()
dir := filepath.Join(t.TempDir(), "artifactcache") dir := filepath.Join(t.TempDir(), "artifactcache")
handler, err := StartHandler(Options{Dir: dir}) handler, err := StartHandler(Options{Dir: dir})
@@ -993,6 +1012,8 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"}) handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"}) handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"})
handler.RegisterJob("token-a-pull", JobCredential{Repo: "owner/repoA", Ref: "refs/pull/1/head"}) handler.RegisterJob("token-a-pull", JobCredential{Repo: "owner/repoA", Ref: "refs/pull/1/head"})
handler.RegisterJob("token-a-read", JobCredential{Repo: "owner/repoA", CacheMode: "read"})
handler.RegisterJob("token-a-write-only", JobCredential{Repo: "owner/repoA", CacheMode: "write-only"})
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
key := "shared-key" key := "shared-key"
@@ -1022,12 +1043,31 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
resp.Body.Close() resp.Body.Close()
assert.Equal(t, http.StatusNoContent, resp.StatusCode) assert.Equal(t, http.StatusNoContent, resp.StatusCode)
resp, err = clientA.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version)) clientRead := &http.Client{Transport: &bearerTransport{token: "token-a-read"}}
require.NoError(t, err) for _, client := range []*http.Client{clientA, clientRead} {
resp.Body.Close() resp, err = client.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version))
assert.Equal(t, http.StatusOK, resp.StatusCode) require.NoError(t, err)
resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
}
for _, client := range []*http.Client{clientB, {Transport: &bearerTransport{token: "token-a-pull"}}} { requireDenied := func(prefix string, do func() (*http.Response, error)) {
resp, err := do()
require.NoError(t, err)
defer resp.Body.Close()
denied := map[string]string{}
require.NoError(t, json.UnmarshalRead(resp.Body, &denied))
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
assert.True(t, strings.HasPrefix(denied["message"], prefix), denied["message"])
}
requireDenied("cache write denied:", func() (*http.Response, error) {
return clientRead.Post(base+"/caches", "application/json", strings.NewReader(`{"key":"new-key","version":"`+version+`"}`))
})
requireDenied("cache read denied:", func() (*http.Response, error) {
return (&http.Client{Transport: &bearerTransport{token: "token-a-write-only"}}).Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version))
})
for _, client := range []*http.Client{clientB, {Transport: &bearerTransport{token: "token-a-pull"}}, clientRead} {
req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison"))) req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison")))
require.NoError(t, err) require.NoError(t, err)
req.Header.Set("Content-Range", "bytes 0-5/*") req.Header.Set("Content-Range", "bytes 0-5/*")
+6 -3
View File
@@ -40,12 +40,13 @@ const (
twirpInternal = "internal" twirpInternal = "internal"
twirpUnavailable = "unavailable" twirpUnavailable = "unavailable"
twirpUnauthenticated = "unauthenticated" twirpUnauthenticated = "unauthenticated"
twirpDenied = "permission_denied"
) )
func (h *Handler) registerV2Routes(router *http.ServeMux) { func (h *Handler) registerV2Routes(router *http.ServeMux) {
router.HandleFunc("POST "+cacheServiceV2Path+"/CreateCacheEntry", h.bearerAuth(h.v2CreateCacheEntry)) router.HandleFunc("POST "+cacheServiceV2Path+"/CreateCacheEntry", h.cacheModeAuth(true, h.v2CreateCacheEntry))
router.HandleFunc("POST "+cacheServiceV2Path+"/FinalizeCacheEntryUpload", h.bearerAuth(h.v2FinalizeCacheEntryUpload)) router.HandleFunc("POST "+cacheServiceV2Path+"/FinalizeCacheEntryUpload", h.cacheModeAuth(true, h.v2FinalizeCacheEntryUpload))
router.HandleFunc("POST "+cacheServiceV2Path+"/GetCacheEntryDownloadURL", h.bearerAuth(h.v2GetCacheEntryDownloadURL)) router.HandleFunc("POST "+cacheServiceV2Path+"/GetCacheEntryDownloadURL", h.cacheModeAuth(false, h.v2GetCacheEntryDownloadURL))
router.HandleFunc("PUT "+blobPath+"/{id}", h.signedAuth(blobUploadPurpose, h.v2UploadBlob)) router.HandleFunc("PUT "+blobPath+"/{id}", h.signedAuth(blobUploadPurpose, h.v2UploadBlob))
} }
@@ -267,6 +268,8 @@ func (h *Handler) twirpError(w http.ResponseWriter, r *http.Request, code string
status = http.StatusServiceUnavailable status = http.StatusServiceUnavailable
case twirpUnauthenticated: case twirpUnauthenticated:
status = http.StatusUnauthorized status = http.StatusUnauthorized
case twirpDenied:
status = http.StatusForbidden
} }
h.responseJSON(w, r, status, map[string]any{"code": code, "msg": err.Error()}) h.responseJSON(w, r, status, map[string]any{"code": code, "msg": err.Error()})
} }
+24
View File
@@ -227,6 +227,30 @@ func TestCacheServiceV2Lookups(t *testing.T) {
assert.NotEmpty(t, got["archiveLocation"]) assert.NotEmpty(t, got["archiveLocation"])
}) })
t.Run("a read-only job only restores and a write-only one only saves", func(t *testing.T) {
handler.RegisterJob("read-only", JobCredential{Repo: testRepo, CacheMode: "read"})
readOnlyClient := &http.Client{Transport: &bearerTransport{token: "read-only"}}
assert.Equal(t, true, v2Call(t, handler, readOnlyClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})["ok"])
for _, method := range []string{"CreateCacheEntry", "FinalizeCacheEntryUpload"} {
got := v2Call(t, handler, readOnlyClient, method, map[string]any{"key": "deps-new", "version": "v1", "size_bytes": 1})
assert.Equal(t, false, got["ok"], method)
assert.True(t, strings.HasPrefix(got["message"].(string), "cache write denied:"), method)
}
handler.RegisterJob("write-only", JobCredential{Repo: testRepo, CacheMode: "write-only"})
writeOnlyClient := &http.Client{Transport: &bearerTransport{token: "write-only"}}
assert.Equal(t, true, v2Call(t, handler, writeOnlyClient, "CreateCacheEntry", map[string]any{"key": "deps-write-only", "version": "v1"})["ok"])
resp, err := writeOnlyClient.Post(handler.ExternalURL()+cacheServiceV2Path+"/GetCacheEntryDownloadURL", "application/json", strings.NewReader(`{"key":"deps-abc","version":"v1"}`))
require.NoError(t, err)
defer resp.Body.Close()
got := map[string]string{}
require.NoError(t, json.UnmarshalRead(resp.Body, &got))
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
assert.Equal(t, twirpDenied, got["code"])
assert.True(t, strings.HasPrefix(got["msg"], "cache read denied:"), got["msg"])
})
t.Run("another repository sees nothing", func(t *testing.T) { t.Run("another repository sees nothing", func(t *testing.T) {
handler.RegisterJob("other-runtime-token", JobCredential{Repo: "other/repo"}) handler.RegisterJob("other-runtime-token", JobCredential{Repo: "other/repo"})
otherClient := &http.Client{Transport: &bearerTransport{token: "other-runtime-token"}} otherClient := &http.Client{Transport: &bearerTransport{token: "other-runtime-token"}}
+1 -1
View File
@@ -218,7 +218,7 @@ func Serve(ctx context.Context, artifactPath, addr, port string) context.CancelF
logger.Debugf("Artifacts base path '%s'", artifactPath) logger.Debugf("Artifacts base path '%s'", artifactPath)
uploads(router, artifactPath) uploads(router, artifactPath)
downloads(router, artifactPath) downloads(router, artifactPath)
RegisterV4(router, artifactPath) registerV4(router, artifactPath)
server := &http.Server{ server := &http.Server{
Addr: net.JoinHostPort(addr, port), Addr: net.JoinHostPort(addr, port),
+2 -2
View File
@@ -28,7 +28,7 @@ func TestArtifactFlow(t *testing.T) {
router := http.NewServeMux() router := http.NewServeMux()
uploads(router, artifactPath) uploads(router, artifactPath)
downloads(router, artifactPath) downloads(router, artifactPath)
RegisterV4(router, artifactPath) registerV4(router, artifactPath)
server := httptest.NewServer(router) server := httptest.NewServer(router)
defer server.Close() defer server.Close()
@@ -256,7 +256,7 @@ func TestArtifactFlow(t *testing.T) {
require.Equal(t, "attachment; filename*=UTF-8''50%25%20art%40v1", header.Get("Content-Disposition")) require.Equal(t, "attachment; filename*=UTF-8''50%25%20art%40v1", header.Get("Content-Disposition"))
rerun := http.NewServeMux() rerun := http.NewServeMux()
RegisterV4(rerun, artifactPath) registerV4(rerun, artifactPath)
recorder := httptest.NewRecorder() recorder := httptest.NewRecorder()
rerun.ServeHTTP(recorder, httptest.NewRequest(http.MethodPost, artifactServicePath+"/CreateArtifact", strings.NewReader("{"+artifactRef+"}"))) rerun.ServeHTTP(recorder, httptest.NewRequest(http.MethodPost, artifactServicePath+"/CreateArtifact", strings.NewReader("{"+artifactRef+"}")))
require.Equal(t, http.StatusOK, recorder.Code, recorder.Body.String()) require.Equal(t, http.StatusOK, recorder.Code, recorder.Body.String())
+1 -1
View File
@@ -35,7 +35,7 @@ const (
maxTwirpBytes = 1 << 20 maxTwirpBytes = 1 << 20
) )
func RegisterV4(router *http.ServeMux, baseDir string) { func registerV4(router *http.ServeMux, baseDir string) {
server := &serverV4{dir: filepath.Join(baseDir, "v4"), lastID: time.Now().UnixMilli(), created: map[string]bool{}} // IDs stay unique across invocations sharing baseDir server := &serverV4{dir: filepath.Join(baseDir, "v4"), lastID: time.Now().UnixMilli(), created: map[string]bool{}} // IDs stay unique across invocations sharing baseDir
router.HandleFunc("POST "+artifactServicePath+"/CreateArtifact", twirp(server.createArtifact)) router.HandleFunc("POST "+artifactServicePath+"/CreateArtifact", twirp(server.createArtifact))
router.HandleFunc("POST "+artifactServicePath+"/FinalizeArtifact", twirp(server.finalizeArtifact)) router.HandleFunc("POST "+artifactServicePath+"/FinalizeArtifact", twirp(server.finalizeArtifact))
+8 -4
View File
@@ -6,6 +6,7 @@ package common
import ( import (
"net" "net"
"slices"
"sort" "sort"
"strings" "strings"
) )
@@ -55,19 +56,22 @@ func GetOutboundIP() net.IP {
} }
func preferredIP(ips []interfaceIP) net.IP { func preferredIP(ips []interfaceIP) net.IP {
ips = slices.DeleteFunc(ips, func(ip interfaceIP) bool {
return len(ip.Name) == 15 && strings.HasPrefix(ip.Name, "br-") && strings.Trim(ip.Name[3:], "0123456789abcdef") == "" // a Docker network's bridge, removed with the network
})
if len(ips) == 0 { if len(ips) == 0 {
return nil return nil
} }
sort.Slice(ips, func(i, j int) bool { sort.Slice(ips, func(i, j int) bool {
first, second := ips[i], ips[j] first, second := ips[i], ips[j]
// ethernet is preferred // IPv4 is preferred, as job networks are IPv4-only by default
if vi, vj := strings.HasPrefix(first.Name, "e"), strings.HasPrefix(second.Name, "e"); vi != vj { if vi, vj := first.To4() != nil, second.To4() != nil; vi != vj {
return vi return vi
} }
// IPv4 is preferred // ethernet is preferred
if vi, vj := first.To4() != nil, second.To4() != nil; vi != vj { if vi, vj := strings.HasPrefix(first.Name, "e"), strings.HasPrefix(second.Name, "e"); vi != vj {
return vi return vi
} }
+2 -1
View File
@@ -13,8 +13,9 @@ import (
func TestPreferredIP(t *testing.T) { func TestPreferredIP(t *testing.T) {
assert.Nil(t, preferredIP(nil)) assert.Nil(t, preferredIP(nil))
assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{{IP: net.ParseIP("2001:db8::1")}})) assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{{IP: net.ParseIP("2001:db8::1")}}))
assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{ assert.Equal(t, net.ParseIP("192.0.2.1"), preferredIP([]interfaceIP{
{IP: net.ParseIP("192.0.2.1"), Name: "wlan0"}, {IP: net.ParseIP("192.0.2.1"), Name: "wlan0"},
{IP: net.ParseIP("2001:db8::1"), Name: "eth0"}, {IP: net.ParseIP("2001:db8::1"), Name: "eth0"},
{IP: net.ParseIP("172.18.0.1"), Name: "br-0123456789ab"},
})) }))
} }
+6 -35
View File
@@ -5,7 +5,6 @@
package container package container
import ( import (
"bytes"
"context" "context"
"errors" "errors"
"fmt" "fmt"
@@ -15,6 +14,7 @@ import (
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
"time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
@@ -24,6 +24,9 @@ import (
var hostOS = runtime.GOOS // replaced in tests var hostOS = runtime.GOOS // replaced in tests
// DrainGracePeriod caps the wait for container output to drain so the log tail is not truncated
const DrainGracePeriod = 2 * time.Second
// ExitCodeError reports a non-zero process exit code from a container command. // ExitCodeError reports a non-zero process exit code from a container command.
type ExitCodeError int type ExitCodeError int
@@ -31,38 +34,6 @@ func (e ExitCodeError) Error() string {
return fmt.Sprintf("Process completed with exit code %d.", int(e)) return fmt.Sprintf("Process completed with exit code %d.", int(e))
} }
type commandOutput struct {
*io.PipeReader
cancel context.CancelFunc
done chan struct{}
err error
}
func (output *commandOutput) Close() error {
_, _ = io.Copy(io.Discard, output.PipeReader)
output.PipeReader.Close()
output.cancel()
<-output.done
return output.err
}
func streamOutput(ctx context.Context, run func(context.Context, io.Writer, io.Writer) error) io.ReadCloser {
ctx, cancel := context.WithCancel(ctx)
reader, writer := io.Pipe()
output := &commandOutput{PipeReader: reader, cancel: cancel, done: make(chan struct{})}
go func() {
defer close(output.done)
defer cancel()
var stderr bytes.Buffer
output.err = run(ctx, writer, &stderr)
if output.err != nil && strings.TrimSpace(stderr.String()) != "" {
output.err = fmt.Errorf("%w: %s", output.err, strings.TrimSpace(stderr.String()))
}
writer.CloseWithError(output.err)
}()
return output
}
// NewContainerInput the input for the New function // NewContainerInput the input for the New function
type NewContainerInput struct { type NewContainerInput struct {
Image string Image string
@@ -163,9 +134,7 @@ type Container interface {
Create(capAdd, capDrop []string) common.Executor Create(capAdd, capDrop []string) common.Executor
Copy(destPath string, files ...*FileEntry) common.Executor Copy(destPath string, files ...*FileEntry) common.Executor
CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor
ExecOutput(ctx context.Context, command []string) (io.ReadCloser, error)
GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error)
CopyTarStream(ctx context.Context, destPath string, write func(*TarWriter) error) error
Inspect(ctx context.Context) (*Info, error) Inspect(ctx context.Context) (*Info, error)
DumpLogs(ctx context.Context) error DumpLogs(ctx context.Context) error
Pull(forcePull bool) common.Executor Pull(forcePull bool) common.Executor
@@ -183,6 +152,8 @@ type NewDockerBuildExecutorInput struct {
ContextDir string ContextDir string
Dockerfile string Dockerfile string
BuildContext io.Reader BuildContext io.Reader
Dockerignore []byte // applied to BuildContext as the docker CLI applies it
Stdout io.Writer
ImageTag string ImageTag string
Platform string Platform string
BuildArgs map[string]*string BuildArgs map[string]*string
+92 -2
View File
@@ -8,10 +8,13 @@ package container
import ( import (
"archive/tar" "archive/tar"
"bytes"
"context" "context"
"fmt"
"io" "io"
"io/fs" "io/fs"
"os" "os"
"path"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strings" "strings"
@@ -63,7 +66,7 @@ func NewDockerBuildExecutor(input NewDockerBuildExecutorInput) common.Executor {
} }
var buildContext io.ReadCloser var buildContext io.ReadCloser
if input.BuildContext != nil { if input.BuildContext != nil {
buildContext = io.NopCloser(input.BuildContext) buildContext, err = filterBuildContext(input.BuildContext, input.Dockerignore, input.Dockerfile)
} else { } else {
buildContext, err = createBuildContext(ctx, input.ContextDir, input.Dockerfile) buildContext, err = createBuildContext(ctx, input.ContextDir, input.Dockerfile)
} }
@@ -78,10 +81,97 @@ func NewDockerBuildExecutor(input NewDockerBuildExecutorInput) common.Executor {
if err != nil { if err != nil {
return err return err
} }
return logDockerResponse(logger, resp.Body, false) err = logDockerResponse(logger, resp.Body, func(format string, args ...any) { fmt.Fprintf(input.Stdout, format, args...) })
common.FlushWriter(input.Stdout)
return err
} }
} }
func filterBuildContext(buildContext io.Reader, dockerignore []byte, relDockerfile string) (io.ReadCloser, error) {
excludes, err := ignorefile.ReadAll(bytes.NewReader(dockerignore))
if err != nil || len(excludes) == 0 {
return io.NopCloser(buildContext), err
}
pm, err := patternmatcher.New(excludes)
if err != nil {
return nil, err
}
reader, writer := io.Pipe()
go func() {
writer.CloseWithError(writeFilteredBuildContext(writer, buildContext, pm, path.Clean(filepath.ToSlash(relDockerfile))))
}()
return reader, nil
}
func writeFilteredBuildContext(w io.Writer, buildContext io.Reader, pm *patternmatcher.PatternMatcher, relDockerfile string) error {
spool, err := os.CreateTemp("", "gitea-runner-build-") // excluded content, for a kept hardlink to it
if err != nil {
return err
}
defer os.Remove(spool.Name())
defer spool.Close()
reader := tar.NewReader(buildContext)
writer := tar.NewWriter(w)
buffer := make([]byte, 32*1024)
excluded := map[string]*excludedBuildContextEntry{}
for {
header, err := reader.Next()
if err == io.EOF {
return writer.Close()
}
if err != nil {
return err
}
name := path.Clean(header.Name)
var target *excludedBuildContextEntry
if header.Typeflag == tar.TypeLink {
target = excluded[path.Clean(header.Linkname)]
}
if name != ".dockerignore" && name != relDockerfile {
skip, err := pm.MatchesOrParentMatches(name)
if err != nil {
return err
}
if skip {
if target != nil {
excluded[name] = target
} else if header.Typeflag != tar.TypeDir {
offset, err := spool.Seek(0, io.SeekCurrent)
if err != nil {
return err
}
excluded[name] = &excludedBuildContextEntry{header: header, offset: offset}
if _, err := io.CopyBuffer(spool, reader, buffer); err != nil {
return err
}
}
continue
}
}
var content io.Reader = reader
if target != nil && target.keptName != "" {
header.Linkname = target.keptName
} else if target != nil {
header.Typeflag, header.Linkname, header.Size = target.header.Typeflag, target.header.Linkname, target.header.Size
content = io.NewSectionReader(spool, target.offset, header.Size)
target.keptName = header.Name
}
if err := writer.WriteHeader(header); err != nil {
return err
}
if _, err := io.CopyBuffer(writer, content, buffer); err != nil {
return err
}
}
}
type excludedBuildContextEntry struct {
header *tar.Header
offset int64
keptName string
}
func createBuildContext(ctx context.Context, contextDir, relDockerfile string) (io.ReadCloser, error) { func createBuildContext(ctx context.Context, contextDir, relDockerfile string) (io.ReadCloser, error) {
common.Logger(ctx).Debugf("Creating archive for build context dir '%s' with relative dockerfile '%s'", contextDir, relDockerfile) common.Logger(ctx).Debugf("Creating archive for build context dir '%s' with relative dockerfile '%s'", contextDir, relDockerfile)
+56 -1
View File
@@ -7,6 +7,8 @@ package container
import ( import (
"archive/tar" "archive/tar"
"bytes"
"fmt"
"io" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
@@ -35,7 +37,7 @@ func TestDockerBuildLabelsImageWithItsTagAndReportsDaemonError(t *testing.T) {
t.Setenv("DOCKER_AUTH_CONFIG", "") t.Setenv("DOCKER_AUTH_CONFIG", "")
require.ErrorContains(t, NewDockerBuildExecutor(NewDockerBuildExecutorInput{ require.ErrorContains(t, NewDockerBuildExecutor(NewDockerBuildExecutorInput{
ImageTag: "action:test", Dockerfile: "Dockerfile", BuildContext: strings.NewReader(""), ImageTag: "action:test", Dockerfile: "Dockerfile", BuildContext: strings.NewReader(""), Stdout: io.Discard,
})(t.Context()), "build rejected") })(t.Context()), "build rejected")
require.JSONEq(t, `{"com.gitea.runner.image":"action:test"}`, labels) require.JSONEq(t, `{"com.gitea.runner.image":"action:test"}`, labels)
} }
@@ -72,3 +74,56 @@ func TestCreateBuildContextFollowsDockerignoreAndSymlinkedRootButKeepsDockerfile
require.Equal(t, []string{".dockerignore", "a.txt", "build/keep.txt", "empty/", "Dockerfile"}, names, contextDir) require.Equal(t, []string{".dockerignore", "a.txt", "build/keep.txt", "empty/", "Dockerfile"}, names, contextDir)
} }
} }
func TestFilterBuildContextAppliesDockerignoreAndMaterializesHardlinksToExcludedFiles(t *testing.T) {
const dockerignore = "*.log\nbuild\n!build/keep.txt\nActionfile\n.dockerignore\n"
for _, prefix := range []string{"", "./"} {
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
for _, entry := range []struct {
name, body, link string
typeflag byte
}{
{"Actionfile", "FROM scratch", "", tar.TypeReg},
{"output.log", "log", "", tar.TypeReg},
{"build/", "", "", tar.TypeDir},
{"build/drop.txt", "drop", "", tar.TypeReg},
{"build/keep.txt", "keep", "", tar.TypeReg},
{"entrypoint", "", prefix + "output.log", tar.TypeLink},
{"other-entrypoint", "", prefix + "output.log", tar.TypeLink},
{".dockerignore", dockerignore, "", tar.TypeReg},
} {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: prefix + entry.name, Linkname: entry.link, Typeflag: entry.typeflag, Size: int64(len(entry.body)), Mode: 0o644}))
_, err := io.WriteString(writer, entry.body)
require.NoError(t, err)
}
require.NoError(t, writer.Close())
unfiltered, err := filterBuildContext(bytes.NewReader(archive.Bytes()), nil, "Actionfile")
require.NoError(t, err)
passedThrough, err := io.ReadAll(unfiltered)
require.NoError(t, err)
require.Equal(t, archive.Bytes(), passedThrough)
filtered, err := filterBuildContext(bytes.NewReader(archive.Bytes()), []byte(dockerignore), "Actionfile")
require.NoError(t, err)
var entries []string
for reader := tar.NewReader(filtered); ; {
header, err := reader.Next()
if err == io.EOF {
break
}
require.NoError(t, err)
body, err := io.ReadAll(reader)
require.NoError(t, err)
entries = append(entries, fmt.Sprintf("%s %c %s %s", strings.TrimPrefix(header.Name, prefix), header.Typeflag, strings.TrimPrefix(header.Linkname, prefix), body))
}
require.Equal(t, []string{
"Actionfile 0 FROM scratch",
"build/keep.txt 0 keep",
"entrypoint 0 log",
"other-entrypoint 1 entrypoint ",
".dockerignore 0 " + dockerignore,
}, entries, prefix)
}
}
+1 -1
View File
@@ -38,7 +38,7 @@ func loadScratchImage(t *testing.T, platform string) string {
response, err := cli.ImageLoad(t.Context(), &archive) response, err := cli.ImageLoad(t.Context(), &archive)
require.NoError(t, err) require.NoError(t, err)
defer response.Close() defer response.Close()
require.NoError(t, logDockerResponse(common.Logger(t.Context()), response, false)) require.NoError(t, logDockerResponse(common.Logger(t.Context()), response, common.Logger(t.Context()).Debugf))
t.Cleanup(func() { t.Cleanup(func() {
_, _ = cli.ImageRemove(context.WithoutCancel(t.Context()), tag, mobyclient.ImageRemoveOptions{Force: true}) _, _ = cli.ImageRemove(context.WithoutCancel(t.Context()), tag, mobyclient.ImageRemoveOptions{Force: true})
}) })
+10 -25
View File
@@ -8,6 +8,7 @@ package container
import ( import (
"bufio" "bufio"
"cmp"
"encoding/json/v2" "encoding/json/v2"
"errors" "errors"
"io" "io"
@@ -26,10 +27,7 @@ type dockerMessage struct {
Progress string `json:"progress"` Progress string `json:"progress"`
} }
func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, isError bool) error { func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, log func(string, ...any)) error {
if dockerResponse == nil {
return nil
}
defer dockerResponse.Close() defer dockerResponse.Close()
scanner := bufio.NewScanner(dockerResponse) scanner := bufio.NewScanner(dockerResponse)
@@ -38,41 +36,28 @@ func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, isE
line := scanner.Bytes() line := scanner.Bytes()
var msg dockerMessage var msg dockerMessage
if err := json.Unmarshal(line, &msg); err != nil { if err := json.Unmarshal(line, &msg); err != nil {
writeLog(logger, false, "Unable to unmarshal line [%s] ==> %v", string(line), err) logger.Debugf("Unable to unmarshal line [%s] ==> %v", string(line), err)
continue continue
} }
if msg.Error != "" { if message := cmp.Or(msg.Error, msg.ErrorDetail.Message); message != "" {
writeLog(logger, isError, "%s", msg.Error) log("%s", message)
return errors.New(msg.Error) return errors.New(message)
}
if msg.ErrorDetail.Message != "" {
writeLog(logger, isError, "%s", msg.ErrorDetail.Message)
return errors.New(msg.ErrorDetail.Message)
} }
switch { switch {
case msg.Status != "": case msg.Status != "":
if msg.Progress != "" { if msg.Progress != "" {
writeLog(logger, isError, "%s :: %s :: %s\n", msg.Status, msg.ID, msg.Progress) log("%s :: %s :: %s\n", msg.Status, msg.ID, msg.Progress)
} else { } else {
writeLog(logger, isError, "%s :: %s\n", msg.Status, msg.ID) log("%s :: %s\n", msg.Status, msg.ID)
} }
case msg.Stream != "": case msg.Stream != "":
writeLog(logger, isError, "%s", msg.Stream) log("%s", msg.Stream)
default: default:
writeLog(logger, false, "Unable to handle line: %s", string(line)) logger.Debugf("Unable to handle line: %s", string(line))
} }
} }
return scanner.Err() return scanner.Err()
} }
func writeLog(logger *logging.Logger, isError bool, format string, args ...any) {
if isError {
logger.Errorf(format, args...)
} else {
logger.Debugf(format, args...)
}
}
+1 -2
View File
@@ -61,11 +61,10 @@ func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
// stream rather than on the call itself, so both have to be checked // stream rather than on the call itself, so both have to be checked
pullOnce := func(opts client.ImagePullOptions) error { pullOnce := func(opts client.ImagePullOptions) error {
reader, err := cli.ImagePull(ctx, imageRef, opts) reader, err := cli.ImagePull(ctx, imageRef, opts)
streamErr := logDockerResponse(logger, reader, err != nil)
if err != nil { if err != nil {
return err return err
} }
return streamErr return logDockerResponse(logger, reader, logger.Debugf)
} }
err = pullOnce(imagePullOptions) err = pullOnce(imagePullOptions)
+4 -3
View File
@@ -95,15 +95,16 @@ func TestIsPinnedImage(t *testing.T) {
// The pull path reports a failure the daemon sent mid-stream, so it must carry the reason // The pull path reports a failure the daemon sent mid-stream, so it must carry the reason
// whichever of the two shapes the daemon used. // whichever of the two shapes the daemon used.
func TestLogDockerResponseError(t *testing.T) { func TestLogDockerResponseError(t *testing.T) {
logger := logging.New(logtest.New()) logger, hook := logtest.NewLogger()
streamErr := func(line string) error { streamErr := func(line string) error {
return logDockerResponse(logger, io.NopCloser(strings.NewReader(line)), false) return logDockerResponse(logger, io.NopCloser(strings.NewReader(line)), logger.Infof)
} }
require.EqualError(t, streamErr(`{"error":"toomanyrequests: rate limit exceeded"}`), "toomanyrequests: rate limit exceeded") require.EqualError(t, streamErr(`{"error":"toomanyrequests: rate limit exceeded"}`), "toomanyrequests: rate limit exceeded")
require.EqualError(t, streamErr(`{"errorDetail":{"message":"unexpected EOF"}}`), "unexpected EOF") require.EqualError(t, streamErr(`{"errorDetail":{"message":"unexpected EOF"}}`), "unexpected EOF")
require.NoError(t, streamErr(`{"status":"Downloading"}`)) require.NoError(t, streamErr(`{"status":"Downloading"}`))
require.NoError(t, streamErr(`{"stream":"`+strings.Repeat("x", 70000)+`"}`)) require.NoError(t, streamErr(`{"stream":"`+strings.Repeat("x", 70000)+`"}`))
assert.Equal(t, logging.LevelInfo, hook.LastRecord().Level)
require.ErrorIs(t, logDockerResponse(logger, io.NopCloser(io.MultiReader( require.ErrorIs(t, logDockerResponse(logger, io.NopCloser(io.MultiReader(
strings.NewReader(`{"status":"Downloading"}`+"\n"), iotest.ErrReader(io.ErrUnexpectedEOF), strings.NewReader(`{"status":"Downloading"}`+"\n"), iotest.ErrReader(io.ErrUnexpectedEOF),
)), false), io.ErrUnexpectedEOF) )), logger.Infof), io.ErrUnexpectedEOF)
} }
+23 -92
View File
@@ -23,6 +23,7 @@ import (
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
"sync"
"time" "time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
@@ -41,13 +42,6 @@ import (
specs "github.com/opencontainers/image-spec/specs-go/v1" specs "github.com/opencontainers/image-spec/specs-go/v1"
) )
// drainGracePeriod bounds how long we wait for an output-copy goroutine to
// finish draining a container's output before returning, so that neither a
// cancellation (waitForCommand) nor a normal container exit (wait) truncates
// the tail of the log. It is a safety bound: in the common case the stream
// reaches EOF and the goroutine returns well before this elapses.
const drainGracePeriod = 2 * time.Second
// NewContainer creates a reference to a container // NewContainer creates a reference to a container
func NewContainer(input *NewContainerInput) ExecutionsEnvironment { func NewContainer(input *NewContainerInput) ExecutionsEnvironment {
// Resolved up front because the image pull runs before the container is created. // Resolved up front because the image pull runs before the container is created.
@@ -90,8 +84,13 @@ func (cr *containerReference) Start(attach bool) common.Executor {
cr.attach().IfBool(attach), cr.attach().IfBool(attach),
cr.start(), cr.start(),
cr.wait().IfBool(attach), cr.wait().IfBool(attach),
cr.tryReadID("-u", func(id int) { cr.UID = id }), func(ctx context.Context) error {
cr.tryReadID("-g", func(id int) { cr.GID = id }), var wg sync.WaitGroup
wg.Go(func() { _ = cr.tryReadID("-u", func(id int) { cr.UID = id })(ctx) })
_ = cr.tryReadID("-g", func(id int) { cr.GID = id })(ctx)
wg.Wait()
return nil
},
func(ctx context.Context) error { func(ctx context.Context) error {
// If this fails, then folders have wrong permissions on non root container // If this fails, then folders have wrong permissions on non root container
if cr.UID != 0 || cr.GID != 0 { if cr.UID != 0 || cr.GID != 0 {
@@ -146,47 +145,6 @@ func (cr *containerReference) CopyDir(destPath, srcPath string, useGitIgnore, sk
).IfNot(common.Dryrun) ).IfNot(common.Dryrun)
} }
func (cr *containerReference) ExecOutput(ctx context.Context, command []string) (io.ReadCloser, error) {
if common.Dryrun(ctx) {
return nil, errors.New("DRYRUN is not supported in ExecOutput")
}
if err := cr.connect()(ctx); err != nil {
return nil, err
}
if err := cr.find()(ctx); err != nil {
return nil, err
}
if cr.id == "" {
return nil, cr.missingContainerError("exec %v", command)
}
return streamOutput(ctx, func(ctx context.Context, stdout, stderr io.Writer) error {
created, err := cr.cli.ExecCreate(ctx, cr.id, client.ExecCreateOptions{
Cmd: command, AttachStdout: true, AttachStderr: true,
})
if err != nil {
return err
}
attached, err := cr.cli.ExecAttach(ctx, created.ID, client.ExecAttachOptions{})
if err != nil {
return err
}
defer attached.Close()
stop := context.AfterFunc(ctx, func() { attached.Close() })
defer stop()
if _, err := stdcopy.StdCopy(stdout, stderr, attached.Reader); err != nil {
return err
}
inspected, err := cr.cli.ExecInspect(ctx, created.ID, client.ExecInspectOptions{})
if err != nil {
return err
}
if inspected.ExitCode != 0 {
return ExitCodeError(inspected.ExitCode)
}
return nil
}), nil
}
func (cr *containerReference) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) { func (cr *containerReference) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
if common.Dryrun(ctx) { if common.Dryrun(ctx) {
return nil, errors.New("DRYRUN is not supported in GetContainerArchive") return nil, errors.New("DRYRUN is not supported in GetContainerArchive")
@@ -232,6 +190,9 @@ func (cr *containerReference) Inspect(ctx context.Context) (*Info, error) {
} else if err != nil { } else if err != nil {
return nil, err return nil, err
} }
if cr.containerEnvID != cr.id && result.Container.Config != nil {
cr.containerEnv, cr.containerEnvID = opts.ConvertKVStringsToMap(result.Container.Config.Env), cr.id
}
return containerInfoFromInspect(result.Container), nil return containerInfoFromInspect(result.Container), nil
} }
@@ -355,15 +316,13 @@ func (cr *containerReference) UpdateFromEnv(srcPath string, env *map[string]stri
func (cr *containerReference) UpdateFromImageEnv(env *map[string]string) common.Executor { func (cr *containerReference) UpdateFromImageEnv(env *map[string]string) common.Executor {
envMap := *env envMap := *env
return common.Executor(func(ctx context.Context) error { return common.Executor(func(ctx context.Context) error {
inspect, err := cr.cli.ContainerInspect(ctx, cr.id, client.ContainerInspectOptions{}) if cr.containerEnvID != cr.id {
if err != nil { if _, err := cr.Inspect(ctx); err != nil {
common.Logger(ctx).Error(err) common.Logger(ctx).Error(err)
return fmt.Errorf("inspect container: %w", err) return fmt.Errorf("inspect container: %w", err)
}
} }
if inspect.Container.Config == nil { for key, value := range cr.containerEnv {
return nil
}
for key, value := range opts.ConvertKVStringsToMap(inspect.Container.Config.Env) {
if envMap[key] == "" { if envMap[key] == "" {
envMap[key] = value envMap[key] = value
} else if key == "PATH" { } else if key == "PATH" {
@@ -413,7 +372,9 @@ type containerReference struct {
// attachDone is closed by the attach() streaming goroutine once it has // attachDone is closed by the attach() streaming goroutine once it has
// drained and flushed the container's output. wait() blocks on it so the // drained and flushed the container's output. wait() blocks on it so the
// tail of the log lands before the step proceeds. // tail of the log lands before the step proceeds.
attachDone chan struct{} attachDone chan struct{}
containerEnv map[string]string // fixed at creation, cached by inspect for containerEnvID
containerEnvID string
LinuxContainerEnvironmentExtensions LinuxContainerEnvironmentExtensions
} }
@@ -902,6 +863,7 @@ func (cr *containerReference) tryReadID(opt string, cbk func(id int)) common.Exe
return nil return nil
} }
defer resp.Close() defer resp.Close()
defer context.AfterFunc(ctx, resp.Close)()
sid, err := resp.Reader.ReadString('\n') sid, err := resp.Reader.ReadString('\n')
if err != nil { if err != nil {
@@ -947,7 +909,7 @@ func (cr *containerReference) waitForCommand(ctx context.Context, resp client.Hi
if err != nil { if err != nil {
logger.Debugf("Failed to drain command output after cancellation: %v", err) logger.Debugf("Failed to drain command output after cancellation: %v", err)
} }
case <-time.After(drainGracePeriod): case <-time.After(DrainGracePeriod):
logger.Warn("Timed out draining command output after cancellation") logger.Warn("Timed out draining command output after cancellation")
} }
@@ -999,37 +961,6 @@ func (cr *containerReference) extractionRoot(dstPath string) string {
return "/" return "/"
} }
func (cr *containerReference) CopyTarStream(ctx context.Context, destPath string, write func(*TarWriter) error) error {
if common.Dryrun(ctx) {
return nil
}
if err := cr.connect()(ctx); err != nil {
return err
}
if err := cr.find()(ctx); err != nil {
return err
}
if cr.id == "" {
return cr.missingContainerError("copy to %s", destPath)
}
destPath = path.Clean(destPath)
root := cr.extractionRoot(destPath)
rel := func(fullPath string) string { return strings.TrimPrefix(strings.TrimPrefix(fullPath, root), "/") }
var parents []string // sent as entries so they belong to the job user, the daemon creates missing ones as root
for dir := destPath; dir != root && dir != "/" && dir != "."; dir = path.Dir(dir) {
if _, err := cr.cli.ContainerStatPath(ctx, cr.id, client.ContainerStatPathOptions{Path: dir}); err == nil {
break
} else if !cerrdefs.IsNotFound(err) {
return fmt.Errorf("stat %s: %w", dir, err)
}
parents = append(parents, rel(dir))
}
slices.Reverse(parents)
content := pipeTar(cr.UID, cr.GID, rel(destPath), parents, write)
defer content.Close()
return cr.upload(ctx, root, content)
}
func (cr *containerReference) copyContent(dstPath string, files ...*FileEntry) common.Executor { func (cr *containerReference) copyContent(dstPath string, files ...*FileEntry) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
if cr.id == "" { if cr.id == "" {
@@ -1122,7 +1053,7 @@ func (cr *containerReference) wait() common.Executor {
if cr.attachDone != nil { if cr.attachDone != nil {
select { select {
case <-cr.attachDone: case <-cr.attachDone:
case <-time.After(drainGracePeriod): case <-time.After(DrainGracePeriod):
logger.Warn("Timed out draining container output") logger.Warn("Timed out draining container output")
} }
cr.attachDone = nil cr.attachDone = nil
+22 -66
View File
@@ -45,14 +45,15 @@ func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
Return(mobyclient.ContainerInspectResult{Container: container.InspectResponse{Config: &container.Config{Env: []string{ Return(mobyclient.ContainerInspectResult{Container: container.InspectResponse{Config: &container.Config{Env: []string{
"PATH=/opt/node/x64/bin:/usr/bin:/bin", "CONFLICT_VAR=container-value", "SOME_RANDOM_VAR=", "PATH=/opt/node/x64/bin:/usr/bin:/bin", "CONFLICT_VAR=container-value", "SOME_RANDOM_VAR=",
"ANOTHER_ONE=old-value", "ANOTHER_ONE=BUT_I_HAVE_VALUE", "LITERAL='quoted'=$HOME\nsecond line", "ANOTHER_ONE=old-value", "ANOTHER_ONE=BUT_I_HAVE_VALUE", "LITERAL='quoted'=$HOME\nsecond line",
}}}}, nil) }}}}, nil).Once()
env := map[string]string{ env := map[string]string{
"PATH": "/custom/bin", "PATH": "/custom/bin",
"RANDOM_VAR": "WITH_VALUE", "RANDOM_VAR": "WITH_VALUE",
"ANOTHER_VAR": "", "ANOTHER_VAR": "",
"CONFLICT_VAR": "step-value", "CONFLICT_VAR": "step-value",
} }
require.NoError(t, (&containerReference{cli: client, id: "amd64-job"}).UpdateFromImageEnv(&env)(t.Context())) cr := &containerReference{cli: client, id: "amd64-job"}
require.NoError(t, cr.UpdateFromImageEnv(&env)(t.Context()))
assert.Equal(t, map[string]string{ assert.Equal(t, map[string]string{
"PATH": "/custom/bin:/opt/node/x64/bin:/usr/bin:/bin", "PATH": "/custom/bin:/opt/node/x64/bin:/usr/bin:/bin",
"RANDOM_VAR": "WITH_VALUE", "RANDOM_VAR": "WITH_VALUE",
@@ -62,6 +63,10 @@ func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
"CONFLICT_VAR": "step-value", "CONFLICT_VAR": "step-value",
"LITERAL": "'quoted'=$HOME\nsecond line", "LITERAL": "'quoted'=$HOME\nsecond line",
}, env) }, env)
pathEnv := map[string]string{}
require.NoError(t, cr.UpdateFromImageEnv(&pathEnv)(t.Context()))
assert.Equal(t, "/opt/node/x64/bin:/usr/bin:/bin", pathEnv["PATH"])
client.AssertExpectations(t)
} }
type mockDockerClient struct { type mockDockerClient struct {
@@ -260,28 +265,6 @@ func TestDockerExecFailure(t *testing.T) {
assert.Equal(t, ExitCodeError(1), exitErr) assert.Equal(t, ExitCodeError(1), exitErr)
assert.Equal(t, "Process completed with exit code 1.", err.Error()) assert.Equal(t, "Process completed with exit code 1.", err.Error())
client.On("ContainerInspect", ctx, "123", mobyclient.ContainerInspectOptions{}).
Return(mobyclient.ContainerInspectResult{}, nil)
client.On("ExecCreate", mock.Anything, "123", mobyclient.ExecCreateOptions{
Cmd: []string{"printf", `a\b`}, AttachStdout: true, AttachStderr: true,
}).Return(mobyclient.ExecCreateResult{ID: "output"}, nil)
client.On("ExecAttach", mock.Anything, "output", mobyclient.ExecAttachOptions{}).
Return(mobyclient.ExecAttachResult{
Conn: &mockConn{},
Reader: bufio.NewReader(bytes.NewReader(append(stdcopyFrame(stdcopy.Stdout, "a\x00b"), stdcopyFrame(stdcopy.Stderr, " failed \n")...))),
}, nil)
client.On("ExecInspect", mock.Anything, "output", mobyclient.ExecInspectOptions{}).
Return(mobyclient.ExecInspectResult{ExitCode: 3}, nil)
output, err := cr.ExecOutput(ctx, []string{"printf", `a\b`})
require.NoError(t, err)
content, err := io.ReadAll(output)
require.ErrorAs(t, err, &exitErr)
assert.Equal(t, ExitCodeError(3), exitErr)
assert.Equal(t, "Process completed with exit code 3.: failed", err.Error())
assert.Equal(t, "a\x00b", string(content))
require.ErrorAs(t, output.Close(), &exitErr)
assert.Equal(t, ExitCodeError(3), exitErr)
client.AssertExpectations(t) client.AssertExpectations(t)
} }
@@ -378,6 +361,21 @@ func TestDockerAttachStreamClosesWithItsContext(t *testing.T) {
client.AssertExpectations(t) client.AssertExpectations(t)
} }
func TestDockerReadIDStreamClosesWithItsContext(t *testing.T) {
conn, peer := net.Pipe()
t.Cleanup(func() { _ = peer.Close() })
client := &mockDockerClient{}
client.On("ExecCreate", mock.Anything, "123", mock.AnythingOfType("client.ExecCreateOptions")).
Return(mobyclient.ExecCreateResult{ID: "id"}, nil)
client.On("ExecAttach", mock.Anything, "id", mobyclient.ExecAttachOptions{}).
Return(mobyclient.ExecAttachResult{HijackedResponse: mobyclient.NewHijackedResponse(conn, "")}, nil)
ctx, cancel := context.WithCancel(t.Context())
cancel()
require.NoError(t, (&containerReference{id: "123", cli: client}).tryReadID("-u", func(int) { t.Error("read an ID after cancellation") })(ctx))
client.AssertExpectations(t)
}
func TestDockerWaitFailure(t *testing.T) { func TestDockerWaitFailure(t *testing.T) {
ctx := t.Context() ctx := t.Context()
statusCh := make(chan container.WaitResponse, 1) statusCh := make(chan container.WaitResponse, 1)
@@ -555,8 +553,6 @@ func TestRejectsMissingContainer(t *testing.T) {
check("exec", cr.exec([]string{"echo"}, nil, "", "")(ctx)) check("exec", cr.exec([]string{"echo"}, nil, "", "")(ctx))
_, err := cr.GetContainerArchive(ctx, "/var/run/act/x") _, err := cr.GetContainerArchive(ctx, "/var/run/act/x")
check("GetContainerArchive", err) check("GetContainerArchive", err)
_, err = cr.ExecOutput(ctx, []string{"true"})
check("ExecOutput", err)
_, err = cr.Inspect(ctx) _, err = cr.Inspect(ctx)
check("Inspect", err) check("Inspect", err)
@@ -568,46 +564,6 @@ func TestRejectsMissingContainer(t *testing.T) {
check("Inspect after removal", err) check("Inspect after removal", err)
} }
func TestCopyTarStreamOwnsMissingParentsAndRejectsEscapes(t *testing.T) {
ctx := t.Context()
client := &mockDockerClient{}
client.On("ContainerInspect", ctx, "job", mobyclient.ContainerInspectOptions{}).
Return(mobyclient.ContainerInspectResult{Container: container.InspectResponse{ID: "job", Name: "/job"}}, nil)
stat := func(p string, err error) {
client.On("ContainerStatPath", ctx, "job", mobyclient.ContainerStatPathOptions{Path: p}).
Return(mobyclient.ContainerStatPathResult{}, err).Once()
}
stat("/w/out/a", cerrdefs.ErrNotFound)
stat("/w/out", cerrdefs.ErrNotFound)
stat("/w", nil)
var entries []string
client.On("CopyToContainer", ctx, "job", mock.Anything).Run(func(args mock.Arguments) {
opts := args.Get(2).(mobyclient.CopyToContainerOptions)
assert.Equal(t, "/", opts.DestinationPath)
reader := tar.NewReader(opts.Content)
for {
header, err := reader.Next()
if errors.Is(err, io.EOF) {
return
}
require.NoError(t, err)
assert.Equal(t, []any{1001, 1002, ""}, []any{header.Uid, header.Gid, header.Uname}, header.Name)
entries = append(entries, header.Name)
}
}).Return(mobyclient.CopyToContainerResult{}, nil).Once()
cr := &containerReference{id: "job", cli: client, input: &NewContainerInput{Name: "job"}, UID: 1001, GID: 1002}
require.NoError(t, cr.CopyTarStream(ctx, "/w/out/a/", func(tw *TarWriter) error {
return tw.WriteHeader(&tar.Header{Typeflag: tar.TypeReg, Name: "dir/f.txt", Mode: 0o644, Uname: "nobody"})
}))
assert.Equal(t, []string{"w/out/", "w/out/a/", "w/out/a/dir/", "w/out/a/dir/f.txt"}, entries)
client.AssertExpectations(t)
require.ErrorContains(t, writeTarStream(io.Discard, 0, 0, "w/out/a", nil, func(tw *TarWriter) error {
return tw.WriteHeader(&tar.Header{Typeflag: tar.TypeReg, Name: "../../../etc/passwd"})
}), "escapes")
}
func TestPublicCopyPipelineHandlesStaleID(t *testing.T) { func TestPublicCopyPipelineHandlesStaleID(t *testing.T) {
source := t.TempDir() source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "x"), nil, 0o644)) require.NoError(t, os.WriteFile(filepath.Join(source, "x"), nil, 0o644))
+15 -94
View File
@@ -16,7 +16,6 @@ import (
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"runtime" "runtime"
"slices"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -32,18 +31,15 @@ import (
) )
type HostEnvironment struct { type HostEnvironment struct {
Path string Path string
TmpDir string TmpDir string
ToolCache string ToolCache string
Workdir string Workdir string
// CleanWorkdir means teardown owns Workdir and may delete it. Leave false ActPath string
// when Workdir points at a caller-owned checkout (e.g. `act` local mode). Root string
CleanWorkdir bool TrackingID string // RUNNER_TRACKING_ID of the job, a step with another value keeps its leftovers
ActPath string StdOut io.Writer
Root string AllocatePTY bool // allocate a pseudo-TTY for each step's process
TrackingID string // RUNNER_TRACKING_ID of the job, a step with another value keeps its leftovers
StdOut io.Writer
AllocatePTY bool // allocate a pseudo-TTY for each step's process
// procGroup owns every process the job's steps start. Atomic: Remove may read // procGroup owns every process the job's steps start. Atomic: Remove may read
// it while a step is still starting. // it while a step is still starting.
@@ -116,20 +112,6 @@ func (e *HostEnvironment) Inspect(_ context.Context) (*Info, error) {
return &Info{Health: HealthNone, Ports: map[string]string{}}, nil return &Info{Health: HealthNone, Ports: map[string]string{}}, nil
} }
func (e *HostEnvironment) ExecOutput(ctx context.Context, command []string) (io.ReadCloser, error) {
return streamOutput(ctx, func(ctx context.Context, stdout, stderr io.Writer) error {
cmd := exec.CommandContext(ctx, command[0], command[1:]...)
cmd.Dir, cmd.Stdout, cmd.Stderr = e.Workdir, stdout, stderr
if err := cmd.Run(); err != nil {
if exit, ok := errors.AsType[*exec.ExitError](err); ok {
return ExitCodeError(exit.ExitCode())
}
return err
}
return nil
}), nil
}
func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) { func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
srcPath = filepath.Clean(srcPath) srcPath = filepath.Clean(srcPath)
fi, err := os.Lstat(srcPath) fi, err := os.Lstat(srcPath)
@@ -177,58 +159,6 @@ func writeHostArchive(ctx context.Context, tc *filecollector.TarCollector, srcPa
return tc.WriteFile(fi.Name(), fi, linkname, f) return tc.WriteFile(fi.Name(), fi, linkname, f)
} }
func (e *HostEnvironment) CopyTarStream(ctx context.Context, destPath string, write func(*TarWriter) error) error {
if err := os.MkdirAll(destPath, 0o777); err != nil {
return err
}
root, err := os.OpenRoot(destPath)
if err != nil {
return err
}
defer root.Close()
content := pipeTar(0, 0, "", nil, write)
defer content.Close()
tr := tar.NewReader(content)
var dirs []*tar.Header
for {
if err := ctx.Err(); err != nil {
return err
}
hdr, err := tr.Next()
if errors.Is(err, io.EOF) {
break
} else if err != nil {
return err
}
hdr.Name = filepath.FromSlash(strings.TrimSuffix(hdr.Name, "/"))
if hdr.Typeflag == tar.TypeDir {
err = root.MkdirAll(hdr.Name, 0o777) // modes apply after the contents, so read-only directories can be filled
dirs = append(dirs, hdr)
} else {
err = writeRootFile(root, hdr.Name, hdr.FileInfo().Mode().Perm(), tr)
}
if err != nil {
return err
}
}
for _, dir := range slices.Backward(dirs) {
if err := root.Chmod(dir.Name, dir.FileInfo().Mode().Perm()); err != nil {
return err
}
}
return nil
}
func writeRootFile(root *os.Root, name string, mode fs.FileMode, body io.Reader) error {
_ = root.Remove(name) // replaces read-only files and applies mode, as the daemon and tar do
file, err := root.OpenFile(name, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, mode)
if err != nil {
return err
}
_, err = io.Copy(file, body)
return errors.Join(err, file.Close())
}
func (e *HostEnvironment) Pull(_ bool) common.Executor { func (e *HostEnvironment) Pull(_ bool) common.Executor {
return common.NewPipelineExecutor() return common.NewPipelineExecutor()
} }
@@ -350,6 +280,7 @@ func (e *HostEnvironment) exec(ctx context.Context, command []string, cmdline st
// wait, so an orphan holding cmd's stdout pipe cannot hang cmd.Wait(). // wait, so an orphan holding cmd's stdout pipe cannot hang cmd.Wait().
treeKill := process.NewTreeKill(cmd) treeKill := process.NewTreeKill(cmd)
treeKill.KeepLeftovers = !tracked treeKill.KeepLeftovers = !tracked
treeKill.GracefulCancel = true
var ppty *os.File var ppty *os.File
var tty *os.File var tty *os.File
@@ -502,24 +433,14 @@ func (e *HostEnvironment) Remove() common.Executor {
rmCtx, rmCancel := context.WithTimeout(context.Background(), hostCleanupTimeout) rmCtx, rmCancel := context.WithTimeout(context.Background(), hostCleanupTimeout)
defer rmCancel() defer rmCancel()
var errs []error err := removePathWithRetry(rmCtx, cmp.Or(e.Root, e.Path))
if err := removePathWithRetry(rmCtx, cmp.Or(e.Root, e.Path)); err != nil { if err != nil {
logger.Warnf("failed to remove host misc state %s: %v", cmp.Or(e.Root, e.Path), err) logger.Warnf("failed to remove host misc state %s: %v", cmp.Or(e.Root, e.Path), err)
errs = append(errs, err)
} }
if e.CleanWorkdir { if errors.Is(err, context.DeadlineExceeded) {
if err := removePathWithRetry(rmCtx, e.Workdir); err != nil { return nil // a stalled delete must not fail job completion
logger.Warnf("failed to remove host workspace %s: %v", e.Workdir, err)
errs = append(errs, err)
}
} }
for _, err := range errs { return err
if !errors.Is(err, context.DeadlineExceeded) {
return errors.Join(errs...)
}
}
// Teardown timed out; warned above. Do not fail job completion over it.
return nil
} }
} }
+6 -80
View File
@@ -8,7 +8,6 @@ import (
"archive/tar" "archive/tar"
"bytes" "bytes"
"context" "context"
"fmt"
"io" "io"
"os" "os"
"path/filepath" "path/filepath"
@@ -84,69 +83,11 @@ func TestGetContainerArchive(t *testing.T) {
assert.ErrorIs(t, err, os.ErrNotExist) assert.ErrorIs(t, err, os.ErrNotExist)
} }
func TestHostCopyTarStream(t *testing.T) {
dest := filepath.Join(t.TempDir(), "out")
write := func(name string, mode int64, body string) func(*TarWriter) error {
return func(tw *TarWriter) error {
if err := tw.WriteHeader(&tar.Header{Typeflag: tar.TypeReg, Name: name, Mode: mode, Size: int64(len(body))}); err != nil {
return err
}
_, err := tw.Write([]byte(body))
return err
}
}
e := &HostEnvironment{}
require.NoError(t, e.CopyTarStream(t.Context(), dest, write("dir/run.sh", 0o755, "#!/bin/sh\n")))
content, err := os.ReadFile(filepath.Join(dest, "dir", "run.sh"))
require.NoError(t, err)
assert.Equal(t, "#!/bin/sh\n", string(content))
if runtime.GOOS != "windows" {
require.NoError(t, os.Chmod(filepath.Join(dest, "dir", "run.sh"), 0o444))
require.NoError(t, e.CopyTarStream(t.Context(), dest, func(tw *TarWriter) error {
if err := tw.WriteHeader(&tar.Header{Typeflag: tar.TypeDir, Name: "dir/", Mode: 0o555}); err != nil {
return err
}
return write("dir/run.sh", 0o755, "")(tw)
}))
t.Cleanup(func() { require.NoError(t, os.Chmod(filepath.Join(dest, "dir"), 0o755)) })
for name, mode := range map[string]os.FileMode{"dir": 0o555, filepath.Join("dir", "run.sh"): 0o755} {
info, err := os.Stat(filepath.Join(dest, name))
require.NoError(t, err)
assert.Equal(t, mode, info.Mode().Perm(), name)
}
require.NoError(t, os.Symlink(t.TempDir(), filepath.Join(dest, "link")))
require.ErrorContains(t, e.CopyTarStream(t.Context(), dest, write("link/x", 0o644, "x")), "escapes")
}
require.ErrorContains(t, e.CopyTarStream(t.Context(), dest, write("../escape", 0o644, "x")), "escapes")
assert.NoFileExists(t, filepath.Join(filepath.Dir(dest), "escape"))
require.Error(t, e.CopyTarStream(t.Context(), dest, func(_tw *TarWriter) error { return fmt.Errorf("read body: %w", io.EOF) }))
canceled, cancel := context.WithCancel(t.Context())
cancel()
require.ErrorIs(t, e.CopyTarStream(canceled, dest, write("late", 0o644, "x")), context.Canceled)
assert.NoFileExists(t, filepath.Join(dest, "late"))
}
func TestHostEnvironmentExecExitCode(t *testing.T) { func TestHostEnvironmentExecExitCode(t *testing.T) {
t.Parallel() t.Parallel()
if runtime.GOOS == "windows" { if runtime.GOOS == "windows" {
t.Skip("uses POSIX shell") t.Skip("uses POSIX shell")
} }
for _, read := range []bool{true, false} {
output, err := (&HostEnvironment{AllocatePTY: true}).ExecOutput(t.Context(), []string{"sh", "-c", `test ! -t 1; printf 'out\000'; printf ' failed \n' >&2; exit 3`})
require.NoError(t, err)
if read {
content, err := io.ReadAll(output)
var exit ExitCodeError
require.ErrorAs(t, err, &exit)
assert.Equal(t, ExitCodeError(3), exit)
assert.Equal(t, "out\x00", string(content))
assert.Equal(t, "Process completed with exit code 3.: failed", err.Error())
}
var exit ExitCodeError
require.ErrorAs(t, output.Close(), &exit)
assert.Equal(t, ExitCodeError(3), exit)
}
for name, allocatePTY := range map[string]bool{"without PTY": false, "with PTY": true} { for name, allocatePTY := range map[string]bool{"without PTY": false, "with PTY": true} {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -217,27 +158,12 @@ func TestHostEnvironmentAllocatePTY(t *testing.T) {
} }
func TestHostEnvironmentRemoveWorkdir(t *testing.T) { func TestHostEnvironmentRemoveWorkdir(t *testing.T) {
for name, owned := range map[string]bool{"preserves caller workdir": false, "cleans owned workdir": true} { miscRoot, workdir := t.TempDir(), t.TempDir()
t.Run(name, func(t *testing.T) { e := &HostEnvironment{Path: filepath.Join(miscRoot, "hostexecutor"), Workdir: workdir, Root: miscRoot}
miscRoot, workdir := t.TempDir(), t.TempDir() require.NoError(t, os.MkdirAll(e.Path, 0o700))
e := &HostEnvironment{ require.NoError(t, e.Remove()(t.Context()))
Path: filepath.Join(miscRoot, "hostexecutor"), assert.NoDirExists(t, miscRoot)
Workdir: workdir, assert.DirExists(t, workdir)
CleanWorkdir: owned,
Root: miscRoot,
}
require.NoError(t, os.MkdirAll(e.Path, 0o700))
require.NoError(t, os.MkdirAll(filepath.Join(miscRoot, "tmp"), 0o700))
require.NoError(t, e.Remove()(t.Context()))
assert.NoDirExists(t, miscRoot)
_, err := os.Stat(workdir)
if owned {
assert.ErrorIs(t, err, os.ErrNotExist)
} else {
require.NoError(t, err)
}
})
}
} }
func TestRemoveAllWithContextDoesNotHangOnStuckDelete(t *testing.T) { func TestRemoveAllWithContextDoesNotHangOnStuckDelete(t *testing.T) {
+18 -17
View File
@@ -50,6 +50,7 @@ type KubernetesPod struct {
job *kubernetesContainer job *kubernetesContainer
services []*kubernetesContainer services []*kubernetesContainer
arch, path string arch, path string
imageOS string
ips []net.IP ips []net.IP
transfer *transferServer transfer *transferServer
transferImage string transferImage string
@@ -432,11 +433,12 @@ func (c *kubernetesContainer) Start(_ bool) common.Executor {
if err != nil { if err != nil {
return err return err
} }
probed := strings.SplitN(output, "\n", 3) probed := strings.SplitN(output, "\x00", 5)
for len(probed) < 3 { for len(probed) < 5 {
probed = append(probed, "") probed = append(probed, "")
} }
c.pod.arch, c.gzip, c.pod.path = goArchToActionArch(probed[0]), probed[1] == "gzip", probed[2] c.pod.arch, c.gzip, c.pod.path = goArchToActionArch(probed[0]), probed[1] == "gzip", probed[2]
c.pod.imageOS = cmp.Or(probed[3], ImageOSFromOSRelease(probed[4]))
return nil return nil
} }
} }
@@ -447,7 +449,7 @@ func (p *KubernetesPod) probe(ctx context.Context, container string) (string, er
deadline := time.Now().Add(time.Minute) deadline := time.Now().Add(time.Minute)
for delay := 100 * time.Millisecond; ; delay = min(2*delay, 2*time.Second) { for delay := 100 * time.Millisecond; ; delay = min(2*delay, 2*time.Second) {
var stdout bytes.Buffer var stdout bytes.Buffer
err := p.client.exec(ctx, p.name, container, []string{"sh", "-c", `printf '%s\n%s\n%s' "$(uname -m)" "$(command -v gzip >/dev/null && echo gzip)" "$PATH"`}, nil, &stdout, io.Discard) err := p.client.exec(ctx, p.name, container, []string{"sh", "-c", `printf '%s\0%s\0%s\0%s\0%s' "$(uname -m)" "$(command -v gzip >/dev/null && echo gzip)" "$PATH" "${ImageOS-}" "$(cat /etc/os-release 2>/dev/null || cat /usr/lib/os-release 2>/dev/null)"`}, nil, &stdout, io.Discard)
var exitErr ExitCodeError var exitErr ExitCodeError
var apiErr *apiError var apiErr *apiError
if err == nil || ctx.Err() != nil || errors.As(err, &exitErr) || time.Now().After(deadline) || if err == nil || ctx.Err() != nil || errors.As(err, &exitErr) || time.Now().After(deadline) ||
@@ -553,10 +555,6 @@ func (c *kubernetesContainer) Copy(destPath string, files ...*FileEntry) common.
} }
} }
func (c *kubernetesContainer) CopyTarStream(ctx context.Context, destPath string, write func(*TarWriter) error) error {
return c.extract(ctx, destPath, true, func(writer io.Writer) error { return writeTarStream(writer, 0, 0, "", nil, write) }) // streams carry large content like artifacts
}
func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor { func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
return c.extract(ctx, destPath, true, func(writer io.Writer) error { // directories are actions and workspaces, megabytes of compressible sources return c.extract(ctx, destPath, true, func(writer io.Writer) error { // directories are actions and workspaces, megabytes of compressible sources
@@ -565,17 +563,17 @@ func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, sk
} }
} }
func (c *kubernetesContainer) ExecOutput(ctx context.Context, command []string) (io.ReadCloser, error) { // GetContainerArchive streams the archive, checking the path first as tar writes an empty archive for a missing one.
return streamOutput(ctx, func(ctx context.Context, stdout, stderr io.Writer) error {
return c.pod.client.exec(ctx, c.pod.name, c.name, command, nil, stdout, stderr)
}), nil
}
func (c *kubernetesContainer) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) { func (c *kubernetesContainer) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
reader, err := c.ExecOutput(ctx, []string{"sh", "-c", `test -e "$1" -o -L "$1" && exec tar -cf - -C "$2" "$3"`, "sh", srcPath, path.Dir(srcPath), path.Base(srcPath)}) reader, writer := io.Pipe()
if err != nil { go func() {
return nil, err var stderr bytes.Buffer
} err := c.pod.client.exec(ctx, c.pod.name, c.name, []string{"sh", "-c", `test -e "$1" -o -L "$1" && exec tar -cf - -C "$2" "$3"`, "sh", srcPath, path.Dir(srcPath), path.Base(srcPath)}, nil, writer, &stderr)
if err != nil {
err = fmt.Errorf("archive %s: %w: %s", srcPath, err, strings.TrimSpace(stderr.String()))
}
writer.CloseWithError(err)
}()
return startedArchive(reader) return startedArchive(reader)
} }
@@ -586,6 +584,9 @@ func (c *kubernetesContainer) UpdateFromEnv(srcPath string, env *map[string]stri
func (c *kubernetesContainer) UpdateFromImageEnv(env *map[string]string) common.Executor { func (c *kubernetesContainer) UpdateFromImageEnv(env *map[string]string) common.Executor {
return func(context.Context) error { return func(context.Context) error {
(*env)["PATH"] = cmp.Or((*env)["PATH"], c.pod.path) (*env)["PATH"] = cmp.Or((*env)["PATH"], c.pod.path)
if c.pod.imageOS != "" {
(*env)["ImageOS"] = cmp.Or((*env)["ImageOS"], c.pod.imageOS)
}
return nil return nil
} }
} }
+5 -28
View File
@@ -193,7 +193,7 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
path := workdir + ":" + os.Getenv("PATH") path := workdir + ":" + os.Getenv("PATH")
kubeconfig, cluster := fakeKubernetes(t, `{"spec":{"containers":[{},{}]},"status":{"phase":"Running","containerStatuses":[ kubeconfig, cluster := fakeKubernetes(t, `{"spec":{"containers":[{},{}]},"status":{"phase":"Running","containerStatuses":[
{"name":"job","ready":true,"state":{"running":{}}}, {"name":"job","ready":true,"state":{"running":{}}},
{"name":"my-db","ready":true,"state":{"running":{}}}]}}`, "PATH="+path) {"name":"my-db","ready":true,"state":{"running":{}}}]}}`, "PATH="+path, "ImageOS=ubuntu24-arm64")
options := KubernetesOptions{Kubeconfig: kubeconfig, RunnerUUID: "runner-uuid", PodTemplates: []map[string]any{ options := KubernetesOptions{Kubeconfig: kubeconfig, RunnerUUID: "runner-uuid", PodTemplates: []map[string]any{
{"spec": map[string]any{ {"spec": map[string]any{
"automountServiceAccountToken": true, "automountServiceAccountToken": true,
@@ -273,17 +273,6 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
assert.Equal(t, "hunter2\nit's\nme\n5\n"+workdir+"\n", jobLog.String()) assert.Equal(t, "hunter2\nit's\nme\n5\n"+workdir+"\n", jobLog.String())
assert.Equal(t, ExitCodeError(3), job.Exec([]string{"sh", "-c", "echo failing >&2; exit 3"}, nil, "", "")(ctx)) assert.Equal(t, ExitCodeError(3), job.Exec([]string{"sh", "-c", "echo failing >&2; exit 3"}, nil, "", "")(ctx))
assert.Contains(t, jobLog.String(), "failing\n") assert.Contains(t, jobLog.String(), "failing\n")
output, err := job.ExecOutput(ctx, []string{"sh", "-c", `printf 'out\000'; printf ' failed \n' >&2; exit 3`})
require.NoError(t, err)
content, err := io.ReadAll(output)
var exit ExitCodeError
require.ErrorAs(t, err, &exit)
assert.Equal(t, ExitCodeError(3), exit)
assert.Contains(t, err.Error(), ": failed")
assert.Equal(t, "out\x00", string(content))
require.ErrorAs(t, output.Close(), &exit)
assert.Equal(t, ExitCodeError(3), exit)
for _, query := range cluster.requests["exec"] { for _, query := range cluster.requests["exec"] {
assert.NotContains(t, query, "hunter2") assert.NotContains(t, query, "hunter2")
} }
@@ -296,7 +285,7 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
reader := tar.NewReader(archive) reader := tar.NewReader(archive)
header, err := reader.Next() header, err := reader.Next()
require.NoError(t, err) require.NoError(t, err)
content, err = io.ReadAll(reader) content, err := io.ReadAll(reader)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, "event.json", header.Name) assert.Equal(t, "event.json", header.Name)
assert.JSONEq(t, `{"ref":"main"}`, string(content)) assert.JSONEq(t, `{"ref":"main"}`, string(content))
@@ -320,21 +309,9 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
assert.Contains(t, lastExec(), "command="+url.QueryEscape(workdir+"/act/actions/sha/")) assert.Contains(t, lastExec(), "command="+url.QueryEscape(workdir+"/act/actions/sha/"))
assert.Contains(t, lastExec(), url.QueryEscape("gzip -dc |")) assert.Contains(t, lastExec(), url.QueryEscape("gzip -dc |"))
require.NoError(t, job.CopyTarStream(ctx, workdir+"/download/new", func(tw *TarWriter) error { imageEnv := map[string]string{}
if err := tw.WriteHeader(&tar.Header{Typeflag: tar.TypeReg, Name: "dir/data.txt", Mode: 0o644, Size: 7}); err != nil { require.NoError(t, job.UpdateFromImageEnv(&imageEnv)(ctx))
return err assert.Equal(t, map[string]string{"PATH": path, "ImageOS": "ubuntu24-arm64"}, imageEnv)
}
_, err := tw.Write([]byte("payload"))
return err
}))
downloaded, err := os.ReadFile(workdir + "/download/new/dir/data.txt")
require.NoError(t, err)
assert.Equal(t, "payload", string(downloaded))
assert.Contains(t, lastExec(), url.QueryEscape("gzip -dc |"))
pathEnv := map[string]string{}
require.NoError(t, job.UpdateFromImageEnv(&pathEnv)(ctx))
assert.Equal(t, path, pathEnv["PATH"])
info, err := service.Inspect(ctx) info, err := service.Inspect(ctx)
require.NoError(t, err) require.NoError(t, err)
@@ -108,3 +108,22 @@ func (*LinuxContainerEnvironmentExtensions) GetRunnerContext(ctx context.Context
func (*LinuxContainerEnvironmentExtensions) IsEnvironmentCaseInsensitive() bool { func (*LinuxContainerEnvironmentExtensions) IsEnvironmentCaseInsensitive() bool {
return false return false
} }
// ImageOSFromOSRelease derives ImageOS from an os-release file, empty for anything but Ubuntu, the only Linux GitHub names one.
func ImageOSFromOSRelease(osRelease string) string {
var id, major string
for line := range strings.Lines(osRelease) {
key, value, _ := strings.Cut(strings.TrimSpace(line), "=")
value = strings.Trim(value, `"'`)
switch key {
case "ID":
id = value
case "VERSION_ID":
major, _, _ = strings.Cut(value, ".")
}
}
if id != "ubuntu" || major == "" {
return ""
}
return id + major
}
+1 -70
View File
@@ -8,11 +8,7 @@ import (
"archive/tar" "archive/tar"
"bufio" "bufio"
"context" "context"
"errors"
"fmt"
"io" "io"
"io/fs"
"path"
"path/filepath" "path/filepath"
"strings" "strings"
@@ -20,73 +16,8 @@ import (
"gitea.com/gitea/runner/act/filecollector" "gitea.com/gitea/runner/act/filecollector"
) )
// TarWriter takes directories and files named relative to the destination, owned by the job user.
type TarWriter struct {
writer *tar.Writer
uid, gid int
prefix string
dirs map[string]bool
}
func (w *TarWriter) Write(data []byte) (int, error) {
return w.writer.Write(data)
}
func (w *TarWriter) WriteHeader(hdr *tar.Header) error {
name := strings.TrimSuffix(hdr.Name, "/")
if !fs.ValidPath(name) {
return fmt.Errorf("tar entry %q escapes the destination", hdr.Name)
}
if hdr.Typeflag != tar.TypeDir && hdr.Typeflag != tar.TypeReg {
return fmt.Errorf("tar entry %q: unsupported type %q", hdr.Name, hdr.Typeflag)
}
for index := range len(name) {
if dir := name[:index]; name[index] == '/' && !w.dirs[dir] {
if err := w.write(&tar.Header{Typeflag: tar.TypeDir, Name: path.Join(w.prefix, dir), Mode: 0o755}); err != nil {
return err
}
w.dirs[dir] = true
}
}
if hdr.Typeflag == tar.TypeDir {
w.dirs[name] = true
}
entry := *hdr
entry.Name = path.Join(w.prefix, name)
return w.write(&entry)
}
func (w *TarWriter) write(hdr *tar.Header) error {
hdr.Uid, hdr.Gid, hdr.Uname, hdr.Gname = w.uid, w.gid, "", "" // tar running as root prefers names over ids
if hdr.Typeflag == tar.TypeDir {
hdr.Name += "/"
}
return w.writer.WriteHeader(hdr)
}
func writeTarStream(dst io.Writer, uid, gid int, prefix string, parents []string, write func(*TarWriter) error) error {
tw := &TarWriter{writer: tar.NewWriter(dst), uid: uid, gid: gid, prefix: prefix, dirs: map[string]bool{}}
for _, name := range parents {
if err := tw.write(&tar.Header{Typeflag: tar.TypeDir, Name: name, Mode: 0o755}); err != nil {
return err
}
}
if err := write(tw); errors.Is(err, io.EOF) {
return fmt.Errorf("tar stream: %v", err) // %v, as a wrapped io.EOF reads as the archive's end
} else if err != nil {
return err
}
return tw.writer.Close()
}
func pipeTar(uid, gid int, prefix string, parents []string, write func(*TarWriter) error) *io.PipeReader {
reader, writer := io.Pipe()
go func() { writer.CloseWithError(writeTarStream(writer, uid, gid, prefix, parents, write)) }()
return reader
}
// startedArchive returns reader once its first byte arrives, so failing to start archiving errors here, not on read. // startedArchive returns reader once its first byte arrives, so failing to start archiving errors here, not on read.
func startedArchive(reader io.ReadCloser) (io.ReadCloser, error) { func startedArchive(reader *io.PipeReader) (io.ReadCloser, error) {
buffered := bufio.NewReader(reader) buffered := bufio.NewReader(reader)
if _, err := buffered.Peek(1); err != nil { if _, err := buffered.Peek(1); err != nil {
reader.Close() reader.Close()
+5 -2
View File
@@ -346,8 +346,10 @@ func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, b
ImageTag: image, ImageTag: image,
Platform: rc.Config.ContainerArchitecture, Platform: rc.Config.ContainerArchitecture,
BuildArgs: rc.proxyBuildArgs(), BuildArgs: rc.proxyBuildArgs(),
Stdout: common.NewLineWriter(rawLogLine(ctx)),
} }
if localAction { if localAction {
input.Dockerignore, _ = readSingleFileFromContainerArchive(ctx, rc.JobContainer, contextDir+"/.dockerignore", 1<<20)
buildContext, err := rc.JobContainer.GetContainerArchive(ctx, contextDir+"/.") buildContext, err := rc.JobContainer.GetContainerArchive(ctx, contextDir+"/.")
if err != nil { if err != nil {
return err return err
@@ -472,7 +474,7 @@ func newStepContainer(ctx context.Context, step step, image string, cmd, entrypo
envList = append(envList, rc.runnerEnv(ctx)...) envList = append(envList, rc.runnerEnv(ctx)...)
binds, mounts := rc.GetBindsAndMounts(ctx) binds, mounts, _ := rc.GetBindsAndMounts(ctx)
networkMode := "container:" + rc.jobContainerName() networkMode := "container:" + rc.jobContainerName()
if rc.IsHostEnv() { if rc.IsHostEnv() {
networkMode = "default" networkMode = "default"
@@ -587,7 +589,8 @@ func shouldRunPreStep(step actionStep) common.Conditional {
func hasPreStep(step actionStep) common.Conditional { func hasPreStep(step actionStep) common.Conditional {
return func(ctx context.Context) bool { return func(ctx context.Context) bool {
action := step.getActionModel() action := step.getActionModel()
return action.Runs.Using.IsComposite() || // a pre stage evaluates `with` before earlier steps run, so only a composite nesting actions gets one
return (action.Runs.Using.IsComposite() && slices.ContainsFunc(action.Runs.Steps, func(compositeStep model.Step) bool { return compositeStep.Uses != "" })) ||
(action.Runs.Using.IsNode() && (action.Runs.Using.IsNode() &&
action.Runs.Pre != "") || action.Runs.Pre != "") ||
(action.Runs.Using.IsDocker() && (action.Runs.Using.IsDocker() &&
+2 -2
View File
@@ -217,11 +217,11 @@ func (rc *RunContext) compositeExecutor(action *model.Action) *compositeSteps {
steps = append(steps, newCompositeStepLogExecutor(rc.newCompositeCommandExecutor(step.main()), stepID)) steps = append(steps, newCompositeStepLogExecutor(rc.newCompositeCommandExecutor(step.main()), stepID))
// run the post executor in reverse order // run the post executor in reverse order
stepPost := rc.newCompositeCommandExecutor(step.post()) stepPost := newCompositeStepLogExecutor(rc.newCompositeCommandExecutor(step.post()), stepID)
if postExecutor != nil { if postExecutor != nil {
stepPost = stepPost.Finally(postExecutor) stepPost = stepPost.Finally(postExecutor)
} }
postExecutor = newCompositeStepLogExecutor(stepPost, stepID) postExecutor = stepPost
} }
steps = append(steps, common.JobError) steps = append(steps, common.JobError)
+13 -20
View File
@@ -11,8 +11,6 @@ import (
"gitea.com/gitea/runner/act/common/git" "gitea.com/gitea/runner/act/common/git"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest" "gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.com/gitea/runner/act/container"
"gitea.com/gitea/runner/internal/pkg/action"
"gitea.dev/actionslib/pkg/model" "gitea.dev/actionslib/pkg/model"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -101,6 +99,19 @@ func TestCompositeActionParity(t *testing.T) {
require.ErrorIs(t, rc.compositeExecutor(&model.Action{Runs: model.ActionRuns{Using: "composite", Steps: []model.Step{{ID: "nested", Uses: "org/action@v1"}}}}).pre(t.Context()), assert.AnError) require.ErrorIs(t, rc.compositeExecutor(&model.Action{Runs: model.ActionRuns{Using: "composite", Steps: []model.Step{{ID: "nested", Uses: "org/action@v1"}}}}).pre(t.Context()), assert.AnError)
}) })
t.Run("post steps log under their own step ids", func(t *testing.T) {
hook := logtest.NewWithLevel(logging.LevelDebug)
rc := &RunContext{Config: &Config{}, JobContainer: &jobContainerMock{}}
steps := rc.compositeExecutor(&model.Action{Runs: model.ActionRuns{Using: "composite", Steps: []model.Step{{ID: "first", Uses: "./first"}, {ID: "second", Uses: "./second"}}}})
require.NoError(t, steps.post(common.WithLogger(t.Context(), logging.New(hook))))
var stepIDs []any
for _, record := range hook.AllRecords() {
ids, _ := record.Attr("stepID")
stepIDs = append(stepIDs, ids)
}
assert.Equal(t, []any{[]string{"second"}, []string{"first"}}, stepIDs)
})
} }
func TestAppendUniqueMasks(t *testing.T) { func TestAppendUniqueMasks(t *testing.T) {
@@ -184,21 +195,3 @@ func newTestCompositeRunContext(t *testing.T, parent *RunContext) *RunContext {
require.NoError(t, err) require.NoError(t, err)
return composite return composite
} }
func TestBuiltinActionGetsLowercaseInputsAndTheJobTokenInComposites(t *testing.T) {
job := makeTestRC(t, "runs-on: ubuntu-latest")
job.Config.Secrets = map[string]string{"ACTIONS_RUNTIME_TOKEN": "daemon-token"}
job.JobContainer = &container.HostEnvironment{Path: t.TempDir(), ActPath: t.TempDir(), Workdir: t.TempDir()}
composite := newTestCompositeRunContext(t, job)
composite.JobContainer = job.JobContainer
for _, rc := range []*RunContext{job, composite} {
var got *action.Context
sab := &stepActionBuiltin{Step: &model.Step{ID: "x", With: map[string]string{"Name": "x"}}, RunContext: rc, run: func(_ctx context.Context, c *action.Context) error {
got = c
return nil
}}
require.NoError(t, sab.main()(t.Context()))
assert.Equal(t, "daemon-token", got.RuntimeToken)
assert.Equal(t, map[string]string{"name": "x"}, got.Inputs)
}
}
+4 -1
View File
@@ -751,12 +751,15 @@ func TestExecAsDockerStageEntrypoint(t *testing.T) {
} }
} }
func TestDockerActionHasPreAndPostStep(t *testing.T) { func TestActionHasPreAndPostStep(t *testing.T) {
newStep := func(runs model.ActionRuns) actionStep { newStep := func(runs model.ActionRuns) actionStep {
return &stepActionRemote{action: &model.Action{Runs: runs}} return &stepActionRemote{action: &model.Action{Runs: runs}}
} }
ctx := t.Context() ctx := t.Context()
assert.False(t, hasPreStep(newStep(model.ActionRuns{Using: "composite", Steps: []model.Step{{Run: "true"}}}))(ctx))
assert.True(t, hasPreStep(newStep(model.ActionRuns{Using: "composite", Steps: []model.Step{{Run: "true"}, {Uses: "actions/cache@v4"}}}))(ctx))
assert.False(t, hasPreStep(newStep(model.ActionRuns{Using: "docker", Image: "Dockerfile"}))(ctx)) assert.False(t, hasPreStep(newStep(model.ActionRuns{Using: "docker", Image: "Dockerfile"}))(ctx))
assert.False(t, hasPostStep(newStep(model.ActionRuns{Using: "docker", Image: "Dockerfile"}))(ctx)) assert.False(t, hasPostStep(newStep(model.ActionRuns{Using: "docker", Image: "Dockerfile"}))(ctx))
+8 -5
View File
@@ -101,10 +101,10 @@ func TestMainStepsExecutorRunsAlwaysStepsAfterInterruption(t *testing.T) {
assert.Equal(t, []string{"step1", "always-step"}, ran) assert.Equal(t, []string{"step1", "always-step"}, ran)
require.NoError(t, laterStepCtxErr) require.NoError(t, laterStepCtxErr)
assert.Equal(t, wantError == context.Canceled, rc.jobCancelled) assert.Equal(t, wantError == context.Canceled, rc.jobCancelled)
assert.Equal(t, "cancelled", rc.getJobContext().Status)
setJobResult(t.Context(), rc, rc, true) setJobResult(t.Context(), rc, rc, true)
if wantError == context.DeadlineExceeded { if wantError == context.DeadlineExceeded {
assert.True(t, rc.jobFailed) assert.True(t, rc.jobFailed)
assert.Equal(t, "failure", rc.getJobContext().Status)
assert.Equal(t, "failure", rc.Run.Job().Result) assert.Equal(t, "failure", rc.Run.Job().Result)
} else { } else {
assert.Equal(t, "cancelled", rc.Run.Job().Result) assert.Equal(t, "cancelled", rc.Run.Job().Result)
@@ -259,8 +259,11 @@ func TestReportStepErrorTreatsCancelAsInterruption(t *testing.T) {
assert.True(t, rc.jobCancelled) assert.True(t, rc.jobCancelled)
// a real error still fails the job // a real error still fails the job
failed := common.WithJobErrorContainer(t.Context()) for _, stepErr := range []error{assert.AnError, context.DeadlineExceeded} {
reportStepError(failed, rc, assert.AnError) rc := &RunContext{}
require.ErrorIs(t, common.JobError(failed), assert.AnError) failed := common.WithJobErrorContainer(t.Context())
assert.True(t, rc.jobFailed) reportStepError(failed, rc, stepErr)
require.ErrorIs(t, common.JobError(failed), stepErr)
assert.Equal(t, "failure", rc.getJobContext().Status)
}
} }
+5 -1
View File
@@ -118,7 +118,11 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
rawLogger.Infof("%s", formatAnnotation(command, kvPairs, arg)) rawLogger.Infof("%s", formatAnnotation(command, kvPairs, arg))
return false return false
case "group", "endgroup": case "group", "endgroup":
return true if strings.HasPrefix(line, "::"+command+"::") || strings.HasPrefix(line, "##["+command+"]") {
return true
}
rawLogger.Infof("::%s::%s", command, EscapeCommandData(arg))
return false
} }
return echoed return echoed
} }
+15 -2
View File
@@ -240,10 +240,23 @@ func TestCommandLogWriter(t *testing.T) {
"::notice::title: m", "::error::x: m", "::error::x:1: m", "::notice::std::vector", "::error::x;y: m]", "::notice::title: m", "::error::x: m", "::error::x:1: m", "::notice::std::vector", "::error::x;y: m]",
}, },
}, },
{
name: "groups normalize unsupported prefixes and preserve recognized prefixes",
steps: [][]string{{
"::group::a%3Bb%5Dc%250A", "::endgroup::", "##[group]a%3Bb%5Dc%250A", "##[endgroup]",
"::GrOuP::a%25b%0Ac", "::EnDgRoUp::", " \t::group::indented", " \t::endgroup::",
"prefix ##[GrOuP]a%3Bb%5Dc%253B%0A", "prefix ##[EnDgRoUp]",
}},
log: []string{
"::group::a%3Bb%5Dc%250A", "::endgroup::", "##[group]a%3Bb%5Dc%250A", "##[endgroup]",
"::group::a%25b%0Ac", "::endgroup::", "::group::indented", "::endgroup::",
"::group::a;b]c%253B%0A", "::endgroup::",
},
},
{ {
name: "stop-commands keeps lines verbatim until resumed", name: "stop-commands keeps lines verbatim until resumed",
steps: [][]string{{"::stop-commands::resume-token", "::add-mask::kept", "::set-output name=a::b", "::RESUME-TOKEN::", "::add-mask::masked"}}, steps: [][]string{{"::stop-commands::resume-token", "::add-mask::kept", "::set-output name=a::b", " \t::GrOuP::kept%3B", "prefix ##[EnDgRoUp]", "::RESUME-TOKEN::", "::add-mask::masked"}},
log: []string{"::stop-commands::resume-token", "::add-mask::kept", "::set-output name=a::b", "::RESUME-TOKEN::"}, log: []string{"::stop-commands::resume-token", "::add-mask::kept", "::set-output name=a::b", " \t::GrOuP::kept%3B", "prefix ##[EnDgRoUp]", "::RESUME-TOKEN::"},
masks: []string{"resume-token", "masked"}, masks: []string{"resume-token", "masked"},
}, },
{ {
+9 -9
View File
@@ -54,6 +54,10 @@ func (cm *containerMock) UpdateFromEnv(srcPath string, env *map[string]string) c
return cm.Called(srcPath, env).Get(0).(func(context.Context) error) return cm.Called(srcPath, env).Get(0).(func(context.Context) error)
} }
func (cm *containerMock) UpdateFromImageEnv(_env *map[string]string) common.Executor {
return noopExecutor
}
func (cm *containerMock) Copy(destPath string, files ...*container.FileEntry) common.Executor { func (cm *containerMock) Copy(destPath string, files ...*container.FileEntry) common.Executor {
return cm.Called(destPath, files).Get(0).(func(context.Context) error) return cm.Called(destPath, files).Get(0).(func(context.Context) error)
} }
@@ -66,16 +70,11 @@ func (cm *containerMock) Exec(command []string, env map[string]string, user, wor
return cm.Called(command, env, user, workdir).Get(0).(func(context.Context) error) return cm.Called(command, env, user, workdir).Get(0).(func(context.Context) error)
} }
func (cm *containerMock) ExecOutput(ctx context.Context, command []string) (io.ReadCloser, error) {
args := cm.Called(ctx, command)
output, _ := args.Get(0).(io.ReadCloser)
return output, args.Error(1)
}
func (cm *containerMock) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) { func (cm *containerMock) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
args := cm.Called(ctx, srcPath) args := cm.Called(ctx, srcPath)
reader, _ := args.Get(0).(io.ReadCloser)
err, _ := args.Get(1).(error) err, _ := args.Get(1).(error)
return args.Get(0).(io.ReadCloser), err return reader, err
} }
func (cm *containerMock) DumpLogs(ctx context.Context) error { func (cm *containerMock) DumpLogs(ctx context.Context) error {
@@ -89,6 +88,7 @@ func (cm *containerMock) Inspect(ctx context.Context) (*container.Info, error) {
return info, err return info, err
} }
func (cm *containerMock) CopyTarStream(ctx context.Context, destPath string, write func(*container.TarWriter) error) error { func (cm *containerMock) ReplaceLogWriter(stdout, stderr io.Writer) (io.Writer, io.Writer) {
return cm.Called(ctx, destPath, write).Error(0) args := cm.Called(stdout, stderr)
return args.Get(0).(io.Writer), args.Get(1).(io.Writer)
} }
+7 -1
View File
@@ -136,7 +136,7 @@ func getHashFilesFunction(ctx context.Context, rc *RunContext) func(v []reflect.
followSymlink := false followSymlink := false
for i, p := range v { for i, p := range v {
s := p.String() s := exprparser.CoerceToString(p)
if i == 0 && strings.HasPrefix(s, "--") { if i == 0 && strings.HasPrefix(s, "--") {
if strings.EqualFold(s, "--follow-symbolic-links") { if strings.EqualFold(s, "--follow-symbolic-links") {
followSymlink = true followSymlink = true
@@ -324,6 +324,12 @@ func (rc *RunContext) resolveWorkflowCall(ctx context.Context) error {
if rc.workflowCallSecrets, err = callerEval.EvaluateCallerSecrets(secrets); err != nil { if rc.workflowCallSecrets, err = callerEval.EvaluateCallerSecrets(secrets); err != nil {
return fmt.Errorf("unable to interpolate %w", err) return fmt.Errorf("unable to interpolate %w", err)
} }
for _, name := range []string{"GITHUB_TOKEN", "GITEA_TOKEN"} {
token, ok := rc.Config.Secrets[name]
if _, mapped := common.LookupFold(rc.workflowCallSecrets, name, true); ok && !mapped {
rc.workflowCallSecrets[name] = token
}
}
return nil return nil
} }
+17 -8
View File
@@ -178,7 +178,7 @@ func (c *hashFilesContainer) ReplaceLogWriter(stdout, stderr io.Writer) (io.Writ
} }
func TestGetHashFilesFunction(t *testing.T) { func TestGetHashFilesFunction(t *testing.T) {
const failureMessage = "hashFiles('dir''file, *.sum') failed. Fail to hash files under directory '/workspace'" const failureMessage = "hashFiles('dir''file, *.sum, 123, true, ') failed. Fail to hash files under directory '/workspace'"
for _, test := range []struct { for _, test := range []struct {
name string name string
output string output string
@@ -192,7 +192,7 @@ func TestGetHashFilesFunction(t *testing.T) {
{name: "success", output: "patterns __OUTPUT__wrong__OUTPUT__ echoed\n__OUTPUT__digest__OUTPUT__", hash: "digest"}, {name: "success", output: "patterns __OUTPUT__wrong__OUTPUT__ echoed\n__OUTPUT__digest__OUTPUT__", hash: "digest"},
{name: "no matches", output: "__OUTPUT____OUTPUT__"}, {name: "no matches", output: "__OUTPUT____OUTPUT__"},
{name: "script failure", output: "Cannot find: node in PATH\n__OUTPUT__digest__OUTPUT__", execErr: container.ExitCodeError(1), err: failureMessage}, {name: "script failure", output: "Cannot find: node in PATH\n__OUTPUT__digest__OUTPUT__", execErr: container.ExitCodeError(1), err: failureMessage},
{name: "timeout", execErr: context.DeadlineExceeded, err: "hashFiles('dir''file, *.sum') couldn't finish within 120 seconds"}, {name: "timeout", execErr: context.DeadlineExceeded, err: "hashFiles('dir''file, *.sum, 123, true, ') couldn't finish within 120 seconds"},
{name: "job deadline", execErr: context.DeadlineExceeded, expired: true, err: context.DeadlineExceeded.Error()}, {name: "job deadline", execErr: context.DeadlineExceeded, expired: true, err: context.DeadlineExceeded.Error()},
{name: "node missing", execErr: fmt.Errorf("wrapped: %w", container.ExitCodeError(127)), warning: true}, {name: "node missing", execErr: fmt.Errorf("wrapped: %w", container.ExitCodeError(127)), warning: true},
{name: "node not executable", execErr: container.ExitCodeError(126), warning: true}, {name: "node not executable", execErr: container.ExitCodeError(126), warning: true},
@@ -205,7 +205,7 @@ func TestGetHashFilesFunction(t *testing.T) {
jobContainer := &hashFilesContainer{stdout: stdout, stderr: stderr, copyErr: test.copyErr} jobContainer := &hashFilesContainer{stdout: stdout, stderr: stderr, copyErr: test.copyErr}
jobContainer.execute = func(command []string, env map[string]string, user, workdir string) common.Executor { jobContainer.execute = func(command []string, env map[string]string, user, workdir string) common.Executor {
assert.Equal(t, []string{"node", "/var/run/act/workflow/hashfiles/index.js"}, command) assert.Equal(t, []string{"node", "/var/run/act/workflow/hashfiles/index.js"}, command)
assert.Equal(t, map[string]string{"patterns": "dir'file\n*.sum", "followSymbolicLinks": "true"}, env) assert.Equal(t, map[string]string{"patterns": "dir'file\n*.sum\n123\ntrue\n", "followSymbolicLinks": "true"}, env)
assert.Empty(t, user) assert.Empty(t, user)
assert.Empty(t, workdir) assert.Empty(t, workdir)
return func(_ctx context.Context) error { return func(_ctx context.Context) error {
@@ -223,7 +223,7 @@ func TestGetHashFilesFunction(t *testing.T) {
result, err := getHashFilesFunction(ctx, &RunContext{ result, err := getHashFilesFunction(ctx, &RunContext{
Config: &Config{Workdir: "/workspace"}, Config: &Config{Workdir: "/workspace"},
JobContainer: jobContainer, JobContainer: jobContainer,
})([]reflect.Value{reflect.ValueOf("--follow-symbolic-links"), reflect.ValueOf("dir'file"), reflect.ValueOf("*.sum")}) })([]reflect.Value{reflect.ValueOf("--follow-symbolic-links"), reflect.ValueOf("dir'file"), reflect.ValueOf("*.sum"), reflect.ValueOf(float64(123)), reflect.ValueOf(true), {}})
if test.err == "" { if test.err == "" {
require.NoError(t, err) require.NoError(t, err)
} else { } else {
@@ -503,7 +503,7 @@ jobs:
assert.Equal(t, "workflow", env["INPUT_FALLBACK"]) assert.Equal(t, "workflow", env["INPUT_FALLBACK"])
} }
func TestResolveWorkflowCallToleratesMismatchesAndPassesInheritedSecretsAndRuntimeToken(t *testing.T) { func TestResolveWorkflowCallToleratesMismatchesAndPassesSecretsAndTokens(t *testing.T) {
callerWorkflow, err := model.ReadWorkflow(strings.NewReader(` callerWorkflow, err := model.ReadWorkflow(strings.NewReader(`
jobs: jobs:
call: call:
@@ -513,6 +513,9 @@ jobs:
inherit: inherit:
uses: ./reuse.yml uses: ./reuse.yml
secrets: inherit secrets: inherit
override:
uses: ./reuse.yml
secrets: {github_token: "${{ secrets.token }}", GITEA_TOKEN: ""}
`)) `))
require.NoError(t, err) require.NoError(t, err)
callee, err := model.ReadWorkflow(strings.NewReader(` callee, err := model.ReadWorkflow(strings.NewReader(`
@@ -527,7 +530,7 @@ jobs:
notCallable, err := model.ReadWorkflow(strings.NewReader("on: push\njobs:\n test: {}\n")) notCallable, err := model.ReadWorkflow(strings.NewReader("on: push\njobs:\n test: {}\n"))
require.NoError(t, err) require.NoError(t, err)
config := &Config{ config := &Config{
Secrets: map[string]string{"token": "${{ github.token }}"}, Secrets: map[string]string{"token": "${{ github.token }}", "GITHUB_TOKEN": "github-token", "GITEA_TOKEN": "gitea-token"},
PresetGitHubContext: &model.GithubContext{}, PresetGitHubContext: &model.GithubContext{},
RuntimeTokens: map[*model.Workflow]string{callerWorkflow: "caller-run"}, RuntimeTokens: map[*model.Workflow]string{callerWorkflow: "caller-run"},
} }
@@ -541,11 +544,17 @@ jobs:
called := resolve("call", callee) called := resolve("call", callee)
assert.Equal(t, map[string]any{"flag": true}, called.workflowCallInputs) assert.Equal(t, map[string]any{"flag": true}, called.workflowCallInputs)
assert.Equal(t, map[string]string{"undeclared": "${{ github.token }}"}, called.workflowCallSecrets) assert.Equal(t, map[string]string{"undeclared": "${{ github.token }}", "GITHUB_TOKEN": "github-token", "GITEA_TOKEN": "gitea-token"}, called.workflowCallSecrets)
assert.Equal(t, "caller-run", called.GetEnv()["ACTIONS_RUNTIME_TOKEN"]) assert.Equal(t, "caller-run", called.GetEnv()["ACTIONS_RUNTIME_TOKEN"])
inherited := resolve("inherit", notCallable) inherited := resolve("inherit", notCallable)
assert.Empty(t, inherited.workflowCallInputs) assert.Empty(t, inherited.workflowCallInputs)
assert.Equal(t, map[string]string{"token": "${{ github.token }}", "ACTIONS_RUNTIME_TOKEN": "caller-run"}, inherited.workflowCallSecrets) assert.Equal(t, map[string]string{"token": "${{ github.token }}", "GITHUB_TOKEN": "github-token", "GITEA_TOKEN": "gitea-token", "ACTIONS_RUNTIME_TOKEN": "caller-run"}, inherited.workflowCallSecrets)
assert.Equal(t, "${{ github.token }}:", resolve("override", callee).ExprEval.InterpolateName(t.Context(), "${{ secrets.GITHUB_TOKEN }}:${{ secrets.GITEA_TOKEN }}"))
middle, err := model.ReadWorkflow(strings.NewReader("on: workflow_call\njobs:\n test: {uses: ./reuse.yml}\n"))
require.NoError(t, err)
nested, err := (&runnerImpl{config: config, caller: &caller{runContext: resolve("override", middle)}}).newRunContext(t.Context(), &model.Run{Workflow: callee, JobID: "test"}, nil)
require.NoError(t, err)
assert.Equal(t, "github-token:gitea-token", nested.ExprEval.InterpolateName(t.Context(), "${{ secrets.GITHUB_TOKEN }}:${{ secrets.GITEA_TOKEN }}"))
} }
func TestJobNameMasksSecrets(t *testing.T) { func TestJobNameMasksSecrets(t *testing.T) {
+10 -8
View File
@@ -44,7 +44,11 @@ func (rc *RunContext) runJobHook(ctx context.Context, hookPath, name string) err
rc.CurrentStep = "" rc.CurrentStep = ""
defer func() { rc.CurrentStep = previousStep }() defer func() { rc.CurrentStep = previousStep }()
cmd, shell := hookCommand(hookPath) env := rc.mergeEnv(rc.defaultEnv, rc.containerSpec.Env, rc.GetEnv())
rc.withGithubEnv(ctx, rc.getGithubContext(ctx), env)
rc.ApplyExtraPath(ctx, &env)
cmd, shell := hookCommand(hookPath, func(preferred, fallback string) string { return rc.shellOrFallback(ctx, env, preferred, fallback) })
rawLogger := common.Logger(ctx).With(rawOutputField, true) rawLogger := common.Logger(ctx).With(rawOutputField, true)
rawLogger.Infof("A %s hook has been configured by the runner administrator", name) rawLogger.Infof("A %s hook has been configured by the runner administrator", name)
rawLogger.Infof("::group::Run '%s'", EscapeCommandData(hookPath)) rawLogger.Infof("::group::Run '%s'", EscapeCommandData(hookPath))
@@ -53,10 +57,6 @@ func (rc *RunContext) runJobHook(ctx context.Context, hookPath, name string) err
} }
rawLogger.Infof("::endgroup::") rawLogger.Infof("::endgroup::")
env := rc.mergeEnv(rc.caEnv, rc.containerSpec.Env, rc.GetEnv())
rc.withGithubEnv(ctx, rc.getGithubContext(ctx), env)
rc.ApplyExtraPath(ctx, &env)
err := rc.setupHookFileCommands(ctx, env) err := rc.setupHookFileCommands(ctx, env)
if err == nil { if err == nil {
err = rc.runWithCommands(ctx, env, func(ctx context.Context) error { err = rc.runWithCommands(ctx, env, func(ctx context.Context) error {
@@ -104,12 +104,14 @@ func (rc *RunContext) processHookFileCommands(ctx context.Context) error {
// gives `run:` steps — a hook sets its own. See docs/adrs/1751-runner-job-hooks.md there. // gives `run:` steps — a hook sets its own. See docs/adrs/1751-runner-job-hooks.md there.
// The second return value is how the invocation is shown in the log, empty when the file is // The second return value is how the invocation is shown in the log, empty when the file is
// executed directly. // executed directly.
func hookCommand(hookPath string) (cmd []string, shell string) { func hookCommand(hookPath string, shellOrFallback func(preferred, fallback string) string) (cmd []string, shell string) {
switch strings.ToLower(path.Ext(hookPath)) { switch strings.ToLower(path.Ext(hookPath)) {
case ".sh": case ".sh":
return []string{"bash", "-e", hookPath}, "bash -e {0}" shellName := shellOrFallback("bash", "sh")
return []string{shellName, "-e", hookPath}, shellName + " -e {0}"
case ".ps1": case ".ps1":
return []string{"pwsh", "-command", ". '" + expressionStringEscape(hookPath) + "'"}, `pwsh -command ". '{0}'"` shellName := shellOrFallback("pwsh", "powershell")
return []string{shellName, "-command", ". '" + expressionStringEscape(hookPath) + "'"}, shellName + ` -command ". '{0}'"`
default: default:
return []string{hookPath}, "" return []string{hookPath}, ""
} }
+20 -16
View File
@@ -71,11 +71,6 @@ func (c *hookContainer) UpdateFromEnv(_ string, env *map[string]string) common.E
} }
} }
func (c *hookContainer) ExecOutput(_ctx context.Context, command []string) (io.ReadCloser, error) {
c.cmd = command
return io.NopCloser(bytes.NewBufferString(c.output)), c.err
}
func (c *hookContainer) GetContainerArchive(context.Context, string) (io.ReadCloser, error) { func (c *hookContainer) GetContainerArchive(context.Context, string) (io.ReadCloser, error) {
return io.NopCloser(bytes.NewReader(c.pathTar)), nil return io.NopCloser(bytes.NewReader(c.pathTar)), nil
} }
@@ -230,7 +225,7 @@ func TestRunJobHook(t *testing.T) {
rc, ctx := newHookRunContext(jobContainer, &Config{JobStartedHook: "/hooks/started.sh"}) rc, ctx := newHookRunContext(jobContainer, &Config{JobStartedHook: "/hooks/started.sh"})
require.ErrorContains(t, rc.runJobStartedHook(ctx), "boom") require.ErrorContains(t, rc.runJobStartedHook(ctx), "boom")
assert.Nil(t, jobContainer.cmd) assert.NotContains(t, jobContainer.cmd, "/hooks/started.sh")
assert.NotContains(t, rc.Env, "STALE") assert.NotContains(t, rc.Env, "STALE")
}) })
@@ -247,17 +242,26 @@ func TestRunJobHook(t *testing.T) {
// actions/runner deliberately runs a hook without the flags it gives `run:` steps, and an // actions/runner deliberately runs a hook without the flags it gives `run:` steps, and an
// executable without a known extension speaks for itself through its shebang. // executable without a known extension speaks for itself through its shebang.
func TestHookCommand(t *testing.T) { func TestHookCommand(t *testing.T) {
for hookPath, want := range map[string]struct { for _, tc := range []struct {
cmd []string hookPath string
shell string hasShell bool
cmd []string
shell string
}{ }{
"/hooks/started.sh": {[]string{"bash", "-e", "/hooks/started.sh"}, "bash -e {0}"}, {"/hooks/started.sh", true, []string{"bash", "-e", "/hooks/started.sh"}, "bash -e {0}"},
"/hooks/started.PS1": {[]string{"pwsh", "-command", ". '/hooks/started.PS1'"}, `pwsh -command ". '{0}'"`}, {"/hooks/started.sh", false, []string{"sh", "-e", "/hooks/started.sh"}, "sh -e {0}"},
"/hooks/started": {[]string{"/hooks/started"}, ""}, {"/hooks/started.PS1", true, []string{"pwsh", "-command", ". '/hooks/started.PS1'"}, `pwsh -command ". '{0}'"`},
`C:\hooks\O'Brien\started.ps1`: {[]string{"pwsh", "-command", `. 'C:\hooks\O''Brien\started.ps1'`}, `pwsh -command ". '{0}'"`}, {"/hooks/started.PS1", false, []string{"powershell", "-command", ". '/hooks/started.PS1'"}, `powershell -command ". '{0}'"`},
{"/hooks/started", false, []string{"/hooks/started"}, ""},
{`C:\hooks\O'Brien\started.ps1`, true, []string{"pwsh", "-command", `. 'C:\hooks\O''Brien\started.ps1'`}, `pwsh -command ". '{0}'"`},
} { } {
cmd, shell := hookCommand(hookPath) cmd, shell := hookCommand(tc.hookPath, func(preferred, fallback string) string {
assert.Equal(t, want.cmd, cmd, hookPath) if tc.hasShell {
assert.Equal(t, want.shell, shell, hookPath) return preferred
}
return fallback
})
assert.Equal(t, tc.cmd, cmd, tc.hookPath)
assert.Equal(t, tc.shell, shell, tc.hookPath)
} }
} }
+10 -5
View File
@@ -165,6 +165,10 @@ func setReusedWorkflowCallerResult(rc *RunContext, runner *runnerImpl) common.Ex
hasFailure = true hasFailure = true
} }
} }
if err := setJobOutputs(ctx, rc); err != nil { // a bad output fails this job, as for ordinary jobs
common.Logger(ctx).Errorf("##[error]%s", EscapeCommandData(err.Error()))
hasFailure = true
}
reusedWorkflowJobResult := "success" reusedWorkflowJobResult := "success"
reusedWorkflowJobResultMessage := "succeeded" reusedWorkflowJobResultMessage := "succeeded"
@@ -173,14 +177,15 @@ func setReusedWorkflowCallerResult(rc *RunContext, runner *runnerImpl) common.Ex
reusedWorkflowJobResultMessage = "failed" reusedWorkflowJobResultMessage = "failed"
} }
// Serialize this shared Job.Result write against the other matrix combos
// and setJobResult (same lockJob key).
unlock := lockJob(rc.Run.Job())
rc.result(reusedWorkflowJobResult)
unlock()
if rc.caller != nil { if rc.caller != nil {
rc.caller.setReusedWorkflowJobResult(rc.Run.JobID, reusedWorkflowJobResult) rc.caller.setReusedWorkflowJobResult(rc.Run.JobID, reusedWorkflowJobResult)
} else { } else {
// Serialize this shared Job.Result write against the other matrix combos
// and setJobResult (same lockJob key).
unlock := lockJob(rc.Run.Job())
rc.result(reusedWorkflowJobResult)
unlock()
common.Logger(ctx).With("jobResult", reusedWorkflowJobResult).Infof("Job %s", reusedWorkflowJobResultMessage) common.Logger(ctx).With("jobResult", reusedWorkflowJobResult).Infof("Job %s", reusedWorkflowJobResultMessage)
} }
+15
View File
@@ -13,6 +13,7 @@ import (
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"strings"
"sync" "sync"
"testing" "testing"
"time" "time"
@@ -213,3 +214,17 @@ func gitMust(t *testing.T, dir string, args ...string) {
out, err := cmd.CombinedOutput() out, err := cmd.CombinedOutput()
require.NoError(t, err, "git %v: %s", args, string(out)) require.NoError(t, err, "git %v: %s", args, string(out))
} }
func TestSetReusedWorkflowCallerResultFailsOnBadOutputs(t *testing.T) {
callerWorkflow, err := model.ReadWorkflow(strings.NewReader("jobs:\n call: {uses: ./reuse.yml}\n"))
require.NoError(t, err)
called, err := model.ReadWorkflow(strings.NewReader("on:\n workflow_call:\n outputs:\n bad: {value: \"${{ fromJSON('not-json') }}\"}\njobs:\n inner: {uses: ./leaf.yml}\n"))
require.NoError(t, err)
config := &Config{PresetGitHubContext: &model.GithubContext{}}
parent, err := (&runnerImpl{config: config}).newRunContext(t.Context(), &model.Run{Workflow: callerWorkflow, JobID: "call"}, nil)
require.NoError(t, err)
rc, err := (&runnerImpl{config: config, caller: &caller{runContext: parent, reusedWorkflowJobResults: map[string]string{}}}).newRunContext(t.Context(), &model.Run{Workflow: called, JobID: "inner"}, nil)
require.NoError(t, err)
require.NoError(t, setReusedWorkflowCallerResult(rc, &runnerImpl{caller: &caller{reusedWorkflowJobResults: map[string]string{"leaf": "success"}}})(t.Context()))
require.Equal(t, "failure", rc.Run.Job().Result)
}
+117 -35
View File
@@ -30,6 +30,7 @@ import (
"time" "time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/container" "gitea.com/gitea/runner/act/container"
"gitea.com/gitea/runner/act/ghcontext" "gitea.com/gitea/runner/act/ghcontext"
"gitea.com/gitea/runner/internal/pkg/lock" "gitea.com/gitea/runner/internal/pkg/lock"
@@ -68,13 +69,14 @@ type RunContext struct {
kubernetes bool kubernetes bool
containerSpec model.ContainerSpec // container:, resolved once with platformImage containerSpec model.ContainerSpec // container:, resolved once with platformImage
containerEnv yaml.Node // container.env of the kept containerSpec, decoded once the job env resolves containerEnv yaml.Node // container.env of the kept containerSpec, decoded once the job env resolves
caEnv map[string]string // NODE_EXTRA_CA_CERTS for the runner CA, beneath any env the job and its image set defaultEnv map[string]string // ImageOS and the runner CA's NODE_EXTRA_CA_CERTS, beneath any env the job sets
JobName string JobName string
ActionPath string ActionPath string
Parent *RunContext Parent *RunContext
actionInputs map[string]any // inputs of the composite action this runs, nil for a job actionInputs map[string]any // inputs of the composite action this runs, nil for a job
Masks []string Masks []string
cleanUpJobContainer common.Executor cleanUpJobContainer common.Executor
taskWorkspace string
deferVolumeCleanup func(common.Executor) deferVolumeCleanup func(common.Executor)
caller *caller // job calling this RunContext (reusable workflows) caller *caller // job calling this RunContext (reusable workflows)
workflowCallInputs map[string]any // the caller's with:, resolved once by resolveWorkflowCall workflowCallInputs map[string]any // the caller's with:, resolved once by resolveWorkflowCall
@@ -101,6 +103,7 @@ type RunContext struct {
// getJobContext report the "cancelled" status so cancelled()/always() evaluate the way // getJobContext report the "cancelled" status so cancelled()/always() evaluate the way
// GitHub Actions does, letting cleanup and always() steps run while normal steps skip. // GitHub Actions does, letting cleanup and always() steps run while normal steps skip.
jobCancelled bool jobCancelled bool
jobTimedOut bool
// jobFailed records failures outside normal main-step results, such as action pre-step // jobFailed records failures outside normal main-step results, such as action pre-step
// failures. Those failures must still make success() false and failure() true for later // failures. Those failures must still make success() false and failure() true for later
// main-step if evaluation. // main-step if evaluation.
@@ -158,12 +161,13 @@ func (rc *RunContext) markFailed() {
// markInterrupted records the job's interruption status from a context error so later step `if` evaluations and the job result observe it, // markInterrupted records the job's interruption status from a context error so later step `if` evaluations and the job result observe it,
// keeping the timeout path symmetric with the cancel path: // keeping the timeout path symmetric with the cancel path:
// - context.Canceled (server cancel) marks the job cancelled, matching GitHub's "only always()/cancelled() run on cancel". // - context.Canceled (server cancel) marks the job cancelled, matching GitHub's "only always()/cancelled() run on cancel".
// - context.DeadlineExceeded (job timeout-minutes) marks the job failed, matching the "Timeout -> FAILURE" reporting semantics. // - context.DeadlineExceeded (job timeout-minutes) evaluates later steps as cancelled like GitHub, but reports the job failed.
func (rc *RunContext) markInterrupted(err error) { func (rc *RunContext) markInterrupted(err error) {
switch { switch {
case errors.Is(err, context.Canceled): case errors.Is(err, context.Canceled):
rc.markCancelled() rc.markCancelled()
case errors.Is(err, context.DeadlineExceeded): case errors.Is(err, context.DeadlineExceeded):
rc.jobTimedOut = true
rc.markFailed() rc.markFailed()
} }
} }
@@ -271,7 +275,7 @@ func (rc *RunContext) validVolumes() []string {
volumes = append(volumes, hostEnv.Path, hostEnv.GetActPath()) volumes = append(volumes, hostEnv.Path, hostEnv.GetActPath())
} }
if rc.Config.SharedToolCache { if rc.Config.SharedToolCache {
volumes = append(volumes, sharedToolCacheVolume) volumes = append(volumes, rc.toolCacheVolume())
} }
if rc.Config.BindWorkdir { if rc.Config.BindWorkdir {
volumes = append(volumes, rc.Config.Workdir, rc.workdirMountRoot()) volumes = append(volumes, rc.Config.Workdir, rc.workdirMountRoot())
@@ -331,6 +335,13 @@ func (rc *RunContext) startDockerProxy(ctx context.Context) {
rc.dockerProxy = container.NewDockerProxy(ctx, rc.jobContainerName()) rc.dockerProxy = container.NewDockerProxy(ctx, rc.jobContainerName())
} }
func (rc *RunContext) toolCacheVolume() string {
if rc.Config.ToolCacheSlot == "" {
return sharedToolCacheVolume
}
return sharedToolCacheVolume + "-" + rc.Config.ToolCacheSlot
}
// toolCache returns the tool cache path the job sees, relocatable through RUNNER_TOOL_CACHE. // toolCache returns the tool cache path the job sees, relocatable through RUNNER_TOOL_CACHE.
func (rc *RunContext) toolCache(fallback string) string { func (rc *RunContext) toolCache(fallback string) string {
return cmp.Or(rc.GetEnv()["RUNNER_TOOL_CACHE"], fallback) return cmp.Or(rc.GetEnv()["RUNNER_TOOL_CACHE"], fallback)
@@ -378,8 +389,8 @@ func splitVolumes(specs []string) ([]string, map[string]string, map[string]bool)
return binds, mounts, targets return binds, mounts, targets
} }
// Returns the binds and mounts for the container, resolving paths as appopriate // Returns the container's binds, mounts and bound task workspace, resolving paths as appropriate
func (rc *RunContext) GetBindsAndMounts(ctx context.Context) ([]string, map[string]string) { func (rc *RunContext) GetBindsAndMounts(ctx context.Context) ([]string, map[string]string, string) {
name := rc.jobContainerName() name := rc.jobContainerName()
ext := container.LinuxContainerEnvironmentExtensions{} ext := container.LinuxContainerEnvironmentExtensions{}
@@ -391,7 +402,7 @@ func (rc *RunContext) GetBindsAndMounts(ctx context.Context) ([]string, map[stri
} }
if rc.Config.SharedToolCache { if rc.Config.SharedToolCache {
if toolCache := rc.toolCache(container.DefaultToolCache); !claimed[toolCache] { if toolCache := rc.toolCache(container.DefaultToolCache); !claimed[toolCache] {
mounts[sharedToolCacheVolume] = toolCache mounts[rc.toolCacheVolume()] = toolCache
} }
} }
if hostEnv, ok := rc.JobContainer.(*container.HostEnvironment); ok { if hostEnv, ok := rc.JobContainer.(*container.HostEnvironment); ok {
@@ -399,12 +410,16 @@ func (rc *RunContext) GetBindsAndMounts(ctx context.Context) ([]string, map[stri
} }
mounts[name+"-env"] = ext.GetActPath() // runner-internal, never overridable mounts[name+"-env"] = ext.GetActPath() // runner-internal, never overridable
taskWorkspace := ""
if workdir := ext.ToContainerPath(rc.Config.Workdir); !claimed[workdir] { if workdir := ext.ToContainerPath(rc.Config.Workdir); !claimed[workdir] {
source := rc.workdirMountRoot() source := rc.workdirMountRoot()
target := ext.ToContainerPath(source) target := ext.ToContainerPath(source)
if claimed[target] { if claimed[target] {
source, target = rc.Config.Workdir, workdir source, target = rc.Config.Workdir, workdir
} }
if rc.Config.BindWorkdir && rc.Config.PresetGitHubContext != nil {
taskWorkspace = target
}
if rc.Config.BindWorkdir { if rc.Config.BindWorkdir {
bindModifiers := "" bindModifiers := ""
if runtime.GOOS == "darwin" { if runtime.GOOS == "darwin" {
@@ -419,7 +434,7 @@ func (rc *RunContext) GetBindsAndMounts(ctx context.Context) ([]string, map[stri
} }
} }
return binds, mounts return binds, mounts, taskWorkspace
} }
func (rc *RunContext) startHostEnvironment() common.Executor { func (rc *RunContext) startHostEnvironment() common.Executor {
@@ -447,20 +462,23 @@ func (rc *RunContext) startHostEnvironment() common.Executor {
toolCacheParent = cacheDir toolCacheParent = cacheDir
} }
toolCache := rc.toolCache(filepath.Join(toolCacheParent, "tool_cache")) toolCache := rc.toolCache(filepath.Join(toolCacheParent, "tool_cache"))
if rc.Config.ToolCacheSlot != "" {
toolCache = filepath.Join(toolCache, rc.Config.ToolCacheSlot)
rc.Env["RUNNER_TOOL_CACHE"] = toolCache // overrides a RUNNER_TOOL_CACHE the slots would otherwise share
}
if err := os.MkdirAll(toolCache, 0o777); err != nil { if err := os.MkdirAll(toolCache, 0o777); err != nil {
return err return err
} }
rc.JobContainer = &container.HostEnvironment{ rc.JobContainer = &container.HostEnvironment{
Path: path, Path: path,
TmpDir: runnerTmp, TmpDir: runnerTmp,
ToolCache: toolCache, ToolCache: toolCache,
Workdir: rc.Config.Workdir, Workdir: rc.Config.Workdir,
CleanWorkdir: rc.Config.CleanWorkdir, ActPath: actPath,
ActPath: actPath, Root: miscpath,
Root: miscpath, TrackingID: rc.runnerTrackingID(),
TrackingID: rc.runnerTrackingID(), StdOut: logWriter,
StdOut: logWriter, AllocatePTY: rc.Config.AllocatePTY,
AllocatePTY: rc.Config.AllocatePTY,
} }
rc.cleanUpJobContainer = rc.JobContainer.Remove() rc.cleanUpJobContainer = rc.JobContainer.Remove()
rc.setRunnerContextEnv(ctx) rc.setRunnerContextEnv(ctx)
@@ -638,6 +656,11 @@ func (rc *RunContext) startJobContainer() common.Executor {
rc.serviceContainers = append(rc.serviceContainers, &serviceContainer{name: serviceID, image: input.Image, container: newContainer(input)}) rc.serviceContainers = append(rc.serviceContainers, &serviceContainer{name: serviceID, image: input.Image, container: newContainer(input)})
} }
exposedPorts, portBindings, err := nat.ParsePortSpecs(rc.containerSpec.Ports)
if err != nil {
return fmt.Errorf("failed to parse container ports: %w", err)
}
ext := container.LinuxContainerEnvironmentExtensions{} ext := container.LinuxContainerEnvironmentExtensions{}
containerInput := &container.NewContainerInput{ containerInput := &container.NewContainerInput{
Entrypoint: []string{"sleep", strconv.FormatInt(int64(rc.Config.ContainerMaxLifetime.Round(time.Second).Seconds()), 10)}, Entrypoint: []string{"sleep", strconv.FormatInt(int64(rc.Config.ContainerMaxLifetime.Round(time.Second).Seconds()), 10)},
@@ -649,6 +672,8 @@ func (rc *RunContext) startJobContainer() common.Executor {
Env: envList, Env: envList,
NetworkMode: networkName, NetworkMode: networkName,
NetworkAliases: []string{rc.Name}, NetworkAliases: []string{rc.Name},
ExposedPorts: exposedPorts,
PortBindings: portBindings,
Stdout: logWriter, Stdout: logWriter,
Stderr: logWriter, Stderr: logWriter,
Privileged: rc.Config.Privileged, Privileged: rc.Config.Privileged,
@@ -672,7 +697,8 @@ func (rc *RunContext) startJobContainer() common.Executor {
return err return err
} }
rc.startDockerProxy(ctx) rc.startDockerProxy(ctx)
containerInput.Binds, containerInput.Mounts = rc.GetBindsAndMounts(ctx) var taskWorkspace string
containerInput.Binds, containerInput.Mounts, taskWorkspace = rc.GetBindsAndMounts(ctx)
containerInput.ValidVolumes = rc.validVolumes() containerInput.ValidVolumes = rc.validVolumes()
rc.jobNetworkName = networkName rc.jobNetworkName = networkName
@@ -687,6 +713,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
func(ctx context.Context) error { return rc.createJobVolumes(ctx, containerInput.Mounts) }, func(ctx context.Context) error { return rc.createJobVolumes(ctx, containerInput.Mounts) },
rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop), rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
rc.JobContainer.Start(false), rc.JobContainer.Start(false),
func(context.Context) error { rc.taskWorkspace = taskWorkspace; return nil },
rc.captureJobContainerInfo(), rc.captureJobContainerInfo(),
rc.copyWorkflowFiles(), rc.copyWorkflowFiles(),
rc.waitForServiceContainers(), rc.waitForServiceContainers(),
@@ -801,7 +828,10 @@ func (rc *RunContext) cleanupJobResources(networkName string, createAndDeleteNet
errs := []error{rc.closeDockerProxy(ctx)} errs := []error{rc.closeDockerProxy(ctx)}
var containers []common.Executor var containers []common.Executor
if rc.JobContainer != nil { if rc.JobContainer != nil {
containers = append(containers, rc.JobContainer.Remove()) containers = append(containers, func(ctx context.Context) error {
rc.emptyTaskWorkspace(ctx)
return rc.JobContainer.Remove()(ctx)
})
} }
if len(rc.serviceContainers) > 0 { if len(rc.serviceContainers) > 0 {
logger.Infof("Cleaning up services for job %s", rc.JobName) logger.Infof("Cleaning up services for job %s", rc.JobName)
@@ -831,6 +861,42 @@ func (rc *RunContext) cleanupJobResources(networkName string, createAndDeleteNet
} }
} }
// exits 3 unless workspace $1 is safe to empty per mountinfo $2
const emptyTaskWorkspaceScript = `case "$1" in /?*/?*) ;; *) exit 3 ;; esac
mounted=
while read -r _ _ _ _ point _; do
case "$point" in "$1") mounted=1 ;; "$1"/*) exit 3 ;; esac
done <"$2"
[ "$mounted" ] || exit 3
rm -rf "$1"/* "$1"/.[!.]* "$1"/..?*`
var runnerCanRemoveJobFiles = runtime.GOOS != "linux" || os.Geteuid() == 0 // root, or a desktop VM mapping file ownership, replaced in tests
// runs as root with root-owned PATH dirs only, skipped for called workflows sharing the workspace
func (rc *RunContext) emptyTaskWorkspace(ctx context.Context) {
if rc.taskWorkspace == "" || rc.caller != nil || runnerCanRemoveJobFiles {
return
}
budget := 30 * time.Second
if deadline, ok := ctx.Deadline(); ok {
budget = min(budget, time.Until(deadline)/2-container.DrainGracePeriod)
}
ctx, cancel := context.WithTimeout(ctx, budget)
defer cancel()
stdout, stderr := rc.JobContainer.ReplaceLogWriter(io.Discard, io.Discard)
defer rc.JobContainer.ReplaceLogWriter(stdout, stderr)
err := rc.JobContainer.Exec(
[]string{"sh", "-c", emptyTaskWorkspaceScript, "sh", rc.taskWorkspace, "/proc/self/mountinfo"},
map[string]string{"PATH": "/usr/sbin:/usr/bin:/sbin:/bin"}, "0", "/",
)(common.WithLogger(ctx, logging.New(logging.DiscardHandler)))
switch {
case errors.Is(err, container.ExitCodeError(3)):
logging.Infof("not emptying workspace %s of %s: under two levels deep, not a mountpoint as given, or a mount below it", rc.taskWorkspace, rc.jobContainerName())
case err != nil:
logging.Infof("could not empty workspace %s of %s: %v", rc.taskWorkspace, rc.jobContainerName(), err)
}
}
func runAll(ctx context.Context, executors []common.Executor) error { func runAll(ctx context.Context, executors []common.Executor) error {
errs := make([]error, len(executors)) errs := make([]error, len(executors))
var wg sync.WaitGroup var wg sync.WaitGroup
@@ -1152,6 +1218,24 @@ func (svc *serviceContainer) healthOutputSuffix() string {
func (rc *RunContext) copyWorkflowFiles() common.Executor { func (rc *RunContext) copyWorkflowFiles() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
imageEnv := map[string]string{}
imageEnvErr := rc.JobContainer.UpdateFromImageEnv(&imageEnv)(ctx)
rc.defaultEnv = map[string]string{}
imageOS, set := imageEnv["ImageOS"]
if !set {
if !rc.kubernetes { // a pod reports its release through its startup probe
for _, osReleasePath := range []string{"/usr/lib/os-release", "/etc/os-release"} { // the archive skips symlinks, as /etc's is on Ubuntu
if osRelease, ok := readSingleFileFromContainerArchive(ctx, rc.JobContainer, osReleasePath, 64<<10); ok {
imageOS = container.ImageOSFromOSRelease(string(osRelease))
break
}
}
}
imageOS = cmp.Or(imageOS, rc.imageOS(ctx))
}
if set || imageOS != "" {
rc.defaultEnv["ImageOS"] = imageOS
}
files := []*container.FileEntry{ files := []*container.FileEntry{
{Name: "workflow/event.json", Mode: 0o644, Body: rc.EventJSON}, {Name: "workflow/event.json", Mode: 0o644, Body: rc.EventJSON},
{Name: "workflow/envs.txt", Mode: 0o666}, {Name: "workflow/envs.txt", Mode: 0o666},
@@ -1162,12 +1246,11 @@ func (rc *RunContext) copyWorkflowFiles() common.Executor {
return fmt.Errorf("read runner CA file: %w", err) return fmt.Errorf("read runner CA file: %w", err)
} }
files = append(files, &container.FileEntry{Name: "ca.crt", Mode: 0o644, Body: string(ca)}) files = append(files, &container.FileEntry{Name: "ca.crt", Mode: 0o644, Body: string(ca)})
imageEnv := map[string]string{} if imageEnvErr != nil {
if err = rc.JobContainer.UpdateFromImageEnv(&imageEnv)(ctx); err != nil { return imageEnvErr
return err
} }
if _, set := imageEnv["NODE_EXTRA_CA_CERTS"]; !set { if _, set := imageEnv["NODE_EXTRA_CA_CERTS"]; !set {
rc.caEnv = map[string]string{"NODE_EXTRA_CA_CERTS": rc.jobCAFile()} rc.defaultEnv["NODE_EXTRA_CA_CERTS"] = rc.jobCAFile()
} }
} }
return rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", files...)(ctx) return rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", files...)(ctx)
@@ -1540,7 +1623,7 @@ func (rc *RunContext) getJobContext() *model.JobContext {
// A cancelled run takes precedence over success/failure so cancelled() is true and // A cancelled run takes precedence over success/failure so cancelled() is true and
// success()/failure() are false, matching GitHub Actions: on cancellation only // success()/failure() are false, matching GitHub Actions: on cancellation only
// always() and cancelled() steps run. // always() and cancelled() steps run.
if job.jobCancelled { if job.jobCancelled || job.jobTimedOut {
jobStatus = "cancelled" jobStatus = "cancelled"
} }
@@ -1577,7 +1660,7 @@ func (rc *RunContext) getRunnerContext(ctx context.Context) map[string]any {
runnerContext["name"] = rc.Config.RunnerName runnerContext["name"] = rc.Config.RunnerName
runnerContext["environment"] = "self-hosted" runnerContext["environment"] = "self-hosted"
runnerContext["workspace"] = parentDir(rc.githubWorkspace()) runnerContext["workspace"] = parentDir(rc.githubWorkspace())
if rc.Config.RunnerDebug() { if rc.topLevelRunContext().Config.RunnerDebug() { // a composite's config has no secrets
runnerContext["debug"] = "1" runnerContext["debug"] = "1"
} }
return runnerContext return runnerContext
@@ -1621,7 +1704,9 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
Workspace: rc.githubWorkspace(), Workspace: rc.githubWorkspace(),
} }
if rc.JobContainer != nil { if rc.JobContainer != nil {
ghc.EventPath = rc.JobContainer.GetActPath() + "/workflow/event.json" actPath := rc.JobContainer.GetActPath()
ghc.EventPath = actPath + "/workflow/event.json"
ghc.Env, ghc.Path = container.JoinPath(actPath, envFileCommand), container.JoinPath(actPath, pathFileCommand)
} }
ghc.RunnerTrackingID = rc.runnerTrackingID() ghc.RunnerTrackingID = rc.runnerTrackingID()
@@ -1629,6 +1714,7 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
if preset := rc.Config.PresetGitHubContext; preset != nil { if preset := rc.Config.PresetGitHubContext; preset != nil {
merged := *preset merged := *preset
merged.Workflow, merged.Job, merged.Workspace, merged.EventPath = ghc.Workflow, ghc.Job, ghc.Workspace, ghc.EventPath merged.Workflow, merged.Job, merged.Workspace, merged.EventPath = ghc.Workflow, ghc.Job, ghc.Workspace, ghc.EventPath
merged.Env, merged.Path = ghc.Env, ghc.Path
merged.Action, merged.ActionPath, merged.ActionRepository, merged.ActionRef = ghc.Action, ghc.ActionPath, ghc.ActionRepository, ghc.ActionRef merged.Action, merged.ActionPath, merged.ActionRepository, merged.ActionRef = ghc.Action, ghc.ActionPath, ghc.ActionRepository, ghc.ActionRef
merged.RunnerPerflog, merged.RunnerTrackingID = ghc.RunnerPerflog, ghc.RunnerTrackingID merged.RunnerPerflog, merged.RunnerTrackingID = ghc.RunnerPerflog, ghc.RunnerTrackingID
*ghc = merged *ghc = merged
@@ -1722,7 +1808,7 @@ func aliasGiteaEnv(env map[string]string) {
} }
} }
func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubContext, env map[string]string) { func (rc *RunContext) withGithubEnv(_ctx context.Context, github *model.GithubContext, env map[string]string) {
if _, ok := env["CI"]; !ok { if _, ok := env["CI"]; !ok {
env["CI"] = "true" env["CI"] = "true"
} }
@@ -1783,7 +1869,7 @@ func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubCon
if workspace := parentDir(github.Workspace); workspace != "" { if workspace := parentDir(github.Workspace); workspace != "" {
env["RUNNER_WORKSPACE"] = workspace env["RUNNER_WORKSPACE"] = workspace
} }
if rc.Config.RunnerDebug() { if rc.topLevelRunContext().Config.RunnerDebug() {
env["RUNNER_DEBUG"] = "1" env["RUNNER_DEBUG"] = "1"
} }
@@ -1791,10 +1877,6 @@ func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubCon
env["ACTIONS_RUNTIME_URL"] = cmp.Or(os.Getenv("ACTIONS_RUNTIME_URL"), "http://"+net.JoinHostPort(rc.Config.ArtifactServerAddr, rc.Config.ArtifactServerPort)+"/") env["ACTIONS_RUNTIME_URL"] = cmp.Or(os.Getenv("ACTIONS_RUNTIME_URL"), "http://"+net.JoinHostPort(rc.Config.ArtifactServerAddr, rc.Config.ArtifactServerPort)+"/")
env["ACTIONS_RUNTIME_TOKEN"] = cmp.Or(rc.runtimeToken, os.Getenv("ACTIONS_RUNTIME_TOKEN"), "token") env["ACTIONS_RUNTIME_TOKEN"] = cmp.Or(rc.runtimeToken, os.Getenv("ACTIONS_RUNTIME_TOKEN"), "token")
} }
if imageOS := rc.imageOS(ctx); imageOS != "" {
env["ImageOS"] = imageOS
}
} }
// parentDir returns the directory containing p, or "" when p names no parent. Both // parentDir returns the directory containing p, or "" when p names no parent. Both
@@ -1811,7 +1893,7 @@ func parentDir(p string) string {
// from another. The resolved image tag is preferred over the runs-on label because it // from another. The resolved image tag is preferred over the runs-on label because it
// still names a release when the label is a rolling one such as ubuntu-latest. // still names a release when the label is a rolling one such as ubuntu-latest.
func (rc *RunContext) imageOS(ctx context.Context) string { func (rc *RunContext) imageOS(ctx context.Context) string {
if rc.Run.Job().RunsOn() == nil { if job := rc.Run.Job(); job == nil || job.RunsOn() == nil {
return "" // a composite action runs on a synthetic job without runs-on return "" // a composite action runs on a synthetic job without runs-on
} }
if imageOS := imageOSFromImage(rc.platformImage); imageOS != "" { if imageOS := imageOSFromImage(rc.platformImage); imageOS != "" {
@@ -1820,8 +1902,8 @@ func (rc *RunContext) imageOS(ctx context.Context) string {
for _, platformName := range slices.Backward(rc.runsOnPlatformNames(ctx)) { for _, platformName := range slices.Backward(rc.runsOnPlatformNames(ctx)) {
if platformName == "ubuntu-latest" { if platformName == "ubuntu-latest" {
// Rolling label whose image names no release either, so keep the historical value. // Rolling label whose image names no release either, so report GitHub's current ubuntu-latest.
return "ubuntu20" return "ubuntu24"
} else if platformName != "" { } else if platformName != "" {
return strings.SplitN(strings.Replace(platformName, `-`, ``, 1), `.`, 2)[0] return strings.SplitN(strings.Replace(platformName, `-`, ``, 1), `.`, 2)[0]
} }
+211 -17
View File
@@ -16,6 +16,7 @@ import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
"os/exec"
"path/filepath" "path/filepath"
"runtime" "runtime"
"slices" "slices"
@@ -167,6 +168,14 @@ func (fakeContainer) Copy(string, ...*container.FileEntry) common.Executor {
return func(context.Context) error { return nil } return func(context.Context) error { return nil }
} }
func (fakeContainer) UpdateFromImageEnv(_env *map[string]string) common.Executor {
return noopExecutor
}
func (fakeContainer) GetContainerArchive(_ctx context.Context, _srcPath string) (io.ReadCloser, error) {
return nil, os.ErrNotExist
}
func (fakeContainer) Inspect(context.Context) (*container.Info, error) { func (fakeContainer) Inspect(context.Context) (*container.Info, error) {
return &container.Info{ID: "fake", State: "running", Health: container.HealthNone}, nil return &container.Info{ID: "fake", State: "running", Health: container.HealthNone}, nil
} }
@@ -341,6 +350,7 @@ jobs:
container: container:
image: node:20 image: node:20
credentials: {username: "${{ env.REG_USER }}", password: pass} credentials: {username: "${{ env.REG_USER }}", password: pass}
ports: ["8080:80"]
services: services:
redis: redis:
image: redis:latest image: redis:latest
@@ -350,6 +360,7 @@ jobs:
`, &Config{Vars: map[string]string{"REG_USER": "user", "TEXT": "${{ 'literal' }}"}}) `, &Config{Vars: map[string]string{"REG_USER": "user", "TEXT": "${{ 'literal' }}"}})
redis, job = inputs[0], inputs[1] redis, job = inputs[0], inputs[1]
require.Equal(t, "user", job.Username) require.Equal(t, "user", job.Username)
require.Equal(t, "8080", job.PortBindings["80/tcp"][0].HostPort)
require.Equal(t, "user", redis.Username) require.Equal(t, "user", redis.Username)
require.Equal(t, []string{"TEXT=${{ 'literal' }}"}, redis.Env) require.Equal(t, []string{"TEXT=${{ 'literal' }}"}, redis.Env)
require.Equal(t, "--volumes-from "+job.Name, redis.WorkflowOptions) require.Equal(t, "--volumes-from "+job.Name, redis.WorkflowOptions)
@@ -455,7 +466,7 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
t.Run(testcase.name+testBindSuffix, func(t *testing.T) { t.Run(testcase.name+testBindSuffix, func(t *testing.T) {
rctemplate.Config.Workdir = testcase.name rctemplate.Config.Workdir = testcase.name
rctemplate.Config.BindWorkdir = bindWorkDir rctemplate.Config.BindWorkdir = bindWorkDir
gotbind, gotmount := rctemplate.GetBindsAndMounts(t.Context()) gotbind, gotmount, _ := rctemplate.GetBindsAndMounts(t.Context())
if bindWorkDir { if bindWorkDir {
fullBind := testcase.name + ":" + testcase.want fullBind := testcase.name + ":" + testcase.want
@@ -507,7 +518,7 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
rc.ExprEval = rc.NewExpressionEvaluator(t.Context()) rc.ExprEval = rc.NewExpressionEvaluator(t.Context())
require.NoError(t, rc.resolvePlatformImage(t.Context())) require.NoError(t, rc.resolvePlatformImage(t.Context()))
gotbind, gotmount := rc.GetBindsAndMounts(t.Context()) gotbind, gotmount, _ := rc.GetBindsAndMounts(t.Context())
assert.Contains(t, gotbind, "/host/mame/roms:/root/.mame/roms:ro") assert.Contains(t, gotbind, "/host/mame/roms:/root/.mame/roms:ro")
assert.NotContains(t, gotbind, "${{ secrets.MAME }}") assert.NotContains(t, gotbind, "${{ secrets.MAME }}")
assert.NotContains(t, gotmount, "${{ secrets.MAME }}") assert.NotContains(t, gotmount, "${{ secrets.MAME }}")
@@ -522,7 +533,7 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
rc.containerSpec.Volumes = testcase.volumes rc.containerSpec.Volumes = testcase.volumes
rc.Run.JobID = "job1" rc.Run.JobID = "job1"
gotbind, gotmount := rc.GetBindsAndMounts(t.Context()) gotbind, gotmount, _ := rc.GetBindsAndMounts(t.Context())
if len(testcase.wantbind) > 0 { if len(testcase.wantbind) > 0 {
assert.Contains(t, gotbind, testcase.wantbind) assert.Contains(t, gotbind, testcase.wantbind)
@@ -551,13 +562,19 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
t.Run("ToolCacheMount", func(t *testing.T) { t.Run("ToolCacheMount", func(t *testing.T) {
rc := newRunContext(&Config{}) rc := newRunContext(&Config{})
_, gotmount := rc.GetBindsAndMounts(t.Context()) _, gotmount, _ := rc.GetBindsAndMounts(t.Context())
assert.NotContains(t, gotmount, sharedToolCacheVolume) assert.NotContains(t, gotmount, sharedToolCacheVolume)
rc.Config.SharedToolCache = true rc.Config.SharedToolCache = true
rc.containerSpec.Volumes = []string{"/unapproved:" + container.DefaultToolCache} rc.containerSpec.Volumes = []string{"/unapproved:" + container.DefaultToolCache}
_, gotmount = rc.GetBindsAndMounts(t.Context()) _, gotmount, _ = rc.GetBindsAndMounts(t.Context())
assert.Equal(t, container.DefaultToolCache, gotmount[sharedToolCacheVolume]) assert.Equal(t, container.DefaultToolCache, gotmount[sharedToolCacheVolume])
rc.Config.ToolCacheSlot = "1"
_, gotmount, _ = rc.GetBindsAndMounts(t.Context())
assert.Equal(t, container.DefaultToolCache, gotmount[sharedToolCacheVolume+"-1"])
assert.NotContains(t, gotmount, sharedToolCacheVolume)
assert.Contains(t, rc.validVolumes(), sharedToolCacheVolume+"-1")
}) })
t.Run("DaemonMountsAboveOwnerRepo", func(t *testing.T) { t.Run("DaemonMountsAboveOwnerRepo", func(t *testing.T) {
@@ -569,16 +586,26 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
Config: &Config{BindWorkdir: true, Workdir: "/workspace/1/owner/repo", PresetGitHubContext: &model.GithubContext{}, ValidVolumes: []string{"claimed"}}, Config: &Config{BindWorkdir: true, Workdir: "/workspace/1/owner/repo", PresetGitHubContext: &model.GithubContext{}, ValidVolumes: []string{"claimed"}},
} }
gotbind, _ := rc.GetBindsAndMounts(t.Context()) gotbind, _, workspace := rc.GetBindsAndMounts(t.Context())
assert.True(t, slices.ContainsFunc(gotbind, func(bind string) bool { return strings.HasPrefix(bind, "/workspace/1:/workspace/1") }), gotbind) assert.True(t, slices.ContainsFunc(gotbind, func(bind string) bool { return strings.HasPrefix(bind, "/workspace/1:/workspace/1") }), gotbind)
assert.Equal(t, "/workspace/1", workspace)
rc.Config.BindWorkdir = false rc.Config.BindWorkdir = false
_, gotmount := rc.GetBindsAndMounts(t.Context()) _, gotmount, workspace := rc.GetBindsAndMounts(t.Context())
assert.Equal(t, "/workspace/1", gotmount[rc.jobContainerName()]) assert.Equal(t, "/workspace/1", gotmount[rc.jobContainerName()])
assert.Empty(t, workspace)
rc.containerSpec.Volumes = []string{"claimed:/workspace/1"} rc.containerSpec.Volumes = []string{"claimed:/workspace/1"}
_, gotmount = rc.GetBindsAndMounts(t.Context()) _, gotmount, _ = rc.GetBindsAndMounts(t.Context())
assert.Equal(t, "/workspace/1/owner/repo", gotmount[rc.jobContainerName()]) assert.Equal(t, "/workspace/1/owner/repo", gotmount[rc.jobContainerName()])
rc.Config.BindWorkdir = true
_, _, workspace = rc.GetBindsAndMounts(t.Context())
assert.Equal(t, "/workspace/1/owner/repo", workspace)
rc.containerSpec.Volumes = nil
rc.Config.PresetGitHubContext = nil
_, _, workspace = rc.GetBindsAndMounts(t.Context())
assert.Empty(t, workspace)
}) })
} }
@@ -611,7 +638,7 @@ func TestRunContextValidVolumes(t *testing.T) {
rc.JobContainer = hostEnv rc.JobContainer = hostEnv
rc.Config.BindWorkdir = false rc.Config.BindWorkdir = false
rc.Config.SharedToolCache = true rc.Config.SharedToolCache = true
binds, mounts := rc.GetBindsAndMounts(t.Context()) binds, mounts, _ := rc.GetBindsAndMounts(t.Context())
assert.Subset(t, binds, []string{hostEnv.Path + ":" + hostEnv.Path, hostEnv.GetActPath() + ":" + hostEnv.GetActPath()}) assert.Subset(t, binds, []string{hostEnv.Path + ":" + hostEnv.Path, hostEnv.GetActPath() + ":" + hostEnv.GetActPath()})
assert.Contains(t, mounts, sharedToolCacheVolume) assert.Contains(t, mounts, sharedToolCacheVolume)
assert.Equal(t, binds, container.SanitizeBinds(t.Context(), rc.validVolumes(), binds)) assert.Equal(t, binds, container.SanitizeBinds(t.Context(), rc.validVolumes(), binds))
@@ -683,6 +710,110 @@ func TestCleanupJobVolumesReapsAbandonedDeferredCleanup(t *testing.T) {
assert.ElementsMatch(t, []string{"became-active", "remove-failed", "foreign", "fresh", "unknown-age"}, slices.Collect(maps.Keys(volumes))) assert.ElementsMatch(t, []string{"became-active", "remove-failed", "foreign", "fresh", "unknown-age"}, slices.Collect(maps.Keys(volumes)))
} }
func TestCleanupJobResourcesEmptiesTaskWorkspace(t *testing.T) {
defer func(previous bool) { runnerCanRemoveJobFiles = previous }(runnerCanRemoveJobFiles)
for _, tt := range []struct {
name, workspace string
caller *caller
runnerCanRemove bool
execErr error
log string
}{
{name: "refusal is logged", workspace: "/workspace/1", execErr: container.ExitCodeError(3), log: "not emptying workspace /workspace/1"},
{name: "failure does not fail the job", workspace: "/workspace/1", execErr: errors.New("no rm"), log: "could not empty workspace /workspace/1"},
{name: "no runner bind"},
{name: "called workflow shares it", workspace: "/workspace/1", caller: &caller{}},
{name: "runner removes it itself", workspace: "/workspace/1", runnerCanRemove: true},
} {
t.Run(tt.name, func(t *testing.T) {
runnerCanRemoveJobFiles = tt.runnerCanRemove
processLog := logtest.SetDefault(t)
var jobLog bytes.Buffer
var calls []string
job := &containerMock{}
if tt.log != "" {
job.On("ReplaceLogWriter", io.Discard, io.Discard).Return(&jobLog, &jobLog).Once()
job.On("ReplaceLogWriter", &jobLog, &jobLog).Return(io.Discard, io.Discard).Once()
job.On("Exec", mock.MatchedBy(func(command []string) bool { return command[4] == tt.workspace }), map[string]string{"PATH": "/usr/sbin:/usr/bin:/sbin:/bin"}, "0", "/").Return(func(ctx context.Context) error {
calls = append(calls, "exec")
deadline, ok := ctx.Deadline()
assert.True(t, ok)
assert.LessOrEqual(t, time.Until(deadline), 10*time.Second-container.DrainGracePeriod)
common.Logger(ctx).Infof("workspace script output")
return tt.execErr
})
}
job.On("Remove").Return(func(context.Context) error {
calls = append(calls, "remove")
return nil
})
rc := &RunContext{
Config: &Config{},
Run: &model.Run{Workflow: &model.Workflow{Name: "workflow"}, JobID: "job"},
JobContainer: job,
taskWorkspace: tt.workspace,
caller: tt.caller,
deferVolumeCleanup: func(common.Executor) {},
}
ctx, cancel := context.WithTimeout(t.Context(), 20*time.Second)
defer cancel()
ctx = common.WithLogger(ctx, logging.New(logging.NewLevelHandler(newJobLogHandler(&jobLog, cyan), logging.LevelInfo)))
require.NoError(t, rc.cleanupJobResources("", false, false)(ctx))
job.AssertExpectations(t)
assert.NotContains(t, jobLog.String(), "workspace")
if tt.log == "" {
assert.Equal(t, []string{"remove"}, calls)
return
}
assert.Equal(t, []string{"exec", "remove"}, calls)
assert.True(t, slices.ContainsFunc(processLog.AllRecords(), func(record logging.Record) bool {
return strings.HasPrefix(record.Message, tt.log)
}), processLog.AllRecords())
})
}
}
func TestEmptyTaskWorkspaceScript(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("the script runs in the linux job container")
}
run := func(t *testing.T, workspace string, mountpoints ...string) error {
t.Helper()
var mountinfo strings.Builder
for _, mountpoint := range mountpoints {
fmt.Fprintf(&mountinfo, "36 35 98:0 /src %s rw - ext4 /dev/sda1 rw\n", mountpoint)
}
info := filepath.Join(t.TempDir(), "mountinfo")
require.NoError(t, os.WriteFile(info, []byte(mountinfo.String()), 0o600))
return exec.Command("sh", "-c", emptyTaskWorkspaceScript, "sh", workspace, info).Run()
}
workspace := t.TempDir()
for _, file := range []string{"owner/repo/file", ".hidden", "..odd"} {
require.NoError(t, os.MkdirAll(filepath.Dir(filepath.Join(workspace, file)), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(workspace, file), nil, 0o600))
}
require.NoError(t, run(t, workspace, workspace))
entries, err := os.ReadDir(workspace)
require.NoError(t, err)
assert.Empty(t, entries)
for _, tt := range []struct {
name, workspace string
mountpoints []string
}{
{"a mount below it", "/nonexistent/1", []string{"/nonexistent/1", "/nonexistent/1/owner/repo/cache"}},
{"a workspace that is not a mountpoint", "/nonexistent/1", []string{"/nonexistent/2"}},
{"a top-level directory", "/nonexistent", []string{"/nonexistent"}},
} {
t.Run("refuses "+tt.name, func(t *testing.T) {
var exitErr *exec.ExitError
require.ErrorAs(t, run(t, tt.workspace, tt.mountpoints...), &exitErr)
assert.Equal(t, 3, exitErr.ExitCode())
})
}
}
func TestRemoveDockerActionImagesByTag(t *testing.T) { func TestRemoveDockerActionImagesByTag(t *testing.T) {
var removed []string var removed []string
fakeDockerDaemon(t, func(writer http.ResponseWriter, request *http.Request) { fakeDockerDaemon(t, func(writer http.ResponseWriter, request *http.Request) {
@@ -1181,6 +1312,26 @@ func TestPrintStartJobContainerGroupGolden(t *testing.T) {
assert.Equal(t, want, buf.String()) assert.Equal(t, want, buf.String())
} }
func TestStartHostEnvironmentToolCacheSlotIsSubdirOfSharedCache(t *testing.T) {
cacheDir := t.TempDir()
override := filepath.Join(cacheDir, "override")
for sharedCache, env := range map[string]map[string]string{
filepath.Join(cacheDir, "tool_cache"): {},
override: {"RUNNER_TOOL_CACHE": override},
} {
rc := &RunContext{
Config: &Config{ActionCacheDir: cacheDir, SharedToolCache: true, ToolCacheSlot: "1"},
Env: env,
Run: &model.Run{Workflow: &model.Workflow{Name: "wf"}},
}
require.NoError(t, rc.startHostEnvironment()(t.Context()))
t.Cleanup(func() { require.NoError(t, rc.cleanUpJobContainer(context.Background())) })
want := filepath.Join(sharedCache, "1")
assert.Equal(t, want, rc.JobContainer.(*container.HostEnvironment).ToolCache)
assert.Equal(t, want, rc.Env["RUNNER_TOOL_CACHE"])
}
}
func TestRunContext_cleanupFailedStart(t *testing.T) { func TestRunContext_cleanupFailedStart(t *testing.T) {
type ctxKey string type ctxKey string
const sentinel = ctxKey("sentinel") const sentinel = ctxKey("sentinel")
@@ -1252,6 +1403,7 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
instance.On("Close").Return(record(name+".Close", nil)) instance.On("Close").Return(record(name+".Close", nil))
} }
job.On("Copy", mock.Anything, mock.Anything).Return(record("job.Copy", nil)) job.On("Copy", mock.Anything, mock.Anything).Return(record("job.Copy", nil))
job.On("GetContainerArchive", mock.Anything, mock.Anything).Return(nil, os.ErrNotExist)
job.On("Inspect", mock.Anything).Return(&container.Info{ID: "job-id"}, nil). job.On("Inspect", mock.Anything).Return(&container.Info{ID: "job-id"}, nil).
Run(func(mock.Arguments) { operations = append(operations, "job.Inspect") }) Run(func(mock.Arguments) { operations = append(operations, "job.Inspect") })
for _, health := range []string{container.HealthStarting, testcase.health} { for _, health := range []string{container.HealthStarting, testcase.health} {
@@ -1465,13 +1617,21 @@ func TestGetJobContextReportsContainers(t *testing.T) {
type imageEnvContainer struct { type imageEnvContainer struct {
*container.HostEnvironment *container.HostEnvironment
imageEnv map[string]string imageEnv map[string]string
osRelease []byte
} }
func (c imageEnvContainer) UpdateFromImageEnv(env *map[string]string) common.Executor { func (c imageEnvContainer) UpdateFromImageEnv(env *map[string]string) common.Executor {
return func(_ctx context.Context) error { maps.Copy(*env, c.imageEnv); return nil } return func(_ctx context.Context) error { maps.Copy(*env, c.imageEnv); return nil }
} }
func (c imageEnvContainer) GetContainerArchive(_ctx context.Context, srcPath string) (io.ReadCloser, error) {
if srcPath != "/usr/lib/os-release" || c.osRelease == nil {
return nil, os.ErrNotExist
}
return io.NopCloser(bytes.NewReader(c.osRelease)), nil
}
func TestCopyWorkflowFilesHandsNodeTheRunnerCAUnlessTheJobOrImageSetsIt(t *testing.T) { func TestCopyWorkflowFilesHandsNodeTheRunnerCAUnlessTheJobOrImageSetsIt(t *testing.T) {
caFile, actPath := filepath.Join(t.TempDir(), "ca.crt"), t.TempDir() caFile, actPath := filepath.Join(t.TempDir(), "ca.crt"), t.TempDir()
require.NoError(t, os.WriteFile(caFile, []byte("PEM\n"), 0o600)) require.NoError(t, os.WriteFile(caFile, []byte("PEM\n"), 0o600))
@@ -1481,7 +1641,7 @@ func TestCopyWorkflowFilesHandsNodeTheRunnerCAUnlessTheJobOrImageSetsIt(t *testi
Config: &Config{CAFile: caFile}, Config: &Config{CAFile: caFile},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}}, Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},
Env: maps.Clone(jobEnv), Env: maps.Clone(jobEnv),
JobContainer: imageEnvContainer{&container.HostEnvironment{Path: t.TempDir(), ActPath: actPath, StdOut: io.Discard}, imageEnv}, JobContainer: imageEnvContainer{&container.HostEnvironment{Path: t.TempDir(), ActPath: actPath, StdOut: io.Discard}, imageEnv, nil},
} }
rc.containerSpec.Env = containerEnv rc.containerSpec.Env = containerEnv
require.NoError(t, rc.copyWorkflowFiles()(t.Context())) require.NoError(t, rc.copyWorkflowFiles()(t.Context()))
@@ -1598,6 +1758,30 @@ func TestRunContextContainerImageCannotSelectBackend(t *testing.T) {
func TestRunContextImageOS(t *testing.T) { func TestRunContextImageOS(t *testing.T) {
ctx := t.Context() ctx := t.Context()
t.Run("keeps an ImageOS from the image, the job env or the job's Ubuntu release", func(t *testing.T) {
ubuntu := tarArchive(t, tarEntry{name: "os-release", body: "ID=ubuntu\nVERSION_ID=\"22.04\"\n"})
for _, tc := range []struct {
imageEnv map[string]string
jobEnv string
osRelease []byte
want string
}{
{imageEnv: map[string]string{"ImageOS": "ubuntu24-arm64"}, osRelease: ubuntu, want: "ubuntu24-arm64"},
{imageEnv: map[string]string{"ImageOS": "ubuntu24-arm64"}, jobEnv: "ImageOS: ubuntu20", want: "ubuntu20"},
{osRelease: ubuntu, want: "ubuntu22"},
{osRelease: tarArchive(t, tarEntry{name: "os-release", body: "ID=debian\nVERSION_ID=\"12\"\n"}), want: "ubuntu24"},
} {
rc := createRunsOnRunContext(t, "ubuntu-latest\nenv: {"+tc.jobEnv+"}")
rc.platformImage = "docker.gitea.com/runner-images:ubuntu-latest"
rc.JobContainer = imageEnvContainer{&container.HostEnvironment{ActPath: t.TempDir()}, tc.imageEnv, tc.osRelease}
require.NoError(t, rc.copyWorkflowFiles()(ctx))
require.NoError(t, evaluateJobEnvAndDefaults(ctx, rc))
env := map[string]string{}
require.NoError(t, setupEnv(ctx, &stepRun{RunContext: rc, Step: &model.Step{}, env: env}))
assert.Equal(t, tc.want, env["ImageOS"])
}
})
t.Run("prefers the release in the resolved image tag", func(t *testing.T) { t.Run("prefers the release in the resolved image tag", func(t *testing.T) {
rc := createRunsOnRunContext(t, "ubuntu-latest") rc := createRunsOnRunContext(t, "ubuntu-latest")
rc.Config.PlatformPicker = func([]string) string { return "docker.gitea.com/runner-images:ubuntu-24.04" } rc.Config.PlatformPicker = func([]string) string { return "docker.gitea.com/runner-images:ubuntu-24.04" }
@@ -1612,8 +1796,8 @@ func TestRunContextImageOS(t *testing.T) {
assert.Equal(t, "ubuntu22", rc.imageOS(ctx)) assert.Equal(t, "ubuntu22", rc.imageOS(ctx))
}) })
t.Run("keeps the historical value for a rolling label with no release", func(t *testing.T) { t.Run("reports the current release for the rolling ubuntu-latest label", func(t *testing.T) {
assert.Equal(t, "ubuntu20", createRunsOnRunContext(t, "ubuntu-latest").imageOS(ctx)) assert.Equal(t, "ubuntu24", createRunsOnRunContext(t, "ubuntu-latest").imageOS(ctx))
}) })
t.Run("is empty for the synthetic job of a composite action", func(t *testing.T) { t.Run("is empty for the synthetic job of a composite action", func(t *testing.T) {
@@ -1640,23 +1824,33 @@ func TestRunContextGetRunnerContext(t *testing.T) {
assert.NotContains(t, runnerContext, "debug") assert.NotContains(t, runnerContext, "debug")
}) })
t.Run("reports debug when step debugging is on", func(t *testing.T) { t.Run("reports debug from the secret, else the variable, also in composites", func(t *testing.T) {
rc := createRunsOnRunContext(t, "ubuntu-latest") rc := createRunsOnRunContext(t, "ubuntu-latest")
rc.Config.Secrets = map[string]string{"ACTIONS_STEP_DEBUG": "true"} rc.Config.Secrets = map[string]string{"ACTIONS_STEP_DEBUG": "true"}
assert.Equal(t, "1", rc.getRunnerContext(ctx)["debug"]) assert.Equal(t, "1", rc.getRunnerContext(ctx)["debug"])
assert.Equal(t, "1", (&RunContext{Parent: rc, Config: &Config{}}).getRunnerContext(ctx)["debug"])
rc.Config.Secrets = map[string]string{}
rc.Config.Vars = map[string]string{"ACTIONS_STEP_DEBUG": "true"}
assert.Equal(t, "1", rc.getRunnerContext(ctx)["debug"])
rc.Config.Secrets = map[string]string{"ACTIONS_STEP_DEBUG": "false"}
assert.NotContains(t, rc.getRunnerContext(ctx), "debug")
}) })
t.Run("keeps the execution environment values", func(t *testing.T) { t.Run("keeps the execution environment values", func(t *testing.T) {
rc := createRunsOnRunContext(t, "ubuntu-latest") rc := createRunsOnRunContext(t, "ubuntu-latest")
rc.Config.Workdir = "/host/work/owner/repo" rc.Config.Workdir = "/host/work/owner/repo"
rc.JobContainer = &container.HostEnvironment{Workdir: rc.Config.Workdir, Path: "/container/work/owner/repo", TmpDir: "/tmp/act", ToolCache: "/tmp/tool_cache"} rc.JobContainer = &container.HostEnvironment{Workdir: rc.Config.Workdir, Path: "/container/work/owner/repo", TmpDir: "/tmp/act", ToolCache: "/tmp/tool_cache", ActPath: "/tmp/act-files"}
runnerContext := rc.getRunnerContext(ctx) runnerContext := rc.getRunnerContext(ctx)
assert.Equal(t, "/tmp/act", runnerContext["temp"]) assert.Equal(t, "/tmp/act", runnerContext["temp"])
assert.Equal(t, "/tmp/tool_cache", runnerContext["tool_cache"]) assert.Equal(t, "/tmp/tool_cache", runnerContext["tool_cache"])
assert.Equal(t, "/container/work/owner", runnerContext["workspace"]) assert.Equal(t, "/container/work/owner", runnerContext["workspace"])
assert.Equal(t, "/container/work/owner/repo", rc.getGithubContext(ctx).Workspace) githubContext := rc.getGithubContext(ctx)
assert.Equal(t, "/container/work/owner/repo", githubContext.Workspace)
assert.Equal(t, "/tmp/act-files/workflow/envs.txt", githubContext.Env)
assert.Equal(t, "/tmp/act-files/workflow/pathcmd.txt", githubContext.Path)
assert.NotEmpty(t, runnerContext["os"]) assert.NotEmpty(t, runnerContext["os"])
}) })
} }
+7 -4
View File
@@ -70,7 +70,6 @@ type Config struct {
EventJSON string // the content of JSON file to use for event.json in containers, overrides EventPath EventJSON string // the content of JSON file to use for event.json in containers, overrides EventPath
ContainerNamePrefix string // the prefix of container name ContainerNamePrefix string // the prefix of container name
ContainerMaxLifetime time.Duration // the max lifetime of job containers ContainerMaxLifetime time.Duration // the max lifetime of job containers
CleanWorkdir bool // remove host executor workdir on teardown
DefaultActionInstance string // the default actions web site DefaultActionInstance string // the default actions web site
// DefaultActionInstanceIsSelfHosted reports whether DefaultActionInstance is this // DefaultActionInstanceIsSelfHosted reports whether DefaultActionInstance is this
// self-hosted Gitea (DEFAULT_ACTIONS_URL=self). It gates token trust: only then may the // self-hosted Gitea (DEFAULT_ACTIONS_URL=self). It gates token trust: only then may the
@@ -81,11 +80,11 @@ type Config struct {
PlatformPicker func(labels []string) string PlatformPicker func(labels []string) string
KubernetesPicker func(labels []string) container.KubernetesOptions KubernetesPicker func(labels []string) container.KubernetesOptions
InstanceTransport http.RoundTripper InstanceTransport http.RoundTripper
ResultsURL string // the results service as the runner reaches it, never the job's ACTIONS_RESULTS_URL, which a step can rewrite
JobLoggerLevel *logging.Level // the level of job logger JobLoggerLevel *logging.Level // the level of job logger
ValidVolumes []string // only volumes (and bind mounts) in this slice can be mounted on the job container or service containers ValidVolumes []string // only volumes (and bind mounts) in this slice can be mounted on the job container or service containers
ContainerVolumes []string // mounted into every job, service and docker action container ContainerVolumes []string // mounted into every job, service and docker action container
SharedToolCache bool // one tool cache for all jobs instead of one per job SharedToolCache bool // keeps the tool cache across jobs instead of one per job
ToolCacheSlot string // keeps concurrent tasks' tool caches apart, empty for slot 0 and in exec
InsecureSkipTLS bool // whether to skip verifying TLS certificate of the Gitea instance InsecureSkipTLS bool // whether to skip verifying TLS certificate of the Gitea instance
ClientCertFile, ClientKeyFile string // PEM client certificate for the Gitea instance ClientCertFile, ClientKeyFile string // PEM client certificate for the Gitea instance
CAFile string // PEM CA certificates the Gitea instance's certificate chains to CAFile string // PEM CA certificates the Gitea instance's certificate chains to
@@ -100,7 +99,11 @@ type Config struct {
// RunnerDebug reports whether debug logging is on, exposed as `runner.debug` and RUNNER_DEBUG. // RunnerDebug reports whether debug logging is on, exposed as `runner.debug` and RUNNER_DEBUG.
func (c Config) RunnerDebug() bool { func (c Config) RunnerDebug() bool {
return c.Secrets["ACTIONS_STEP_DEBUG"] == "true" || c.Env["ACTIONS_STEP_DEBUG"] == "true" stepDebug, ok := c.Secrets["ACTIONS_STEP_DEBUG"]
if !ok {
stepDebug = c.Vars["ACTIONS_STEP_DEBUG"]
}
return stepDebug == "true" || c.Env["ACTIONS_STEP_DEBUG"] == "true"
} }
// GetToken: Adapt to Gitea // GetToken: Adapt to Gitea
-1
View File
@@ -455,7 +455,6 @@ func TestRunEventHostEnvironment(t *testing.T) {
{workdir, "nix-prepend-path", "push", "", platforms, secrets}, {workdir, "nix-prepend-path", "push", "", platforms, secrets},
{workdir, "uses-docker-url", "push", "", platforms, secrets}, {workdir, "uses-docker-url", "push", "", platforms, secrets},
{workdir, "local-action-dockerfile", "push", "", platforms, secrets}, {workdir, "local-action-dockerfile", "push", "", platforms, secrets},
{workdir, "builtin-checkout", "push", "", platforms, secrets},
{workdir, "inputs-via-env-context", "push", "", platforms, secrets}, {workdir, "inputs-via-env-context", "push", "", platforms, secrets},
{workdir, "do-not-leak-step-env-in-composite", "push", "", platforms, secrets}, {workdir, "do-not-leak-step-env-in-composite", "push", "", platforms, secrets},
{workdir, "set-env-step-env-override", "push", "", platforms, secrets}, {workdir, "set-env-step-env-override", "push", "", platforms, secrets},
+6 -4
View File
@@ -21,6 +21,11 @@ import (
"gitea.dev/actionslib/pkg/model" "gitea.dev/actionslib/pkg/model"
) )
const (
pathFileCommand = "workflow/pathcmd.txt"
envFileCommand = "workflow/envs.txt"
)
type step interface { type step interface {
pre() common.Executor pre() common.Executor
main() common.Executor main() common.Executor
@@ -149,10 +154,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
stateFileCommand := path.Join("workflow", "statecmd.txt") stateFileCommand := path.Join("workflow", "statecmd.txt")
(*step.getEnv())["GITHUB_STATE"] = container.JoinPath(actPath, stateFileCommand) (*step.getEnv())["GITHUB_STATE"] = container.JoinPath(actPath, stateFileCommand)
pathFileCommand := path.Join("workflow", "pathcmd.txt")
(*step.getEnv())["GITHUB_PATH"] = container.JoinPath(actPath, pathFileCommand) (*step.getEnv())["GITHUB_PATH"] = container.JoinPath(actPath, pathFileCommand)
envFileCommand := path.Join("workflow", "envs.txt")
(*step.getEnv())["GITHUB_ENV"] = container.JoinPath(actPath, envFileCommand) (*step.getEnv())["GITHUB_ENV"] = container.JoinPath(actPath, envFileCommand)
// Per-step summary file. Composite sub-steps share the outer job step's index // Per-step summary file. Composite sub-steps share the outer job step's index
@@ -263,7 +265,7 @@ func setupEnv(ctx context.Context, step step) error {
env := step.getEnv() env := step.getEnv()
// container env is the image's baseline, which job env and $GITHUB_ENV override // container env is the image's baseline, which job env and $GITHUB_ENV override
mergeIntoMap(step, env, rc.caEnv, rc.containerSpec.Env, rc.GetEnv()) mergeIntoMap(step, env, rc.defaultEnv, rc.containerSpec.Env, rc.GetEnv())
rc.withGithubEnv(ctx, getStepGithubContext(ctx, step), *env) rc.withGithubEnv(ctx, getStepGithubContext(ctx, step), *env)
var err error var err error
-87
View File
@@ -1,87 +0,0 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package runner
import (
"cmp"
"context"
"fmt"
"strings"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/internal/pkg/action"
"gitea.com/gitea/runner/internal/pkg/action/checkout"
"gitea.com/gitea/runner/internal/pkg/action/downloadartifact"
"gitea.com/gitea/runner/internal/pkg/action/uploadartifact"
"gitea.dev/actionslib/pkg/model"
)
var builtinActions = map[string]func(context.Context, *action.Context) error{
"checkout": checkout.Main,
"download-artifact": downloadartifact.Main,
"upload-artifact": uploadartifact.Main,
}
type stepActionBuiltin struct {
Step *model.Step
RunContext *RunContext
run func(context.Context, *action.Context) error
env map[string]string
}
func (sab *stepActionBuiltin) pre() common.Executor { return common.NewPipelineExecutor() }
func (sab *stepActionBuiltin) main() common.Executor {
sab.env = map[string]string{}
return runStepExecutor(sab, stepStageMain, func(ctx context.Context) error {
if common.Dryrun(ctx) {
return nil
}
printRunActionHeader(ctx, sab.Step, nil, sab.env, sab.RunContext)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
rc := sab.RunContext
rc.ApplyExtraPath(ctx, &sab.env)
inputs := make(map[string]string, len(sab.Step.With))
for name, value := range sab.Step.With {
inputs[strings.ToLower(name)] = value // input names are case-insensitive on GitHub
}
return sab.run(ctx, &action.Context{
Container: rc.JobContainer,
Github: getStepGithubContext(ctx, sab),
Inputs: inputs,
HostWorkdir: rc.Config.Workdir,
Workspace: rc.JobContainer.ToContainerPath(rc.Config.Workdir),
BindWorkdir: rc.Config.BindWorkdir,
NoSkipCheckout: rc.Config.NoSkipCheckout,
UseGitIgnore: rc.Config.UseGitIgnore,
InsecureSkipTLS: rc.Config.InsecureSkipTLS,
ClientCertFile: rc.Config.ClientCertFile,
ClientKeyFile: rc.Config.ClientKeyFile,
CAFile: rc.jobCAFile(),
Env: sab.env,
ResultsURL: rc.Config.ResultsURL,
RuntimeToken: cmp.Or(rc.runtimeToken, rc.topLevelRunContext().Config.Secrets["ACTIONS_RUNTIME_TOKEN"]), // composite contexts have no secrets
Transport: rc.Config.InstanceTransport,
Infof: rawLogger.Infof,
Warning: func(format string, args ...any) {
rawLogger.Infof("%s", formatAnnotation("warning", nil, fmt.Sprintf(format, args...)))
},
})
})
}
func (sab *stepActionBuiltin) post() common.Executor { return common.NewPipelineExecutor() }
func (sab *stepActionBuiltin) getRunContext() *RunContext { return sab.RunContext }
func (sab *stepActionBuiltin) getStepModel() *model.Step { return sab.Step }
func (sab *stepActionBuiltin) getEnv() *map[string]string { return &sab.env }
func (sab *stepActionBuiltin) getIfExpression(_ context.Context, _ stepStage) string {
return sab.Step.If.Value
}
-15
View File
@@ -6,9 +6,6 @@ package runner
import ( import (
"fmt" "fmt"
"maps"
"slices"
"strings"
"gitea.dev/actionslib/pkg/model" "gitea.dev/actionslib/pkg/model"
) )
@@ -36,18 +33,6 @@ func (sf *stepFactoryImpl) newStep(stepModel *model.Step, rc *RunContext) (step,
runAction: runActionImpl, runAction: runActionImpl,
}, nil }, nil
case model.StepTypeUsesActionRemote: case model.StepTypeUsesActionRemote:
if name, ok := strings.CutPrefix(stepModel.Uses, "builtin:"); ok {
run, ok := builtinActions[name]
if !ok {
return nil, fmt.Errorf("unknown built-in action %q (known: %v) for job:%s step:%+v",
name, slices.Sorted(maps.Keys(builtinActions)), rc.Name, stepModel)
}
return &stepActionBuiltin{
Step: stepModel,
RunContext: rc,
run: run,
}, nil
}
return &stepActionRemote{ return &stepActionRemote{
Step: stepModel, Step: stepModel,
RunContext: rc, RunContext: rc,
Loaded 50 of 121 files, more files were not shown because too many files have changed in this diff. Show more