Compare commits

..
40 Commits
Author SHA1 Message Date
silverwindandbircni 0e4c8b6b78 fix: misc improvements (#1415)
- Stop overwriting the image's `ImageOS`. `ubuntu-latest` got `ubuntu20` on the 24.04 default image.
- Derive `ImageOS` from `os-release` when the image sets none. Fall back to `ubuntu24` for `ubuntu-latest`.
- Keep an `ImageOS` set in the job, step or container env.
- Honor the image's `ImageOS` on Kubernetes.
- Treat steps after a job timeout as cancelled. `cancelled()` and `always()` cleanup now runs. The job still fails.
- Record nested reusable workflow results. Jobs that `needs:` a nested call now run.
- Forward outputs through nested reusable workflows. A bad output fails the call.
- Pass the job's `GITHUB_TOKEN` and `GITEA_TOKEN` to called workflows by default. An explicit mapping overrides them.
- Probe a started container's uid and gid concurrently.
- Cache the Docker container env from its first inspect.
- Check `ImageOS` in the Kubernetes e2e test.

---------

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1415
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-16 20:51:26 +00:00
silverwind 1f2ea1b83a fix: honor zero and empty values for fetch backoff and idle cleanup (#1418)
`fetch_interval_max: 0` is documented to disable polling backoff but became 5s. Zeros for `workdir_cleanup_age` and `idle_cleanup_interval` were ignored when they came from a YAML merge key or alias, so cleanup still ran, while an empty or null value turned cleanup off. These defaults are now seeded before the file is read, as `cache` already does, so a written 0 means off and an empty value keeps the default.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1418
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-16 20:43:37 +00:00
Renovate Bot 891f3fde38 fix(deps): update dependencies (#1414)
This PR contains the following updates:

| Package | Type | Update | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|---|---|
| docker.gitea.com/runner-images |  | digest | `b96ba46` → `139605d` |  |  |
| [docker/login-action](https://github.com/docker/login-action) ([changelog](https://github.com/docker/login-action/compare/81c6f93d4bc991e510b342a00b419b3bb4efc5c6..8aa5831fb8d48d8f76594c7ddde61ae58019f550)) | action | digest | `81c6f93` → `8aa5831` |  |  |
| [github.com/spf13/pflag](https://github.com/spf13/pflag) | require | minor | `v1.0.10` → `v1.1.0` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fspf13%2fpflag/v1.1.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fspf13%2fpflag/v1.0.10/v1.1.0?slim=true) |
| [go](https://go.dev/) ([source](https://github.com/golang/go)) | toolchain | patch | `1.27.3` → `1.27.4` | ![age](https://developer.mend.io/api/mc/badges/age/golang-version/go/1.27.4?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/golang-version/go/1.27.3/1.27.4?slim=true) |
| [golang.org/x/sync](https://pkg.go.dev/golang.org/x/sync) | require | minor | [`v0.24.0` → `v0.25.0`](https://cs.opensource.google/go/x/sync/+/refs/tags/v0.24.0...refs/tags/v0.25.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fsync/v0.25.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fsync/v0.24.0/v0.25.0?slim=true) |
| [golang.org/x/sys](https://pkg.go.dev/golang.org/x/sys) | require | minor | [`v0.49.0` → `v0.50.0`](https://cs.opensource.google/go/x/sys/+/refs/tags/v0.49.0...refs/tags/v0.50.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fsys/v0.50.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fsys/v0.49.0/v0.50.0?slim=true) |
| [golang.org/x/term](https://pkg.go.dev/golang.org/x/term) | require | minor | [`v0.47.0` → `v0.48.0`](https://cs.opensource.google/go/x/term/+/refs/tags/v0.47.0...refs/tags/v0.48.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fterm/v0.48.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fterm/v0.47.0/v0.48.0?slim=true) |
| [golang.org/x/text](https://pkg.go.dev/golang.org/x/text) | require | minor | [`v0.43.0` → `v0.44.0`](https://cs.opensource.google/go/x/text/+/refs/tags/v0.43.0...refs/tags/v0.44.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2ftext/v0.44.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2ftext/v0.43.0/v0.44.0?slim=true) |
| [quay.io/containers/podman](https://quay.io/containers/podman) ([source](https://github.com/podman-container-tools/podman)) |  | major | `v5.8.9-immutable` → `v6.1.3-immutable` | ![age](https://developer.mend.io/api/mc/badges/age/docker/quay.io%2fcontainers%2fpodman/v6.1.3?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/docker/quay.io%2fcontainers%2fpodman/v5.8.9/v6.1.3?slim=true) |

---

### HTTP/1 server crash due to MaxBytesReader header race in net/http
[CVE-2026-97054](https://nvd.nist.gov/vuln/detail/CVE-2026-97054) / [GO-2026-6859](https://pkg.go.dev/vuln/GO-2026-6859)

<details>
<summary>More information</summary>

#### Details
For HTTP/1 servers, when MaxBytesReader's limit is exceeded, "Connection: close" header is automatically set without synchronization, which might race with another header being set explicitly in the server handler. As a result, an HTTP/1 server that utilizes MaxBytesReader or MaxBytesHandler can crash due to runtime panic if its handler reads the request body concurrently with writing the response header.

#### Severity
Unknown

#### References
- [https://go.dev/cl/862185](https://go.dev/cl/862185)
- [https://go.dev/issue/82563](https://go.dev/issue/82563)
- [https://groups.google.com/g/golang-announce/c/iY8TY0wWgbM](https://groups.google.com/g/golang-announce/c/iY8TY0wWgbM)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6859) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Release Notes

<details>
<summary>spf13/pflag (github.com/spf13/pflag)</summary>

### [`v1.1.0`](https://github.com/spf13/pflag/releases/tag/v1.1.0)

[Compare Source](https://github.com/spf13/pflag/compare/v1.0.10...v1.1.0)

It's been long enough, and enough things have been fixed, that it might be time to release a new version, even if there isn't any particular big new issue to market as the reason for this release.

If you notice anything breaking, it is entirely un-intentional, but you might also have been relying on [undocumented](https://www.hyrumslaw.com/) or [buggy](https://xkcd.com/1172/) behavior. Please file an issue describing what you're doing, how it worked before this release and why it doesn't work anymore, and pin your projects to `v1.0.10` (the latest release before this one) while we decide how to address your concerns.

#### What's Changed

- README: update badges and minor linting fixes by [@&#8203;thaJeztah](https://github.com/thaJeztah) in [#&#8203;452](https://github.com/spf13/pflag/pull/452)
- Show 'correct' explicit bool value syntax in help by [@&#8203;alimpfard](https://github.com/alimpfard) in [#&#8203;453](https://github.com/spf13/pflag/pull/453)
- Fix for Go 1.24 by [@&#8203;pterjan](https://github.com/pterjan) in [#&#8203;460](https://github.com/spf13/pflag/pull/460)
- fix(map): ensure string-to-string flags maintain stable pointers by [@&#8203;brandon1024](https://github.com/brandon1024) in [#&#8203;457](https://github.com/spf13/pflag/pull/457)
- Fix getFlagType error message by [@&#8203;wellitonscheer](https://github.com/wellitonscheer) in [#&#8203;462](https://github.com/spf13/pflag/pull/462)
- update actions, and fix linting issues by [@&#8203;thaJeztah](https://github.com/thaJeztah) in [#&#8203;450](https://github.com/spf13/pflag/pull/450)
- fix: README.md indicating the use of godoc(deprecated) by [@&#8203;malklera](https://github.com/malklera) in [#&#8203;467](https://github.com/spf13/pflag/pull/467)
- Fix broken link to POSIX argument conventions by [@&#8203;tomasaschan](https://github.com/tomasaschan) in [#&#8203;464](https://github.com/spf13/pflag/pull/464)
- Replace raw stdlib parse errors in typed flag Set methods ([#&#8203;269](https://github.com/spf13/pflag/issues/269)) by [@&#8203;reuvenharrison](https://github.com/reuvenharrison) in [#&#8203;469](https://github.com/spf13/pflag/pull/469)
- Extend `UnknownFlags` to `UnknownFlagsHandling` and introduce `PassUnknownFlagToArgs` ([#&#8203;337](https://github.com/spf13/pflag/issues/337)) by [@&#8203;ikedam](https://github.com/ikedam) in [#&#8203;440](https://github.com/spf13/pflag/pull/440)
- fix: allow nil default IP flags in GetIP by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;483](https://github.com/spf13/pflag/pull/483)
- fix: allow hex input in UintSlice by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;484](https://github.com/spf13/pflag/pull/484)
- fix: honor custom IsBoolFlag compatibility by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;487](https://github.com/spf13/pflag/pull/487)
- docs: clarify SortFlags example with complete runnable program by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;488](https://github.com/spf13/pflag/pull/488)
- docs: add release process documentation by [@&#8203;happysnaker](https://github.com/happysnaker) in [#&#8203;490](https://github.com/spf13/pflag/pull/490)
- fix: print to set destination, not hardcoded os.Stderr by [@&#8203;quite](https://github.com/quite) in [#&#8203;502](https://github.com/spf13/pflag/pull/502)
- ci: bump golangci-lint to v2.13 for Go 1.27 by [@&#8203;hdimer](https://github.com/hdimer) in [#&#8203;508](https://github.com/spf13/pflag/pull/508)
- Initialize the target map when a StringToString flag has a nil default by [@&#8203;nikolauspschuetz](https://github.com/nikolauspschuetz) in [#&#8203;506](https://github.com/spf13/pflag/pull/506)
- fix: honour deprecated ParseErrorsWhitelist.UnknownFlagsHandling by [@&#8203;lenamonj](https://github.com/lenamonj) in [#&#8203;507](https://github.com/spf13/pflag/pull/507)
- test: add StringSlice Append coverage by [@&#8203;LY-19418](https://github.com/LY-19418) in [#&#8203;504](https://github.com/spf13/pflag/pull/504)
- Keep wrapping after an unbreakable usage word by [@&#8203;AzazelSensei](https://github.com/AzazelSensei) in [#&#8203;503](https://github.com/spf13/pflag/pull/503)
- Validate IP slice values before appending or replacing by [@&#8203;agammann](https://github.com/agammann) in [#&#8203;513](https://github.com/spf13/pflag/pull/513)
- fix(ci): Use the latest golangci-lint release by [@&#8203;tomasaschan](https://github.com/tomasaschan) in [#&#8203;525](https://github.com/spf13/pflag/pull/525)
- Reject IPv6 addresses as IPv4 masks by [@&#8203;yuefdev](https://github.com/yuefdev) in [#&#8203;524](https://github.com/spf13/pflag/pull/524)
- Report the flag name as typed in deprecation warnings by [@&#8203;hdimer](https://github.com/hdimer) in [#&#8203;500](https://github.com/spf13/pflag/pull/500)
- Document non-interspersed argument parsing by [@&#8203;yuefdev](https://github.com/yuefdev) in [#&#8203;523](https://github.com/spf13/pflag/pull/523)
- Preserve trailing whitespace in StringToString formatting by [@&#8203;r4venrudy](https://github.com/r4venrudy) in [#&#8203;522](https://github.com/spf13/pflag/pull/522)
- Reject extra CSV records in StringToString arguments by [@&#8203;Harshil-1603](https://github.com/Harshil-1603) in [#&#8203;521](https://github.com/spf13/pflag/pull/521)
- Preserve current string values in CopyToGoFlagSet by [@&#8203;adenzhou1350](https://github.com/adenzhou1350) in [#&#8203;520](https://github.com/spf13/pflag/pull/520)
- Fix type identification and mismatch checks for text flags by [@&#8203;dewanshshekhar](https://github.com/dewanshshekhar) in [#&#8203;526](https://github.com/spf13/pflag/pull/526)
- fix: enforce native integer flag bounds on 32-bit platforms by [@&#8203;gdpu11](https://github.com/gdpu11) in [#&#8203;534](https://github.com/spf13/pflag/pull/534)
- fix: preserve count flag values when parsing fails by [@&#8203;gdpu11](https://github.com/gdpu11) in [#&#8203;533](https://github.com/spf13/pflag/pull/533)

#### New Contributors

- [@&#8203;alimpfard](https://github.com/alimpfard) made their first contribution in [#&#8203;453](https://github.com/spf13/pflag/pull/453)
- [@&#8203;pterjan](https://github.com/pterjan) made their first contribution in [#&#8203;460](https://github.com/spf13/pflag/pull/460)
- [@&#8203;brandon1024](https://github.com/brandon1024) made their first contribution in [#&#8203;457](https://github.com/spf13/pflag/pull/457)
- [@&#8203;wellitonscheer](https://github.com/wellitonscheer) made their first contribution in [#&#8203;462](https://github.com/spf13/pflag/pull/462)
- [@&#8203;malklera](https://github.com/malklera) made their first contribution in [#&#8203;467](https://github.com/spf13/pflag/pull/467)
- [@&#8203;reuvenharrison](https://github.com/reuvenharrison) made their first contribution in [#&#8203;469](https://github.com/spf13/pflag/pull/469)
- [@&#8203;ikedam](https://github.com/ikedam) made their first contribution in [#&#8203;440](https://github.com/spf13/pflag/pull/440)
- [@&#8203;happysnaker](https://github.com/happysnaker) made their first contribution in [#&#8203;483](https://github.com/spf13/pflag/pull/483)
- [@&#8203;quite](https://github.com/quite) made their first contribution in [#&#8203;502](https://github.com/spf13/pflag/pull/502)
- [@&#8203;hdimer](https://github.com/hdimer) made their first contribution in [#&#8203;508](https://github.com/spf13/pflag/pull/508)
- [@&#8203;nikolauspschuetz](https://github.com/nikolauspschuetz) made their first contribution in [#&#8203;506](https://github.com/spf13/pflag/pull/506)
- [@&#8203;lenamonj](https://github.com/lenamonj) made their first contribution in [#&#8203;507](https://github.com/spf13/pflag/pull/507)
- [@&#8203;LY-19418](https://github.com/LY-19418) made their first contribution in [#&#8203;504](https://github.com/spf13/pflag/pull/504)
- [@&#8203;AzazelSensei](https://github.com/AzazelSensei) made their first contribution in [#&#8203;503](https://github.com/spf13/pflag/pull/503)
- [@&#8203;agammann](https://github.com/agammann) made their first contribution in [#&#8203;513](https://github.com/spf13/pflag/pull/513)
- [@&#8203;yuefdev](https://github.com/yuefdev) made their first contribution in [#&#8203;524](https://github.com/spf13/pflag/pull/524)
- [@&#8203;r4venrudy](https://github.com/r4venrudy) made their first contribution in [#&#8203;522](https://github.com/spf13/pflag/pull/522)
- [@&#8203;Harshil-1603](https://github.com/Harshil-1603) made their first contribution in [#&#8203;521](https://github.com/spf13/pflag/pull/521)
- [@&#8203;adenzhou1350](https://github.com/adenzhou1350) made their first contribution in [#&#8203;520](https://github.com/spf13/pflag/pull/520)
- [@&#8203;dewanshshekhar](https://github.com/dewanshshekhar) made their first contribution in [#&#8203;526](https://github.com/spf13/pflag/pull/526)
- [@&#8203;gdpu11](https://github.com/gdpu11) made their first contribution in [#&#8203;534](https://github.com/spf13/pflag/pull/534)

**Full Changelog**: <https://github.com/spf13/pflag/compare/v1.0.10...v1.1.0>

</details>

<details>
<summary>podman-container-tools/podman (quay.io/containers/podman)</summary>

### [`v6.1.3`](https://github.com/podman-container-tools/podman/releases/tag/v6.1.3)

[Compare Source](https://github.com/podman-container-tools/podman/compare/v5.8.9...v6.1.3)

##### Security

- This release addresses [CVE-2026-94603](https://github.com/podman-container-tools/podman/security/advisories/GHSA-2cvf-wqm6-wr9g), where a `podman run` on a checkpoint image (any image with the `io.podman.annotations.checkpoint.runtime.name` annotation) could disable all sandboxing, including sandboxing specified by the user, when the container was created.

##### Breaking Changes

- Removed support for checkpoint images in `podman run` due to serious security concerns with the different security models of running images and running checkpoints. Checkpoints ignore user-specified security configuration and are very difficult to run safely.

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4yMjIuNSIsInVwZGF0ZWRJblZlciI6IjQ0LjIzOS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1414
Reviewed-by: silverwind <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
2026-11-16 15:15:09 +00:00
silverwind 4f59e3e418 fix: support cache-mode and make low-trust runs cache read-only (#1412)
Low-trust runs like `pull_request_target` on the default branch could poison caches that `push` runs restore. As on GitHub, they now only restore unless `cache-mode` allows saving.

This and the Gitea change can be deployed in any order.

Refs:
https://gitea.com/gitea/actionslib/pulls/33
https://github.com/go-gitea/gitea/pull/40343

_Written by Claude (Opus 5.5)._
Reviewed-on: https://gitea.com/gitea/runner/pulls/1412
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-13 09:33:32 +00:00
clintlmartinandsilverwind c2046d597c fix: stop host teardown from deleting a workdir it never created (#1409)
Host jobs run in a per-job directory under `host.workdir_parent`, which teardown removes. Teardown also deleted `<container.workdir_parent>/<owner>/<repo>`, a path host jobs never create. With an empty parent and no repository that was the filesystem root, and any existing directory at that path was deleted although the job never used it.

This removes that delete instead of guarding it, so teardown only removes what the job created.

---------

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1409
Reviewed-by: bircni <[email protected]>
Co-authored-by: clintlmartin <[email protected]>
2026-11-13 06:50:44 +00:00
silverwind 645b17de72 fix: align common behaviour with GitHub's runner (#1413)
Closes differences from GitHub's runner that common workflows hit:

- `ACTIONS_STEP_DEBUG` set as a repository variable enables debug output as on GitHub, where a secret of the same name still takes precedence. `runner.debug` and `RUNNER_DEBUG` are now also set inside composite actions, which lost the secret.
- `github.env` and `github.path` hold the file-command paths instead of being empty, so `echo "A=1" >> "${{ github.env }}"` works.
- Remote composite actions only get a pre stage when they nest actions. That stage evaluated `with:` before earlier steps ran, so an input like `${{ fromJSON(steps.meta.outputs.json).v }}` failed the job.
- `ports` on a job container are published, they were ignored while service ports worked.
- Dockerfile action build output reaches the job log. It was logged at debug and dropped, so a failing `RUN` showed only the final error.
- Local Docker actions honor their `.dockerignore`, as `docker build` does.
- Cancelling or timing out a host-mode step sends SIGINT, then SIGTERM after 7.5 seconds and SIGKILL after 2.5 more, as GitHub's runner does, so `trap` handlers and shutdown hooks run.
- `.sh` job hooks fall back to `sh` where bash is missing, and `.ps1` hooks to `powershell` where pwsh is missing, as on GitHub.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1413
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-12 15:28:47 +00:00
silverwind 0dfec377d2 fix: keep composite post steps in their own step scope (#1410)
Each composite step's post ran its earlier siblings' posts inside its own step-ID scope, so their log records listed siblings as ancestors, e.g. `outer.results.nodepost` instead of `outer.nodepost`. Every post now gets its own scope. Execution order and run-all-posts behavior are unchanged.

Prerequisite for https://gitea.com/gitea/runner/issues/809.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1410
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-11 19:31:21 +00:00
silverwind b5305c6785 docs: add a launchd example for macOS (#1408)
Adds `examples/launchd` with a plist and the steps to run the runner on macOS, next to the existing systemd example.

The runner runs in the login session of the user that jobs should run as, which gives jobs the login keychain and the GUI. The README also covers the two problems that setup runs into: macOS blocks the runner from hosts on the Mac's own subnet, and Git's keychain helper can stop a job at a dialog.

_Written by Claude (Fable 5.2)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1408
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-10 17:17:35 +00:00
bircni a021d3139d enhance: make shared tool cache safe at any capacity (#1401)
Co-authored-by: bircni <[email protected]>
2026-11-10 14:25:50 +00:00
bircniandsilverwind 84c0fcd1fa fix!: remove the dbfs log workaround and require Gitea 1.26.2 (#1395)
Gitea 1.26.2 finalizes logs on an empty `UpdateLog` with `NoMore` (https://github.com/go-gitea/gitea/pull/37687), so the sentinel row is no longer needed. On older servers, the final log of most jobs would never be archived and stays in `dbfs_data`, so this raises the minimum Gitea version. It lives in `ver.MinGiteaVersion`, which the error for servers without `Declare` now names.

Fixes: https://gitea.com/gitea/runner/issues/953
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1395
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-10 11:35:16 +00:00
bircniandsilverwind c59caa9689 refactor!: remove the builtin actions (#1407)
Removes `builtin:checkout`, `builtin:upload-artifact` and `builtin:download-artifact`, along with the container helpers only they used.

Builtin actions are tied to the runner version, so users cannot pin them, and they diverge from the shareable actions ecosystem that tools like zizmor build on. Workflows should use `actions/checkout`, `actions/upload-artifact` and `actions/download-artifact`, or lightweight third-party actions such as https://github.com/taiki-e/checkout-action.

**Breaking:** steps with `uses: builtin:checkout`, released in v4.0.0, now fail and need to switch to one of the actions above. The artifact actions were never released.

Docs: https://gitea.com/gitea/docs/pulls/643
Fixes: https://gitea.com/gitea/runner/issues/1405
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1407
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-10 11:05:38 +00:00
Renovate Bot 85d007c47f chore(deps): update dependencies (#1406)
This PR contains the following updates:

| Package | Type | Update | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|---|---|
| docker.gitea.com/runner-images |  | digest | `657378e` → `b96ba46` |  |  |
| [github.com/docker/cli](https://github.com/docker/cli) | require | patch | `v29.9.2+incompatible` → `v29.9.3+incompatible` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fdocker%2fcli/v29.9.3+incompatible?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fdocker%2fcli/v29.9.2+incompatible/v29.9.3+incompatible?slim=true) |
| [rancher/k3s](https://github.com/k3s-io/k3s) |  | patch | `v1.37.1-k3s1` → `v1.37.2-k3s1` | ![age](https://developer.mend.io/api/mc/badges/age/docker/rancher%2fk3s/v1.37.2?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/docker/rancher%2fk3s/v1.37.1/v1.37.2?slim=true) |

---

### Release Notes

<details>
<summary>docker/cli (github.com/docker/cli)</summary>

### [`v29.9.3+incompatible`](https://github.com/docker/cli/compare/v29.9.2...v29.9.3)

[Compare Source](https://github.com/docker/cli/compare/v29.9.2...v29.9.3)

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xOTkuMiIsInVwZGF0ZWRJblZlciI6IjQ0LjE5OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1406
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
2026-11-09 04:31:50 +00:00
silverwindandbircni d34e4cf11a fix: stop a stalled docker daemon from hanging jobs and the e2e suite (#1404)
When the docker daemon stopped answering on https://gitea.com/gitea/runner/actions/runs/1204971, unbounded exec runs hung the e2e suite until the 20m `go test` panic, which skipped teardown. Exec runs and teardown are now bounded, and attached exec streams, including the runner's user ID probe, close with their context.

CLI daemons now stop with SIGTERM, so job volumes no longer leak.

_Written by Claude (Opus 5.5)._

---------

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1404
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-07 08:07:41 +00:00
8e999cd039 fix: re-detect the cache address per job and prefer IPv4 (#1398)
The cache address was detected once at startup, and the interface fallback ranked ethernet names above IPv4. A runner that started before DHCPv4 finished could advertise a global IPv6 address to IPv4-only job networks until it restarted, failing every artifact and cache call.

IPv4 now ranks first, and the address is detected again for each job unless `cache.host` sets it. Detecting an IPv6 address logs a warning.

Fixes: https://gitea.com/gitea/runner/issues/1379
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1398
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-07 06:41:25 +00:00
silverwind cd748053b0 docs: trim comments and docs from workspace emptying change (#1403)
Follow-up to https://gitea.com/gitea/runner/commit/79faf5e55df7a3b312fb4888b7405a58e01f567c. Reverts the post-task-script doc change, which is too much detail for that page. Also shortens the `DrainGracePeriod` comment and the other comments that commit added, since the code and log messages already say most of it.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1403
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-07 06:14:41 +00:00
bircni 9bb72bbe5c enhance: add ubuntu-26.04 to the default labels (#1396)
Newly registered runners get an `ubuntu-26.04` label alongside the existing defaults.

Ref: https://gitea.com/gitea/runner/issues/1334
Reviewed-on: https://gitea.com/gitea/runner/pulls/1396
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-07 05:12:04 +00:00
79faf5e55d fix: empty root-owned bind workspaces for non-root runners (#1399)
A job running as root leaves root-owned files in the bind-mounted workspace. A non-root runner on Linux cannot remove them, so workspaces pile up with only a warning. Such a runner now empties the workspace as root from inside the job container, right before removing the container. Root runners and Docker Desktop already remove these files themselves and skip this.

The workspace is only emptied when it is exactly the runner's own mountpoint with no mount below it. Otherwise, or when the command fails, the existing removal and warning apply. Jobs of a reusable workflow share the workspace, so they are not covered and still end in the existing warning.

Fixes: https://gitea.com/gitea/runner/issues/879
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1399
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-07 04:33:19 +00:00
bircni 993389d428 docs: trust a private registry CA in the dind images (#1397)
The bundled Docker daemon pulls job images, so it needs to trust a private registry's CA itself, which `runner.ca_file` doesn't cover. Documents the `certs.d` mount for both dind images.

Ref: https://gitea.com/gitea/runner/issues/1390
Reviewed-on: https://gitea.com/gitea/runner/pulls/1397
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-06 21:36:06 +00:00
silverwind db76fd5159 enhance: improve hashFiles coercion and log grouping (#1394)
- Fix `hashFiles()` argument conversion for numbers, booleans, and null to match GitHub expression semantics.
- Fix rendering of mixed-case, indented, and prefixed legacy group commands while preserving recognized prefixes and escaping.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1394
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-06 20:03:39 +00:00
bircni c8b16f4792 enhance: add builtin:download-artifact (#1344)
Co-authored-by: bircni <[email protected]>
2026-11-06 03:32:59 +00:00
bircni 02446620df enhance: add builtin:upload-artifact (#1343)
Co-authored-by: bircni <[email protected]>
2026-11-06 03:17:49 +00:00
8c8950d9ef refactor(container): stream copies into the job and archives out of the host (#1340)
Adds `CopyTarStream` to the job environments. It extracts directories and files into the job as they are written, kept below the destination and owned by the job user along with any directories it creates, so built-in actions can put large content in place without buffering it. The host's `GetContainerArchive` now streams instead of building the archive in memory.

---------

Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Co-authored-by: Lunny Xiao <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1340
Reviewed-by: Lunny Xiao <[email protected]>
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-06 02:22:40 +00:00
bircniandsilverwind b88d111cc5 feat: port the path patterns of @actions/glob (#1342)
Built-in actions that select files need to match what the JavaScript actions match. This ports @actions/glob 0.6.1 and the minimatch 3 subset it uses, as bundled in `actions/upload-artifact` v7. It only answers questions about paths; walking the files stays with the caller. Only malformed bracket expressions and some non-ASCII case folding on Windows can match differently.

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1342
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-06 01:13:07 +00:00
silverwind bb6d793cb9 fix: make container.sweep opt-in (#1389)
Sweeping removed every container a job started through its Docker proxy, which silently tore down services that jobs deploy to the runner's own Docker daemon. Deploying from a runner on the target host is common, so `container.sweep` now defaults to `false` and CI-only runners can opt in with `container.sweep: true`.

Fixes https://gitea.com/gitea/runner/issues/1385

Reviewed-on: https://gitea.com/gitea/runner/pulls/1389
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-05 13:50:46 +00:00
silverwind 95ef17d022 fix: prevent masking gaps in workflow command logs (#1384)
Mask partially escaped workflow command output and complete one-character values while preserving literal output and stream line endings.

Fixes: https://gitea.com/gitea/runner/issues/300
Reviewed-on: https://gitea.com/gitea/runner/pulls/1384
Reviewed-by: bircni <[email protected]>
2026-11-05 08:01:44 +00:00
Lunny Xiaoandsilverwind ee03687688 fix: upgrade Alpine packages in the runner images (#1388)
All images run `apk upgrade` before installing packages, so they pick up Alpine fixes their base images lag on. `docker:29.9.2-dind` shipped libexpat 2.9.0-r0 while Alpine 3.24 provides 2.10.0-r0 with the security fixes, and both `docker:29.9.3-dind` and `alpine:3.24` still ship apk-tools 3.0.8-r0 where 3.0.9-r0 is available. The dind bases are also bumped to `docker:29.9.3`, which drops several vulnerable bundled binaries' findings.

Verified by building the targets locally: libexpat is 2.10.0-r0 (was 2.9.0-r0), apk-tools is 3.0.9-r0 (was 3.0.8-r0), and Trivy reports no Alpine package findings on the dind images.

Assisted-by: Codet:claude
Assisted-by: Claude Code:claude-opus-5-5
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1388
Reviewed-by: silverwind <[email protected]>
Co-authored-by: Lunny Xiao <[email protected]>
2026-11-05 07:45:27 +00:00
silverwind 30e600d8f6 test: speed up (#1386)
Reduce redundant fixture setup, Git subprocesses, and large data copies while preserving test coverage.

Local macOS measurements against main:

| Measurement | Before | After |
| --- | ---: | ---: |
| Git package time, median of 3 alternating runs | 4.09 s | 3.48 s (-15%) |
| Allocations across 18 action-bundle cases | 489 MiB | 251 MiB (-49%) |
| Traced Git starts per checkout table run | 234 | 175 |

Reviewed-on: https://gitea.com/gitea/runner/pulls/1386
Reviewed-by: bircni <[email protected]>
2026-11-05 07:33:17 +00:00
silverwind 664978c819 test: fix windows path test (#1387)
`TestCopyWorkflowFilesHandsNodeTheRunnerCAUnlessTheJobOrImageSetsIt` from https://gitea.com/gitea/runner/pulls/1366 fails on Windows. `HostEnvironment.GetActPath` turns backslashes into slashes there, so `NODE_EXTRA_CA_CERTS` gets the slash form while the test expects the raw temp dir path. The runner behaviour is correct, only the expectation is wrong.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1387
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-05 07:14:56 +00:00
aa56f46c89 enhance: trust a private CA with runner.ca_file and add container.volumes (#1366)
Adds `container.volumes`, mounted into every Docker job, service and action container. Sources are paths on the Docker daemon's host and need no `valid_volumes` entry, and workflow volumes may use them only with the same access.

Adds `runner.ca_file`, a private CA trusted for the Gitea instance besides the system's by the runner's connections and its artifact proxy. Runner-side git and `builtin:checkout` use it as their CA bundle for the instance, persisted with the checkout's credentials. Jobs get it at `/var/run/act/ca.crt` with `NODE_EXTRA_CA_CERTS` pointing there unless the job or its image sets it. Git in jobs, as run by `actions/checkout`, needs the CA in the image's bundle, for example through `container.volumes`.

Fixes: https://gitea.com/gitea/runner/issues/568
Fixes: https://gitea.com/gitea/runner/issues/671
Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1366
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-11-05 05:28:40 +00:00
silverwindandLunny Xiao 7a9fccb842 ci: sign release checksums with the imported GPG key (#1382)
The release job has imported the Teabot GPG key since https://gitea.com/gitea/runner/pulls/116, but `.goreleaser.yaml` never had a `signs:` block, so releases were never signed. Add it so `checksums.txt` gets a detached `checksums.txt.sig`, the same way https://gitea.com/gitea/cli signs its releases.

The nightly job imports the key as well since the `signs:` block applies to nightly builds, and the R2 publisher mirrors the signature to dl.gitea.com.

_Written by Claude (Opus 5.5)._

---------

Co-authored-by: Lunny Xiao <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1382
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-05 04:28:22 +00:00
silverwindandLunny Xiao 7d6e1ff021 test: fix TestRunExecListUsesJobEventAndAllPlans hang on macOS (#1378)
`TestRunExecListUsesJobEventAndAllPlans` hangs until the test timeout on macOS.

Regression from https://gitea.com/gitea/runner/pulls/1338, which added a fifth `runExecList` call to the test to cover `exec --list --detect-event` without events. That took the captured output from 498 to 560 bytes. `captureStdout` swapped `os.Stdout` for a pipe it only read after the callback returned, and on macOS 27.2 an unread pipe holds 512 bytes, so the write blocked forever. Linux pipes hold 64 KiB, so was unaffected.

`printList` and `runExecList` now write to a passed writer, `exec --list` hands them `cmd.OutOrStdout()`, and the tests pass a buffer, so no test swaps `os.Stdout` anymore.

---------

Co-authored-by: Lunny Xiao <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1378
Reviewed-by: Lunny Xiao <[email protected]>
2026-11-05 04:22:01 +00:00
silverwind 319567213d docs: use latest image tags in examples (#1383)
The README's Docker example and the Docker and Kubernetes examples pointed at `nightly` images, while the README recommends `latest-dind`. Deployments copied from them should run the latest release, not a build of `main`.

None of the examples changed since v5.3.0, so they work as written with the current `latest` tags.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1383
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 23:00:45 +00:00
silverwind 0093731295 chore(deps): update go.yaml.in/yaml/v4 to v4.0.0 (#1381)
Update go-yaml from the rc.6 pseudo-version to the stable v4.0.0 release, and actionslib to v1.5.2, which does the same in https://gitea.com/gitea/actionslib/pulls/32.

v4 accepts `:` as a mapping with an empty key, so the reusable workflow tests now use an unclosed `[` as their invalid workflow. Gitea counterpart: https://github.com/go-gitea/gitea/pull/40222.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1381
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 20:05:17 +00:00
silverwind 3e1b4b42c2 fix(cache): stop the cache server from running its volume out of space (#1377)
Committing a cache upload copied every part into a new archive before deleting the parts, so a 5 GB save needed 10 GB free. Failed writes and assemblies left their partial bytes until the next stale sweep, up to an hour later, and every other save on the runner failed meanwhile. Nothing made room for an upload, so once the cache had filled its volume down to the free space floor, larger saves kept failing.

- Assemble in place, deleting each part once appended.
- Drop partial parts and archives on failure.
- Before a part would cross the floor, evict the least recently used entries.
- Reject uploads to reservations the sweep already deleted, which left files nothing removed.

_Written by Claude (Opus 5.5)._

Reviewed-on: https://gitea.com/gitea/runner/pulls/1377
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 16:38:21 +00:00
silverwind dcba546cb3 fix: align process handling and fix Windows crashes (#1376)
The intermittent Windows test crashes came from step errors formatting a nil `rc.Run`. `fmt` recovers the nil dereference, but on CPUs with AMX the recovered fault overflows into the heap (https://github.com/golang/go/issues/81238). The errors now name the job, and the Windows test VM hides AMX and runs on UTC, so cached boots no longer jump back 8 hours mid-test.

Windows commands now start suspended and resume once they are in their job objects, so no child escapes them. That drops the process scan reading other processes' memory and the last `unsafe` code, which `depguard` now denies. Step processes outlive a crashed runner, and commands are no longer looked up in the runner's working directory, both as with GitHub's runner.

Like GitHub's runner, host jobs on Linux and macOS now end the processes carrying the job's `RUNNER_TRACKING_ID`, and image pulls and Dockerfile action builds are retried.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1376
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 16:32:22 +00:00
silverwind 142d9f8c88 fix: give each Dockerfile action image its own ID (#1372)
Identical Dockerfile builds in concurrent jobs share one image ID under their per-job tags. When a job ends and removes its tag, the daemon deletes the image while another job's build is about to tag it, failing that job with `No such image`. This also hits `TestRunEvent` in CI, where two parallel subtests build the same action.

Label each build with its tag, so every job gets its own image on top of the shared cached layers and removing it never touches another job's.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1372
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 07:08:45 +00:00
silverwindandbircni 54b2cb944c fix: give git-lfs the checkout token once (#1371)
`builtin:checkout` set the token both in its persisted credentials file and in the step's environment. git-lfs before 3.8.0 then sent `Authorization` twice on the LFS batch request, which nginx rejects with 400.

Write the credentials file once the checkout succeeded, since it only serves later steps. A failed checkout no longer persists credentials.

Users can also upgrade git-lfs to fix the root bug, but this fix is worth to keep regardless.

Fixes: https://gitea.com/gitea/runner/issues/1370
Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1371
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-04 06:42:00 +00:00
Zettat123andsilverwind b65fd0f8f8 ci: run the Windows tests on a runner with KVM (#1369)
Run the Windows tests on `ubuntu-latest-kvm` as a PR job, with the installed Windows cached, minutes instead of ~1h40m. Aligned with https://gitea.com/gitea/gitea-mcp/pulls/321 and https://gitea.com/gitea/cli/pulls/79.

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1369
Reviewed-by: silverwind <[email protected]>
Co-authored-by: Zettat123 <[email protected]>
2026-11-04 02:08:38 +00:00
silverwind f2fbd67962 enhance: support Docker on FreeBSD and restructure the README (#1367)
1. Restructures the README into installation, quickstart, deployment, configuration, jobs and operation, and shortens it to user-facing prose. Option details are left to `config.example.yaml` and `docs/`, existing heading anchors are kept where possible.
2. FreeBSD builds used the Docker stub, so `docker://` labels, services and Docker actions failed there. The Docker code works on FreeBSD against a remote daemon, so it is enabled, and host jobs report `runner.os` as `FreeBSD`.
3. `make build` on Windows now writes `gitea-runner.exe`, taking the suffix from `go env GOEXE`.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1367
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-11-03 02:42:25 +00:00
silverwind d6f5b68a0d enhance: test on Windows and fix what it found (#1348)
Adds a weekly and on-demand workflow that runs the cross-compiled tests on Windows Server 2025 Core in a [dockur/windows](https://github.com/dockur/windows) QEMU VM, and fixes the Windows bugs it found in host and UNC paths, env handling, shell quoting, post-task scripts, health checks, checkout, action symlinks, step summaries, Git client certificates and docker mode from a Windows host.

Host-mode steps now emit UTF-8, `bash` falls back to Git's bash instead of the WSL launcher, the runner can run as a native Windows service, steps can keep background helpers with `RUNNER_TRACKING_ID`, and Windows arm64 binaries are released.

Fixes: https://gitea.com/gitea/runner/issues/1339
Fixes: https://gitea.com/gitea/runner/issues/637
Fixes: https://gitea.com/gitea/runner/issues/1172
Fixes: https://gitea.com/gitea/runner/issues/347
Reviewed-on: https://gitea.com/gitea/runner/pulls/1348
Reviewed-by: bircni <[email protected]>
2026-11-03 01:26:10 +00:00
196 changed files with 5138 additions and 3245 deletions

No files matched your search

+9 -1
View File
@@ -34,6 +34,13 @@ jobs:
with: with:
go-version-file: "go.mod" go-version-file: "go.mod"
check-latest: true check-latest: true
- name: Import GPG key
id: import_gpg
uses: crazy-max/ghaction-import-gpg@2dc316deee8e90f13e1a351ab510b4d5bc0c82cd # v7
with:
gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
passphrase: ${{ secrets.PASSPHRASE }}
fingerprint: CC64B1DB67ABBEECAB24B6455FC346329753F4B0
- name: goreleaser - name: goreleaser
uses: goreleaser/goreleaser-action@a4f02ab5355a589de60d917a659ce617ed19b3de # v7 uses: goreleaser/goreleaser-action@a4f02ab5355a589de60d917a659ce617ed19b3de # v7
with: with:
@@ -47,6 +54,7 @@ jobs:
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }} R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
GORELEASER_FORCE_TOKEN: "gitea" GORELEASER_FORCE_TOKEN: "gitea"
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GPG_FINGERPRINT: ${{ steps.import_gpg.outputs.fingerprint }}
release-image: release-image:
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -79,7 +87,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
- name: Login to DockerHub - name: Login to DockerHub
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4
with: with:
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}
+1 -1
View File
@@ -82,7 +82,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
- name: Login to DockerHub - name: Login to DockerHub
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4
with: with:
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}
+26
View File
@@ -57,6 +57,32 @@ jobs:
make coverage-report make coverage-report
cat .tmp/coverage.md >> "$GITHUB_STEP_SUMMARY" cat .tmp/coverage.md >> "$GITHUB_STEP_SUMMARY"
windows:
name: test on windows
runs-on: ubuntu-latest-kvm
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: ./.gitea/actions/go-setup
- name: prepare anonymous docker config
run: mkdir -p "$DOCKER_CONFIG" && echo '{}' > "$DOCKER_CONFIG/config.json"
- name: windows vm cache key
id: windows-vm
run: echo "key=windows-vm-$(grep '^WINDOWS_TEST_IMAGE' Makefile | cat - scripts/test-windows.sh scripts/test-windows-setup.ps1 | sha256sum | cut -c1-16)" >> "$GITHUB_OUTPUT"
- uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/windows-vm
key: ${{ steps.windows-vm.outputs.key }}
- name: test on windows
id: test-windows
run: WINDOWS_TEST_CACHE="$HOME/.cache/windows-vm" make test-windows
env:
WINDOWS_TEST_TIMEOUT: 45
- if: always() && steps.test-windows.outputs.save-cache == 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/windows-vm
key: ${{ steps.windows-vm.outputs.key }}
e2e: e2e:
name: gitea compatibility (${{ matrix.gitea_image }}) name: gitea compatibility (${{ matrix.gitea_image }})
runs-on: ubuntu-latest runs-on: ubuntu-latest
+1
View File
@@ -1,4 +1,5 @@
/gitea-runner /gitea-runner
/gitea-runner.exe
.env .env
!/act/runner/testdata/secrets/.env !/act/runner/testdata/secrets/.env
.runner .runner
+2
View File
@@ -39,6 +39,8 @@ linters:
deny: deny:
- pkg: golang.org/x/exp - pkg: golang.org/x/exp
desc: it's experimental and unreliable desc: it's experimental and unreliable
- pkg: unsafe
desc: use typed APIs instead
logging: logging:
files: files:
- $all - $all
+5 -2
View File
@@ -34,8 +34,6 @@ builds:
goarch: s390x goarch: s390x
- goos: windows - goos: windows
goarch: arm goarch: arm
- goos: windows
goarch: arm64
- goos: freebsd - goos: freebsd
goarch: s390x goarch: s390x
- goos: freebsd - goos: freebsd
@@ -82,6 +80,7 @@ builds:
publishers: publishers:
- name: cloudflare-r2 - name: cloudflare-r2
checksum: true checksum: true
signature: true
extra_files: extra_files:
- glob: ./**.xz - glob: ./**.xz
- glob: ./**.sha256 - glob: ./**.sha256
@@ -102,6 +101,10 @@ checksum:
extra_files: extra_files:
- glob: ./**.xz - glob: ./**.xz
signs:
- artifacts: checksum
args: ["--batch", "-u", "{{ .Env.GPG_FINGERPRINT }}", "--output", "${signature}", "--detach-sign", "${artifact}"]
snapshot: snapshot:
version_template: "{{ .Branch }}-devel" version_template: "{{ .Branch }}-devel"
+17 -9
View File
@@ -23,11 +23,14 @@ depending on the prefix:
covers both forms: it escapes `%` first, so a literal `%3B` becomes `%253B` that the extra covers both forms: it escapes `%` first, so a literal `%3B` becomes `%253B` that the extra
`##[…]` rules cannot match, and a raw `;` or `]` is never decoded. It is also what makes `##[…]` rules cannot match, and a raw `;` or `]` is never decoded. It is also what makes
multi-line work, `\n` becomes `%0A` and the UI turns it back into a line break. multi-line work, `\n` becomes `%0A` and the UI turns it back into a line break.
- **Forwarding a command from step output?** Leave the payload alone, it arrived escaped and is - **Forwarding a command from step output?** Preserve its escaping unless masking changes it.
decoded once. Decoding here double-decodes and destroys multi-line. Decoding for display happens in the UI.
- **No prefix?** Do not escape, and split multi-line values into one row each. - **No prefix?** Do not escape, and split multi-line values into one row each.
- **Interpolating a secret?** Masking runs after escaping, so `AppendSecretMasker` registers the - **Interpolating a secret?** Masking runs after escaping. Recognised command payloads are
encoded forms too. normalised before masking to cover partially escaped values. Their original escaping is
preserved when no replacement occurs. `AppendSecretMasker` also registers encoded forms for
other output. Complete nonblank one-character values are masked, while one-byte fragments
of multiline values are skipped.
- Command *properties* also escape `%3A` and `%2C`, which the UI never decodes, so the command - Command *properties* also escape `%3A` and `%2C`, which the UI never decodes, so the command
handler folds an annotation's location into its decoded message and escapes the result again. handler folds an annotation's location into its decoded message and escapes the result again.
@@ -48,11 +51,6 @@ fields.
- Tests read what was logged with `logtest.NewLogger()`, or `logtest.SetDefault(t)` for the - Tests read what was logged with `logtest.NewLogger()`, or `logtest.SetDefault(t)` for the
package functions. package functions.
## Built-in actions
Each lives in `internal/pkg/action/<name>` and is registered in `builtinActions` in
`act/runner/step_action_builtin.go`.
## End-to-end compatibility tests ## End-to-end compatibility tests
`make test-e2e` runs the runner against `E2E_GITEA_IMAGE`. It defaults to the nightly image. `make test-e2e` runs the runner against `E2E_GITEA_IMAGE`. It defaults to the nightly image.
@@ -69,3 +67,13 @@ Docker and Kubernetes.
The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated
repository runners. A run path is `<workflow>@<ref>` and a log row is `<timestamp>Z <payload>`. repository runners. A run path is `<workflow>@<ref>` and a log row is `<timestamp>Z <payload>`.
## Windows tests
`make test-windows` cross-compiles the test binaries, installs Windows Server 2025 Core in a
[dockur/windows](https://github.com/dockur/windows) QEMU VM on the docker host and runs them there.
The host must be amd64 and allow privileged containers. `WINDOWS_TEST_TIMEOUT` sets the minutes
allowed for installing Windows and running the tests, 165 by default. The VM uses KVM when the host
has `/dev/kvm`, otherwise Windows is emulated and much slower. Docker-backed tests skip in the VM
and the race detector is off, it needs cgo. `WINDOWS_TEST_CACHE` names a directory to keep the
installed Windows in, so later runs skip the install. CI runs this on every pull request.
+7 -5
View File
@@ -19,14 +19,15 @@ RUN --mount=type=cache,target=/go/pkg/mod \
### DIND VARIANT ### DIND VARIANT
# #
# #
FROM docker:29.9.2-dind AS dind FROM docker:29.9.3-dind AS dind
ARG VERSION=dev ARG VERSION=dev
LABEL org.opencontainers.image.source="https://gitea.com/gitea/runner" LABEL org.opencontainers.image.source="https://gitea.com/gitea/runner"
LABEL org.opencontainers.image.version="${VERSION}" LABEL org.opencontainers.image.version="${VERSION}"
RUN apk add --no-cache s6 bash git tzdata nftables # upgrade first: the docker base image can lag behind Alpine security fixes
RUN apk upgrade --no-cache && apk add --no-cache s6 bash git tzdata nftables
COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner
COPY scripts/run.sh /usr/local/bin/run.sh COPY scripts/run.sh /usr/local/bin/run.sh
@@ -41,7 +42,7 @@ ENTRYPOINT ["s6-svscan","/etc/s6"]
### DIND-ROOTLESS VARIANT ### DIND-ROOTLESS VARIANT
# #
# #
FROM docker:29.9.2-dind-rootless AS dind-rootless FROM docker:29.9.3-dind-rootless AS dind-rootless
ARG VERSION=dev ARG VERSION=dev
@@ -49,7 +50,8 @@ LABEL org.opencontainers.image.source="https://gitea.com/gitea/runner"
LABEL org.opencontainers.image.version="${VERSION}" LABEL org.opencontainers.image.version="${VERSION}"
USER root USER root
RUN apk add --no-cache s6 bash git tzdata nftables # upgrade first: the docker base image can lag behind Alpine security fixes
RUN apk upgrade --no-cache && apk add --no-cache s6 bash git tzdata nftables
COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner
COPY scripts/run.sh /usr/local/bin/run.sh COPY scripts/run.sh /usr/local/bin/run.sh
@@ -75,7 +77,7 @@ ARG VERSION=dev
LABEL org.opencontainers.image.source="https://gitea.com/gitea/runner" LABEL org.opencontainers.image.source="https://gitea.com/gitea/runner"
LABEL org.opencontainers.image.version="${VERSION}" LABEL org.opencontainers.image.version="${VERSION}"
RUN apk add --no-cache tini bash git tzdata RUN apk upgrade --no-cache && apk add --no-cache tini bash git tzdata
COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner COPY --from=builder /opt/src/runner/gitea-runner /usr/local/bin/gitea-runner
COPY scripts/run.sh /usr/local/bin/run.sh COPY scripts/run.sh /usr/local/bin/run.sh
+9 -4
View File
@@ -1,13 +1,14 @@
DIST := dist DIST := dist
EXECUTABLE := gitea-runner
GO ?= go GO ?= go
EXECUTABLE := gitea-runner$(shell $(GO) env GOEXE)
HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" ) HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" )
PODMAN_TEST_IMAGE ?= quay.io/containers/podman:v5.8.9-immutable@sha256:87f42f37fa85b421664c30b8f5f9cfc23920057f5e5fcadf0b6f68d9edc6d415 # renovate: datasource=docker PODMAN_TEST_IMAGE ?= quay.io/containers/podman:v6.1.3-immutable@sha256:1d844fd1fc6ce68c1e72ae19550cc3caef039306219ee007d88f7e67c5df4dd1 # renovate: datasource=docker
E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:657378e5cfd898e36f820459c60ef8e69be8fcda9fd354e4381a8b654d091bbc # renovate: datasource=docker E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:139605d4a6ab61e48dfd7687b11184881c2d7fcf06bf739e9cadb34f73deb51d # renovate: datasource=docker
SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker
E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly
E2E_K3S_IMAGE ?= rancher/k3s:v1.37.1-k3s1@sha256:ca7f37d993d82ef0dcdcfecb2e0e2618ea541dbaffc620c8cedebe01a82acd0d # renovate: datasource=docker E2E_K3S_IMAGE ?= rancher/k3s:v1.37.2-k3s1@sha256:21ea6e0fac4e3930c2ce8aa40ddd41a8b18cb3e9a0222c030dc005e7e88c4332 # renovate: datasource=docker
WINDOWS_TEST_IMAGE ?= dockurr/windows:6.06@sha256:16fd7c39a3e48bb946d29c59dc6015942e5b900a1fa5d7e5552053bfc48ee2ea # renovate: datasource=docker
GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*") GOFILES := $(shell find . -type f -name "*.go" -o -name "go.mod" ! -name "generated.*")
@@ -150,6 +151,10 @@ coverage-report: ## turn coverage.txt from `make test` into .tmp/coverage.md
test-dind: ## run the daemon-facing tests against the built dind image or podman (TARGET=dind|dind-rootless|podman) test-dind: ## run the daemon-facing tests against the built dind image or podman (TARGET=dind|dind-rootless|podman)
@PODMAN_TEST_IMAGE=$(PODMAN_TEST_IMAGE) ./scripts/test-dind.sh $(TARGET) @PODMAN_TEST_IMAGE=$(PODMAN_TEST_IMAGE) ./scripts/test-dind.sh $(TARGET)
.PHONY: test-windows
test-windows: ## run the tests on Windows Server Core in a QEMU VM on the docker host
@GO=$(GO) WINDOWS_TEST_IMAGE=$(WINDOWS_TEST_IMAGE) ./scripts/test-windows.sh
E2E_CONCURRENCY ?= 8 E2E_CONCURRENCY ?= 8
.PHONY: test-e2e .PHONY: test-e2e
+142 -220
View File
File diff suppressed because it is too large. Load diff
+88 -32
View File
@@ -62,6 +62,7 @@ type JobCredential struct {
Ref string `json:"ref"` Ref string `json:"ref"`
BaseRef string `json:"base_ref"` BaseRef string `json:"base_ref"`
DefaultRef string `json:"default_ref"` DefaultRef string `json:"default_ref"`
CacheMode string `json:"cache_mode"` // empty allows everything
// Results is the instance whose artifact service this server forwards for the job, and // Results is the instance whose artifact service this server forwards for the job, and
// InsecureTLS how the runner reaches it; see results.go. // InsecureTLS how the runner reaches it; see results.go.
@@ -104,7 +105,10 @@ type Handler struct {
stopSync context.CancelFunc stopSync context.CancelFunc
syncing sync.WaitGroup syncing sync.WaitGroup
outboundIP string outboundIPMu sync.RWMutex
outboundIP string
detectOutboundIP func() net.IP // nil when the address is configured
instanceTransport http.RoundTripper
// internalSecret guards /_internal/{register,revoke}. When set, a remote // internalSecret guards /_internal/{register,revoke}. When set, a remote
// runner can use these endpoints to pre-register per-job // runner can use these endpoints to pre-register per-job
@@ -124,7 +128,7 @@ type Handler struct {
policy Policy policy Policy
// freeDisk is a field so tests can drive evictForFreeSpace without a full volume. // freeDisk is a field so tests can drive makeRoom without a full volume.
freeDisk func(string) (uint64, error) freeDisk func(string) (uint64, error)
} }
@@ -143,19 +147,22 @@ type Options struct {
// in-process RegisterJob method directly. // in-process RegisterJob method directly.
InternalSecret string InternalSecret string
Policy Policy Policy Policy
Logger *logging.Logger Logger *logging.Logger
InstanceTransport http.RoundTripper // reaches the instance artifact requests are forwarded to, nil for http.DefaultTransport
} }
// StartHandler starts an embedded cache or forwards cache requests to Upstream. // StartHandler starts an embedded cache or forwards cache requests to Upstream.
func StartHandler(opts Options) (*Handler, error) { func StartHandler(opts Options) (*Handler, error) {
dir, logger := opts.Dir, opts.Logger dir, logger := opts.Dir, opts.Logger
h := &Handler{ h := &Handler{
creds: make(map[string]*credEntry), creds: make(map[string]*credEntry),
internalSecret: opts.InternalSecret, internalSecret: opts.InternalSecret,
policy: opts.Policy.withDefaults(), policy: opts.Policy.withDefaults(),
freeDisk: disk.FreeBytes, freeDisk: disk.FreeBytes,
dbLock: make(chan struct{}, 1), dbLock: make(chan struct{}, 1),
outboundIP: opts.OutboundIP,
instanceTransport: opts.InstanceTransport,
} }
if logger == nil { if logger == nil {
@@ -164,12 +171,11 @@ func StartHandler(opts Options) (*Handler, error) {
logger = logger.With("module", "artifactcache") logger = logger.With("module", "artifactcache")
h.logger = logger h.logger = logger
if opts.OutboundIP != "" { if opts.OutboundIP == "" {
h.outboundIP = opts.OutboundIP h.detectOutboundIP = common.GetOutboundIP
} else if ip := common.GetOutboundIP(); ip == nil { if h.RefreshExternalURL() == "" {
return nil, errors.New("unable to determine outbound IP address") logger.Warn("unable to determine an outbound IP address for the cache server yet, set cache.host to configure one")
} else { }
h.outboundIP = ip.String()
} }
if opts.Upstream != "" { if opts.Upstream != "" {
@@ -213,10 +219,10 @@ func StartHandler(opts Options) (*Handler, error) {
} }
router := http.NewServeMux() router := http.NewServeMux()
router.HandleFunc("GET "+apiPath+"/cache", h.bearerAuth(h.find)) router.HandleFunc("GET "+apiPath+"/cache", h.cacheModeAuth(false, h.find))
router.HandleFunc("POST "+apiPath+"/caches", h.bearerAuth(h.reserve)) router.HandleFunc("POST "+apiPath+"/caches", h.cacheModeAuth(true, h.reserve))
router.HandleFunc("PATCH "+apiPath+"/caches/{id}", h.bearerAuth(h.upload)) router.HandleFunc("PATCH "+apiPath+"/caches/{id}", h.cacheModeAuth(true, h.upload))
router.HandleFunc("POST "+apiPath+"/caches/{id}", h.bearerAuth(h.commit)) router.HandleFunc("POST "+apiPath+"/caches/{id}", h.cacheModeAuth(true, h.commit))
router.HandleFunc("POST "+apiPath+"/clean", h.bearerAuth(h.clean)) router.HandleFunc("POST "+apiPath+"/clean", h.bearerAuth(h.clean))
// Artifact GET is signed via query-string HMAC because @actions/cache // Artifact GET is signed via query-string HMAC because @actions/cache
// does not attach Authorization when downloading archiveLocation. // does not attach Authorization when downloading archiveLocation.
@@ -301,11 +307,36 @@ func (h *Handler) cacheForwarder(upstream string) (http.Handler, error) {
}), nil }), nil
} }
// ExternalURL is "" while no address is known.
func (h *Handler) ExternalURL() string { func (h *Handler) ExternalURL() string {
// TODO: make the external url configurable if necessary if h == nil {
return ""
}
h.outboundIPMu.RLock()
defer h.outboundIPMu.RUnlock()
if h.outboundIP == "" {
return ""
}
return "http://" + net.JoinHostPort(h.outboundIP, strconv.Itoa(h.port)) return "http://" + net.JoinHostPort(h.outboundIP, strconv.Itoa(h.port))
} }
// RefreshExternalURL re-detects an unconfigured address, which can change after startup.
func (h *Handler) RefreshExternalURL() string {
if h != nil && h.detectOutboundIP != nil {
if ip := h.detectOutboundIP(); ip != nil {
h.outboundIPMu.Lock()
if address := ip.String(); address != h.outboundIP {
h.outboundIP = address
if ip.To4() == nil {
h.logger.Warnf("detected IPv6 address %s for the cache server, jobs on IPv4-only networks cannot reach it, set cache.host if they fail to", address)
}
}
h.outboundIPMu.Unlock()
}
}
return h.ExternalURL()
}
func (h *Handler) baseURL(cred JobCredential) string { func (h *Handler) baseURL(cred JobCredential) string {
if base := strings.TrimRight(cred.PublicURL, "/"); base != "" { if base := strings.TrimRight(cred.PublicURL, "/"); base != "" {
return base return base
@@ -773,6 +804,7 @@ func (h *Handler) upload(w http.ResponseWriter, r *http.Request) {
h.responseJSON(w, r, 400, err) h.responseJSON(w, r, 400, err)
return return
} }
h.makeRoom(r.ContentLength)
if err := h.storage.Write(cache.ID, start, r.Body); err != nil { if err := h.storage.Write(cache.ID, start, r.Body); err != nil {
h.responseJSON(w, r, 500, err) h.responseJSON(w, r, 500, err)
return return
@@ -875,11 +907,8 @@ func (h *Handler) commitCache(cache *Cache) error {
if h.s3 != nil { if h.s3 != nil {
return nil // lifecycle rules on the bucket bound its size return nil // lifecycle rules on the bucket bound its size
} }
// A commit is the only thing that grows the store, so the only thing that can push the
// volume under the floor.
h.evictRepo(db, cache.Repo) h.evictRepo(db, cache.Repo)
h.evictTotal(db) h.evictTotal(db)
h.evictForFreeSpace(db)
return nil return nil
} }
@@ -928,6 +957,30 @@ func (h *Handler) bearerAuth(handler http.HandlerFunc) http.HandlerFunc {
} }
} }
// cacheModeAuth is bearerAuth refusing what the job's cache-mode forbids, in the shapes the toolkit reports as a warning.
func (h *Handler) cacheModeAuth(write bool, handler http.HandlerFunc) http.HandlerFunc {
return h.bearerAuth(func(w http.ResponseWriter, r *http.Request) {
mode := credFromContext(r.Context()).CacheMode
access, forbidden := "read", "write-only"
if write {
access, forbidden = "write", "read"
}
if mode != "none" && mode != forbidden {
handler(w, r)
return
}
message := fmt.Sprintf("cache %s denied: the job's cache-mode is %s", access, mode)
switch {
case !strings.HasPrefix(r.URL.Path, cacheServiceV2Path):
h.responseJSON(w, r, http.StatusForbidden, map[string]any{"message": message})
case write:
h.responseJSON(w, r, http.StatusOK, map[string]any{"ok": false, "message": message})
default:
h.twirpError(w, r, twirpDenied, errors.New(message))
}
})
}
func (h *Handler) unauthorized(w http.ResponseWriter, r *http.Request, err error) { func (h *Handler) unauthorized(w http.ResponseWriter, r *http.Request, err error) {
if strings.HasPrefix(r.URL.Path, cacheServiceV2Path) { if strings.HasPrefix(r.URL.Path, cacheServiceV2Path) {
h.twirpError(w, r, twirpUnauthenticated, err) h.twirpError(w, r, twirpUnauthenticated, err)
@@ -1173,9 +1226,6 @@ func (h *Handler) touchCache(id uint64, requireIncomplete bool) error {
defer db.Close() defer db.Close()
cache, err := db.get(id) cache, err := db.get(id)
if errors.Is(err, errCacheNotFound) {
return nil
}
if err != nil { if err != nil {
return err return err
} }
@@ -1260,23 +1310,29 @@ func (h *Handler) gcCache() {
h.evictExpired(db) h.evictExpired(db)
h.evictSuperseded(db) h.evictSuperseded(db)
h.evictOversized(db) h.evictOversized(db)
h.evictForFreeSpace(db) _ = db.Close()
h.makeRoom(0)
} }
// evictForFreeSpace bounds the volume itself, so it also covers bytes the cache never // makeRoom evicts the least recently used entries to keep need bytes above the floor, as GitHub saves new over old.
// accounted for. func (h *Handler) makeRoom(need int64) {
func (h *Handler) evictForFreeSpace(db *cacheDB) {
free, err := h.freeDisk(h.dir) free, err := h.freeDisk(h.dir)
if err != nil { if err != nil {
h.logger.Debugf("free disk check: %v", err) // unsupported platform, treat as unavailable rather than full h.logger.Debugf("free disk check: %v", err) // unsupported platform, treat as unavailable rather than full
return return
} }
if free >= uint64(h.policy.MinFreeDisk) { shortfall := h.policy.MinFreeDisk + max(need, 0) - int64(free)
if h.s3 != nil || shortfall <= 0 {
return return
} }
db, err := h.openDB()
if err != nil {
return
}
defer db.Close()
caches := h.completedByUse(db) caches := h.completedByUse(db)
total, shortfall := totalSize(caches), h.policy.MinFreeDisk-int64(free) total := totalSize(caches)
if total <= shortfall { if total <= shortfall {
// Say so, or shedding everything and still being short reads as the backstop working. // Say so, or shedding everything and still being short reads as the backstop working.
h.logger.Warnf("cache volume is %d MiB short of the free space floor with only %d MiB of cache on it; something else is filling it", shortfall/miB, total/miB) h.logger.Warnf("cache volume is %d MiB short of the free space floor with only %d MiB of cache on it; something else is filling it", shortfall/miB, total/miB)
+72 -23
View File
@@ -11,6 +11,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"net"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
@@ -18,9 +19,12 @@ import (
"strconv" "strconv"
"strings" "strings"
"testing" "testing"
"testing/iotest"
"testing/synctest" "testing/synctest"
"time" "time"
"gitea.com/gitea/runner/act/common/logging/logtest"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.etcd.io/bbolt" "go.etcd.io/bbolt"
@@ -49,7 +53,23 @@ func signArtifactURL(h *Handler, id int64) string {
func TestHandler_ExternalURL(t *testing.T) { func TestHandler_ExternalURL(t *testing.T) {
t.Parallel() t.Parallel()
assert.Equal(t, "http://[2001:db8::1]:8080", (&Handler{outboundIP: "2001:db8::1", port: 8080}).ExternalURL()) assert.Empty(t, (*Handler)(nil).RefreshExternalURL())
assert.Equal(t, "http://cache.example:8080", (&Handler{outboundIP: "cache.example", port: 8080}).RefreshExternalURL())
logger, logs := logtest.NewLogger()
var ip net.IP
handler := &Handler{port: 8080, logger: logger, detectOutboundIP: func() net.IP { return ip }}
assert.Empty(t, handler.RefreshExternalURL())
ip = net.ParseIP("2001:db8::1")
assert.Equal(t, "http://[2001:db8::1]:8080", handler.RefreshExternalURL())
assert.Equal(t, "http://[2001:db8::1]:8080", handler.RefreshExternalURL())
ip = net.ParseIP("192.0.2.1")
assert.Equal(t, "http://[2001:db8::1]:8080", handler.ExternalURL())
assert.Equal(t, "http://192.0.2.1:8080", handler.RefreshExternalURL())
ip = nil
assert.Equal(t, "http://192.0.2.1:8080", handler.RefreshExternalURL())
require.Len(t, logs.AllRecords(), 1)
assert.Contains(t, logs.LastRecord().Message, "IPv6")
} }
func TestHandler_Upstream(t *testing.T) { func TestHandler_Upstream(t *testing.T) {
@@ -673,7 +693,7 @@ func TestHandler_evictPolicy(t *testing.T) {
} }
} }
func TestHandler_evictForFreeSpace(t *testing.T) { func TestHandler_makeRoom(t *testing.T) {
t.Parallel() t.Parallel()
free := func(n int64) func(string) (uint64, error) { free := func(n int64) func(string) (uint64, error) {
return func(string) (uint64, error) { return uint64(n), nil } return func(string) (uint64, error) { return uint64(n), nil }
@@ -682,14 +702,16 @@ func TestHandler_evictForFreeSpace(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
name string name string
freeDisk func(string) (uint64, error) freeDisk func(string) (uint64, error)
upload int64
kept []string kept []string
}{ }{
{"ample free space evicts nothing", free(defaultMinFreeDisk), []string{"oldest", "middle", "newest"}}, {"ample free space evicts nothing", free(defaultMinFreeDisk), 0, []string{"oldest", "middle", "newest"}},
{"a small shortfall sheds one entry", free(defaultMinFreeDisk - 4*miB), []string{"middle", "newest"}}, {"room for an upload sheds the oldest entry", free(defaultMinFreeDisk), 4 * miB, []string{"middle", "newest"}},
{"a shortfall the cache cannot cover sheds all of it", free(0), nil}, {"a shortfall the cache cannot cover sheds all of it", free(0), 0, nil},
{ {
"an unreadable volume is treated as unavailable, not as full", "an unreadable volume is treated as unavailable, not as full",
func(string) (uint64, error) { return 0, errors.New("unsupported") }, func(string) (uint64, error) { return 0, errors.New("unsupported") },
0,
[]string{"oldest", "middle", "newest"}, []string{"oldest", "middle", "newest"},
}, },
} { } {
@@ -702,11 +724,7 @@ func TestHandler_evictForFreeSpace(t *testing.T) {
} }
handler := newTestHandler(t, Policy{}, entries...) handler := newTestHandler(t, Policy{}, entries...)
handler.freeDisk = tc.freeDisk handler.freeDisk = tc.freeDisk
handler.makeRoom(tc.upload)
db, err := handler.openDB()
require.NoError(t, err)
handler.evictForFreeSpace(db)
require.NoError(t, db.Close())
assert.ElementsMatch(t, tc.kept, keptKeys(t, handler, entries)) assert.ElementsMatch(t, tc.kept, keptKeys(t, handler, entries))
}) })
@@ -852,6 +870,9 @@ func TestHandler_touchCacheSkipsFreshCompletedEntries(t *testing.T) {
func newTestHandler(t *testing.T, policy Policy, entries ...*Cache) *Handler { func newTestHandler(t *testing.T, policy Policy, entries ...*Cache) *Handler {
t.Helper() t.Helper()
handler := startTestHandler(t, Options{Policy: policy}) handler := startTestHandler(t, Options{Policy: policy})
if len(entries) == 0 {
return handler
}
db, err := handler.openDB() db, err := handler.openDB()
require.NoError(t, err) require.NoError(t, err)
@@ -888,6 +909,13 @@ func keptKeys(t *testing.T, handler *Handler, entries []*Cache) []string {
return kept return kept
} }
func TestStorageDropsFailedWrites(t *testing.T) {
storage, err := NewStorage(t.TempDir())
require.NoError(t, err)
require.Error(t, storage.Write(1, 0, io.MultiReader(strings.NewReader("partial"), iotest.ErrReader(io.ErrUnexpectedEOF))))
assert.NoFileExists(t, storage.tempName(1, 0))
}
func writeBlob(t *testing.T, handler *Handler, id uint64) { func writeBlob(t *testing.T, handler *Handler, id uint64) {
t.Helper() t.Helper()
require.NoError(t, handler.storage.Write(id, 0, strings.NewReader("a"))) require.NoError(t, handler.storage.Write(id, 0, strings.NewReader("a")))
@@ -934,6 +962,10 @@ func TestHandler_RejectsBearer(t *testing.T) {
func TestHandler_UnregisterRevokes(t *testing.T) { func TestHandler_UnregisterRevokes(t *testing.T) {
t.Parallel() t.Parallel()
handler, err := StartHandler(Options{Dir: filepath.Join(t.TempDir(), "artifactcache")})
require.NoError(t, err)
defer handler.Close()
for _, tc := range []struct { for _, tc := range []struct {
name string name string
token string token string
@@ -944,9 +976,6 @@ func TestHandler_UnregisterRevokes(t *testing.T) {
{"reference counted", "shared", "v", 2}, {"reference counted", "shared", "v", 2},
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
handler, err := StartHandler(Options{Dir: filepath.Join(t.TempDir(), "artifactcache")})
require.NoError(t, err)
defer handler.Close()
revokers := make([]func(), tc.registrations) revokers := make([]func(), tc.registrations)
for i := range revokers { for i := range revokers {
revokers[i] = handler.RegisterJob(tc.token, JobCredential{Repo: testRepo}) revokers[i] = handler.RegisterJob(tc.token, JobCredential{Repo: testRepo})
@@ -974,7 +1003,7 @@ func TestHandler_UnregisterRevokes(t *testing.T) {
} }
} }
func TestHandler_CrossRepoIsolation(t *testing.T) { func TestHandler_CrossRepoAndCacheModeIsolation(t *testing.T) {
t.Parallel() t.Parallel()
dir := filepath.Join(t.TempDir(), "artifactcache") dir := filepath.Join(t.TempDir(), "artifactcache")
handler, err := StartHandler(Options{Dir: dir}) handler, err := StartHandler(Options{Dir: dir})
@@ -983,6 +1012,8 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"}) handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"}) handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"})
handler.RegisterJob("token-a-pull", JobCredential{Repo: "owner/repoA", Ref: "refs/pull/1/head"}) handler.RegisterJob("token-a-pull", JobCredential{Repo: "owner/repoA", Ref: "refs/pull/1/head"})
handler.RegisterJob("token-a-read", JobCredential{Repo: "owner/repoA", CacheMode: "read"})
handler.RegisterJob("token-a-write-only", JobCredential{Repo: "owner/repoA", CacheMode: "write-only"})
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
key := "shared-key" key := "shared-key"
@@ -1012,12 +1043,31 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
resp.Body.Close() resp.Body.Close()
assert.Equal(t, http.StatusNoContent, resp.StatusCode) assert.Equal(t, http.StatusNoContent, resp.StatusCode)
resp, err = clientA.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version)) clientRead := &http.Client{Transport: &bearerTransport{token: "token-a-read"}}
require.NoError(t, err) for _, client := range []*http.Client{clientA, clientRead} {
resp.Body.Close() resp, err = client.Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version))
assert.Equal(t, http.StatusOK, resp.StatusCode) require.NoError(t, err)
resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
}
for _, client := range []*http.Client{clientB, {Transport: &bearerTransport{token: "token-a-pull"}}} { requireDenied := func(prefix string, do func() (*http.Response, error)) {
resp, err := do()
require.NoError(t, err)
defer resp.Body.Close()
denied := map[string]string{}
require.NoError(t, json.UnmarshalRead(resp.Body, &denied))
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
assert.True(t, strings.HasPrefix(denied["message"], prefix), denied["message"])
}
requireDenied("cache write denied:", func() (*http.Response, error) {
return clientRead.Post(base+"/caches", "application/json", strings.NewReader(`{"key":"new-key","version":"`+version+`"}`))
})
requireDenied("cache read denied:", func() (*http.Response, error) {
return (&http.Client{Transport: &bearerTransport{token: "token-a-write-only"}}).Get(fmt.Sprintf("%s/cache?keys=%s&version=%s", base, key, version))
})
for _, client := range []*http.Client{clientB, {Transport: &bearerTransport{token: "token-a-pull"}}, clientRead} {
req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison"))) req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison")))
require.NoError(t, err) require.NoError(t, err)
req.Header.Set("Content-Range", "bytes 0-5/*") req.Header.Set("Content-Range", "bytes 0-5/*")
@@ -1043,9 +1093,8 @@ func TestHandler_ArtifactSignature(t *testing.T) {
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
good := signArtifactURL(handler, 1) good := signArtifactURL(handler, 1)
other, err := StartHandler(Options{Dir: filepath.Join(t.TempDir(), "artifactcache2")}) secret := bytes.Clone(handler.secret)
require.NoError(t, err) secret[0] ^= 1
defer other.Close()
for _, tc := range []struct { for _, tc := range []struct {
name string name string
@@ -1054,7 +1103,7 @@ func TestHandler_ArtifactSignature(t *testing.T) {
{"missing signature", fmt.Sprintf("%s/artifacts/%d", base, 1)}, {"missing signature", fmt.Sprintf("%s/artifacts/%d", base, 1)},
{"tampered signature", good[:len(good)-4] + "dead"}, {"tampered signature", good[:len(good)-4] + "dead"},
{"expired signature", handler.signedArtifactURL(JobCredential{}, 1, time.Now().Add(-time.Second))}, {"expired signature", handler.signedArtifactURL(JobCredential{}, 1, time.Now().Add(-time.Second))},
{"signature from a different server", base + strings.SplitN(signArtifactURL(other, 1), apiPath, 2)[1]}, {"signature from a different server", base + strings.SplitN(signArtifactURL(&Handler{secret: secret}, 1), apiPath, 2)[1]},
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
resp, err := testClient.Get(tc.url) resp, err := testClient.Get(tc.url)
+7 -3
View File
@@ -40,12 +40,13 @@ const (
twirpInternal = "internal" twirpInternal = "internal"
twirpUnavailable = "unavailable" twirpUnavailable = "unavailable"
twirpUnauthenticated = "unauthenticated" twirpUnauthenticated = "unauthenticated"
twirpDenied = "permission_denied"
) )
func (h *Handler) registerV2Routes(router *http.ServeMux) { func (h *Handler) registerV2Routes(router *http.ServeMux) {
router.HandleFunc("POST "+cacheServiceV2Path+"/CreateCacheEntry", h.bearerAuth(h.v2CreateCacheEntry)) router.HandleFunc("POST "+cacheServiceV2Path+"/CreateCacheEntry", h.cacheModeAuth(true, h.v2CreateCacheEntry))
router.HandleFunc("POST "+cacheServiceV2Path+"/FinalizeCacheEntryUpload", h.bearerAuth(h.v2FinalizeCacheEntryUpload)) router.HandleFunc("POST "+cacheServiceV2Path+"/FinalizeCacheEntryUpload", h.cacheModeAuth(true, h.v2FinalizeCacheEntryUpload))
router.HandleFunc("POST "+cacheServiceV2Path+"/GetCacheEntryDownloadURL", h.bearerAuth(h.v2GetCacheEntryDownloadURL)) router.HandleFunc("POST "+cacheServiceV2Path+"/GetCacheEntryDownloadURL", h.cacheModeAuth(false, h.v2GetCacheEntryDownloadURL))
router.HandleFunc("PUT "+blobPath+"/{id}", h.signedAuth(blobUploadPurpose, h.v2UploadBlob)) router.HandleFunc("PUT "+blobPath+"/{id}", h.signedAuth(blobUploadPurpose, h.v2UploadBlob))
} }
@@ -218,6 +219,7 @@ func (h *Handler) v2UploadBlob(w http.ResponseWriter, r *http.Request) {
h.responseJSON(w, r, http.StatusBadRequest, err) h.responseJSON(w, r, http.StatusBadRequest, err)
return return
} }
h.makeRoom(r.ContentLength)
query := r.URL.Query() query := r.URL.Query()
switch strings.ToLower(query.Get("comp")) { switch strings.ToLower(query.Get("comp")) {
@@ -266,6 +268,8 @@ func (h *Handler) twirpError(w http.ResponseWriter, r *http.Request, code string
status = http.StatusServiceUnavailable status = http.StatusServiceUnavailable
case twirpUnauthenticated: case twirpUnauthenticated:
status = http.StatusUnauthorized status = http.StatusUnauthorized
case twirpDenied:
status = http.StatusForbidden
} }
h.responseJSON(w, r, status, map[string]any{"code": code, "msg": err.Error()}) h.responseJSON(w, r, status, map[string]any{"code": code, "msg": err.Error()})
} }
+27
View File
@@ -14,6 +14,7 @@ import (
"strconv" "strconv"
"strings" "strings"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -84,6 +85,8 @@ func TestCacheServiceV2RoundTrip(t *testing.T) {
unsigned := fmt.Sprintf("%s%s/1", handler.ExternalURL(), blobPath) unsigned := fmt.Sprintf("%s%s/1", handler.ExternalURL(), blobPath)
assert.Equal(t, http.StatusUnauthorized, putBlob(t, unsigned, content)) assert.Equal(t, http.StatusUnauthorized, putBlob(t, unsigned, content))
unreserved := handler.signedURL(JobCredential{Repo: testRepo}, blobPath, blobUploadPurpose, 1, time.Now().Add(time.Minute))
assert.Equal(t, http.StatusBadRequest, putBlob(t, unreserved, content))
finalized, uploadURL := saveV2(t, handler, "deps-v1", "abc123", content) finalized, uploadURL := saveV2(t, handler, "deps-v1", "abc123", content)
require.Equal(t, true, finalized["ok"]) require.Equal(t, true, finalized["ok"])
@@ -224,6 +227,30 @@ func TestCacheServiceV2Lookups(t *testing.T) {
assert.NotEmpty(t, got["archiveLocation"]) assert.NotEmpty(t, got["archiveLocation"])
}) })
t.Run("a read-only job only restores and a write-only one only saves", func(t *testing.T) {
handler.RegisterJob("read-only", JobCredential{Repo: testRepo, CacheMode: "read"})
readOnlyClient := &http.Client{Transport: &bearerTransport{token: "read-only"}}
assert.Equal(t, true, v2Call(t, handler, readOnlyClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})["ok"])
for _, method := range []string{"CreateCacheEntry", "FinalizeCacheEntryUpload"} {
got := v2Call(t, handler, readOnlyClient, method, map[string]any{"key": "deps-new", "version": "v1", "size_bytes": 1})
assert.Equal(t, false, got["ok"], method)
assert.True(t, strings.HasPrefix(got["message"].(string), "cache write denied:"), method)
}
handler.RegisterJob("write-only", JobCredential{Repo: testRepo, CacheMode: "write-only"})
writeOnlyClient := &http.Client{Transport: &bearerTransport{token: "write-only"}}
assert.Equal(t, true, v2Call(t, handler, writeOnlyClient, "CreateCacheEntry", map[string]any{"key": "deps-write-only", "version": "v1"})["ok"])
resp, err := writeOnlyClient.Post(handler.ExternalURL()+cacheServiceV2Path+"/GetCacheEntryDownloadURL", "application/json", strings.NewReader(`{"key":"deps-abc","version":"v1"}`))
require.NoError(t, err)
defer resp.Body.Close()
got := map[string]string{}
require.NoError(t, json.UnmarshalRead(resp.Body, &got))
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
assert.Equal(t, twirpDenied, got["code"])
assert.True(t, strings.HasPrefix(got["msg"], "cache read denied:"), got["msg"])
})
t.Run("another repository sees nothing", func(t *testing.T) { t.Run("another repository sees nothing", func(t *testing.T) {
handler.RegisterJob("other-runtime-token", JobCredential{Repo: "other/repo"}) handler.RegisterJob("other-runtime-token", JobCredential{Repo: "other/repo"})
otherClient := &http.Client{Transport: &bearerTransport{token: "other-runtime-token"}} otherClient := &http.Client{Transport: &bearerTransport{token: "other-runtime-token"}}
+1 -1
View File
@@ -67,7 +67,7 @@ func (h *Handler) forwardOrNotFound(w http.ResponseWriter, r *http.Request) {
h.twirpError(w, r, twirpInternal, fmt.Errorf("cache server cannot reach the artifact service at %s: %w", target, err)) h.twirpError(w, r, twirpInternal, fmt.Errorf("cache server cannot reach the artifact service at %s: %w", target, err))
}, },
} }
if cred.InsecureTLS { if proxy.Transport = h.instanceTransport; proxy.Transport == nil && cred.InsecureTLS { // the instance transport carries runner.insecure and the client cert
proxy.Transport = insecureTransport proxy.Transport = insecureTransport
} }
proxy.ServeHTTP(w, r) proxy.ServeHTTP(w, r)
+17
View File
@@ -133,3 +133,20 @@ func TestFrontResultsService(t *testing.T) {
}) })
} }
} }
func TestFrontResultsServiceForwardsThroughTheInstanceTransport(t *testing.T) {
t.Parallel()
gitea := httptest.NewTLSServer(http.HandlerFunc(func(writer http.ResponseWriter, _request *http.Request) { _, _ = io.WriteString(writer, "{}") }))
defer gitea.Close()
handler, err := StartHandler(Options{Dir: t.TempDir(), OutboundIP: "127.0.0.1", InstanceTransport: gitea.Client().Transport})
require.NoError(t, err)
defer handler.Close()
defer handler.RegisterJob("token", JobCredential{Results: gitea.URL})()
req, err := http.NewRequestWithContext(t.Context(), http.MethodPost, handler.ExternalURL()+artifactServicePath+"CreateArtifact", nil)
require.NoError(t, err)
resp, err := (&http.Client{Transport: &bearerTransport{token: "token"}}).Do(req)
require.NoError(t, err)
defer resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
}
+25 -27
View File
@@ -9,6 +9,7 @@ import (
"crypto/sha256" "crypto/sha256"
"encoding/hex" "encoding/hex"
"encoding/json/v2" "encoding/json/v2"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
@@ -51,9 +52,10 @@ func (s *Storage) writeFile(name string, reader io.Reader) error {
if err != nil { if err != nil {
return err return err
} }
defer file.Close()
_, err = io.Copy(file, reader) _, err = io.Copy(file, reader)
if err = errors.Join(err, file.Close()); err != nil {
_ = os.Remove(name)
}
return err return err
} }
@@ -102,50 +104,46 @@ func (s *Storage) Commit(id uint64, size int64) (int64, error) {
return 0, err return 0, err
} }
written, err := assemble(name, tempNames) written, err := assemble(name, tempNames)
if err != nil { if err == nil && size >= 0 && written != size { // old clients like actions/cache@v2 send no size
return 0, err err = fmt.Errorf("broken file: %v != %v", written, size)
} }
// If size is less than 0, it means the size is unknown. if err != nil {
// We can't check the size of the file, just skip the check.
// It happens when the request comes from old versions of actions, like `actions/cache@v2`.
if size >= 0 && written != size {
_ = os.Remove(name) _ = os.Remove(name)
return 0, fmt.Errorf("broken file: %v != %v", written, size) return 0, err
} }
return written, nil return written, nil
} }
// assemble concatenates the uploaded parts into name. A single part, which is what the v2 API // assemble appends the parts into the first, deleting each once copied, so it never needs twice the space.
// produces below the client's block threshold, is already the whole archive and is moved.
func assemble(name string, tempNames []string) (int64, error) { func assemble(name string, tempNames []string) (int64, error) {
if len(tempNames) == 1 { if len(tempNames) == 0 {
info, err := os.Stat(tempNames[0]) return 0, errors.New("nothing was uploaded")
if err != nil {
return 0, err
}
return info.Size(), os.Rename(tempNames[0], name)
} }
if err := os.Rename(tempNames[0], name); err != nil {
file, err := os.Create(name) return 0, err
}
file, err := os.OpenFile(name, os.O_WRONLY|os.O_APPEND, 0)
if err != nil { if err != nil {
return 0, err return 0, err
} }
defer file.Close() defer file.Close()
for _, v := range tempNames[1:] {
var written int64 part, err := os.Open(v)
for _, v := range tempNames {
f, err := os.Open(v)
if err != nil { if err != nil {
return 0, err return 0, err
} }
n, err := io.Copy(file, f) _, err = io.Copy(file, part)
_ = f.Close() _ = part.Close()
if err != nil { if err != nil {
return 0, err return 0, err
} }
written += n _ = os.Remove(v)
} }
return written, nil info, err := file.Stat()
if err != nil {
return 0, err
}
return info.Size(), file.Close()
} }
func (s *Storage) Serve(w http.ResponseWriter, r *http.Request, id uint64) { func (s *Storage) Serve(w http.ResponseWriter, r *http.Request, id uint64) {
+1 -1
View File
@@ -281,7 +281,7 @@ func TestSafeResolve(t *testing.T) {
for name, tc := range tests { for name, tc := range tests {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
require.Equal(t, tc.want, safeResolve(baseDir, tc.input)) require.Equal(t, filepath.FromSlash(tc.want), safeResolve(baseDir, tc.input))
}) })
} }
} }
+24
View File
@@ -7,7 +7,9 @@ package common
import ( import (
"context" "context"
"fmt" "fmt"
"math/rand/v2"
"runtime/debug" "runtime/debug"
"time"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
) )
@@ -158,6 +160,28 @@ func (e Executor) IfBool(conditional bool) Executor {
}) })
} }
// RetryBackoff is the wait between Retry's attempts, random between 1 and 10 seconds like GitHub's runner.
var RetryBackoff = func() time.Duration { return time.Second + rand.N(9*time.Second) }
// Retry runs the executor up to attempts times until it succeeds.
func (e Executor) Retry(attempts int) Executor {
return func(ctx context.Context) error {
for attempt := 1; ; attempt++ {
err := e(ctx)
if err == nil || attempt == attempts || ctx.Err() != nil {
return err
}
backoff := RetryBackoff()
Logger(ctx).Warnf("%v, retrying in %s", err, backoff.Round(time.Second))
select {
case <-ctx.Done():
return err
case <-time.After(backoff):
}
}
}
}
// Finally adds an executor to run after other executor // Finally adds an executor to run after other executor
func (e Executor) Finally(finally Executor) Executor { func (e Executor) Finally(finally Executor) Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
+26
View File
@@ -11,6 +11,7 @@ import (
"sync" "sync"
"sync/atomic" "sync/atomic"
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -134,6 +135,31 @@ func TestExecutorConditionalsAndFinally(t *testing.T) {
assert.Equal(t, []string{"if-true", "if-not", "if-bool", "main", "finally"}, calls) assert.Equal(t, []string{"if-true", "if-not", "if-bool", "main", "finally"}, calls)
} }
func TestExecutorRetryStopsOnSuccessOrAfterAllAttempts(t *testing.T) {
backoff := RetryBackoff
RetryBackoff = func() time.Duration { return 0 }
t.Cleanup(func() { RetryBackoff = backoff })
transient := errors.New("transient")
for _, tc := range []struct {
failures, calls int
err error
}{
{failures: 2, calls: 3},
{failures: 5, calls: 3, err: transient},
} {
var calls int
err := Executor(func(context.Context) error {
calls++
if calls <= tc.failures {
return transient
}
return nil
}).Retry(3)(t.Context())
require.Equal(t, tc.err, err)
require.Equal(t, tc.calls, calls)
}
}
func TestExecutorFinallyReturnsFinallyErrorWithOriginal(t *testing.T) { func TestExecutorFinallyReturnsFinallyErrorWithOriginal(t *testing.T) {
finalErr := errors.New("cleanup failed") finalErr := errors.New("cleanup failed")
for _, mainErr := range []error{context.Canceled, nil} { for _, mainErr := range []error{context.Canceled, nil} {
+163 -3
View File
@@ -10,6 +10,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"io" "io"
"math/rand/v2" "math/rand/v2"
"net" "net"
@@ -29,6 +30,10 @@ import (
var actionFetches sync.Map var actionFetches sync.Map
var symlink = (*os.Root).Symlink
var symlinkFallback = runtime.GOOS == "windows"
const treeSizes = ".sizes" // beside the tree rather than in it, which is copied into job containers as it is const treeSizes = ".sizes" // beside the tree rather than in it, which is copied into job containers as it is
type treeFile struct { type treeFile struct {
@@ -229,6 +234,8 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
sha := "" sha := ""
sizes := map[string]int64{} sizes := map[string]int64{}
links := map[string]string{}
var deferred []string
reader := tar.NewReader(archive) reader := tar.NewReader(archive)
for { for {
header, err := reader.Next() header, err := reader.Next()
@@ -257,9 +264,8 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
case tar.TypeReg: case tar.TypeReg:
sizes[name], err = writeTreeFile(root, name, header.Mode, reader) sizes[name], err = writeTreeFile(root, name, header.Mode, reader)
case tar.TypeSymlink: case tar.TypeSymlink:
if err = root.Symlink(header.Linkname, name); err != nil && runtime.GOOS == "windows" { links[filepath.Clean(name)] = header.Linkname
sizes[name], err = writeTreeFile(root, name, 0, strings.NewReader(header.Linkname)) deferred = append(deferred, filepath.Clean(name))
}
} }
if err != nil { if err != nil {
return "", err return "", err
@@ -268,12 +274,166 @@ func extractTree(ctx context.Context, archive io.Reader, dir string, strip bool)
if !isFullObjectID(sha) { if !isFullObjectID(sha) {
return "", errors.New("the archive names no commit") return "", errors.New("the archive names no commit")
} }
linksByParent := map[string][]string{}
if symlinkFallback {
for name := range links {
linksByParent[filepath.Dir(name)] = append(linksByParent[filepath.Dir(name)], filepath.Base(name))
}
}
for requireTarget := symlinkFallback; len(deferred) > 0; {
unresolved := deferred[:0]
for _, name := range deferred {
if err := ctx.Err(); err != nil {
return "", err
}
if requireTarget {
if _, err := root.Stat(filepath.Dir(name) + string(filepath.Separator) + links[name]); err != nil {
unresolved = append(unresolved, name)
continue
}
}
if err := symlink(root, links[name], name); err == nil || !symlinkFallback {
if err != nil {
return "", err
}
continue
}
if err := copyTreeLink(ctx, root, name, links, linksByParent, sizes); err != nil {
return "", fmt.Errorf("materialize symlink %q: %w", filepath.ToSlash(name), err)
}
}
if len(unresolved) == len(deferred) {
requireTarget = false
}
deferred = unresolved
}
if err := root.Close(); err != nil { if err := root.Close(); err != nil {
return "", err return "", err
} }
return sha, installTree(ctx, dir, sha, temp, sizes) return sha, installTree(ctx, dir, sha, temp, sizes)
} }
func resolveTreeLink(name string, links map[string]string) (string, bool) {
if !filepath.IsLocal(name) {
return "", false
}
parts := strings.Split(name, string(filepath.Separator))
var resolved []string
for count := 0; len(parts) > 0; {
part := parts[0]
parts = parts[1:]
switch part {
case "", ".":
continue
case "..":
if len(resolved) == 0 {
return "", false
}
resolved = resolved[:len(resolved)-1]
continue
}
target, ok := links[filepath.Join(append(resolved, part)...)]
if !ok {
resolved = append(resolved, part)
continue
}
count++
if target == "" || os.IsPathSeparator(target[0]) || filepath.VolumeName(target) != "" || count > 255 {
return "", false
}
parts = append(strings.Split(filepath.FromSlash(target), string(filepath.Separator)), parts...)
}
return filepath.Join(append([]string{"."}, resolved...)...), true
}
var (
errTreeLinkTarget = errors.New("unmaterializable symlink target")
errTreeLinkCycle = fmt.Errorf("cyclic %w", errTreeLinkTarget)
)
func copyTreeLink(ctx context.Context, root *os.Root, name string, links map[string]string, linksByParent map[string][]string, sizes map[string]int64) error {
writeLinkText := func(destination, target string) error { // as git writes links it cannot create
if err := root.RemoveAll(destination); err != nil {
return err
}
for written := range sizes {
if written == destination || strings.HasPrefix(written, destination+string(filepath.Separator)) {
delete(sizes, written)
}
}
size, err := writeTreeFile(root, destination, 0o644, strings.NewReader(target))
sizes[destination] = size
return err
}
active := map[string]bool{}
var copyTarget func(string, string) error
copyTarget = func(source, destination string) error {
if err := ctx.Err(); err != nil {
return err
}
source, ok := resolveTreeLink(source, links)
if !ok {
return errTreeLinkTarget
}
file, err := root.Open(source)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return errTreeLinkTarget
}
return err
}
defer file.Close()
info, err := file.Stat()
if err != nil {
return err
}
if info.Mode().IsRegular() {
sizes[destination], err = writeTreeFile(root, destination, int64(info.Mode().Perm()), file)
return err
}
if !info.IsDir() {
return errTreeLinkTarget
}
if active[source] {
return errTreeLinkCycle
}
active[source] = true
defer delete(active, source)
entries, err := file.ReadDir(-1)
if err != nil {
return err
}
children := map[string]bool{}
for _, entry := range entries {
children[entry.Name()] = true
}
for _, child := range linksByParent[source] {
children[child] = true
}
if err := root.MkdirAll(destination, info.Mode().Perm()); err != nil {
return err
}
for child := range children {
childSource, childDestination := filepath.Join(source, child), filepath.Join(destination, child)
err := copyTarget(childSource, childDestination)
if target, isLink := links[childSource]; isLink && errors.Is(err, errTreeLinkTarget) && !errors.Is(err, errTreeLinkCycle) {
err = writeLinkText(childDestination, target)
}
if err != nil {
return err
}
}
return nil
}
if err := copyTarget(name, name); !errors.Is(err, errTreeLinkTarget) {
return err
}
if err := ctx.Err(); err != nil {
return err
}
return writeLinkText(name, links[name])
}
var lockTree = AcquireCloneLock // replaced in tests to see installTree wait for a job holding the tree var lockTree = AcquireCloneLock // replaced in tests to see installTree wait for a job holding the tree
// installTree moves temp into place as dir/sha, replacing a tree that failed treeIntact. // installTree moves temp into place as dir/sha, replacing a tree that failed treeIntact.
+165 -4
View File
@@ -8,6 +8,7 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/base64" "encoding/base64"
"errors"
"fmt" "fmt"
"net/http" "net/http"
"net/http/cgi" "net/http/cgi"
@@ -35,11 +36,12 @@ func TestFetchAction(t *testing.T) {
root := t.TempDir() root := t.TempDir()
remote := filepath.Join(root, "org", "action") remote := filepath.Join(root, "org", "action")
require.NoError(t, gitCmd("init", "--initial-branch=main", remote)) require.NoError(t, os.CopyFS(remote, os.DirFS(filepath.Join(gitFixtureDir, "work"))))
require.NoError(t, os.WriteFile(filepath.Join(remote, "action.yml"), []byte("name: action\n"), 0o644)) require.NoError(t, os.WriteFile(filepath.Join(remote, "action.yml"), []byte("name: action\n"), 0o644))
require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755)) require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755))
require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link"))) require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link")))
require.NoError(t, gitCmd("-C", remote, "add", ".")) require.NoError(t, gitCmd("-C", remote, "add", "."))
require.NoError(t, gitCmd("-C", remote, "update-index", "--chmod=+x", "run.sh"))
require.NoError(t, gitCmd("-C", remote, "commit", "-m", "initial")) require.NoError(t, gitCmd("-C", remote, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", remote, "tag", "v1")) require.NoError(t, gitCmd("-C", remote, "tag", "v1"))
expectedSha := gitRevParse(t, remote, "main") expectedSha := gitRevParse(t, remote, "main")
@@ -51,6 +53,7 @@ func TestFetchAction(t *testing.T) {
var archiveRequests []http.Header var archiveRequests []http.Header
var mirror *httptest.Server var mirror *httptest.Server
archiveCache := map[string][]byte{} archiveCache := map[string][]byte{}
revisions := map[string]string{}
requests, archives, failures := 0, "", 0 requests, archives, failures := 0, "", 0
handler := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) { handler := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
mu.Lock() mu.Lock()
@@ -77,7 +80,14 @@ func TestFetchAction(t *testing.T) {
http.Redirect(writer, request, mirror.URL+request.URL.Path, http.StatusFound) http.Redirect(writer, request, mirror.URL+request.URL.Path, http.StatusFound)
return return
} }
sha, _ := gitOutput(request.Context(), remote, "rev-parse", strings.TrimSuffix(ref, ".tar.gz")) sha, ok := revisions[ref]
if !ok {
var err error
sha, err = gitOutput(request.Context(), remote, "rev-parse", strings.TrimSuffix(ref, ".tar.gz"))
if err == nil {
revisions[ref] = sha
}
}
etag := fmt.Sprintf(`W/"%s-tar.gz-%t"`, sha, archives != "unprefixed") etag := fmt.Sprintf(`W/"%s-tar.gz-%t"`, sha, archives != "unprefixed")
if request.Header.Get("If-None-Match") == etag { if request.Header.Get("If-None-Match") == etag {
writer.WriteHeader(http.StatusNotModified) writer.WriteHeader(http.StatusNotModified)
@@ -192,7 +202,9 @@ func TestFetchAction(t *testing.T) {
assert.NoDirExists(t, filepath.Join(tree, ".git")) assert.NoDirExists(t, filepath.Join(tree, ".git"))
info, err := os.Stat(filepath.Join(tree, "run.sh")) info, err := os.Stat(filepath.Join(tree, "run.sh"))
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm()) if runtime.GOOS != "windows" {
assert.EqualValues(t, 0o755, info.Mode().Perm())
}
link, err := os.Readlink(filepath.Join(tree, "link")) link, err := os.Readlink(filepath.Join(tree, "link"))
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, "action.yml", link) assert.Equal(t, "action.yml", link)
@@ -204,6 +216,9 @@ func TestFetchAction(t *testing.T) {
require.NoError(t, gitCmd("-C", remote, "commit", "--allow-empty", "-m", "moved")) require.NoError(t, gitCmd("-C", remote, "commit", "--allow-empty", "-m", "moved"))
require.NoError(t, gitCmd("-C", remote, "tag", "--force", "v1")) require.NoError(t, gitCmd("-C", remote, "tag", "--force", "v1"))
expectedSha = gitRevParse(t, remote, "v1") expectedSha = gitRevParse(t, remote, "v1")
mu.Lock()
clear(revisions)
mu.Unlock()
moved, movedSha, err := FetchAction(t.Context(), input) moved, movedSha, err := FetchAction(t.Context(), input)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, expectedSha, movedSha) assert.Equal(t, expectedSha, movedSha)
@@ -249,7 +264,7 @@ func TestFetchAction(t *testing.T) {
}) })
t.Run("downloads again over a tree whose files changed size or that has no record, not under a job copying it", func(t *testing.T) { t.Run("downloads again over a tree whose files changed size or that has no record, not under a job copying it", func(t *testing.T) {
sha := gitRevParse(t, remote, "main") sha := expectedSha
for _, mode := range []string{"prefixed", "none"} { for _, mode := range []string{"prefixed", "none"} {
serve(mode, 0) serve(mode, 0)
input := newInput(sha) input := newInput(sha)
@@ -391,6 +406,79 @@ func TestExtractTreeConfinesEntries(t *testing.T) {
} }
} }
func TestExtractTreeCopiesDeferredLinks(t *testing.T) {
originalSymlink, originalFallback := symlink, symlinkFallback
symlink = func(_root *os.Root, _target, _name string) error { return errors.New("symlink privilege unavailable") }
symlinkFallback = true
t.Cleanup(func() { symlink, symlinkFallback = originalSymlink, originalFallback })
for _, test := range []struct {
name string
links [][2]string
files []string
text bool
}{
{name: "file chain", links: [][2]string{{"index.js", "entry.js"}, {"entry.js", "dist/index.js"}}, files: []string{"index.js", "entry.js"}},
{name: "directory chain", links: [][2]string{{"lib", "bundle"}, {"bundle", "dist"}, {"dist/entry.js", "index.js"}}, files: []string{"lib/index.js", "lib/entry.js", "lib/nested/index.js", "bundle/index.js", "bundle/entry.js", "bundle/nested/index.js"}},
{name: "directory in target path", links: [][2]string{{"index.js", "bundle/nested/index.js"}, {"bundle", "dist"}}, files: []string{"index.js", "bundle/nested/index.js"}},
{name: "parent after directory link", links: [][2]string{{"index.js", "bundle/../index.js"}, {"bundle", "dist/nested"}}, files: []string{"index.js", "bundle/index.js"}},
{name: "outside", links: [][2]string{{"index.js", "../outside.js"}}, text: true},
{name: "absolute", links: [][2]string{{"index.js", t.TempDir()}}, text: true},
{name: "dangling", links: [][2]string{{"index.js", "missing.js"}}, text: true},
{name: "dangling child of directory link", links: [][2]string{{"lib", "dist"}, {"dist/optional", "missing.js"}}, files: []string{"lib/index.js", "lib/nested/index.js"}},
{name: "cycle", links: [][2]string{{"index.js", "entry.js"}, {"entry.js", "index.js"}}, text: true},
{name: "directory cycle", links: [][2]string{{"dist/loop", "."}}, text: true},
} {
t.Run(test.name, func(t *testing.T) {
sha := strings.Repeat("a", 40)
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
require.NoError(t, writer.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, PAXRecords: map[string]string{"comment": sha}}))
require.NoError(t, writer.WriteHeader(&tar.Header{Name: "dist", Typeflag: tar.TypeDir, Mode: 0o755}))
require.NoError(t, writer.WriteHeader(&tar.Header{Name: "dist/nested", Typeflag: tar.TypeDir, Mode: 0o755}))
for _, link := range test.links {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: link[0], Typeflag: tar.TypeSymlink, Linkname: link[1]}))
}
content := "console.log('action');\n"
for _, name := range []string{"dist/index.js", "dist/nested/index.js"} {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: name, Typeflag: tar.TypeReg, Mode: 0o755, Size: int64(len(content))}))
_, err := writer.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, writer.Close())
dir := t.TempDir()
_, err := extractTree(t.Context(), &archive, dir, false)
require.NoError(t, err)
assert.True(t, treeIntact(dir, sha))
if test.text {
for _, link := range test.links {
data, err := os.ReadFile(filepath.Join(dir, sha, link[0]))
require.NoError(t, err)
assert.Equal(t, link[1], string(data))
info, err := os.Lstat(filepath.Join(dir, sha, link[0]))
require.NoError(t, err)
assert.True(t, info.Mode().IsRegular())
}
}
for _, name := range test.files {
data, err := os.ReadFile(filepath.Join(dir, sha, name))
require.NoError(t, err)
assert.Equal(t, content, string(data))
info, err := os.Lstat(filepath.Join(dir, sha, name))
require.NoError(t, err)
assert.True(t, info.Mode().IsRegular())
if runtime.GOOS != "windows" {
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm())
}
require.NoError(t, os.WriteFile(filepath.Join(dir, sha, name), nil, 0o644))
assert.False(t, treeIntact(dir, sha), name)
require.NoError(t, os.WriteFile(filepath.Join(dir, sha, name), []byte(content), 0o755))
}
})
}
}
func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) { func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) {
if runtime.GOOS != "linux" { if runtime.GOOS != "linux" {
t.Skip("macOS and Windows store no name that is not UTF-8") t.Skip("macOS and Windows store no name that is not UTF-8")
@@ -410,3 +498,76 @@ func TestTreeIntactFindsNamesThatAreNotUTF8(t *testing.T) {
require.FileExists(t, filepath.Join(dir, sha, "caf\xe9.js")) require.FileExists(t, filepath.Join(dir, sha, "caf\xe9.js"))
assert.True(t, treeIntact(dir, sha)) assert.True(t, treeIntact(dir, sha))
} }
func TestExtractTreeCreatesLinksInPlatformOrder(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("requires symlink privileges")
}
original, originalFallback := symlink, symlinkFallback
var created []string
symlink = func(root *os.Root, target, name string) error {
created = append(created, name)
return root.Symlink(target, name)
}
t.Cleanup(func() { symlink, symlinkFallback = original, originalFallback })
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
require.NoError(t, writer.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, PAXRecords: map[string]string{"comment": strings.Repeat("a", 40)}}))
for _, entry := range []*tar.Header{
{Name: "a", Typeflag: tar.TypeSymlink, Linkname: "b"},
{Name: "b", Typeflag: tar.TypeSymlink, Linkname: "dist"},
{Name: "dangling", Typeflag: tar.TypeSymlink, Linkname: "missing"},
{Name: "dist", Typeflag: tar.TypeDir, Mode: 0o755},
} {
require.NoError(t, writer.WriteHeader(entry))
}
require.NoError(t, writer.Close())
for _, test := range []struct {
name string
fallback bool
want []string
}{
{name: "native", want: []string{"a", "b", "dangling"}},
{name: "fallback", fallback: true, want: []string{"b", "a", "dangling"}},
} {
t.Run(test.name, func(t *testing.T) {
symlinkFallback = test.fallback
created = nil
_, err := extractTree(t.Context(), bytes.NewReader(archive.Bytes()), t.TempDir(), false)
require.NoError(t, err)
assert.Equal(t, test.want, created)
})
}
}
func TestCopyTreeLinkCancellationBetweenEntries(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
require.NoError(t, err)
defer root.Close()
require.NoError(t, root.Mkdir("source", 0o755))
for _, name := range []string{"first", "second"} {
_, err := writeTreeFile(root, filepath.Join("source", name), 0o644, strings.NewReader("content"))
require.NoError(t, err)
}
sizes := map[string]int64{}
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
observed := &treeCopyContext{Context: ctx, check: func() {
if len(sizes) == 1 {
cancel()
}
}}
require.ErrorIs(t, copyTreeLink(observed, root, "copy", map[string]string{"copy": "source"}, nil, sizes), context.Canceled)
assert.Len(t, sizes, 1)
}
type treeCopyContext struct {
context.Context
check func()
}
func (ctx *treeCopyContext) Err() error {
ctx.check()
return ctx.Context.Err()
}
+37 -8
View File
@@ -5,6 +5,7 @@
package git package git
import ( import (
"bytes"
"context" "context"
"crypto/sha256" "crypto/sha256"
"encoding/base64" "encoding/base64"
@@ -27,6 +28,7 @@ import (
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/internal/pkg/lock" "gitea.com/gitea/runner/internal/pkg/lock"
"gitea.com/gitea/runner/internal/pkg/process"
) )
var ( var (
@@ -183,6 +185,7 @@ type NewGitCloneExecutorInput struct {
// For Gitea // For Gitea
InsecureSkipTLS bool InsecureSkipTLS bool
ClientCertFile, ClientKeyFile string // PEM, absolute as git runs in other directories ClientCertFile, ClientKeyFile string // PEM, absolute as git runs in other directories
CAFile string // PEM, absolute, replaces git's CA bundle
Transport http.RoundTripper // for FetchAction's archive downloads, nil for http.DefaultTransport Transport http.RoundTripper // for FetchAction's archive downloads, nil for http.DefaultTransport
} }
@@ -387,6 +390,21 @@ func RemoteConfig(token string, insecureSkipTLS bool) []string {
return config return config
} }
// SSLConfig uses OpenSSL on Windows, as Schannel takes no PEM key and ignores sslCAInfo.
func SSLConfig(prefix, certFile, keyFile, caFile, executionOS string) []string {
var config []string
if certFile != "" {
config = append(config, prefix+"sslCert="+certFile, prefix+"sslKey="+keyFile)
}
if caFile != "" {
config = append(config, prefix+"sslCAInfo="+caFile)
}
if config != nil && strings.EqualFold(executionOS, "windows") {
config = append(config, prefix+"sslBackend=openssl")
}
return config
}
// ConfigEnv applies `-c` style entries through the environment, keeping secrets out of argv. // ConfigEnv applies `-c` style entries through the environment, keeping secrets out of argv.
func ConfigEnv(config ...string) map[string]string { func ConfigEnv(config ...string) map[string]string {
env := map[string]string{"GIT_CONFIG_COUNT": strconv.Itoa(len(config))} env := map[string]string{"GIT_CONFIG_COUNT": strconv.Itoa(len(config))}
@@ -444,9 +462,7 @@ func runGitWithStdin(ctx context.Context, dir string, input *NewGitCloneExecutor
config := []string{"core.autocrlf=false", "core.eol=lf"} // keep action files LF config := []string{"core.autocrlf=false", "core.eol=lf"} // keep action files LF
if input != nil { if input != nil {
config = append(config, RemoteConfig(input.Token, input.InsecureSkipTLS)...) config = append(config, RemoteConfig(input.Token, input.InsecureSkipTLS)...)
if input.ClientCertFile != "" { config = append(config, SSLConfig("http.", input.ClientCertFile, input.ClientKeyFile, input.CAFile, runtime.GOOS)...)
config = append(config, "http.sslCert="+input.ClientCertFile, "http.sslKey="+input.ClientKeyFile)
}
} }
if dir != "" { if dir != "" {
args = append([]string{"-C", dir}, args...) args = append([]string{"-C", dir}, args...)
@@ -461,15 +477,28 @@ func runGitWithStdin(ctx context.Context, dir string, input *NewGitCloneExecutor
cmd.Env = append(cmd.Env, key+"="+value) cmd.Env = append(cmd.Env, key+"="+value)
} }
cmd.Env = append(cmd.Env, "GIT_TERMINAL_PROMPT=0") cmd.Env = append(cmd.Env, "GIT_TERMINAL_PROMPT=0")
if runtime.GOOS != "windows" { var treeKill *process.TreeKill
if runtime.GOOS == "windows" {
treeKill = process.NewTreeKill(cmd) // a cancelled git leaves its remote helper holding the repository otherwise
} else {
cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) } // lets git remove its lock files cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) } // lets git remove its lock files
} }
cmd.WaitDelay = time.Second cmd.WaitDelay = time.Second
var stdout bytes.Buffer
var stderr strings.Builder var stderr strings.Builder
cmd.Stdout = &stdout
cmd.Stderr = &stderr cmd.Stderr = &stderr
output, err := cmd.Output() err := cmd.Start()
if err != nil { if err == nil {
return output, fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(stderr.String())) if treeKill != nil {
if killer, captureErr := treeKill.Capture(cmd.Process); captureErr == nil {
defer killer.Close()
}
}
err = cmd.Wait()
} }
return output, nil if err != nil {
return stdout.Bytes(), fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(stderr.String()))
}
return stdout.Bytes(), nil
} }
+46 -35
View File
@@ -54,6 +54,22 @@ func TestMain(m *testing.M) {
os.Exit(code) os.Exit(code)
} }
func TestSSLConfig(t *testing.T) {
for _, testcase := range []struct {
prefix, certFile, caFile, executionOS string
expected []string
}{
{"http.https://example.com/.", "cert.pem", "ca.pem", "Windows", []string{"http.https://example.com/.sslCert=cert.pem", "http.https://example.com/.sslKey=key.pem", "http.https://example.com/.sslCAInfo=ca.pem", "http.https://example.com/.sslBackend=openssl"}},
{"http.", "", "ca.pem", "windows", []string{"http.sslCAInfo=ca.pem", "http.sslBackend=openssl"}},
{"http.", "cert.pem", "ca.pem", "Linux", []string{"http.sslCert=cert.pem", "http.sslKey=key.pem", "http.sslCAInfo=ca.pem"}},
{"http.", "", "", "Windows", nil},
} {
t.Run(testcase.prefix+testcase.certFile+testcase.caFile+testcase.executionOS, func(t *testing.T) {
assert.Equal(t, testcase.expected, SSLConfig(testcase.prefix, testcase.certFile, "key.pem", testcase.caFile, testcase.executionOS))
})
}
}
func TestFindGitSlug(t *testing.T) { func TestFindGitSlug(t *testing.T) {
for _, testcase := range []struct { for _, testcase := range []struct {
remoteURL string remoteURL string
@@ -96,8 +112,7 @@ func TestFindGitMetadataOfSHA256RepositoryUsesOrigin(t *testing.T) {
t.Parallel() t.Parallel()
basedir := t.TempDir() basedir := t.TempDir()
const remoteURL = "https://github.com/owner/repo.git" const remoteURL = "https://github.com/owner/repo.git"
require.NoError(t, gitCmd("init", "--object-format=sha256", "--initial-branch=main", basedir)) require.NoError(t, gitCmd("init", "--template=", "--object-format=sha256", "--initial-branch=main", basedir))
require.NoError(t, os.RemoveAll(filepath.Join(basedir, ".git", "hooks")))
require.NoError(t, gitCmd("-C", basedir, "commit", "--allow-empty", "-m", "init")) require.NoError(t, gitCmd("-C", basedir, "commit", "--allow-empty", "-m", "init"))
require.NoError(t, gitCmd("-C", basedir, "remote", "add", "origin", remoteURL)) require.NoError(t, gitCmd("-C", basedir, "remote", "add", "origin", remoteURL))
require.NoError(t, gitCmd("-C", basedir, "config", "--add", "remote.origin.url", "https://github.com/other/repo.git")) require.NoError(t, gitCmd("-C", basedir, "config", "--add", "remote.origin.url", "https://github.com/other/repo.git"))
@@ -185,22 +200,20 @@ func TestGitFindRef(t *testing.T) {
func TestGitCloneExecutor(t *testing.T) { func TestGitCloneExecutor(t *testing.T) {
remoteDir, workDir := newGitRemote(t) remoteDir, workDir := newGitRemote(t)
require.NoError(t, os.WriteFile(filepath.Join(workDir, "action.sh"), []byte("#!/bin/sh\necho hi\n"), 0o755)) require.NoError(t, os.WriteFile(filepath.Join(workDir, "action.sh"), []byte("#!/bin/sh\necho hi\n"), 0o755))
require.NoError(t, gitCmd("-C", workDir, "add", "action.sh")) require.NoError(t, os.WriteFile(filepath.Join(workDir, ".gitattributes"), []byte("*.sh text\n"), 0o644))
require.NoError(t, gitCmd("-C", workDir, "add", "action.sh", ".gitattributes"))
require.NoError(t, gitCmd("-C", workDir, "commit", "-m", "initial")) require.NoError(t, gitCmd("-C", workDir, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", workDir, "tag", "v2")) require.NoError(t, gitCmd("-C", workDir, "tag", "v2"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "v2"))
require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "act-fails")) require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "act-fails"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "branch-commit")) require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "branch-commit"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "act-fails"))
fullSha := gitRevParse(t, workDir, "main") fullSha := gitRevParse(t, workDir, "main")
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:refs/heads/"+fullSha[:4])) require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main", "v2", "act-fails", "main:refs/heads/"+fullSha[:4]))
hostConfig := filepath.Join(t.TempDir(), "gitconfig") hostConfig := filepath.Join(t.TempDir(), "gitconfig")
require.NoError(t, os.WriteFile(hostConfig, []byte("[core]\n\tautocrlf = true\n[url \""+remoteDir+"\"]\n\tinsteadOf = https://example.invalid/action\n"), 0o644)) require.NoError(t, os.WriteFile(hostConfig, []byte("[core]\n\tautocrlf = true\n\teol = crlf\n[url "+strconv.Quote(remoteDir)+"]\n\tinsteadOf = https://example.invalid/action\n"), 0o644))
t.Setenv("GIT_CONFIG_GLOBAL", hostConfig) t.Setenv("GIT_CONFIG_GLOBAL", hostConfig)
t.Setenv("GIT_DIR", t.TempDir()) t.Setenv("GIT_DIR", t.TempDir())
@@ -265,10 +278,10 @@ func TestGitCloneExecutor(t *testing.T) {
func TestGitCloneExecutorReclonesWhenOriginURLChanges(t *testing.T) { func TestGitCloneExecutorReclonesWhenOriginURLChanges(t *testing.T) {
t.Parallel() t.Parallel()
createRemote := func(message string) string { createRemote := func(message string) string {
remoteDir, workDir := newGitRemote(t, message) dir := t.TempDir()
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main")) require.NoError(t, os.CopyFS(dir, os.DirFS(filepath.Join(gitFixtureDir, "work"))))
require.NoError(t, gitCmd("-C", dir, "commit", "--allow-empty", "-m", message))
return remoteDir return dir
} }
oldRemoteDir := createRemote("old-action") oldRemoteDir := createRemote("old-action")
@@ -388,31 +401,30 @@ func TestGitCloneExecutorShallow(t *testing.T) {
remoteDir, workDir := newGitRemote(t, "c1", "c2", "c3") remoteDir, workDir := newGitRemote(t, "c1", "c2", "c3")
require.NoError(t, gitCmd("-C", workDir, "tag", "v1")) require.NoError(t, gitCmd("-C", workDir, "tag", "v1"))
unadvertisedSHA := gitRevParse(t, workDir, "HEAD~1") unadvertisedSHA := gitRevParse(t, workDir, "HEAD~1")
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main")) require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main", "v1"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "v1"))
shallowMarker := func(dir string) string { return filepath.Join(dir, ".git", "shallow") } shallowMarker := func(dir string) string { return filepath.Join(dir, ".git", "shallow") }
for name, ref := range map[string]string{"branch is cloned shallowly": "main", "tag is cloned shallowly": "v1"} { for name, ref := range map[string]string{"branch is cloned shallowly": "main", "tag is cloned shallowly": "v1"} {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel()
dir := t.TempDir() dir := t.TempDir()
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{ require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir, Ref: ref, Dir: dir, Depth: 1, URL: remoteDir, Ref: ref, Dir: dir, Depth: 1,
})(t.Context())) })(t.Context()))
assert.FileExists(t, shallowMarker(dir), "clone should be shallow") assert.FileExists(t, shallowMarker(dir), "clone should be shallow")
assert.Equal(t, 1, gitRevCount(t, dir), "only the tip commit should be present") assert.Equal(t, []string{"c3"}, gitSubjects(t, dir), "only the tip commit should be present")
assert.Equal(t, "c3", gitHeadSubject(t, dir))
}) })
} }
unadvertisedRemote := func(t *testing.T, allowUnadvertised bool) string { unadvertisedRemote := func(t *testing.T, allowUnadvertised bool) string {
remote := filepath.Join(t.TempDir(), "remote.git") remote := filepath.Join(t.TempDir(), "remote.git")
require.NoError(t, gitCmd("clone", "--bare", remoteDir, remote)) require.NoError(t, gitCmd("clone", "--bare", "--template=", "--config", "uploadpack.allowAnySHA1InWant="+strconv.FormatBool(allowUnadvertised), remoteDir, remote))
require.NoError(t, gitCmd("-C", remote, "config", "uploadpack.allowAnySHA1InWant", strconv.FormatBool(allowUnadvertised)))
return remote return remote
} }
t.Run("commit hash resolves when the remote accepts a direct shallow fetch", func(t *testing.T) { t.Run("commit hash resolves when the remote accepts a direct shallow fetch", func(t *testing.T) {
t.Parallel()
dir := t.TempDir() dir := t.TempDir()
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{ require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: unadvertisedRemote(t, false), Ref: unadvertisedSHA, Dir: dir, Depth: 1, URL: unadvertisedRemote(t, false), Ref: unadvertisedSHA, Dir: dir, Depth: 1,
@@ -421,6 +433,7 @@ func TestGitCloneExecutorShallow(t *testing.T) {
}) })
t.Run("commit hash is fetched shallowly, moved to another hash, then reused without reaching the remote", func(t *testing.T) { t.Run("commit hash is fetched shallowly, moved to another hash, then reused without reaching the remote", func(t *testing.T) {
t.Parallel()
remote := unadvertisedRemote(t, true) remote := unadvertisedRemote(t, true)
dir := t.TempDir() dir := t.TempDir()
cloneAt := func(ref string) error { cloneAt := func(ref string) error {
@@ -431,7 +444,7 @@ func TestGitCloneExecutorShallow(t *testing.T) {
require.NoError(t, cloneAt(unadvertisedSHA)) require.NoError(t, cloneAt(unadvertisedSHA))
assert.FileExists(t, shallowMarker(dir)) assert.FileExists(t, shallowMarker(dir))
assert.Equal(t, 1, gitRevCount(t, dir)) assert.Len(t, gitSubjects(t, dir), 1)
assert.Equal(t, unadvertisedSHA, gitRevParse(t, dir, "HEAD")) assert.Equal(t, unadvertisedSHA, gitRevParse(t, dir, "HEAD"))
olderSHA := gitRevParse(t, workDir, "HEAD~2") olderSHA := gitRevParse(t, workDir, "HEAD~2")
@@ -444,6 +457,8 @@ func TestGitCloneExecutorShallow(t *testing.T) {
}) })
t.Run("moving hexadecimal-named branch updates while staying shallow", func(t *testing.T) { t.Run("moving hexadecimal-named branch updates while staying shallow", func(t *testing.T) {
t.Parallel()
remoteDir, workDir := newGitRemote(t, "c2", "c3")
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:deadbeef")) require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:deadbeef"))
dir := t.TempDir() dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{ clone := NewGitCloneExecutor(NewGitCloneExecutorInput{
@@ -456,19 +471,17 @@ func TestGitCloneExecutorShallow(t *testing.T) {
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:deadbeef")) require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:deadbeef"))
require.NoError(t, clone(t.Context())) require.NoError(t, clone(t.Context()))
assert.Equal(t, "c4", gitHeadSubject(t, dir), "reused shallow clone should update to the new tip") assert.Equal(t, []string{"c4"}, gitSubjects(t, dir), "reused shallow clone should update to the new tip")
assert.FileExists(t, shallowMarker(dir), "repo should remain shallow after update") assert.FileExists(t, shallowMarker(dir), "repo should remain shallow after update")
assert.Equal(t, 1, gitRevCount(t, dir))
}) })
t.Run("sha256 full hash is cloned shallowly, reused without the remote, and its 40-character prefix is short", func(t *testing.T) { t.Run("sha256 full hash is cloned shallowly, reused without the remote, and its 40-character prefix is short", func(t *testing.T) {
t.Parallel()
remote := filepath.Join(t.TempDir(), "remote.git") remote := filepath.Join(t.TempDir(), "remote.git")
require.NoError(t, gitCmd("init", "--bare", "--object-format=sha256", "--initial-branch=main", remote))
work := t.TempDir() work := t.TempDir()
require.NoError(t, gitCmd("init", "--object-format=sha256", "--initial-branch=main", work)) require.NoError(t, gitCmd("init", "--template=", "--object-format=sha256", "--initial-branch=main", work))
require.NoError(t, gitCmd("-C", work, "remote", "add", "origin", remote))
require.NoError(t, gitCmd("-C", work, "commit", "--allow-empty", "-m", "pinned")) require.NoError(t, gitCmd("-C", work, "commit", "--allow-empty", "-m", "pinned"))
require.NoError(t, gitCmd("-C", work, "push", "origin", "main")) require.NoError(t, gitCmd("clone", "--bare", "--template=", work, remote))
sha := gitRevParse(t, work, "HEAD") sha := gitRevParse(t, work, "HEAD")
dir := t.TempDir() dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remote, Ref: sha, Dir: dir, Depth: 1}) clone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remote, Ref: sha, Dir: dir, Depth: 1})
@@ -487,11 +500,11 @@ func TestGitCloneExecutorPinnedHashIgnoresShadowingRef(t *testing.T) {
pinned := gitRevParse(t, workDir, "HEAD") pinned := gitRevParse(t, workDir, "HEAD")
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "decoy")) require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "decoy"))
require.NoError(t, gitCmd("-C", workDir, "tag", pinned)) require.NoError(t, gitCmd("-C", workDir, "tag", pinned))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main")) require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main", pinned))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", pinned))
for name, depth := range map[string]int{"shallow": 1, "full clone": 0} { for name, depth := range map[string]int{"shallow": 1, "full clone": 0} {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel()
dir := t.TempDir() dir := t.TempDir()
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{ require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir, Ref: pinned, Dir: dir, Depth: depth, URL: remoteDir, Ref: pinned, Dir: dir, Depth: depth,
@@ -523,13 +536,11 @@ func gitRevParse(t *testing.T, dir, rev string) string {
return strings.TrimSpace(string(out)) return strings.TrimSpace(string(out))
} }
func gitRevCount(t *testing.T, dir string) int { func gitSubjects(t *testing.T, dir string) []string {
t.Helper() t.Helper()
out, err := exec.Command("git", "-C", dir, "rev-list", "--count", "HEAD").Output() out, err := exec.Command("git", "-C", dir, "log", "--format=%s").Output()
require.NoError(t, err) require.NoError(t, err)
n, err := strconv.Atoi(strings.TrimSpace(string(out))) return strings.Split(strings.TrimSpace(string(out)), "\n")
require.NoError(t, err)
return n
} }
func gitHeadSubject(t *testing.T, dir string) string { func gitHeadSubject(t *testing.T, dir string) string {
@@ -663,7 +674,7 @@ func TestNewGitCloneExecutorFetchSendsTokenAndHonoursContext(t *testing.T) {
}) })
dir := filepath.Join(t.TempDir(), "cached-action") dir := filepath.Join(t.TempDir(), "cached-action")
require.NoError(t, gitCmd("init", dir)) require.NoError(t, os.CopyFS(dir, os.DirFS(filepath.Join(gitFixtureDir, "work"))))
require.NoError(t, gitCmd("-C", dir, "remote", "add", "origin", server.URL)) require.NoError(t, gitCmd("-C", dir, "remote", "add", "origin", server.URL))
ctx, cancel := context.WithCancel(t.Context()) ctx, cancel := context.WithCancel(t.Context())
@@ -676,8 +687,8 @@ func TestNewGitCloneExecutorFetchSendsTokenAndHonoursContext(t *testing.T) {
select { select {
case <-reached: case <-reached:
case <-time.After(10 * time.Second): case err := <-done:
t.Fatal("the executor never reached the remote") t.Fatalf("the executor finished before reaching the remote: %v", err)
} }
assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), authorization) assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), authorization)
cancel() cancel()
+20
View File
@@ -0,0 +1,20 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package common
import "strings"
// LookupFold finds name in values, ignoring case where the environment does, as GitHub's inputs always do.
func LookupFold[V any](values map[string]V, name string, caseInsensitive bool) (V, bool) {
if value, ok := values[name]; ok || !caseInsensitive {
return value, ok
}
for key, value := range values {
if strings.EqualFold(key, name) {
return value, true
}
}
var zero V
return zero, false
}
+8 -4
View File
@@ -6,6 +6,7 @@ package common
import ( import (
"net" "net"
"slices"
"sort" "sort"
"strings" "strings"
) )
@@ -55,19 +56,22 @@ func GetOutboundIP() net.IP {
} }
func preferredIP(ips []interfaceIP) net.IP { func preferredIP(ips []interfaceIP) net.IP {
ips = slices.DeleteFunc(ips, func(ip interfaceIP) bool {
return len(ip.Name) == 15 && strings.HasPrefix(ip.Name, "br-") && strings.Trim(ip.Name[3:], "0123456789abcdef") == "" // a Docker network's bridge, removed with the network
})
if len(ips) == 0 { if len(ips) == 0 {
return nil return nil
} }
sort.Slice(ips, func(i, j int) bool { sort.Slice(ips, func(i, j int) bool {
first, second := ips[i], ips[j] first, second := ips[i], ips[j]
// ethernet is preferred // IPv4 is preferred, as job networks are IPv4-only by default
if vi, vj := strings.HasPrefix(first.Name, "e"), strings.HasPrefix(second.Name, "e"); vi != vj { if vi, vj := first.To4() != nil, second.To4() != nil; vi != vj {
return vi return vi
} }
// IPv4 is preferred // ethernet is preferred
if vi, vj := first.To4() != nil, second.To4() != nil; vi != vj { if vi, vj := strings.HasPrefix(first.Name, "e"), strings.HasPrefix(second.Name, "e"); vi != vj {
return vi return vi
} }
+2 -1
View File
@@ -13,8 +13,9 @@ import (
func TestPreferredIP(t *testing.T) { func TestPreferredIP(t *testing.T) {
assert.Nil(t, preferredIP(nil)) assert.Nil(t, preferredIP(nil))
assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{{IP: net.ParseIP("2001:db8::1")}})) assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{{IP: net.ParseIP("2001:db8::1")}}))
assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{ assert.Equal(t, net.ParseIP("192.0.2.1"), preferredIP([]interfaceIP{
{IP: net.ParseIP("192.0.2.1"), Name: "wlan0"}, {IP: net.ParseIP("192.0.2.1"), Name: "wlan0"},
{IP: net.ParseIP("2001:db8::1"), Name: "eth0"}, {IP: net.ParseIP("2001:db8::1"), Name: "eth0"},
{IP: net.ParseIP("172.18.0.1"), Name: "br-0123456789ab"},
})) }))
} }
+9
View File
@@ -10,9 +10,11 @@ import (
"fmt" "fmt"
"io" "io"
"path" "path"
"runtime"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
"time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
@@ -20,6 +22,11 @@ import (
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
) )
var hostOS = runtime.GOOS // replaced in tests
// DrainGracePeriod caps the wait for container output to drain so the log tail is not truncated
const DrainGracePeriod = 2 * time.Second
// ExitCodeError reports a non-zero process exit code from a container command. // ExitCodeError reports a non-zero process exit code from a container command.
type ExitCodeError int type ExitCodeError int
@@ -145,6 +152,8 @@ type NewDockerBuildExecutorInput struct {
ContextDir string ContextDir string
Dockerfile string Dockerfile string
BuildContext io.Reader BuildContext io.Reader
Dockerignore []byte // applied to BuildContext as the docker CLI applies it
Stdout io.Writer
ImageTag string ImageTag string
Platform string Platform string
BuildArgs map[string]*string BuildArgs map[string]*string
+1 -1
View File
@@ -2,7 +2,7 @@
// Copyright 2021 The nektos/act Authors. All rights reserved. // Copyright 2021 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !windows && !(WITHOUT_DOCKER || !(linux || darwin || netbsd)) //go:build !windows && !(WITHOUT_DOCKER || !(linux || darwin || netbsd || freebsd))
package container package container
+100 -6
View File
@@ -2,16 +2,19 @@
// Copyright 2020 The nektos/act Authors. All rights reserved. // Copyright 2020 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
import ( import (
"archive/tar" "archive/tar"
"bytes"
"context" "context"
"fmt"
"io" "io"
"io/fs" "io/fs"
"os" "os"
"path"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strings" "strings"
@@ -48,6 +51,7 @@ func NewDockerBuildExecutor(input NewDockerBuildExecutorInput) common.Executor {
options := client.ImageBuildOptions{ options := client.ImageBuildOptions{
Tags: []string{input.ImageTag}, Tags: []string{input.ImageTag},
Labels: map[string]string{"com.gitea.runner.image": input.ImageTag}, // own image ID per tag, so removing one never deletes another job's identical build
Remove: true, Remove: true,
AuthConfigs: LoadDockerAuthConfigs(ctx), AuthConfigs: LoadDockerAuthConfigs(ctx),
Dockerfile: input.Dockerfile, Dockerfile: input.Dockerfile,
@@ -62,7 +66,7 @@ func NewDockerBuildExecutor(input NewDockerBuildExecutorInput) common.Executor {
} }
var buildContext io.ReadCloser var buildContext io.ReadCloser
if input.BuildContext != nil { if input.BuildContext != nil {
buildContext = io.NopCloser(input.BuildContext) buildContext, err = filterBuildContext(input.BuildContext, input.Dockerignore, input.Dockerfile)
} else { } else {
buildContext, err = createBuildContext(ctx, input.ContextDir, input.Dockerfile) buildContext, err = createBuildContext(ctx, input.ContextDir, input.Dockerfile)
} }
@@ -77,10 +81,97 @@ func NewDockerBuildExecutor(input NewDockerBuildExecutorInput) common.Executor {
if err != nil { if err != nil {
return err return err
} }
return logDockerResponse(logger, resp.Body, false) err = logDockerResponse(logger, resp.Body, func(format string, args ...any) { fmt.Fprintf(input.Stdout, format, args...) })
common.FlushWriter(input.Stdout)
return err
} }
} }
func filterBuildContext(buildContext io.Reader, dockerignore []byte, relDockerfile string) (io.ReadCloser, error) {
excludes, err := ignorefile.ReadAll(bytes.NewReader(dockerignore))
if err != nil || len(excludes) == 0 {
return io.NopCloser(buildContext), err
}
pm, err := patternmatcher.New(excludes)
if err != nil {
return nil, err
}
reader, writer := io.Pipe()
go func() {
writer.CloseWithError(writeFilteredBuildContext(writer, buildContext, pm, path.Clean(filepath.ToSlash(relDockerfile))))
}()
return reader, nil
}
func writeFilteredBuildContext(w io.Writer, buildContext io.Reader, pm *patternmatcher.PatternMatcher, relDockerfile string) error {
spool, err := os.CreateTemp("", "gitea-runner-build-") // excluded content, for a kept hardlink to it
if err != nil {
return err
}
defer os.Remove(spool.Name())
defer spool.Close()
reader := tar.NewReader(buildContext)
writer := tar.NewWriter(w)
buffer := make([]byte, 32*1024)
excluded := map[string]*excludedBuildContextEntry{}
for {
header, err := reader.Next()
if err == io.EOF {
return writer.Close()
}
if err != nil {
return err
}
name := path.Clean(header.Name)
var target *excludedBuildContextEntry
if header.Typeflag == tar.TypeLink {
target = excluded[path.Clean(header.Linkname)]
}
if name != ".dockerignore" && name != relDockerfile {
skip, err := pm.MatchesOrParentMatches(name)
if err != nil {
return err
}
if skip {
if target != nil {
excluded[name] = target
} else if header.Typeflag != tar.TypeDir {
offset, err := spool.Seek(0, io.SeekCurrent)
if err != nil {
return err
}
excluded[name] = &excludedBuildContextEntry{header: header, offset: offset}
if _, err := io.CopyBuffer(spool, reader, buffer); err != nil {
return err
}
}
continue
}
}
var content io.Reader = reader
if target != nil && target.keptName != "" {
header.Linkname = target.keptName
} else if target != nil {
header.Typeflag, header.Linkname, header.Size = target.header.Typeflag, target.header.Linkname, target.header.Size
content = io.NewSectionReader(spool, target.offset, header.Size)
target.keptName = header.Name
}
if err := writer.WriteHeader(header); err != nil {
return err
}
if _, err := io.CopyBuffer(writer, content, buffer); err != nil {
return err
}
}
}
type excludedBuildContextEntry struct {
header *tar.Header
offset int64
keptName string
}
func createBuildContext(ctx context.Context, contextDir, relDockerfile string) (io.ReadCloser, error) { func createBuildContext(ctx context.Context, contextDir, relDockerfile string) (io.ReadCloser, error) {
common.Logger(ctx).Debugf("Creating archive for build context dir '%s' with relative dockerfile '%s'", contextDir, relDockerfile) common.Logger(ctx).Debugf("Creating archive for build context dir '%s' with relative dockerfile '%s'", contextDir, relDockerfile)
@@ -125,6 +216,10 @@ func createBuildContext(ctx context.Context, contextDir, relDockerfile string) (
} }
func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, includes []string, pm *patternmatcher.PatternMatcher) error { func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, includes []string, pm *patternmatcher.PatternMatcher) error {
contextDir, err := filepath.EvalSymlinks(contextDir)
if err != nil {
return err
}
tw := tar.NewWriter(w) tw := tar.NewWriter(w)
seen := map[string]bool{} seen := map[string]bool{}
hardlinks := map[uint64]string{} hardlinks := map[uint64]string{}
@@ -132,8 +227,7 @@ func writeBuildContext(ctx context.Context, w io.Writer, contextDir string, incl
for _, include := range includes { for _, include := range includes {
include = filepath.FromSlash(include) include = filepath.FromSlash(include)
var parents []parentMatch var parents []parentMatch
root := strings.TrimSuffix(contextDir, string(filepath.Separator)) + string(filepath.Separator) + include // unlike filepath.Join, follows a symlinked context dir as docker build does err := filepath.WalkDir(filepath.Join(contextDir, include), func(file string, entry fs.DirEntry, err error) error {
err := filepath.WalkDir(root, func(file string, entry fs.DirEntry, err error) error {
if err != nil { if err != nil {
return leaveOut(ctx, file, err) return leaveOut(ctx, file, err)
} }
@@ -210,7 +304,7 @@ func addBuildContextEntry(ctx context.Context, tw *tar.Writer, file, rel string,
return leaveOut(ctx, file, err) return leaveOut(ctx, file, err)
} }
} }
header, err := tar.FileInfoHeader(noNameLookups{info}, link) header, err := tar.FileInfoHeader(noNameLookups{info}, filepath.ToSlash(link))
if err != nil { if err != nil {
return err return err
} }
+63 -5
View File
@@ -1,12 +1,14 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
import ( import (
"archive/tar" "archive/tar"
"bytes"
"fmt"
"io" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
@@ -18,10 +20,12 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
func TestDockerBuildReportsDaemonError(t *testing.T) { func TestDockerBuildLabelsImageWithItsTagAndReportsDaemonError(t *testing.T) {
var labels string
daemon := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) { daemon := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
writer.Header().Set("API-Version", "1.47") writer.Header().Set("API-Version", "1.47")
if strings.HasSuffix(request.URL.Path, "/build") { if strings.HasSuffix(request.URL.Path, "/build") {
labels = request.URL.Query().Get("labels")
writer.Header().Set("Content-Type", "application/json") writer.Header().Set("Content-Type", "application/json")
writer.WriteHeader(http.StatusInternalServerError) writer.WriteHeader(http.StatusInternalServerError)
_, _ = io.WriteString(writer, `{"message":"build rejected"}`) _, _ = io.WriteString(writer, `{"message":"build rejected"}`)
@@ -33,24 +37,25 @@ func TestDockerBuildReportsDaemonError(t *testing.T) {
t.Setenv("DOCKER_AUTH_CONFIG", "") t.Setenv("DOCKER_AUTH_CONFIG", "")
require.ErrorContains(t, NewDockerBuildExecutor(NewDockerBuildExecutorInput{ require.ErrorContains(t, NewDockerBuildExecutor(NewDockerBuildExecutorInput{
ImageTag: "action:test", Dockerfile: "Dockerfile", BuildContext: strings.NewReader(""), ImageTag: "action:test", Dockerfile: "Dockerfile", BuildContext: strings.NewReader(""), Stdout: io.Discard,
})(t.Context()), "build rejected") })(t.Context()), "build rejected")
require.JSONEq(t, `{"com.gitea.runner.image":"action:test"}`, labels)
} }
func TestCreateBuildContextFollowsDockerignoreAndSymlinkedRootButKeepsDockerfile(t *testing.T) { func TestCreateBuildContextFollowsDockerignoreAndSymlinkedRootButKeepsDockerfile(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
require.NoError(t, os.Mkdir(filepath.Join(dir, "build"), 0o755))
for name, content := range map[string]string{ for name, content := range map[string]string{
".dockerignore": "*.log\nbuild\n!build/keep.txt\nDockerfile", "Dockerfile": "FROM scratch", "a.txt": "a", ".dockerignore": "*.log\nbuild\n!build/keep.txt\nDockerfile", "Dockerfile": "FROM scratch", "a.txt": "a",
"a.log": "log", "build/drop.txt": "drop", "build/keep.txt": "keep", "a.log": "log", "build/drop.txt": "drop", "build/keep.txt": "keep",
} { } {
require.NoError(t, os.MkdirAll(filepath.Join(dir, filepath.Dir(name)), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644)) require.NoError(t, os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644))
} }
require.NoError(t, os.Mkdir(filepath.Join(dir, "empty"), 0o755)) require.NoError(t, os.Mkdir(filepath.Join(dir, "empty"), 0o755))
contexts := []string{dir} contexts := []string{dir}
if link := filepath.Join(t.TempDir(), "link"); os.Symlink(dir, link) == nil { if link := filepath.Join(t.TempDir(), "link"); os.Symlink(dir, link) == nil {
contexts = append(contexts, link+string(filepath.Separator)) contexts = append(contexts, link, link+string(filepath.Separator))
} }
for _, contextDir := range contexts { for _, contextDir := range contexts {
@@ -69,3 +74,56 @@ func TestCreateBuildContextFollowsDockerignoreAndSymlinkedRootButKeepsDockerfile
require.Equal(t, []string{".dockerignore", "a.txt", "build/keep.txt", "empty/", "Dockerfile"}, names, contextDir) require.Equal(t, []string{".dockerignore", "a.txt", "build/keep.txt", "empty/", "Dockerfile"}, names, contextDir)
} }
} }
func TestFilterBuildContextAppliesDockerignoreAndMaterializesHardlinksToExcludedFiles(t *testing.T) {
const dockerignore = "*.log\nbuild\n!build/keep.txt\nActionfile\n.dockerignore\n"
for _, prefix := range []string{"", "./"} {
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
for _, entry := range []struct {
name, body, link string
typeflag byte
}{
{"Actionfile", "FROM scratch", "", tar.TypeReg},
{"output.log", "log", "", tar.TypeReg},
{"build/", "", "", tar.TypeDir},
{"build/drop.txt", "drop", "", tar.TypeReg},
{"build/keep.txt", "keep", "", tar.TypeReg},
{"entrypoint", "", prefix + "output.log", tar.TypeLink},
{"other-entrypoint", "", prefix + "output.log", tar.TypeLink},
{".dockerignore", dockerignore, "", tar.TypeReg},
} {
require.NoError(t, writer.WriteHeader(&tar.Header{Name: prefix + entry.name, Linkname: entry.link, Typeflag: entry.typeflag, Size: int64(len(entry.body)), Mode: 0o644}))
_, err := io.WriteString(writer, entry.body)
require.NoError(t, err)
}
require.NoError(t, writer.Close())
unfiltered, err := filterBuildContext(bytes.NewReader(archive.Bytes()), nil, "Actionfile")
require.NoError(t, err)
passedThrough, err := io.ReadAll(unfiltered)
require.NoError(t, err)
require.Equal(t, archive.Bytes(), passedThrough)
filtered, err := filterBuildContext(bytes.NewReader(archive.Bytes()), []byte(dockerignore), "Actionfile")
require.NoError(t, err)
var entries []string
for reader := tar.NewReader(filtered); ; {
header, err := reader.Next()
if err == io.EOF {
break
}
require.NoError(t, err)
body, err := io.ReadAll(reader)
require.NoError(t, err)
entries = append(entries, fmt.Sprintf("%s %c %s %s", strings.TrimPrefix(header.Name, prefix), header.Typeflag, strings.TrimPrefix(header.Linkname, prefix), body))
}
require.Equal(t, []string{
"Actionfile 0 FROM scratch",
"build/keep.txt 0 keep",
"entrypoint 0 log",
"other-entrypoint 1 entrypoint ",
".dockerignore 0 " + dockerignore,
}, entries, prefix)
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !WITHOUT_DOCKER && (linux || darwin || netbsd) //go:build !WITHOUT_DOCKER && (linux || darwin || netbsd || freebsd)
package container package container
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !WITHOUT_DOCKER && (linux || darwin || netbsd) //go:build !WITHOUT_DOCKER && (linux || darwin || netbsd || freebsd)
package container package container
+9 -5
View File
@@ -2,7 +2,7 @@
// Copyright 2022 The nektos/act Authors. All rights reserved. // Copyright 2022 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
// This file is based on https://github.com/docker/cli/blob/4a63305d74332de5ceba7fcbccbc3cbb7412f5ba/cli/command/container/opts.go with: // This file is based on https://github.com/docker/cli/blob/4a63305d74332de5ceba7fcbccbc3cbb7412f5ba/cli/command/container/opts.go with:
// * appended with license information // * appended with license information
@@ -351,7 +351,7 @@ type containerConfig struct {
// parse parses the args for the specified command and generates a Config, // parse parses the args for the specified command and generates a Config,
// a HostConfig and returns them with the specified command. // a HostConfig and returns them with the specified command.
// If the specified args are not valid, it will return an error. // If the specified args are not valid, it will return an error.
func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*containerConfig, error) { //nolint:gocyclo,unparam // verbatim copy from docker/cli func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*containerConfig, error) { //nolint:gocyclo // verbatim copy from docker/cli
var ( var (
attachStdin = copts.attach.Get("stdin") attachStdin = copts.attach.Get("stdin")
attachStdout = copts.attach.Get("stdout") attachStdout = copts.attach.Get("stdout")
@@ -724,6 +724,10 @@ func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*con
}, nil }, nil
} }
func isUserDefinedNetwork(mode container.NetworkMode) bool { // Docker's helper uses the host OS instead of the daemon's.
return !mode.IsDefault() && mode != network.NetworkBridge && mode != network.NetworkHost && !mode.IsNone() && !mode.IsContainer()
}
// parseNetworkOpts converts --network advanced options to endpoint-specs, and combines // parseNetworkOpts converts --network advanced options to endpoint-specs, and combines
// them with the old --network-alias and --links. If returns an error if conflicting options // them with the old --network-alias and --links. If returns an error if conflicting options
// are found. // are found.
@@ -752,7 +756,7 @@ func parseNetworkOpts(copts *containerOptions) (map[string]*network.EndpointSett
} }
for i, n := range copts.netMode.Value() { for i, n := range copts.netMode.Value() {
if container.NetworkMode(n.Target).IsUserDefined() { if isUserDefinedNetwork(container.NetworkMode(n.Target)) {
hasUserDefined = true hasUserDefined = true
} else { } else {
hasNonUserDefined = true hasNonUserDefined = true
@@ -814,7 +818,7 @@ func applyContainerOptions(n *opts.NetworkAttachmentOpts, copts *containerOption
} }
// For a user-defined network, "--link" is an endpoint option, it creates an alias. But, // For a user-defined network, "--link" is an endpoint option, it creates an alias. But,
// for the default bridge it defines a legacy-link. // for the default bridge it defines a legacy-link.
if container.NetworkMode(n.Target).IsUserDefined() && copts.links.Len() > 0 { if isUserDefinedNetwork(container.NetworkMode(n.Target)) && copts.links.Len() > 0 {
n.Links = slices.Clone(copts.links.GetSlice()) n.Links = slices.Clone(copts.links.GetSlice())
} }
if copts.ipv4Address != nil { if copts.ipv4Address != nil {
@@ -840,7 +844,7 @@ func parseNetworkAttachmentOpt(ep opts.NetworkAttachmentOpts) (*network.Endpoint
if strings.TrimSpace(ep.Target) == "" { if strings.TrimSpace(ep.Target) == "" {
return nil, errors.New("no name set for network") return nil, errors.New("no name set for network")
} }
if !container.NetworkMode(ep.Target).IsUserDefined() { if !isUserDefinedNetwork(container.NetworkMode(ep.Target)) {
if len(ep.Aliases) > 0 { if len(ep.Aliases) > 0 {
return nil, errors.New("network-scoped aliases are only supported for user-defined networks") return nil, errors.New("network-scoped aliases are only supported for user-defined networks")
} }
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
+1 -1
View File
@@ -2,7 +2,7 @@
// Copyright 2020 The nektos/act Authors. All rights reserved. // Copyright 2020 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
+1 -1
View File
@@ -38,7 +38,7 @@ func loadScratchImage(t *testing.T, platform string) string {
response, err := cli.ImageLoad(t.Context(), &archive) response, err := cli.ImageLoad(t.Context(), &archive)
require.NoError(t, err) require.NoError(t, err)
defer response.Close() defer response.Close()
require.NoError(t, logDockerResponse(common.Logger(t.Context()), response, false)) require.NoError(t, logDockerResponse(common.Logger(t.Context()), response, common.Logger(t.Context()).Debugf))
t.Cleanup(func() { t.Cleanup(func() {
_, _ = cli.ImageRemove(context.WithoutCancel(t.Context()), tag, mobyclient.ImageRemoveOptions{Force: true}) _, _ = cli.ImageRemove(context.WithoutCancel(t.Context()), tag, mobyclient.ImageRemoveOptions{Force: true})
}) })
+11 -26
View File
@@ -2,12 +2,13 @@
// Copyright 2020 The nektos/act Authors. All rights reserved. // Copyright 2020 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
import ( import (
"bufio" "bufio"
"cmp"
"encoding/json/v2" "encoding/json/v2"
"errors" "errors"
"io" "io"
@@ -26,10 +27,7 @@ type dockerMessage struct {
Progress string `json:"progress"` Progress string `json:"progress"`
} }
func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, isError bool) error { func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, log func(string, ...any)) error {
if dockerResponse == nil {
return nil
}
defer dockerResponse.Close() defer dockerResponse.Close()
scanner := bufio.NewScanner(dockerResponse) scanner := bufio.NewScanner(dockerResponse)
@@ -38,41 +36,28 @@ func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, isE
line := scanner.Bytes() line := scanner.Bytes()
var msg dockerMessage var msg dockerMessage
if err := json.Unmarshal(line, &msg); err != nil { if err := json.Unmarshal(line, &msg); err != nil {
writeLog(logger, false, "Unable to unmarshal line [%s] ==> %v", string(line), err) logger.Debugf("Unable to unmarshal line [%s] ==> %v", string(line), err)
continue continue
} }
if msg.Error != "" { if message := cmp.Or(msg.Error, msg.ErrorDetail.Message); message != "" {
writeLog(logger, isError, "%s", msg.Error) log("%s", message)
return errors.New(msg.Error) return errors.New(message)
}
if msg.ErrorDetail.Message != "" {
writeLog(logger, isError, "%s", msg.ErrorDetail.Message)
return errors.New(msg.ErrorDetail.Message)
} }
switch { switch {
case msg.Status != "": case msg.Status != "":
if msg.Progress != "" { if msg.Progress != "" {
writeLog(logger, isError, "%s :: %s :: %s\n", msg.Status, msg.ID, msg.Progress) log("%s :: %s :: %s\n", msg.Status, msg.ID, msg.Progress)
} else { } else {
writeLog(logger, isError, "%s :: %s\n", msg.Status, msg.ID) log("%s :: %s\n", msg.Status, msg.ID)
} }
case msg.Stream != "": case msg.Stream != "":
writeLog(logger, isError, "%s", msg.Stream) log("%s", msg.Stream)
default: default:
writeLog(logger, false, "Unable to handle line: %s", string(line)) logger.Debugf("Unable to handle line: %s", string(line))
} }
} }
return scanner.Err() return scanner.Err()
} }
func writeLog(logger *logging.Logger, isError bool, format string, args ...any) {
if isError {
logger.Errorf(format, args...)
} else {
logger.Debugf(format, args...)
}
}
+1 -1
View File
@@ -2,7 +2,7 @@
// Copyright 2023 The nektos/act Authors. All rights reserved. // Copyright 2023 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
+1 -1
View File
@@ -2,7 +2,7 @@
// Copyright 2025 The nektos/act Authors. All rights reserved. // Copyright 2025 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
+8 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
@@ -484,6 +484,10 @@ type probeClient struct {
containers []container.Summary containers []container.Summary
} }
func (c *probeClient) Info(_ctx context.Context, _opts mobyclient.InfoOptions) (mobyclient.SystemInfoResult, error) {
return mobyclient.SystemInfoResult{}, nil
}
func (c *probeClient) Close() error { func (c *probeClient) Close() error {
return nil return nil
} }
@@ -766,6 +770,9 @@ func TestDaemonSeesDir(t *testing.T) {
} }
func TestRunnerContainerWorkdir(t *testing.T) { func TestRunnerContainerWorkdir(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("a runner container is Linux")
}
workdir := t.TempDir() workdir := t.TempDir()
t.Chdir(workdir) t.Chdir(workdir)
volume := container.MountPoint{Type: mount.TypeVolume, Source: "/daemon/_data", Destination: filepath.Dir(workdir)} volume := container.MountPoint{Type: mount.TypeVolume, Source: "/daemon/_data", Destination: filepath.Dir(workdir)}
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !WITHOUT_DOCKER && (linux || darwin || netbsd) //go:build !WITHOUT_DOCKER && (linux || darwin || netbsd || freebsd)
package container package container
+4 -5
View File
@@ -2,7 +2,7 @@
// Copyright 2020 The nektos/act Authors. All rights reserved. // Copyright 2020 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
@@ -22,7 +22,7 @@ import (
// NewDockerPullExecutor function to create a run executor for the container // NewDockerPullExecutor function to create a run executor for the container
func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor { func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
return func(ctx context.Context) error { return common.Executor(func(ctx context.Context) error {
logger := common.Logger(ctx) logger := common.Logger(ctx)
logger.Debugf("docker pull %v", input.Image) logger.Debugf("docker pull %v", input.Image)
@@ -61,11 +61,10 @@ func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
// stream rather than on the call itself, so both have to be checked // stream rather than on the call itself, so both have to be checked
pullOnce := func(opts client.ImagePullOptions) error { pullOnce := func(opts client.ImagePullOptions) error {
reader, err := cli.ImagePull(ctx, imageRef, opts) reader, err := cli.ImagePull(ctx, imageRef, opts)
streamErr := logDockerResponse(logger, reader, err != nil)
if err != nil { if err != nil {
return err return err
} }
return streamErr return logDockerResponse(logger, reader, logger.Debugf)
} }
err = pullOnce(imagePullOptions) err = pullOnce(imagePullOptions)
@@ -83,7 +82,7 @@ func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
return nil return nil
} }
return fmt.Errorf("failed to pull image '%s' (%s): %w", imageRef, input.Platform, err) return fmt.Errorf("failed to pull image '%s' (%s): %w", imageRef, input.Platform, err)
} }).Retry(3)
} }
func getImagePullOptions(ctx context.Context, input NewDockerPullExecutorInput) (client.ImagePullOptions, error) { func getImagePullOptions(ctx context.Context, input NewDockerPullExecutorInput) (client.ImagePullOptions, error) {
+4 -3
View File
@@ -95,15 +95,16 @@ func TestIsPinnedImage(t *testing.T) {
// The pull path reports a failure the daemon sent mid-stream, so it must carry the reason // The pull path reports a failure the daemon sent mid-stream, so it must carry the reason
// whichever of the two shapes the daemon used. // whichever of the two shapes the daemon used.
func TestLogDockerResponseError(t *testing.T) { func TestLogDockerResponseError(t *testing.T) {
logger := logging.New(logtest.New()) logger, hook := logtest.NewLogger()
streamErr := func(line string) error { streamErr := func(line string) error {
return logDockerResponse(logger, io.NopCloser(strings.NewReader(line)), false) return logDockerResponse(logger, io.NopCloser(strings.NewReader(line)), logger.Infof)
} }
require.EqualError(t, streamErr(`{"error":"toomanyrequests: rate limit exceeded"}`), "toomanyrequests: rate limit exceeded") require.EqualError(t, streamErr(`{"error":"toomanyrequests: rate limit exceeded"}`), "toomanyrequests: rate limit exceeded")
require.EqualError(t, streamErr(`{"errorDetail":{"message":"unexpected EOF"}}`), "unexpected EOF") require.EqualError(t, streamErr(`{"errorDetail":{"message":"unexpected EOF"}}`), "unexpected EOF")
require.NoError(t, streamErr(`{"status":"Downloading"}`)) require.NoError(t, streamErr(`{"status":"Downloading"}`))
require.NoError(t, streamErr(`{"stream":"`+strings.Repeat("x", 70000)+`"}`)) require.NoError(t, streamErr(`{"stream":"`+strings.Repeat("x", 70000)+`"}`))
assert.Equal(t, logging.LevelInfo, hook.LastRecord().Level)
require.ErrorIs(t, logDockerResponse(logger, io.NopCloser(io.MultiReader( require.ErrorIs(t, logDockerResponse(logger, io.NopCloser(io.MultiReader(
strings.NewReader(`{"status":"Downloading"}`+"\n"), iotest.ErrReader(io.ErrUnexpectedEOF), strings.NewReader(`{"status":"Downloading"}`+"\n"), iotest.ErrReader(io.ErrUnexpectedEOF),
)), false), io.ErrUnexpectedEOF) )), logger.Infof), io.ErrUnexpectedEOF)
} }
+70 -40
View File
@@ -2,7 +2,7 @@
// Copyright 2020 The nektos/act Authors. All rights reserved. // Copyright 2020 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
@@ -23,6 +23,7 @@ import (
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
"sync"
"time" "time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
@@ -41,13 +42,6 @@ import (
specs "github.com/opencontainers/image-spec/specs-go/v1" specs "github.com/opencontainers/image-spec/specs-go/v1"
) )
// drainGracePeriod bounds how long we wait for an output-copy goroutine to
// finish draining a container's output before returning, so that neither a
// cancellation (waitForCommand) nor a normal container exit (wait) truncates
// the tail of the log. It is a safety bound: in the common case the stream
// reaches EOF and the goroutine returns well before this elapses.
const drainGracePeriod = 2 * time.Second
// NewContainer creates a reference to a container // NewContainer creates a reference to a container
func NewContainer(input *NewContainerInput) ExecutionsEnvironment { func NewContainer(input *NewContainerInput) ExecutionsEnvironment {
// Resolved up front because the image pull runs before the container is created. // Resolved up front because the image pull runs before the container is created.
@@ -90,8 +84,13 @@ func (cr *containerReference) Start(attach bool) common.Executor {
cr.attach().IfBool(attach), cr.attach().IfBool(attach),
cr.start(), cr.start(),
cr.wait().IfBool(attach), cr.wait().IfBool(attach),
cr.tryReadID("-u", func(id int) { cr.UID = id }), func(ctx context.Context) error {
cr.tryReadID("-g", func(id int) { cr.GID = id }), var wg sync.WaitGroup
wg.Go(func() { _ = cr.tryReadID("-u", func(id int) { cr.UID = id })(ctx) })
_ = cr.tryReadID("-g", func(id int) { cr.GID = id })(ctx)
wg.Wait()
return nil
},
func(ctx context.Context) error { func(ctx context.Context) error {
// If this fails, then folders have wrong permissions on non root container // If this fails, then folders have wrong permissions on non root container
if cr.UID != 0 || cr.GID != 0 { if cr.UID != 0 || cr.GID != 0 {
@@ -191,6 +190,9 @@ func (cr *containerReference) Inspect(ctx context.Context) (*Info, error) {
} else if err != nil { } else if err != nil {
return nil, err return nil, err
} }
if cr.containerEnvID != cr.id && result.Container.Config != nil {
cr.containerEnv, cr.containerEnvID = opts.ConvertKVStringsToMap(result.Container.Config.Env), cr.id
}
return containerInfoFromInspect(result.Container), nil return containerInfoFromInspect(result.Container), nil
} }
@@ -314,15 +316,13 @@ func (cr *containerReference) UpdateFromEnv(srcPath string, env *map[string]stri
func (cr *containerReference) UpdateFromImageEnv(env *map[string]string) common.Executor { func (cr *containerReference) UpdateFromImageEnv(env *map[string]string) common.Executor {
envMap := *env envMap := *env
return common.Executor(func(ctx context.Context) error { return common.Executor(func(ctx context.Context) error {
inspect, err := cr.cli.ContainerInspect(ctx, cr.id, client.ContainerInspectOptions{}) if cr.containerEnvID != cr.id {
if err != nil { if _, err := cr.Inspect(ctx); err != nil {
common.Logger(ctx).Error(err) common.Logger(ctx).Error(err)
return fmt.Errorf("inspect container: %w", err) return fmt.Errorf("inspect container: %w", err)
}
} }
if inspect.Container.Config == nil { for key, value := range cr.containerEnv {
return nil
}
for key, value := range opts.ConvertKVStringsToMap(inspect.Container.Config.Env) {
if envMap[key] == "" { if envMap[key] == "" {
envMap[key] = value envMap[key] = value
} else if key == "PATH" { } else if key == "PATH" {
@@ -366,22 +366,30 @@ type containerReference struct {
id string id string
input *NewContainerInput input *NewContainerInput
pullPolicy string pullPolicy string
daemonOS string
UID int UID int
GID int GID int
// attachDone is closed by the attach() streaming goroutine once it has // attachDone is closed by the attach() streaming goroutine once it has
// drained and flushed the container's output. wait() blocks on it so the // drained and flushed the container's output. wait() blocks on it so the
// tail of the log lands before the step proceeds. // tail of the log lands before the step proceeds.
attachDone chan struct{} attachDone chan struct{}
containerEnv map[string]string // fixed at creation, cached by inspect for containerEnvID
containerEnvID string
LinuxContainerEnvironmentExtensions LinuxContainerEnvironmentExtensions
} }
func GetDockerClient(ctx context.Context) (cli client.APIClient, err error) { func GetDockerClient(ctx context.Context) (client.APIClient, error) {
cli, _, err := newDockerClient(ctx)
return cli, err
}
func newDockerClient(ctx context.Context) (cli client.APIClient, daemonOS string, err error) {
dockerHost := os.Getenv("DOCKER_HOST") dockerHost := os.Getenv("DOCKER_HOST")
if strings.HasPrefix(dockerHost, "ssh://") { if strings.HasPrefix(dockerHost, "ssh://") {
var args []string var args []string
if args, err = sshArgs(dockerHost); err != nil { if args, err = sshArgs(dockerHost); err != nil {
return nil, err return nil, "", err
} }
cli, err = client.New( cli, err = client.New(
client.WithHost("http://docker.example.com"), // the dialer below ignores the address client.WithHost("http://docker.example.com"), // the dialer below ignores the address
@@ -391,16 +399,17 @@ func GetDockerClient(ctx context.Context) (cli client.APIClient, err error) {
cli, err = client.New(client.FromEnv) cli, err = client.New(client.FromEnv)
} }
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to connect to docker daemon: %w", err) return nil, "", fmt.Errorf("failed to connect to docker daemon: %w", err)
} }
// Best-effort API version negotiation, matching the old client.NegotiateAPIVersion // Best-effort API version negotiation, matching the old client.NegotiateAPIVersion
// behaviour. Ping failures here are non-fatal: the connection is exercised again on // behaviour. Ping failures here are non-fatal: the connection is exercised again on
// the first real call, and the client falls back to the daemon's default API version. // the first real call, and the client falls back to the daemon's default API version.
if _, err := cli.Ping(ctx, client.PingOptions{NegotiateAPIVersion: true}); err != nil { ping, err := cli.Ping(ctx, client.PingOptions{NegotiateAPIVersion: true})
if err != nil {
common.Logger(ctx).Warnf("docker daemon ping during version negotiation failed, continuing: %v", err) common.Logger(ctx).Warnf("docker daemon ping during version negotiation failed, continuing: %v", err)
} }
return cli, nil return cli, ping.OSType, nil
} }
func GetHostInfo(ctx context.Context) (system.Info, error) { func GetHostInfo(ctx context.Context) (system.Info, error) {
@@ -440,16 +449,32 @@ func RunnerArch(ctx context.Context) string {
return info.Architecture return info.Architecture
} }
func (cr *containerReference) targetOS(ctx context.Context) string {
if cr.daemonOS == "" {
cr.daemonOS = "linux"
if cr.cli != nil {
if result, err := cr.cli.Info(ctx, client.InfoOptions{}); err == nil && result.Info.OSType != "" {
cr.daemonOS = result.Info.OSType
}
}
}
return cr.daemonOS
}
func (cr *containerReference) markExecutable(ctx context.Context) bool {
return hostOS == "windows" && cr.targetOS(ctx) == "linux"
}
func (cr *containerReference) connect() common.Executor { func (cr *containerReference) connect() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
if cr.cli != nil { if cr.cli != nil {
return nil return nil
} }
cli, err := GetDockerClient(ctx) cli, daemonOS, err := newDockerClient(ctx)
if err != nil { if err != nil {
return err return err
} }
cr.cli = cli cr.cli, cr.daemonOS = cli, daemonOS
return nil return nil
} }
} }
@@ -459,6 +484,7 @@ func (cr *containerReference) Close() common.Executor {
if cr.cli != nil { if cr.cli != nil {
err := cr.cli.Close() err := cr.cli.Close()
cr.cli = nil cr.cli = nil
cr.daemonOS = ""
if err != nil { if err != nil {
return fmt.Errorf("failed to close client: %w", err) return fmt.Errorf("failed to close client: %w", err)
} }
@@ -603,7 +629,8 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
copts.privileged = false copts.privileged = false
} }
containerConfig, err := parse(flags, copts, runtime.GOOS) serverOS := cr.targetOS(ctx)
containerConfig, err := parse(flags, copts, serverOS)
if err != nil { if err != nil {
return fmt.Errorf("cannot process container options: '%s': '%w'", options, err) return fmt.Errorf("cannot process container options: '%s': '%w'", options, err)
} }
@@ -618,7 +645,7 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
// For Gitea, forcing --privileged off is not enough, other options reach the host too // For Gitea, forcing --privileged off is not enough, other options reach the host too
if !hostConfig.Privileged { if !hostConfig.Privileged {
trusted, err := parseOptionsHostConfig(cr.input.RunnerOptions) trusted, err := parseOptionsHostConfig(cr.input.RunnerOptions, serverOS)
if err != nil { if err != nil {
return err return err
} }
@@ -731,9 +758,7 @@ func (cr *containerReference) create(capAdd, capDrop []string) common.Executor {
cr.sanitizeConfig(ctx, hostConfig) cr.sanitizeConfig(ctx, hostConfig)
var networkingConfig *network.NetworkingConfig var networkingConfig *network.NetworkingConfig
n := hostConfig.NetworkMode if isUserDefinedNetwork(hostConfig.NetworkMode) && len(input.NetworkAliases) > 0 {
// IsUserDefined and IsHost are broken on windows
if n.IsUserDefined() && n != "host" && len(input.NetworkAliases) > 0 {
networkingConfig = &network.NetworkingConfig{ networkingConfig = &network.NetworkingConfig{
EndpointsConfig: map[string]*network.EndpointSettings{ EndpointsConfig: map[string]*network.EndpointSettings{
input.NetworkMode: {Aliases: input.NetworkAliases}, input.NetworkMode: {Aliases: input.NetworkAliases},
@@ -838,6 +863,7 @@ func (cr *containerReference) tryReadID(opt string, cbk func(id int)) common.Exe
return nil return nil
} }
defer resp.Close() defer resp.Close()
defer context.AfterFunc(ctx, resp.Close)()
sid, err := resp.Reader.ReadString('\n') sid, err := resp.Reader.ReadString('\n')
if err != nil { if err != nil {
@@ -883,7 +909,7 @@ func (cr *containerReference) waitForCommand(ctx context.Context, resp client.Hi
if err != nil { if err != nil {
logger.Debugf("Failed to drain command output after cancellation: %v", err) logger.Debugf("Failed to drain command output after cancellation: %v", err)
} }
case <-time.After(drainGracePeriod): case <-time.After(DrainGracePeriod):
logger.Warn("Timed out draining command output after cancellation") logger.Warn("Timed out draining command output after cancellation")
} }
@@ -915,11 +941,8 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, ski
logger.Error(err) logger.Error(err)
} }
}() }()
extractionRoot := "/" extractionRoot := cr.extractionRoot(dstPath)
if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") { if err := writeDirTar(ctx, tarFile, path.Join("/", strings.TrimPrefix(dstPath, extractionRoot)), srcPath, useGitIgnore, skipGitDir, cr.UID, cr.GID, cr.markExecutable(ctx)); err != nil {
extractionRoot = actPath // keeps tar entries from crossing the image's /var/run symlink
}
if err := writeDirTar(ctx, tarFile, path.Join("/", strings.TrimPrefix(dstPath, extractionRoot)), srcPath, useGitIgnore, skipGitDir, cr.UID, cr.GID); err != nil {
return err return err
} }
@@ -931,6 +954,13 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, ski
} }
} }
func (cr *containerReference) extractionRoot(dstPath string) string {
if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") {
return actPath // keeps tar entries from crossing the image's /var/run symlink
}
return "/"
}
func (cr *containerReference) copyContent(dstPath string, files ...*FileEntry) common.Executor { func (cr *containerReference) copyContent(dstPath string, files ...*FileEntry) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
if cr.id == "" { if cr.id == "" {
@@ -1023,7 +1053,7 @@ func (cr *containerReference) wait() common.Executor {
if cr.attachDone != nil { if cr.attachDone != nil {
select { select {
case <-cr.attachDone: case <-cr.attachDone:
case <-time.After(drainGracePeriod): case <-time.After(DrainGracePeriod):
logger.Warn("Timed out draining container output") logger.Warn("Timed out draining container output")
} }
cr.attachDone = nil cr.attachDone = nil
@@ -1090,12 +1120,12 @@ func resetOption[T any](logger *logging.Logger, option string, field *T, trusted
// parseOptionsHostConfig parses one options string on its own, to see what it alone asks for. // parseOptionsHostConfig parses one options string on its own, to see what it alone asks for.
// Even "" goes through the parser, or its empty slices and maps would differ from a real parse. // Even "" goes through the parser, or its empty slices and maps would differ from a real parse.
func parseOptionsHostConfig(options string) (*container.HostConfig, error) { func parseOptionsHostConfig(options, serverOS string) (*container.HostConfig, error) {
flags, copts, _, err := parseContainerOptions(options) flags, copts, _, err := parseContainerOptions(options)
if err != nil { if err != nil {
return nil, err return nil, err
} }
containerConfig, err := parse(flags, copts, runtime.GOOS) containerConfig, err := parse(flags, copts, serverOS)
if err != nil { if err != nil {
return nil, fmt.Errorf("cannot process container options: '%s': '%w'", options, err) return nil, fmt.Errorf("cannot process container options: '%s': '%w'", options, err)
} }
+166 -6
View File
@@ -27,10 +27,12 @@ import (
"gitea.com/gitea/runner/act/common/logging/logtest" "gitea.com/gitea/runner/act/common/logging/logtest"
cerrdefs "github.com/containerd/errdefs" cerrdefs "github.com/containerd/errdefs"
"github.com/kballard/go-shellquote"
"github.com/moby/moby/api/pkg/stdcopy" "github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount" "github.com/moby/moby/api/types/mount"
"github.com/moby/moby/api/types/network" "github.com/moby/moby/api/types/network"
"github.com/moby/moby/api/types/system"
mobyclient "github.com/moby/moby/client" mobyclient "github.com/moby/moby/client"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
@@ -43,14 +45,15 @@ func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
Return(mobyclient.ContainerInspectResult{Container: container.InspectResponse{Config: &container.Config{Env: []string{ Return(mobyclient.ContainerInspectResult{Container: container.InspectResponse{Config: &container.Config{Env: []string{
"PATH=/opt/node/x64/bin:/usr/bin:/bin", "CONFLICT_VAR=container-value", "SOME_RANDOM_VAR=", "PATH=/opt/node/x64/bin:/usr/bin:/bin", "CONFLICT_VAR=container-value", "SOME_RANDOM_VAR=",
"ANOTHER_ONE=old-value", "ANOTHER_ONE=BUT_I_HAVE_VALUE", "LITERAL='quoted'=$HOME\nsecond line", "ANOTHER_ONE=old-value", "ANOTHER_ONE=BUT_I_HAVE_VALUE", "LITERAL='quoted'=$HOME\nsecond line",
}}}}, nil) }}}}, nil).Once()
env := map[string]string{ env := map[string]string{
"PATH": "/custom/bin", "PATH": "/custom/bin",
"RANDOM_VAR": "WITH_VALUE", "RANDOM_VAR": "WITH_VALUE",
"ANOTHER_VAR": "", "ANOTHER_VAR": "",
"CONFLICT_VAR": "step-value", "CONFLICT_VAR": "step-value",
} }
require.NoError(t, (&containerReference{cli: client, id: "amd64-job"}).UpdateFromImageEnv(&env)(t.Context())) cr := &containerReference{cli: client, id: "amd64-job"}
require.NoError(t, cr.UpdateFromImageEnv(&env)(t.Context()))
assert.Equal(t, map[string]string{ assert.Equal(t, map[string]string{
"PATH": "/custom/bin:/opt/node/x64/bin:/usr/bin:/bin", "PATH": "/custom/bin:/opt/node/x64/bin:/usr/bin:/bin",
"RANDOM_VAR": "WITH_VALUE", "RANDOM_VAR": "WITH_VALUE",
@@ -60,11 +63,23 @@ func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
"CONFLICT_VAR": "step-value", "CONFLICT_VAR": "step-value",
"LITERAL": "'quoted'=$HOME\nsecond line", "LITERAL": "'quoted'=$HOME\nsecond line",
}, env) }, env)
pathEnv := map[string]string{}
require.NoError(t, cr.UpdateFromImageEnv(&pathEnv)(t.Context()))
assert.Equal(t, "/opt/node/x64/bin:/usr/bin:/bin", pathEnv["PATH"])
client.AssertExpectations(t)
} }
type mockDockerClient struct { type mockDockerClient struct {
mobyclient.APIClient mobyclient.APIClient
mock.Mock mock.Mock
info system.Info
infoErr error
infoCalls int
}
func (m *mockDockerClient) Info(_ctx context.Context, _opts mobyclient.InfoOptions) (mobyclient.SystemInfoResult, error) {
m.infoCalls++
return mobyclient.SystemInfoResult{Info: m.info}, m.infoErr
} }
func (m *mockDockerClient) ServerVersion(ctx context.Context, opts mobyclient.ServerVersionOptions) (mobyclient.ServerVersionResult, error) { func (m *mockDockerClient) ServerVersion(ctx context.Context, opts mobyclient.ServerVersionOptions) (mobyclient.ServerVersionResult, error) {
@@ -112,6 +127,11 @@ func (m *mockDockerClient) CopyToContainer(ctx context.Context, id string, optio
return args.Get(0).(mobyclient.CopyToContainerResult), args.Error(1) return args.Get(0).(mobyclient.CopyToContainerResult), args.Error(1)
} }
func (m *mockDockerClient) ContainerStatPath(ctx context.Context, id string, options mobyclient.ContainerStatPathOptions) (mobyclient.ContainerStatPathResult, error) {
args := m.Called(ctx, id, options)
return args.Get(0).(mobyclient.ContainerStatPathResult), args.Error(1)
}
func (m *mockDockerClient) ContainerInspect(ctx context.Context, id string, opts mobyclient.ContainerInspectOptions) (mobyclient.ContainerInspectResult, error) { func (m *mockDockerClient) ContainerInspect(ctx context.Context, id string, opts mobyclient.ContainerInspectOptions) (mobyclient.ContainerInspectResult, error) {
args := m.Called(ctx, id, opts) args := m.Called(ctx, id, opts)
return args.Get(0).(mobyclient.ContainerInspectResult), args.Error(1) return args.Get(0).(mobyclient.ContainerInspectResult), args.Error(1)
@@ -341,6 +361,21 @@ func TestDockerAttachStreamClosesWithItsContext(t *testing.T) {
client.AssertExpectations(t) client.AssertExpectations(t)
} }
func TestDockerReadIDStreamClosesWithItsContext(t *testing.T) {
conn, peer := net.Pipe()
t.Cleanup(func() { _ = peer.Close() })
client := &mockDockerClient{}
client.On("ExecCreate", mock.Anything, "123", mock.AnythingOfType("client.ExecCreateOptions")).
Return(mobyclient.ExecCreateResult{ID: "id"}, nil)
client.On("ExecAttach", mock.Anything, "id", mobyclient.ExecAttachOptions{}).
Return(mobyclient.ExecAttachResult{HijackedResponse: mobyclient.NewHijackedResponse(conn, "")}, nil)
ctx, cancel := context.WithCancel(t.Context())
cancel()
require.NoError(t, (&containerReference{id: "123", cli: client}).tryReadID("-u", func(int) { t.Error("read an ID after cancellation") })(ctx))
client.AssertExpectations(t)
}
func TestDockerWaitFailure(t *testing.T) { func TestDockerWaitFailure(t *testing.T) {
ctx := t.Context() ctx := t.Context()
statusCh := make(chan container.WaitResponse, 1) statusCh := make(chan container.WaitResponse, 1)
@@ -530,6 +565,9 @@ func TestRejectsMissingContainer(t *testing.T) {
} }
func TestPublicCopyPipelineHandlesStaleID(t *testing.T) { func TestPublicCopyPipelineHandlesStaleID(t *testing.T) {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "x"), nil, 0o644))
for _, testcase := range []struct { for _, testcase := range []struct {
name, destination, extractionRoot, entry string name, destination, extractionRoot, entry string
directory bool directory bool
@@ -557,8 +595,6 @@ func TestPublicCopyPipelineHandlesStaleID(t *testing.T) {
cr := &containerReference{id: "stale", cli: client, input: &NewContainerInput{Name: "job-1"}} cr := &containerReference{id: "stale", cli: client, input: &NewContainerInput{Name: "job-1"}}
if testcase.directory { if testcase.directory {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "x"), nil, 0o644))
require.NoError(t, cr.CopyDir(testcase.destination, source+string(filepath.Separator), false, true)(ctx)) require.NoError(t, cr.CopyDir(testcase.destination, source+string(filepath.Separator), false, true)(ctx))
} else { } else {
require.NoError(t, cr.Copy(testcase.destination, &FileEntry{Name: "x", Mode: 0o644})(ctx)) require.NoError(t, cr.Copy(testcase.destination, &FileEntry{Name: "x", Mode: 0o644})(ctx))
@@ -689,11 +725,11 @@ func TestMergeContainerConfigsDropsVolumeDriversFromWorkflows(t *testing.T) {
} }
func TestMergeContainerConfigsKeepsTheRunnersFilesAndEnvToItself(t *testing.T) { func TestMergeContainerConfigsKeepsTheRunnersFilesAndEnvToItself(t *testing.T) {
hostFile := filepath.Join(t.TempDir(), "host.env") hostFile := filepath.Join(t.TempDir(), "host file.env")
require.NoError(t, os.WriteFile(hostFile, []byte("STOLEN=from-the-host\n"), 0o600)) require.NoError(t, os.WriteFile(hostFile, []byte("STOLEN=from-the-host\n"), 0o600))
t.Setenv("RUNNER_SECRET", "s3cr3t") t.Setenv("RUNNER_SECRET", "s3cr3t")
for _, option := range []string{"--env-file " + hostFile, "--label-file " + hostFile} { for _, option := range []string{shellquote.Join("--env-file", hostFile), shellquote.Join("--label-file", hostFile)} {
logger := logging.New(logtest.New()) logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{NetworkMode: "bridge", WorkflowOptions: option}} cr := &containerReference{input: &NewContainerInput{NetworkMode: "bridge", WorkflowOptions: option}}
@@ -933,6 +969,7 @@ func TestContainerInfoFromInspect(t *testing.T) {
"/etc/hosts": "", "/etc/hosts": "",
"/etc/hostname": "", "/etc/hostname": "",
}, info.Mounts) }, info.Mounts)
assert.Equal(t, "/var/lib/docker/volumes/job/_data/subdir", info.DaemonPath("/workspace/owner/repo/subdir"))
}) })
} }
@@ -998,6 +1035,39 @@ func TestDockerCreateKeepsInputEntrypoint(t *testing.T) {
} }
} }
func TestDockerCreateNetworkAliases(t *testing.T) {
for _, test := range []struct {
networkMode string
wantAliases bool
}{
{"job-network", true},
{"host", false},
{"bridge", false},
{"default", false},
{"none", false},
{"container:x", false},
} {
t.Run(test.networkMode, func(t *testing.T) {
cr := &containerReference{
input: &NewContainerInput{NetworkMode: test.networkMode, NetworkAliases: []string{"service"}},
cli: &probeClient{create: func(_ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
assert.Equal(t, container.NetworkMode(test.networkMode), opts.HostConfig.NetworkMode)
if test.wantAliases {
assert.Equal(t, &network.NetworkingConfig{EndpointsConfig: map[string]*network.EndpointSettings{
"job-network": {Aliases: []string{"service"}},
}}, opts.NetworkingConfig)
} else {
assert.Nil(t, opts.NetworkingConfig)
}
return mobyclient.ContainerCreateResult{ID: "created"}, nil
}},
}
require.NoError(t, cr.create(nil, nil)(t.Context()))
assert.Equal(t, "created", cr.id)
})
}
}
func TestMergeContainerConfigsKeepsNetworkAliasesFromOptions(t *testing.T) { func TestMergeContainerConfigsKeepsNetworkAliasesFromOptions(t *testing.T) {
logger := logging.New(logtest.New()) logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{ cr := &containerReference{input: &NewContainerInput{
@@ -1019,3 +1089,93 @@ func TestSupportsContainerImagePlatformDaemonError(t *testing.T) {
require.ErrorContains(t, err, "cannot connect to the Docker daemon") require.ErrorContains(t, err, "cannot connect to the Docker daemon")
assert.False(t, supported) assert.False(t, supported)
} }
func TestContainerOptionsUseDaemonOS(t *testing.T) {
for _, testcase := range []struct {
name string
serverOS string
infoErr error
device string
expected container.DeviceMapping
}{
{name: "Linux daemon", serverOS: "linux", device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "unknown daemon", device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "unavailable daemon info", infoErr: errors.New("unavailable"), device: "/dev/kvm:/dev/kvm:rwm", expected: container.DeviceMapping{PathOnHost: "/dev/kvm", PathInContainer: "/dev/kvm", CgroupPermissions: "rwm"}},
{name: "Windows daemon", serverOS: "windows", device: "class/01234567-89ab-cdef-0123-456789abcdef", expected: container.DeviceMapping{PathOnHost: "class/01234567-89ab-cdef-0123-456789abcdef"}},
} {
t.Run(testcase.name, func(t *testing.T) {
cli := &mockDockerClient{info: system.Info{OSType: testcase.serverOS}, infoErr: testcase.infoErr}
cr := &containerReference{cli: cli, input: &NewContainerInput{RunnerOptions: "--device=" + testcase.device, WorkflowOptions: "--device=" + testcase.device, NetworkMode: "bridge"}}
for range 2 {
hostConfig := &container.HostConfig{}
require.NoError(t, cr.mergeContainerConfigs(t.Context(), &container.Config{}, hostConfig))
assert.Equal(t, []container.DeviceMapping{testcase.expected}, hostConfig.Devices)
}
assert.Equal(t, 1, cli.infoCalls)
})
}
}
func TestConnectTakesDaemonOSFromPing(t *testing.T) {
var infos atomic.Int32
daemon := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
writer.Header().Set("API-Version", "1.47")
writer.Header().Set("OSType", "windows")
if strings.HasSuffix(request.URL.Path, "/info") {
infos.Add(1)
}
}))
t.Cleanup(daemon.Close)
t.Setenv("DOCKER_HOST", daemon.URL)
cr := &containerReference{input: &NewContainerInput{}}
require.NoError(t, cr.connect()(t.Context()))
assert.Equal(t, "windows", cr.targetOS(t.Context()))
assert.Zero(t, infos.Load())
}
func TestContainerCopyModes(t *testing.T) {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "run.sh"), []byte("#!/bin/sh\n"), 0o666))
require.NoError(t, os.Chmod(filepath.Join(source, "run.sh"), 0o666))
for _, testcase := range []struct {
hostOS, serverOS string
expected int64
}{
{"windows", "linux", 0o755},
{"windows", "windows", 0o666},
{"linux", "linux", 0o666},
} {
t.Run(testcase.hostOS+"-"+testcase.serverOS, func(t *testing.T) {
file := &FileEntry{Name: "run.sh", Mode: 0o600, Body: "#!/bin/sh\n"}
original := hostOS
hostOS = testcase.hostOS
t.Cleanup(func() { hostOS = original })
cli := &mockDockerClient{info: system.Info{OSType: testcase.serverOS}}
var modes []int64
cli.On("CopyToContainer", mock.Anything, "job", mock.Anything).
Run(func(args mock.Arguments) {
reader := tar.NewReader(args.Get(2).(mobyclient.CopyToContainerOptions).Content)
header, err := reader.Next()
require.NoError(t, err)
modes = append(modes, header.Mode)
assert.Equal(t, "run.sh", filepath.Base(header.Name))
content, err := io.ReadAll(reader)
require.NoError(t, err)
assert.Equal(t, file.Body, string(content))
_, err = reader.Next()
require.ErrorIs(t, err, io.EOF)
}).Return(mobyclient.CopyToContainerResult{}, nil).Twice()
cr := &containerReference{cli: cli, id: "job", input: &NewContainerInput{}}
require.NoError(t, cr.copyContent("/workspace", file)(t.Context()))
require.NoError(t, cr.copyDir("/workspace", source+string(filepath.Separator), false, true)(t.Context()))
assert.Equal(t, []int64{0o600, testcase.expected}, modes)
if testcase.hostOS == "windows" {
assert.Equal(t, 1, cli.infoCalls)
} else {
assert.Zero(t, cli.infoCalls)
}
cli.AssertExpectations(t)
})
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2026 The Gitea Authors. All rights reserved. // Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd || freebsd))
package container package container
Loaded 50 of 196 files, more files were not shown because too many files have changed in this diff. Show more