Compare commits

...
15 Commits
Author SHA1 Message Date
Renovate Bot 6e42621b4a chore(deps): update dependencies (#1365)
Co-authored-by: Renovate Bot <[email protected]>
2026-11-02 08:16:42 +00:00
adamaruandsilverwind 228effab64 fix(dind): double-fork in s6-notifyoncheck to eliminate zombie processes (#1364)
In `scripts/s6/docker/run`, `s6-notifyoncheck` runs without the `-d` flag:

```bash
exec s6-notifyoncheck -n 0 dockerd-entrypoint.sh
```

Without `-d`, `s6-notifyoncheck` performs a single fork to execute the background checking process. The parent immediately replaces itself via `exec` with `dockerd-entrypoint.sh` (which calls `exec rootlesskit` on `dind-rootless`).

Because only a single fork occurred, the background checking child remains parented to `rootlesskit`. When the check passes and the process terminates:
1. `rootlesskit` did not spawn this process and never reaps it via `waitpid()`.
2. Under standard Linux process semantics, `s6-svscan` (PID 1) cannot adopt or reap the child because its parent (`rootlesskit`) is still running.
3. The terminated process remains as a permanent zombie (`[s6-notifyoncheck] <defunct>`) for the container's entire lifespan.

Adding `-d` instructs `s6-notifyoncheck` to double-fork, detaching the background check so it is immediately adopted by `s6-svscan` (PID 1) and cleanly reaped upon exit.

---------

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1364
Reviewed-by: silverwind <[email protected]>
Co-authored-by: adamaru <[email protected]>
2026-11-01 07:20:20 +00:00
silverwindandbircni a46d3f3eda test(e2e): speed up and cover exec and Kubernetes modes (#1363)
Follow-up to https://gitea.com/gitea/runner/pulls/1362, whose crash e2e missed because it only drove the daemon on Docker.

- `dispatch`, `failure_paths` and `reusable_workflow` run through the daemon and `exec`, each on Docker and one shared k3s cluster.
- Fixed sleeps become gates the test releases, the job timeout uses a 15s runner deadline, k3s boots alongside Gitea, Gitea's data is on tmpfs.
- Outside a container, `make test-e2e` runs `go test` in a Linux container on its own network like CI, so local runs take the same paths.

Local wall time:

| | Before | After |
|---|---|---|
| Suite | 84s | 27–35s |
| `job_timeout` | 64s | 17s |
| `reconnect` | 22s | 4.5s |
| `capacity/serial` | 20s | 2s |
| `cli_shutdown_drains` | 14s | 2.3s |
| `kubernetes` | 31s | 21–23s |

_Written by Claude (Opus 5.5)._

---------

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1363
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-31 10:47:35 +00:00
2e72cf902a enhance(kubernetes): fetch job pod copies from the runner over the pod network (#1354)
The Kubernetes backend copies actions and workspaces into job containers with `tar` over `pods/exec`, so every byte goes through the API server. On managed clusters that often sits in other zones than the nodes, so each byte can cross zones twice, and the copies load the API server.

With `kubernetes.transfer.mode: auto` or `direct`, an init container copies `gitea-runner` into the job pod, and the job container fetches directories and copies of 64 KiB or more gzipped from the runner over the pod network. `auto` falls back to the API server for a job pod that cannot reach the runner, `direct` fails the job.

Copies can hold secrets, so each is served once, over TLS 1.3 pinned to a certificate the runner generates, only to the job pod's IPs.

Copying setup-go@v5, cache@v4, setup-java@v4 and build-push-action@v6 sends 11.6 MB through the API server on main and 532 B with this PR.

Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1354
Reviewed-by: silverwind <[email protected]>
Co-authored-by: Lunny Xiao <[email protected]>
2026-10-31 08:00:36 +00:00
silverwind f4741df90d fix(exec): load local reusable workflows from disk with --no-skip-checkout (#1362)
`gitea-runner exec --no-skip-checkout` panics with a nil pointer dereference on any job that calls a local reusable workflow (`uses: ./.gitea/workflows/x.yml`).

Fetching a local reusable workflow from the instance was gated on `NoSkipCheckout`, but it reads the repository and SHA from the task's preset context, which only the daemon has. `exec` always has the workflow on disk, so the fetch is now gated on the preset instead, and `--no-skip-checkout` only governs checkout again. Daemon behavior is unchanged.

The `exec` e2e test now also runs the existing reusable caller/callee pair with `--no-skip-checkout`.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1362
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-31 05:24:36 +00:00
261ab374ce enhance(cache): scope cache entries by ref like GitHub (#1361)
Cache entries were only scoped by repository, so an entry a pull request saved could be restored on the default branch, for example through a `restore-keys` prefix.

Entries now carry the ref of the job that saved them. As on GitHub ([Restrictions for accessing a cache](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#restrictions-for-accessing-a-cache)), a job restores entries of its own ref first, then of its pull request's base branch, then of the default branch. Pull request events are scoped by their pull request number.

Entries saved before this change are no longer restored and expire through `cache.retention`, so caches start cold once after the upgrade. Runners sharing a cache, and an external `cache-server`, should be upgraded together.

Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1361
Reviewed-by: silverwind <[email protected]>
Co-authored-by: JorgeDLMV <[email protected]>
2026-10-30 22:34:45 +00:00
KenanZhuandsilverwind ca8e9c3c64 enhance: add GITEA_* aliases for built-in workflow environment variables (#1358)
Closes https://gitea.com/gitea/runner/issues/1317, which makes https://github.com/go-gitea/gitea/issues/39813 redundant as the runner sets these variables.

Every built-in `GITHUB_*` environment variable the runner sets is now also available as `GITEA_*` with the same value, so Gitea-native workflows don't have to rely on GitHub naming. The `GITEA_*` names mirror the `GITHUB_*` ones after their last write, so they stay equal and are unset together.

Workflow, job and runner `envs` values for these `GITEA_*` names are now overwritten, the same as for `GITHUB_*`.

---------

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1358
Reviewed-by: silverwind <[email protected]>
Co-authored-by: KenanZhu <[email protected]>
2026-10-30 21:21:02 +00:00
silverwindandbircni d8d4a4657d fix: scope the action identity to the step executing an action (#1359)
`github.action_repository`/`github.action_ref` and their env vars now belong to the step executing an action, like on GitHub: set for remote actions, empty for `run`, `docker://` and local steps. Before, a local action nested in a remote composite resolved `$/` against the remote action instead of the workflow's repository.

Updates actionslib to v1.5.1, which accepts `$/` as the repository root action.

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1359
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-29 17:41:27 +00:00
silverwind 5fd2f82735 test: fix race in TestTreeKillCancelBeforeCaptureKillsTree (#1360)
`TestTreeKillCancelBeforeCaptureKillsTree` fails intermittently on CI, e.g. https://gitea.com/gitea/runner/actions/runs/1155856.

Dash, the Ubuntu `/bin/sh`, points a background job's stdin at `/dev/null` before applying its redirections, so `cat <&0 &` read EOF and exited at once and the liveness check raced it. The pipe is now handed over on fd 3, which dash leaves alone.

Introduced in https://gitea.com/gitea/runner/pulls/1338

Reviewed-on: https://gitea.com/gitea/runner/pulls/1360
Reviewed-by: Zettat123 <[email protected]>
2026-10-29 16:46:53 +00:00
silverwind c6567296a0 fix: stop the docker proxy probe from leaking its container (#1357)
When the probe's deadline expired during `ContainerCreate`, dockerd still created the container but the probe never got its ID, leaving it in `Created` with its image's anonymous volumes. A slow daemon leaked one per job.

The probe now runs detached with its own timeout, so the create is never cancelled and the container is always removed. Callers still fall back after their own deadline, and the runner waits for pending probes before exiting.

Fixes: https://gitea.com/gitea/runner/issues/1353
Reviewed-on: https://gitea.com/gitea/runner/pulls/1357
Reviewed-by: Zettat123 <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-29 15:53:32 +00:00
Renovate Bot e3607b576e fix(deps): update module golang.org/x/net to v0.62.0 [security] (#1356)
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [golang.org/x/net](https://pkg.go.dev/golang.org/x/net) | [`v0.61.0` → `v0.62.0`](https://cs.opensource.google/go/x/net/+/refs/tags/v0.61.0...refs/tags/v0.62.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fnet/v0.62.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fnet/v0.61.0/v0.62.0?slim=true) |

---

### Avoid quadratic complexity when coalescing text nodes in html in golang.org/x/net
[CVE-2026-78661](https://nvd.nist.gov/vuln/detail/CVE-2026-78661) / [GO-2026-6860](https://pkg.go.dev/vuln/GO-2026-6860)

<details>
<summary>More information</summary>

#### Details
When coalescing adjacent text nodes in addText and fosterParent, appending to an existing TextNode's Data via string concatenation (n.Data += text) reallocated and copied the accumulated string on every token. Inputs containing many short text runs separated by ignored end tags (such as "a</x>") caused O(n^2) allocations and byte copies.

Accumulate coalesced text in per-node byte slices during parsing and materialize Data strings once at the end of parse.

#### Severity
Unknown

#### References
- [https://go.dev/cl/849365](https://go.dev/cl/849365)
- [https://go.dev/issue/82224](https://go.dev/issue/82224)
- [https://groups.google.com/g/golang-announce/c/sqrs2vTx_Xs](https://groups.google.com/g/golang-announce/c/sqrs2vTx_Xs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6860) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Check all ancestor namespace transitions when rendering raw text in html in golang.org/x/net
[CVE-2026-97045](https://nvd.nist.gov/vuln/detail/CVE-2026-97045) / [GO-2026-6866](https://pkg.go.dev/vuln/GO-2026-6866)

<details>
<summary>More information</summary>

#### Details
In childTextNodesAreLiteral, walking up the tree previously stopped at the nearest foreign ancestor once an HTML integration point was found. However, that integration point may itself be a descendant of an element that was foster-parented into a foreign namespace (such as <mglyph> or <malignmark> fostered under <mtext>), causing a parser to resolve the subtree in a different foreign namespace upon re-parsing the rendered output.

Continue walking the full ancestor chain and verify that every transition from a non-empty namespace occurs at an HTML integration point before emitting literal text.

#### Severity
Unknown

#### References
- [https://go.dev/cl/855121](https://go.dev/cl/855121)
- [https://go.dev/issue/82622](https://go.dev/issue/82622)
- [https://groups.google.com/g/golang-announce/c/sqrs2vTx_Xs](https://groups.google.com/g/golang-announce/c/sqrs2vTx_Xs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6866) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xODEuNyIsInVwZGF0ZWRJblZlciI6IjQ0LjE4MS43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1356
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
2026-10-28 06:33:58 +00:00
Lunny XiaoandZettat123 ffd80101a3 enhance(kubernetes): gzip copies into job containers over exec (#1355)
The Kubernetes backend copies actions and other files into the job container with `tar` over `pods/exec`, so the data goes through the API server, which on managed clusters often sits in other zones than the nodes (each copied byte then crosses zones twice). Actions are megabytes of JavaScript bundles that gzip about fivefold:

| Action | tar | gzip |
|---|---|---|
| actions/setup-go@v5 | 8.0 MB | 1.5 MB |
| actions/cache@v4 | 11.6 MB | 2.0 MB |
| actions/setup-java@v4 | 15.3 MB | 2.5 MB |
| docker/build-push-action@v6 | 12.9 MB | 3.0 MB |

The startup probe now also reports whether the job container has `gzip`. If it does, `CopyDir` and copies of 64 KiB or more are gzipped at `BestSpeed` and extracted with `gzip -dc | tar -xf -`. Smaller copies (run scripts, file command files) stay uncompressed, where gzip would save next to nothing and cost a process in the pod. Images without `gzip` keep the plain tar stream, so the image requirements do not change.

This is independent of #1354 and gets most of its traffic saving with no new moving parts.

---
*Written by Codet (AI coding agent) on behalf of @lunny.*

---------

Co-authored-by: Zettat123 <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1355
Reviewed-by: Zettat123 <[email protected]>
2026-10-28 06:00:51 +00:00
b712c91c2d test(e2e): cover host mode, reusable workflows, scheduling, reconnect and CLI flags (#1347)
Extends the e2e suite with scenarios for host labels, local `workflow_call`, label routing, `runner.capacity`, a dropped and restored connection to Gitea, and the daemon's `--once`, `--labels` and `shutdown_timeout` behaviour.

Stacked on https://gitea.com/gitea/runner/pulls/1346, so its commit shows here until that merges. The new scenarios pass locally against gitea/gitea:latest and main-nightly.

Assisted-by: Codet:codet
Co-authored-by: Zettat123 <[email protected]>
Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1347
Reviewed-by: Zettat123 <[email protected]>
2026-10-28 04:32:50 +00:00
d77e216674 fix(exec): serve the v4 artifact API so upload-artifact v4 and later work (#1341)
`exec` forwarded the toolkit's ArtifactService calls to an instance it never registered, and its random runtime token held no backend IDs, so `actions/upload-artifact` and `actions/download-artifact` v4 and later failed.

The local artifact server now serves the v4 API, including the single Put Blob uploads docker/build-push-action uses. `exec` gives each workflow its own unsigned JWT runtime token and artifact run, so artifact names conflict and carry over between runs as on GitHub. Artifacts default to a temporary directory instead of the workspace that jobs copy.

Co-authored-by: silverwind <[email protected]>
Co-authored-by: Lunny Xiao <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1341
Reviewed-by: Lunny Xiao <[email protected]>
Reviewed-by: silverwind <[email protected]>
Co-authored-by: bircni <[email protected]>
2026-10-28 03:52:53 +00:00
Lunny Xiao a76ccd7020 test(e2e): cover the register/daemon CLI and failing jobs (#1346)
The e2e suite started the runner in-process and never exercised failing jobs. This adds a scenario that builds the binary and drives `register` then `daemon` against the Gitea fixture, and one for failed, skipped, continue-on-error and timed-out jobs.

Verified locally against gitea/gitea:main-nightly.

Assisted-by: Codet:codet
Reviewed-on: https://gitea.com/gitea/runner/pulls/1346
Reviewed-by: Zettat123 <[email protected]>
2026-10-27 21:23:59 +00:00
97 changed files with 3294 additions and 605 deletions

No files matched your search

+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7
with:
node-version: 24
node-version: 26
- run: make lint-pr-title
env:
PR_TITLE: ${{ github.event.pull_request.title }}
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
# daemon retains them between runs, and their digest pins skip the registry when it has them.
- name: pre-pull test images
env:
TEST_JOB_IMAGE: node:24-bookworm-slim@sha256:55fb5f5a527ea2c5c4a95f963e48ef90aa38c7b346054fdc8ee9957a2b55d205 # renovate: datasource=docker
TEST_JOB_IMAGE: node:26-bookworm-slim@sha256:bc0a2afd3b74ce12222fc8bab99271e66257f021078f5f4a5d687a77d280c1d9 # renovate: datasource=docker
TEST_SERVICE_IMAGE: nginx:alpine@sha256:a3554c0d0b48177b1413c26e01f695334428020d9df01243fbae3e11cfd17417 # renovate: datasource=docker
run: |
for image in "$TEST_JOB_IMAGE" "$TEST_SERVICE_IMAGE"; do
+10 -2
View File
@@ -57,7 +57,15 @@ Each lives in `internal/pkg/action/<name>` and is registered in `builtinActions`
`make test-e2e` runs the runner against `E2E_GITEA_IMAGE`. It defaults to the nightly image.
It requires Docker and is excluded from `make test`. CI runs stable and nightly variants in
parallel.
parallel. Outside a container it runs `go test` in a Linux container on its own network, as CI does,
so both take the same paths. `E2E_ARGS` passes extra flags, e.g. `make test-e2e E2E_ARGS='-run TestCompatibility/dispatch'`.
The suite shares one Gitea and regular runner. Cache and ephemeral scenarios use isolated
The `cli_*` scenarios build the binary and drive `register` and `daemon` as a user would, including `--once`,
`--labels` and shutdown with a running job. `failure_paths` covers failed, skipped and tolerated jobs, `job_timeout`
a job killed at the runner's deadline. `label_routing` and `capacity` cover scheduling, `reconnect` drops the
connection to Gitea through a proxy, and `host_mode` and `reusable_workflow` cover host labels and `workflow_call`.
`dispatch`, `failure_paths` and `reusable_workflow` run in every mode: through the daemon and `exec`, each on
Docker and Kubernetes.
The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated
repository runners. A run path is `<workflow>@<ref>` and a log row is `<timestamp>Z <payload>`.
+2 -2
View File
@@ -19,7 +19,7 @@ RUN --mount=type=cache,target=/go/pkg/mod \
### DIND VARIANT
#
#
FROM docker:29.9.1-dind AS dind
FROM docker:29.9.2-dind AS dind
ARG VERSION=dev
@@ -41,7 +41,7 @@ ENTRYPOINT ["s6-svscan","/etc/s6"]
### DIND-ROOTLESS VARIANT
#
#
FROM docker:29.9.1-dind-rootless AS dind-rootless
FROM docker:29.9.2-dind-rootless AS dind-rootless
ARG VERSION=dev
+2 -2
View File
@@ -4,7 +4,7 @@ GO ?= go
HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" )
PODMAN_TEST_IMAGE ?= quay.io/containers/podman:v5.8.9-immutable@sha256:87f42f37fa85b421664c30b8f5f9cfc23920057f5e5fcadf0b6f68d9edc6d415 # renovate: datasource=docker
E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:8856b200466e9a5af9cc0d4de4ca96f73547833bcdddf9a567189ff772c4d372 # renovate: datasource=docker
E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:657378e5cfd898e36f820459c60ef8e69be8fcda9fd354e4381a8b654d091bbc # renovate: datasource=docker
SERVICE_IMAGE ?= nginx:1.31.7-alpine@sha256:892bb2152464f0c5d8972593a5f5afaf8054552580b699d95f8ff4af08c2d33d # renovate: datasource=docker
E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly
E2E_K3S_IMAGE ?= rancher/k3s:v1.37.1-k3s1@sha256:ca7f37d993d82ef0dcdcfecb2e0e2618ea541dbaffc620c8cedebe01a82acd0d # renovate: datasource=docker
@@ -154,7 +154,7 @@ E2E_CONCURRENCY ?= 8
.PHONY: test-e2e
test-e2e: ## run Gitea compatibility tests against E2E_GITEA_IMAGE
@E2E_CONCURRENCY=$(E2E_CONCURRENCY) E2E_GITEA_IMAGE=$(E2E_GITEA_IMAGE) E2E_K3S_IMAGE=$(E2E_K3S_IMAGE) E2E_JOB_IMAGE=$(E2E_JOB_IMAGE) GO=$(GO) SERVICE_IMAGE=$(SERVICE_IMAGE) ./tools/test-e2e.sh
@E2E_CONCURRENCY=$(E2E_CONCURRENCY) E2E_GITEA_IMAGE=$(E2E_GITEA_IMAGE) E2E_K3S_IMAGE=$(E2E_K3S_IMAGE) E2E_JOB_IMAGE=$(E2E_JOB_IMAGE) GO=$(GO) SERVICE_IMAGE=$(SERVICE_IMAGE) ./tools/test-e2e.sh $(E2E_ARGS)
.PHONY: install
install: $(GOFILES) ## install the runner binary via `go install`
+2
View File
@@ -303,6 +303,8 @@ Use `uses: builtin:checkout` to check out a repository without downloading `acti
Each runner starts its own cache server, so runners do not share cached entries. A runner in a container on the jobs' Docker daemon attaches itself to each job network, so jobs reach its cache without extra setup.
As on GitHub, a job restores caches saved for its own ref, its pull request's base branch and the default branch, so a cache saved by a pull request is only restored by that pull request.
**Sharing a cache between runners**
Run one `gitea-runner cache-server` and point every runner at it, using the same secret everywhere, for example from `openssl rand -hex 32`:
-2
View File
@@ -6,7 +6,5 @@
//
// Inspired by https://github.com/sp-ricard-valverde/github-act-cache-server
//
// TODO: Authorization
// TODO: Restrictions for accessing a cache, see https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache
// TODO: Force deleting cache entries, see https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries
package artifactcache
+65 -47
View File
@@ -59,6 +59,10 @@ type credKey struct{}
type JobCredential struct {
Repo string `json:"repo"`
Ref string `json:"ref"`
BaseRef string `json:"base_ref"`
DefaultRef string `json:"default_ref"`
// Results is the instance whose artifact service this server forwards for the job, and
// InsecureTLS how the runner reaches it; see results.go.
Results string `json:"-"`
@@ -68,6 +72,10 @@ type JobCredential struct {
PublicURL string `json:"public_url"`
}
func (cred JobCredential) owns(cache *Cache) bool {
return cache.Repo == cred.Repo && cache.Ref == cred.Ref
}
// credEntry holds a registered job's credential along with an active
// registration count. RegisterJob is reference-counted so that if two tasks
// briefly share an ACTIONS_RUNTIME_TOKEN — e.g. a runner that retries a task
@@ -653,7 +661,7 @@ func (h *Handler) find(w http.ResponseWriter, r *http.Request) {
}
defer db.Close()
cache, err := h.lookupCache(db, cred.Repo, keys, version)
cache, err := h.lookupCache(db, cred, keys, version)
if err != nil {
h.responseJSON(w, r, 500, err)
return
@@ -669,26 +677,27 @@ func (h *Handler) find(w http.ResponseWriter, r *http.Request) {
})
}
// lookupCache returns the entry to restore for these keys, or (nil, nil) when there is none:
// either nothing matched, or the match had lost its blob to a prune, in which case the dangling
// entry is dropped on the way out.
func (h *Handler) lookupCache(db *cacheDB, repo string, keys []string, version string) (*Cache, error) {
cache, err := findCache(db, repo, keys, version)
if err != nil || cache == nil {
return nil, err
// lookupCache returns the entry to restore or (nil, nil), dropping matches whose blob was pruned.
func (h *Handler) lookupCache(db *cacheDB, cred JobCredential, keys []string, version string) (*Cache, error) {
for {
cache, err := findCache(db, cred, keys, version)
if err != nil || cache == nil {
return nil, err
}
ok, err := h.storage.Exist(cache.ID)
if err != nil {
return nil, err
}
if ok {
// Handing out a download URL counts as access, or eviction could drop the entry between
// this call and the GET that follows it.
h.touch(db, cache)
return cache, nil
}
if !h.deleteCache(db, cache) {
return nil, nil //nolint:nilnil // absence is not an error here
}
}
ok, err := h.storage.Exist(cache.ID)
if err != nil {
return nil, err
}
if !ok {
h.deleteCache(db, cache)
return nil, nil //nolint:nilnil // absence is not an error here
}
// Handing out a download URL counts as access, or eviction could drop the entry between
// this call and the GET that follows it.
h.touch(db, cache)
return cache, nil
}
// POST /_apis/artifactcache/caches
@@ -700,7 +709,7 @@ func (h *Handler) reserve(w http.ResponseWriter, r *http.Request) {
return
}
cache := &Cache{Repo: cred.Repo, Key: api.Key, Version: api.Version, Size: api.Size}
cache := &Cache{Repo: cred.Repo, Ref: cred.Ref, Key: api.Key, Version: api.Version, Size: api.Size}
if cache.Size == 0 {
cache.Size = -1
}
@@ -749,7 +758,7 @@ func (h *Handler) upload(w http.ResponseWriter, r *http.Request) {
return
}
if cache.Repo != cred.Repo {
if !cred.owns(cache) {
h.responseJSON(w, r, 403, fmt.Errorf("cache %d: forbidden", id))
return
}
@@ -808,7 +817,7 @@ func (h *Handler) commit(w http.ResponseWriter, r *http.Request) {
return
}
if cache.Repo != cred.Repo {
if !cred.owns(cache) {
h.responseJSON(w, r, 403, fmt.Errorf("cache %d: forbidden", id))
return
}
@@ -1071,25 +1080,35 @@ func (h *Handler) signedArtifactURL(cred JobCredential, cacheID uint64, exp time
return h.signedURL(cred, apiPath+"/artifacts", "", cacheID, exp)
}
// if not found, return (nil, nil) instead of an error.
func findCache(db *cacheDB, repo string, keys []string, version string) (*Cache, error) {
for _, prefix := range keys {
// if a key in the list matches exactly, don't return partial matches
exact, err := findExactCache(db, repo, prefix, version, true)
if err != nil {
return nil, err
// findCache tries every key in one ref before the next ref, and returns (nil, nil) on a miss.
func findCache(db *cacheDB, cred JobCredential, keys []string, version string) (*Cache, error) {
refs := []string{cred.Ref}
for _, ref := range []string{cred.BaseRef, cred.DefaultRef} {
if ref != "" && !slices.Contains(refs, ref) {
refs = append(refs, ref)
}
if exact != nil {
return exact, nil
}
caches, err := db.find(func(cache *Cache) bool {
return cache.Repo == repo && strings.HasPrefix(cache.Key, prefix) && cache.Version == version && cache.Complete
})
if err != nil {
return nil, fmt.Errorf("find cache: %w", err)
}
if cache := latest(caches, (*Cache).createdAtNano); cache != nil {
return cache, nil
}
caches, err := db.find(func(cache *Cache) bool {
return cache.Repo == cred.Repo && cache.Version == version && cache.Complete && slices.Contains(refs, cache.Ref) &&
slices.ContainsFunc(keys, func(prefix string) bool { return strings.HasPrefix(cache.Key, prefix) })
})
if err != nil {
return nil, fmt.Errorf("find cache: %w", err)
}
for _, ref := range refs {
for _, prefix := range keys {
var exact, partial []*Cache
for _, cache := range caches {
if cache.Ref == ref && cache.Key == prefix {
exact = append(exact, cache)
} else if cache.Ref == ref && strings.HasPrefix(cache.Key, prefix) {
partial = append(partial, cache)
}
}
// if a key in the list matches exactly, don't return partial matches
if cache := cmp.Or(latest(exact, (*Cache).createdAtNano), latest(partial, (*Cache).createdAtNano)); cache != nil {
return cache, nil
}
}
}
return nil, nil //nolint:nilnil // pre-existing issue from nektos/act
@@ -1111,9 +1130,9 @@ func latest(caches []*Cache, stamp func(*Cache) int64) *Cache {
// A completed entry is the one to restore, sorted by when it was written. An incomplete one is a
// reservation being uploaded to, sorted by when it was last written to, because the upload route
// touches UsedAt on every part.
func findExactCache(db *cacheDB, repo, key, version string, complete bool) (*Cache, error) {
func findExactCache(db *cacheDB, cred JobCredential, key, version string, complete bool) (*Cache, error) {
caches, err := db.find(func(cache *Cache) bool {
return cache.Key == key && cache.Repo == repo && cache.Version == version && cache.Complete == complete
return cred.owns(cache) && cache.Key == key && cache.Version == version && cache.Complete == complete
})
if err != nil {
return nil, fmt.Errorf("find cache: %w", err)
@@ -1280,13 +1299,12 @@ func (h *Handler) evictExpired(db *cacheDB) {
h.sweep(db, func(cache *Cache) bool { return cache.UsedAt < expired })
}
// evictSuperseded removes entries a newer one with the same key and version replaced. The
// grouping includes Repo so two repos sharing a (key, version) do not evict each other.
// evictSuperseded removes entries a newer one with the same repo, ref, key and version replaced.
func (h *Handler) evictSuperseded(db *cacheDB) {
type entry struct{ repo, key, version string }
type entry struct{ repo, ref, key, version string }
groups := make(map[entry][]*Cache)
for _, cache := range h.caches(db, func(cache *Cache) bool { return cache.Complete }) {
group := entry{cache.Repo, cache.Key, cache.Version}
group := entry{cache.Repo, cache.Ref, cache.Key, cache.Version}
groups[group] = append(groups[group], cache)
}
for _, caches := range groups {
+70 -47
View File
@@ -759,11 +759,55 @@ func TestHandler_PrefixUsesNewestReservationWithinSecond(t *testing.T) {
db, err := handler.openDB()
require.NoError(t, err)
defer db.Close()
cache, err := findCache(db, testRepo, []string{"deps-"}, "v")
cache, err := findCache(db, JobCredential{Repo: testRepo}, []string{"deps-"}, "v")
require.NoError(t, err)
require.Equal(t, newer, cache)
}
func TestHandler_RestoreRefs(t *testing.T) {
t.Parallel()
const main, release, pull = "refs/heads/main", "refs/heads/release", "refs/pull/1/head"
now := time.Now().Unix()
entry := func(ref, key string, age int64) *Cache {
return &Cache{Repo: testRepo, Ref: ref, Key: key, Version: "v", Complete: true, CreatedAt: now - 3600 + age, UsedAt: now}
}
exactOnMain := entry(main, "deps-exact", -1)
onRelease := entry(release, "deps-release", 0)
onMain := entry(main, "deps-main", 1)
onPull := entry(pull, "deps-pull", 2)
unscoped := entry("", "deps-unscoped", 3)
handler := newTestHandler(t, Policy{}, exactOnMain, onRelease, onMain, onPull, unscoped, entry("refs/pull/2/head", "deps-lost", 4))
for _, cache := range []*Cache{exactOnMain, onRelease, onMain, onPull, unscoped} {
writeBlob(t, handler, cache.ID)
}
db, err := handler.openDB()
require.NoError(t, err)
defer db.Close()
for _, tc := range []struct {
name string
cred JobCredential
keys []string
want *Cache
}{
{"the default branch never restores what a pull request wrote", JobCredential{Ref: main, DefaultRef: main}, []string{"deps-"}, onMain},
{"a pull request restores its own entry first", JobCredential{Ref: pull, BaseRef: release, DefaultRef: main}, []string{"deps-"}, onPull},
{"a pull request falls back past a lost blob to its base branch before the default branch", JobCredential{Ref: "refs/pull/2/head", BaseRef: release, DefaultRef: main}, []string{"deps-"}, onRelease},
{"a branch does not restore a sibling branch", JobCredential{Ref: "refs/heads/other", DefaultRef: main}, []string{"deps-"}, onMain},
{"a prefix in the own ref wins over an exact key in the default branch", JobCredential{Ref: release, DefaultRef: main}, []string{"deps-exact", "deps-"}, onRelease},
{"a job with a ref never restores entries saved without one", JobCredential{Ref: "refs/heads/other"}, []string{"deps-"}, nil},
{"a job without a ref only restores entries saved without one", JobCredential{}, []string{"deps-"}, unscoped},
} {
t.Run(tc.name, func(t *testing.T) {
tc.cred.Repo = testRepo
cache, err := handler.lookupCache(db, tc.cred, tc.keys, "v")
require.NoError(t, err)
assert.Equal(t, tc.want, cache)
})
}
}
func TestHandler_evictOnCommit(t *testing.T) {
t.Parallel()
full := &Cache{Repo: testRepo, Key: "full", Version: "v", Complete: true, Size: 4 * miB, UsedAt: time.Now().Add(-time.Hour).Unix()}
@@ -938,6 +982,7 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
defer handler.Close()
handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"})
handler.RegisterJob("token-a-pull", JobCredential{Repo: "owner/repoA", Ref: "refs/pull/1/head"})
base := handler.ExternalURL() + apiPath
key := "shared-key"
@@ -972,13 +1017,20 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison")))
require.NoError(t, err)
req.Header.Set("Content-Range", "bytes 0-5/*")
resp, err = clientB.Do(req)
require.NoError(t, err)
resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
for _, client := range []*http.Client{clientB, {Transport: &bearerTransport{token: "token-a-pull"}}} {
req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison")))
require.NoError(t, err)
req.Header.Set("Content-Range", "bytes 0-5/*")
resp, err = client.Do(req)
require.NoError(t, err)
resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
resp, err = client.Post(fmt.Sprintf("%s/caches/%d", base, id), "", nil)
require.NoError(t, err)
resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
}
}
func TestHandler_ArtifactSignature(t *testing.T) {
@@ -1083,49 +1135,20 @@ func TestHandler_ArtifactSignatureDownload(t *testing.T) {
assert.Equal(t, content, body)
}
func TestHandler_GC_PerRepoDedup(t *testing.T) {
func TestHandler_GC_DedupPerRepoAndRef(t *testing.T) {
t.Parallel()
dir := filepath.Join(t.TempDir(), "artifactcache")
handler, err := StartHandler(Options{Dir: dir})
require.NoError(t, err)
defer handler.Close()
handler.RegisterJob("tok-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("tok-b", JobCredential{Repo: "owner/repoB"})
key := "shared-dedup-key"
version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20"
db, err := handler.openDB()
require.NoError(t, err)
now := time.Now().Unix()
stale := time.Now().Add(-inUseGrace - time.Minute).Unix()
a := &Cache{Repo: "owner/repoA", Key: key, Version: version, Complete: true, CreatedAt: stale, UsedAt: stale, Size: 1}
b := &Cache{Repo: "owner/repoB", Key: key, Version: version, Complete: true, CreatedAt: now, UsedAt: now, Size: 1}
require.NoError(t, handler.insertCache(db, a))
require.NoError(t, handler.insertCache(db, b))
require.NoError(t, handler.storage.Write(a.ID, 0, strings.NewReader("a")))
_, err = handler.storage.Commit(a.ID, 1)
require.NoError(t, err)
require.NoError(t, handler.storage.Write(b.ID, 0, strings.NewReader("b")))
_, err = handler.storage.Commit(b.ID, 1)
require.NoError(t, err)
require.NoError(t, db.Close())
entries := []*Cache{
{Repo: "owner/repoA", Key: "shared", Version: "v", Complete: true, CreatedAt: stale, UsedAt: stale, Size: 1},
{Repo: "owner/repoB", Key: "shared", Version: "v", Complete: true, CreatedAt: now, UsedAt: now, Size: 1},
{Repo: "owner/repoA", Ref: "refs/pull/1/head", Key: "shared", Version: "v", Complete: true, CreatedAt: now, UsedAt: now, Size: 1},
}
handler := newTestHandler(t, Policy{}, entries...)
handler.gcAt = time.Time{}
handler.gcCache()
db, err = handler.openDB()
require.NoError(t, err)
defer db.Close()
after, err := db.find(func(cache *Cache) bool { return cache.Key == key && cache.Version == version })
require.NoError(t, err)
repos := make(map[string]bool)
for _, c := range after {
repos[c.Repo] = true
}
assert.True(t, repos["owner/repoA"], "repoA's cache must survive dedup against repoB")
assert.True(t, repos["owner/repoB"], "repoB's cache must survive dedup against repoA")
assert.Len(t, keptKeys(t, handler, entries), len(entries))
}
func TestHandler_InternalAPI_Disabled(t *testing.T) {
@@ -1190,10 +1213,10 @@ func TestHandler_InternalAPI_AuthAndUsage(t *testing.T) {
}
assert.Equal(t, http.StatusUnauthorized, probe("via-internal-api"))
assert.Equal(t, http.StatusOK, post("/_internal/register", secret, `{"token":"via-internal-api","repo":"owner/repo","public_url":"http://runner.example","results":"https://gitea.example","insecure_tls":true}`))
assert.Equal(t, http.StatusOK, post("/_internal/register", secret, `{"token":"via-internal-api","repo":"owner/repo","ref":"refs/pull/1/head","base_ref":"refs/heads/release","default_ref":"refs/heads/main","public_url":"http://runner.example","results":"https://gitea.example","insecure_tls":true}`))
cred, ok := handler.lookupCredential("via-internal-api")
require.True(t, ok)
assert.Equal(t, JobCredential{Repo: testRepo, PublicURL: "http://runner.example"}, cred)
assert.Equal(t, JobCredential{Repo: testRepo, Ref: "refs/pull/1/head", BaseRef: "refs/heads/release", DefaultRef: "refs/heads/main", PublicURL: "http://runner.example"}, cred)
assert.NotEqual(t, http.StatusUnauthorized, probe("via-internal-api"))
assert.Equal(t, http.StatusOK, post("/_internal/revoke", secret, `{"token":"via-internal-api"}`))
assert.Equal(t, http.StatusUnauthorized, probe("via-internal-api"))
+6 -5
View File
@@ -73,7 +73,7 @@ func (h *Handler) v2CreateCacheEntry(w http.ResponseWriter, r *http.Request) {
// An exact (key, version) match means the entry is already cached; the client then skips
// the upload. A prefix match must not count here, or a shorter key would be reported as
// existing and silently never saved.
if existing, err := findExactCache(db, cred.Repo, req.Key, req.Version, true); err != nil {
if existing, err := findExactCache(db, cred, req.Key, req.Version, true); err != nil {
h.twirpError(w, r, twirpInternal, err)
return
} else if existing != nil {
@@ -84,7 +84,7 @@ func (h *Handler) v2CreateCacheEntry(w http.ResponseWriter, r *http.Request) {
// A second live reservation is finalized by whichever job calls last, against the other's upload.
owner := hashedToken(bearerToken(r))
if pending, err := findExactCache(db, cred.Repo, req.Key, req.Version, false); err != nil {
if pending, err := findExactCache(db, cred, req.Key, req.Version, false); err != nil {
h.twirpError(w, r, twirpInternal, err)
return
} else if pending != nil && pending.UsedAt > time.Now().Add(-uploadStallTimeout).Unix() {
@@ -103,6 +103,7 @@ func (h *Handler) v2CreateCacheEntry(w http.ResponseWriter, r *http.Request) {
now := time.Now()
cache := &Cache{
Repo: cred.Repo,
Ref: cred.Ref,
Key: req.Key,
Version: req.Version,
Owner: owner,
@@ -137,13 +138,13 @@ func (h *Handler) v2FinalizeCacheEntryUpload(w http.ResponseWriter, r *http.Requ
}
defer db.Close()
cache, err := findExactCache(db, cred.Repo, req.Key, req.Version, false)
cache, err := findExactCache(db, cred, req.Key, req.Version, false)
if err == nil && cache != nil && cache.Owner != hashedToken(bearerToken(r)) {
cache = nil // not the reservation this job made, so not this job's to commit
}
if err == nil && cache == nil {
// A retry whose first response was lost finds the entry already committed.
cache, err = findExactCache(db, cred.Repo, req.Key, req.Version, true)
cache, err = findExactCache(db, cred, req.Key, req.Version, true)
}
if err != nil {
h.twirpError(w, r, twirpInternal, err)
@@ -186,7 +187,7 @@ func (h *Handler) v2GetCacheEntryDownloadURL(w http.ResponseWriter, r *http.Requ
}
defer db.Close()
cache, err := h.lookupCache(db, cred.Repo, req.keys(), req.Version)
cache, err := h.lookupCache(db, cred, req.keys(), req.Version)
if err != nil {
h.twirpError(w, r, twirpInternal, err)
return
+18
View File
@@ -231,6 +231,24 @@ func TestCacheServiceV2Lookups(t *testing.T) {
got := v2Call(t, handler, otherClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})
assert.Equal(t, false, got["ok"])
})
t.Run("each ref saves and restores its own entry for the same key", func(t *testing.T) {
clients := map[string]*http.Client{"x": testClient}
for _, ref := range []string{"y", "z"} {
handler.RegisterJob(ref, JobCredential{Repo: testRepo, Ref: "refs/pull/" + ref + "/head"})
clients[ref] = &http.Client{Transport: &bearerTransport{token: ref}}
created := v2Call(t, handler, clients[ref], "CreateCacheEntry", map[string]any{"key": "deps-abc", "version": "v1"})
require.Equal(t, true, created["ok"])
require.Equal(t, http.StatusCreated, putBlob(t, created["signed_upload_url"].(string), []byte(ref)))
}
for _, ref := range []string{"y", "z"} {
require.Equal(t, true, v2Call(t, handler, clients[ref], "FinalizeCacheEntryUpload", map[string]any{"key": "deps-abc", "version": "v1", "size_bytes": 1})["ok"])
}
for content, client := range clients {
got := v2Call(t, handler, client, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})
assert.Equal(t, content, string(getURL(t, got["signed_download_url"].(string))))
}
})
}
func TestCacheServiceV2RefusalsAreTwirp(t *testing.T) {
+1
View File
@@ -15,6 +15,7 @@ type Request struct {
type Cache struct {
ID uint64 `json:"id"`
Repo string `json:"repo"`
Ref string `json:"ref,omitzero"`
Key string `json:"key"`
Version string `json:"version"`
Size int64 `json:"cacheSize"`
+1
View File
@@ -218,6 +218,7 @@ func Serve(ctx context.Context, artifactPath, addr, port string) context.CancelF
logger.Debugf("Artifacts base path '%s'", artifactPath)
uploads(router, artifactPath)
downloads(router, artifactPath)
registerV4(router, artifactPath)
server := &http.Server{
Addr: net.JoinHostPort(addr, port),
+87
View File
@@ -28,6 +28,7 @@ func TestArtifactFlow(t *testing.T) {
router := http.NewServeMux()
uploads(router, artifactPath)
downloads(router, artifactPath)
registerV4(router, artifactPath)
server := httptest.NewServer(router)
defer server.Close()
@@ -174,6 +175,92 @@ func TestArtifactFlow(t *testing.T) {
require.Equal(t, http.StatusOK, status)
require.Equal(t, content, string(data))
})
t.Run("v4", func(t *testing.T) {
const artifactRef = `"workflow_run_backend_id":"7","name":"50% art@v1"`
call := func(method, body string) (int, map[string]any) {
status, data := request(t, http.MethodPost, baseURL+artifactServicePath+"/"+method, strings.NewReader(body), nil)
var out map[string]any
require.NoError(t, json.Unmarshal(data, &out), string(data))
return status, out
}
put := func(rawURL, body string) int {
status, _ := request(t, http.MethodPut, rawURL, strings.NewReader(body), nil)
return status
}
download := func() (string, http.Header) {
status, signed := call("GetSignedArtifactURL", "{"+artifactRef+"}")
require.Equal(t, http.StatusOK, status, signed)
resp, err := client.Get(signed["signedUrl"].(string))
require.NoError(t, err)
defer resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
return string(body), resp.Header
}
status, created := call("CreateArtifact", "{"+artifactRef+"}")
require.Equal(t, http.StatusOK, status, created)
upload := created["signedUploadUrl"].(string)
require.Equal(t, http.StatusCreated, put(upload+"&comp=block&blockid=Yg", "world"))
require.Equal(t, http.StatusCreated, put(upload+"&comp=block&blockid=YQ", "hello "))
for attempt := range 2 {
require.Equal(t, http.StatusCreated, put(upload+"&comp=blocklist", `<?xml version="1.0"?><BlockList><Latest>YQ</Latest><Latest>Yg</Latest></BlockList>`), "commit attempt %d", attempt)
}
digest := "sha256:b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9"
status, _ = call("FinalizeArtifact", "{"+artifactRef+`,"size":"10"}`)
require.Equal(t, http.StatusBadRequest, status)
status, finalized := call("FinalizeArtifact", "{"+artifactRef+`,"size":"11","hash":"`+digest+`"}`)
require.Equal(t, http.StatusOK, status, finalized)
id := finalized["artifactId"].(string)
status, refinalized := call("FinalizeArtifact", "{"+artifactRef+`,"size":"10"}`)
require.Equal(t, http.StatusOK, status)
require.Equal(t, id, refinalized["artifactId"])
status, conflict := call("CreateArtifact", "{"+artifactRef+"}")
require.Equal(t, http.StatusConflict, status)
require.Equal(t, "already_exists", conflict["code"])
status, listed := call("ListArtifacts", `{"workflowRunBackendId":"7","idFilter":"`+id+`"}`)
require.Equal(t, http.StatusOK, status)
artifacts := listed["artifacts"].([]any)
require.Len(t, artifacts, 1)
artifact := artifacts[0].(map[string]any)
require.Regexp(t, `^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ$`, artifact["createdAt"])
delete(artifact, "createdAt")
require.Equal(t, map[string]any{"workflowRunBackendId": "7", "name": "50% art@v1", "size": "11", "digest": digest, "databaseId": id}, artifact)
_, other := call("ListArtifacts", `{"workflow_run_backend_id":"8"}`)
require.Empty(t, other["artifacts"])
body, header := download()
require.Equal(t, "hello world", body)
require.Equal(t, "application/zip", header.Get("Content-Type"))
require.Equal(t, "attachment; filename*=UTF-8''50%25%20art%40v1.zip", header.Get("Content-Disposition"))
status, _ = call("DeleteArtifact", "{"+artifactRef+"}")
require.Equal(t, http.StatusOK, status)
status, missing := call("GetSignedArtifactURL", "{"+artifactRef+"}")
require.Equal(t, http.StatusNotFound, status)
require.Equal(t, "not_found", missing["code"])
status, created = call("CreateArtifact", "{"+artifactRef+"}")
require.Equal(t, http.StatusOK, status, created)
status, _ = request(t, http.MethodPut, created["signedUploadUrl"].(string), strings.NewReader("gz"), http.Header{"X-Ms-Blob-Content-Type": {"application/gzip"}})
require.Equal(t, http.StatusCreated, status)
status, _ = call("FinalizeArtifact", "{"+artifactRef+`,"size":"2"}`)
require.Equal(t, http.StatusOK, status)
body, header = download()
require.Equal(t, "gz", body)
require.Equal(t, "application/gzip", header.Get("Content-Type"))
require.Equal(t, "attachment; filename*=UTF-8''50%25%20art%40v1", header.Get("Content-Disposition"))
rerun := http.NewServeMux()
registerV4(rerun, artifactPath)
recorder := httptest.NewRecorder()
rerun.ServeHTTP(recorder, httptest.NewRequest(http.MethodPost, artifactServicePath+"/CreateArtifact", strings.NewReader("{"+artifactRef+"}")))
require.Equal(t, http.StatusOK, recorder.Code, recorder.Body.String())
})
}
func TestSafeResolve(t *testing.T) {
+329
View File
@@ -0,0 +1,329 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifacts
import (
"cmp"
"crypto/sha256"
"encoding/hex"
"encoding/json/v2"
"encoding/xml"
"errors"
"fmt"
"io"
"io/fs"
"net/http"
"net/url"
"os"
"path/filepath"
"slices"
"strings"
"sync"
"time"
)
type serverV4 struct { // the subset of Gitea's v4 artifact service that the toolkit uses, unauthenticated like the v3 routes beside it
dir string
mu sync.Mutex
lastID int64
created map[string]bool
}
const (
zipMimeType = "application/zip"
maxTwirpBytes = 1 << 20
)
func registerV4(router *http.ServeMux, baseDir string) {
server := &serverV4{dir: filepath.Join(baseDir, "v4"), lastID: time.Now().UnixMilli(), created: map[string]bool{}} // IDs stay unique across invocations sharing baseDir
router.HandleFunc("POST "+artifactServicePath+"/CreateArtifact", twirp(server.createArtifact))
router.HandleFunc("POST "+artifactServicePath+"/FinalizeArtifact", twirp(server.finalizeArtifact))
router.HandleFunc("POST "+artifactServicePath+"/ListArtifacts", twirp(server.listArtifacts))
router.HandleFunc("POST "+artifactServicePath+"/GetSignedArtifactURL", twirp(server.getSignedArtifactURL))
router.HandleFunc("POST "+artifactServicePath+"/DeleteArtifact", twirp(server.deleteArtifact))
router.HandleFunc("PUT "+artifactServicePath+"/UploadArtifact", server.uploadArtifact)
router.HandleFunc("GET "+artifactServicePath+"/DownloadArtifact", server.downloadArtifact)
}
type twirpError struct {
status int
code string
msg string
}
func (e *twirpError) Error() string { return e.msg }
func notFound(name string) error {
return &twirpError{http.StatusNotFound, "not_found", fmt.Sprintf("artifact %q not found", name)}
}
func badRequest(format string, args ...any) error {
return &twirpError{http.StatusBadRequest, "invalid_argument", fmt.Sprintf(format, args...)}
}
func twirp[Req, Resp any](handle func(*http.Request, Req) (Resp, error)) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
var req Req
var resp Resp
err := json.UnmarshalRead(io.LimitReader(r.Body, maxTwirpBytes), &req, json.MatchCaseInsensitiveNames(true)) // also matches the toolkit's proto field names
if err != nil {
err = badRequest("malformed request: %v", err)
} else {
resp, err = handle(r, req)
}
w.Header().Set("Content-Type", "application/json")
if err != nil {
terr := &twirpError{http.StatusInternalServerError, "internal", err.Error()}
errors.As(err, &terr)
w.WriteHeader(terr.status)
_ = json.MarshalWrite(w, map[string]string{"code": terr.code, "msg": terr.msg})
return
}
_ = json.MarshalWrite(w, resp)
}
}
func fileName(name string) string { // safe on any filesystem for any client-chosen name, case-insensitive ones included
sum := sha256.Sum256([]byte(name))
return hex.EncodeToString(sum[:])
}
func (s *serverV4) runDir(runID string) string {
return filepath.Join(s.dir, fileName(runID))
}
func (s *serverV4) artifactDir(runID, name string) string {
return filepath.Join(s.runDir(runID), fileName(name))
}
func (s *serverV4) urlDir(r *http.Request) string {
return s.artifactDir(r.URL.Query().Get("run"), r.URL.Query().Get("name"))
}
func readMeta(dir string) (*monolithArtifact, error) {
data, err := os.ReadFile(filepath.Join(dir, "meta.json"))
if err != nil {
return nil, err
}
var meta monolithArtifact
return &meta, json.Unmarshal(data, &meta)
}
func writeMeta(dir string, meta *monolithArtifact) error {
data, err := json.Marshal(meta)
if err != nil {
return err
}
return os.WriteFile(filepath.Join(dir, "meta.json"), data, 0o644)
}
func blobURL(r *http.Request, endpoint, runID, name string) string {
return "http://" + r.Host + artifactServicePath + "/" + endpoint + "?" + url.Values{"run": {runID}, "name": {name}}.Encode()
}
func (s *serverV4) createArtifact(r *http.Request, req createArtifactRequest) (createArtifactResponse, error) { // an exec invocation re-runs its workflows, so it replaces earlier invocations' artifacts but conflicts with its own
dir := s.artifactDir(req.WorkflowRunBackendID, req.Name)
s.mu.Lock()
defer s.mu.Unlock()
if s.created[dir] {
return createArtifactResponse{}, &twirpError{http.StatusConflict, "already_exists", "an artifact with this name already exists on the workflow run"}
}
if err := os.RemoveAll(dir); err != nil {
return createArtifactResponse{}, err
}
if err := os.MkdirAll(filepath.Join(dir, "blocks"), 0o755); err != nil {
return createArtifactResponse{}, err
}
s.lastID++
if err := writeMeta(dir, &monolithArtifact{
WorkflowRunBackendID: req.WorkflowRunBackendID,
DatabaseID: s.lastID,
Name: req.Name,
CreatedAt: time.Now().UTC().Truncate(time.Second),
}); err != nil {
return createArtifactResponse{}, err
}
s.created[dir] = true
return createArtifactResponse{Ok: true, SignedUploadURL: blobURL(r, "UploadArtifact", req.WorkflowRunBackendID, req.Name)}, nil
}
type blockList struct {
Blocks []string `xml:",any"` // Latest, Committed and Uncommitted all name a staged block
}
func (s *serverV4) uploadArtifact(w http.ResponseWriter, r *http.Request) {
dir := s.urlDir(r)
if meta, err := readMeta(dir); err != nil || meta.Digest != "" {
http.Error(w, "no artifact awaits an upload", http.StatusNotFound)
return
}
contentType := r.Header.Get("x-ms-blob-content-type")
var err error
switch comp := r.URL.Query().Get("comp"); comp {
case "block":
err = writeFile(filepath.Join(dir, "blocks", fileName(r.URL.Query().Get("blockid"))), r.Body)
case "blocklist":
err = commitBlob(dir, contentType, func(out io.Writer) error { return joinBlocks(out, dir, r.Body) })
case "": // Put Blob, which docker/build-push-action uses
err = commitBlob(dir, contentType, func(out io.Writer) error {
_, err := io.Copy(out, r.Body)
return err
})
default:
http.Error(w, "unsupported comp "+comp, http.StatusBadRequest)
return
}
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusCreated)
}
func writeFile(name string, body io.Reader) error {
file, err := os.Create(name)
if err != nil {
return err
}
_, err = io.Copy(file, body)
return errors.Join(err, file.Close())
}
func commitBlob(dir, contentType string, write func(io.Writer) error) error { // replaces the blob whole and keeps its content type, as Azure does
out, err := os.CreateTemp(dir, "blob")
if err != nil {
return err
}
if err = errors.Join(write(out), out.Close()); err == nil {
err = os.Rename(out.Name(), filepath.Join(dir, "blob"))
}
if err != nil {
return errors.Join(err, os.Remove(out.Name()))
}
return os.WriteFile(filepath.Join(dir, "content-type"), []byte(contentType), 0o644)
}
func joinBlocks(out io.Writer, dir string, body io.Reader) error { // keeps the blocks for finalizeArtifact, so that a retried commit can join them again
var list blockList
if err := xml.NewDecoder(io.LimitReader(body, maxTwirpBytes)).Decode(&list); err != nil {
return err
}
for _, id := range list.Blocks {
if err := appendBlock(out, filepath.Join(dir, "blocks", fileName(id))); err != nil {
return err
}
}
return nil
}
func appendBlock(out io.Writer, name string) error {
in, err := os.Open(name)
if err != nil {
return err
}
defer in.Close()
_, err = io.Copy(out, in)
return err
}
func (s *serverV4) finalizeArtifact(_ *http.Request, req finalizeArtifactRequest) (finalizeArtifactResponse, error) {
dir := s.artifactDir(req.WorkflowRunBackendID, req.Name)
meta, err := readMeta(dir)
if err != nil {
return finalizeArtifactResponse{}, notFound(req.Name)
} else if meta.Digest != "" {
return finalizeArtifactResponse{Ok: true, ArtifactID: meta.DatabaseID}, nil
}
blob, err := os.Open(filepath.Join(dir, "blob"))
if err != nil {
return finalizeArtifactResponse{}, badRequest("artifact %q has no upload", req.Name)
}
defer blob.Close()
hash := sha256.New()
size, err := io.Copy(hash, blob)
if err != nil {
return finalizeArtifactResponse{}, err
}
digest := "sha256:" + hex.EncodeToString(hash.Sum(nil))
if size != req.Size {
return finalizeArtifactResponse{}, badRequest("size %d does not match the %d bytes uploaded", req.Size, size)
}
if req.Hash != "" && req.Hash != digest {
return finalizeArtifactResponse{}, badRequest("hash %s does not match the uploaded %s", req.Hash, digest)
}
s.mu.Lock() // only after hashing, so finalizing one large artifact does not stall the others
defer s.mu.Unlock()
if current, err := readMeta(dir); err != nil || current.DatabaseID != meta.DatabaseID {
return finalizeArtifactResponse{}, notFound(req.Name)
}
meta.Size, meta.Digest = size, digest
if err := writeMeta(dir, meta); err != nil {
return finalizeArtifactResponse{}, err
}
return finalizeArtifactResponse{Ok: true, ArtifactID: meta.DatabaseID}, os.RemoveAll(filepath.Join(dir, "blocks"))
}
func (s *serverV4) listArtifacts(_ *http.Request, req listArtifactsRequest) (listArtifactsResponse, error) {
run := s.runDir(req.WorkflowRunBackendID)
entries, err := os.ReadDir(run)
if err != nil && !errors.Is(err, fs.ErrNotExist) {
return listArtifactsResponse{}, err
}
var resp listArtifactsResponse
for _, entry := range entries {
meta, err := readMeta(filepath.Join(run, entry.Name()))
if err != nil || meta.Digest == "" ||
req.NameFilter != "" && req.NameFilter != meta.Name || req.IDFilter != 0 && req.IDFilter != meta.DatabaseID {
continue
}
resp.Artifacts = append(resp.Artifacts, *meta)
}
slices.SortFunc(resp.Artifacts, func(left, right monolithArtifact) int { return cmp.Compare(left.DatabaseID, right.DatabaseID) })
return resp, nil
}
func (s *serverV4) getSignedArtifactURL(r *http.Request, req getSignedArtifactURLRequest) (getSignedArtifactURLResponse, error) {
if meta, err := readMeta(s.artifactDir(req.WorkflowRunBackendID, req.Name)); err != nil || meta.Digest == "" {
return getSignedArtifactURLResponse{}, notFound(req.Name)
}
return getSignedArtifactURLResponse{SignedURL: blobURL(r, "DownloadArtifact", req.WorkflowRunBackendID, req.Name)}, nil
}
func (s *serverV4) downloadArtifact(w http.ResponseWriter, r *http.Request) {
dir := s.urlDir(r)
meta, err := readMeta(dir)
if err != nil || meta.Digest == "" {
http.Error(w, "artifact not found", http.StatusNotFound)
return
}
blob, err := os.Open(filepath.Join(dir, "blob"))
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
defer blob.Close()
data, _ := os.ReadFile(filepath.Join(dir, "content-type"))
contentType, filename := string(data), meta.Name
if !strings.Contains(contentType, "/") {
contentType = zipMimeType
}
if contentType == zipMimeType && !strings.HasSuffix(filename, ".zip") {
filename += ".zip"
}
w.Header().Set("Content-Type", contentType)
w.Header().Set("Content-Disposition", "attachment; filename*=UTF-8''"+strings.ReplaceAll(url.QueryEscape(filename), "+", "%20")) // RFC 5987 attr-chars only, which the toolkit and mime.ParseMediaType both accept
http.ServeContent(w, r, "", meta.CreatedAt, blob)
}
func (s *serverV4) deleteArtifact(_ *http.Request, req deleteArtifactRequest) (deleteArtifactResponse, error) {
dir := s.artifactDir(req.WorkflowRunBackendID, req.Name)
s.mu.Lock()
defer s.mu.Unlock()
meta, err := readMeta(dir)
if err != nil {
return deleteArtifactResponse{}, notFound(req.Name)
}
delete(s.created, dir)
return deleteArtifactResponse{Ok: true, ArtifactID: meta.DatabaseID}, os.RemoveAll(dir)
}
+23
View File
@@ -0,0 +1,23 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifacts
import (
"crypto/rand"
"encoding/base64"
"encoding/json/v2"
"time"
)
func LocalRuntimeToken(run string, lifetime time.Duration) string { // an unsecured JWT with the claims the toolkit and buildx read, its run hashed for scp and its jti an unguessable bearer
now := time.Now()
claims, _ := json.Marshal(map[string]any{
"scp": "Actions.Results:" + fileName(run) + ":1",
"ac": `[{"Scope":"","Permission":3}]`,
"nbf": now.Unix(),
"exp": now.Add(lifetime).Unix(),
"jti": rand.Text(),
})
return base64.RawURLEncoding.EncodeToString([]byte(`{"alg":"none","typ":"JWT"}`)) + "." + base64.RawURLEncoding.EncodeToString(claims) + "."
}
+55
View File
@@ -0,0 +1,55 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifacts
import "time"
const artifactServicePath = "/twirp/github.actions.results.api.v1.ArtifactService"
type ( // the ArtifactService messages in the protobuf JSON mapping, which writes int64 as strings
createArtifactResponse struct {
Ok bool `json:"ok"`
SignedUploadURL string `json:"signedUploadUrl"`
}
finalizeArtifactRequest struct {
WorkflowRunBackendID string
Name string
Size int64 `json:",string"`
Hash string
}
finalizeArtifactResponse struct {
Ok bool `json:"ok"`
ArtifactID int64 `json:"artifactId,string"`
}
listArtifactsRequest struct {
WorkflowRunBackendID string
NameFilter string
IDFilter int64 `json:",string"`
}
listArtifactsResponse struct {
Artifacts []monolithArtifact `json:"artifacts"`
}
monolithArtifact struct {
WorkflowRunBackendID string `json:"workflowRunBackendId"` // the toolkit sends it back to GetSignedArtifactURL
DatabaseID int64 `json:"databaseId,string"`
Name string `json:"name"`
Size int64 `json:"size,string"`
CreatedAt time.Time `json:"createdAt"` // whole seconds, the toolkit rejects the trimmed fractions json writes
Digest string `json:"digest"` // empty until the upload is finalized
}
getSignedArtifactURLRequest struct {
WorkflowRunBackendID string
Name string
}
getSignedArtifactURLResponse struct {
SignedURL string `json:"signedUrl"`
}
createArtifactRequest = getSignedArtifactURLRequest
deleteArtifactRequest = getSignedArtifactURLRequest
deleteArtifactResponse = finalizeArtifactResponse
)
+72 -36
View File
@@ -40,7 +40,9 @@ const (
jobLabel = "com.gitea.runner.job."
maxCreateBody = 8 << 20
dockerProxyProbeTimeout = 5 * time.Second
dockerProxyProbeTimeout = 5 * time.Second
dockerProxyProbeWorkerTimeout = time.Minute
dockerProxyProbeTarget = "/gitea-runner-probe"
)
var (
@@ -48,7 +50,7 @@ var (
rawStreamPath = regexp.MustCompile(`^(/v[0-9.]+)?/(containers/[^/]+/attach|exec/[^/]+/start)$`)
)
var dockerProxyDirectories, dockerProxyProbes sync.Map
var dockerProxyProbes sync.Map
type dockerProxyDirectory struct {
dir, daemonDir string
@@ -68,7 +70,7 @@ func NewDockerProxy(ctx context.Context, job string) *DockerProxy {
}
probeCtx, cancel := context.WithTimeout(ctx, dockerProxyProbeTimeout)
defer cancel()
cli, err := GetDockerClient(probeCtx)
cli, err := client.New(client.FromEnv) // negotiates on first request, so a cached probe costs no round trip
if err != nil {
return nil
}
@@ -83,65 +85,69 @@ func NewDockerProxy(ctx context.Context, job string) *DockerProxy {
}
workdir, _ := os.Getwd()
key := [3]string{cli.DaemonHost(), workdir, os.TempDir()}
cached, _ := dockerProxyDirectories.Load(key)
directory, seen := cached.(dockerProxyDirectory)
seen = seen && os.SameFile(directory.socket, socket) // daemon replacement invalidates cached paths
if !seen {
directory, seen, err = probeDockerProxyDirectory(probeCtx, cli, key, socket)
}
probe, err := probeDockerProxyDirectory(probeCtx, cli, key, socket)
if err != nil {
common.Logger(ctx).Infof("docker proxy probe failed, jobs get the daemon socket directly: %v", err)
return nil
}
if !seen {
if !probe.seen {
common.Logger(ctx).Infof("the docker daemon cannot reach the runner's temporary or working directory, jobs get the daemon socket directly")
return nil
}
if ctx.Err() != nil {
return nil
}
proxy, err := StartDockerProxy(daemonSocket, directory.dir, job)
proxy, err := StartDockerProxy(daemonSocket, probe.directory.dir, job)
if err != nil {
dockerProxyDirectories.Delete(key)
dockerProxyProbes.CompareAndDelete(key, probe)
common.Logger(ctx).Warnf("docker proxy not started, the job gets the daemon socket directly: %v", err)
return nil
}
dockerProxyDirectories.Store(key, directory)
proxy.Socket = directory.daemonDir + strings.TrimPrefix(proxy.Socket, directory.dir)
proxy.Socket = probe.directory.daemonDir + strings.TrimPrefix(proxy.Socket, probe.directory.dir)
return proxy
}
func probeDockerProxyDirectory(ctx context.Context, cli client.APIClient, key [3]string, socket os.FileInfo) (dockerProxyDirectory, bool, error) {
func probeDockerProxyDirectory(ctx context.Context, cli client.APIClient, key [3]string, socket os.FileInfo) (*dockerProxyProbe, error) {
for {
probe := &dockerProxyProbe{directory: dockerProxyDirectory{socket: socket}, done: make(chan struct{})}
actual, loaded := dockerProxyProbes.LoadOrStore(key, probe)
if loaded {
probe, _ = actual.(*dockerProxyProbe)
if !os.SameFile(probe.directory.socket, socket) {
if !os.SameFile(probe.directory.socket, socket) { // daemon replacement invalidates cached paths
dockerProxyProbes.CompareAndDelete(key, probe)
continue
}
select {
case <-ctx.Done():
return dockerProxyDirectory{}, false, ctx.Err()
case <-probe.done:
}
if ctx.Err() == nil && (errors.Is(probe.err, context.Canceled) || errors.Is(probe.err, context.DeadlineExceeded)) {
continue
}
} else {
probe.directory.dir, probe.directory.daemonDir = runnerContainerWorkdir(ctx, cli)
probe.seen = probe.directory.daemonDir != ""
if !probe.seen {
if probe.directory.dir, probe.err = filepath.Abs(key[2]); probe.err == nil {
probe.directory.daemonDir = probe.directory.dir
probe.seen, probe.err = daemonSeesDir(ctx, cli, probe.directory.dir, probe.directory.daemonDir)
go func() { // outlives callers' deadlines so a slow daemon still gets its result cached
defer cli.Close()
ctx, cancel := context.WithTimeout(context.Background(), dockerProxyProbeWorkerTimeout)
defer cancel()
probe.directory.dir, probe.directory.daemonDir = runnerContainerWorkdir(ctx, cli)
probe.seen = probe.directory.daemonDir != ""
if !probe.seen {
if probe.directory.dir, probe.err = filepath.Abs(key[2]); probe.err == nil {
probe.directory.daemonDir = probe.directory.dir
probe.seen, probe.err = daemonSeesDir(ctx, cli, probe.directory.dir, probe.directory.daemonDir)
}
}
}
dockerProxyProbes.CompareAndDelete(key, probe)
close(probe.done)
if probe.seen {
dockerProxyProbes.LoadOrStore(key, probe)
} else {
dockerProxyProbes.CompareAndDelete(key, probe)
}
close(probe.done)
removeStaleProbeContainers(cli)
}()
}
return probe.directory, probe.seen, probe.err
select {
case <-ctx.Done():
return &dockerProxyProbe{}, ctx.Err()
case <-probe.done:
}
if ctx.Err() == nil && errors.Is(probe.err, context.DeadlineExceeded) {
continue
}
return probe, probe.err
}
}
@@ -200,7 +206,7 @@ func daemonSeesDir(ctx context.Context, cli client.APIClient, dir, daemonDir str
created, err = cli.ContainerCreate(ctx, client.ContainerCreateOptions{
Config: &container.Config{Image: image.ID, Cmd: []string{"true"}},
HostConfig: &container.HostConfig{Mounts: []mount.Mount{
{Type: mount.TypeBind, Source: daemonDir, Target: "/gitea-runner-probe", ReadOnly: true}, // not the marker itself, podman creates a missing bind source where docker rejects it
{Type: mount.TypeBind, Source: daemonDir, Target: dockerProxyProbeTarget, ReadOnly: true}, // not the marker itself, podman creates a missing bind source where docker rejects it
}},
})
if !cerrdefs.IsNotFound(err) {
@@ -213,7 +219,7 @@ func daemonSeesDir(ctx context.Context, cli client.APIClient, dir, daemonDir str
if err != nil {
return false, err
}
seen, err := containerSeesMarker(ctx, cli, dir, created.ID, "/gitea-runner-probe")
seen, err := containerSeesMarker(ctx, cli, dir, created.ID, dockerProxyProbeTarget)
cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), dockerProxyProbeTimeout)
defer cancel()
if _, removeErr := cli.ContainerRemove(cleanupCtx, created.ID, client.ContainerRemoveOptions{Force: true, RemoveVolumes: true}); removeErr != nil {
@@ -222,6 +228,36 @@ func daemonSeesDir(ctx context.Context, cli client.APIClient, dir, daemonDir str
return seen, err
}
// removeStaleProbeContainers removes probe containers left by abandoned creates once any probe in flight when listed has ended.
func removeStaleProbeContainers(cli client.APIClient) {
ctx, cancel := context.WithTimeout(context.Background(), dockerProxyProbeTimeout)
defer cancel()
listed, err := cli.ContainerList(ctx, client.ContainerListOptions{
All: true,
Filters: make(client.Filters).Add("status", string(container.StateCreated)).Add("volume", dockerProxyProbeTarget),
})
if err != nil {
common.Logger(ctx).Warnf("listing stale docker proxy probe containers failed: %v", err)
return
}
listed.Items = slices.DeleteFunc(listed.Items, func(stale container.Summary) bool {
return stale.State != container.StateCreated || !slices.ContainsFunc(stale.Mounts, func(mountPoint container.MountPoint) bool {
return mountPoint.Type == mount.TypeBind && !mountPoint.RW && mountPoint.Destination == dockerProxyProbeTarget
})
})
if len(listed.Items) == 0 {
return
}
time.Sleep(dockerProxyProbeWorkerTimeout + dockerProxyProbeTimeout)
for _, stale := range listed.Items {
ctx, cancel := context.WithTimeout(context.Background(), dockerProxyProbeTimeout)
if err := (&containerReference{cli: cli, id: stale.ID}).remove()(ctx); err != nil {
common.Logger(ctx).Warnf("removing stale docker proxy probe container failed: %v", err)
}
cancel()
}
}
// StartDockerProxy serves a job's docker socket in dir, labelling what the job creates through it.
func StartDockerProxy(daemonSocket, dir, job string) (*DockerProxy, error) {
info, err := os.Stat(daemonSocket)
+122 -56
View File
@@ -94,6 +94,8 @@ func TestDockerProxy(t *testing.T) {
return
}
_, _ = w.Write([]byte(`[{"Id":"probe-image"}]`))
case strings.HasSuffix(r.URL.Path, "/containers/json"):
_, _ = w.Write([]byte(`[]`))
case strings.HasSuffix(r.URL.Path, "/json"):
http.NotFound(w, r)
case strings.HasSuffix(r.URL.Path, "/archive"):
@@ -235,7 +237,8 @@ func TestDockerProxy(t *testing.T) {
t.Skip("automatic proxy discovery requires Linux")
}
t.Setenv("DOCKER_HOST", "unix://"+daemonSocket)
t.Setenv("TMPDIR", shortTempDir(t))
tempDir := shortTempDir(t)
t.Setenv("TMPDIR", tempDir)
require.Nil(t, NewDockerProxy(t.Context(), "failed"))
for _, job := range []string{"first", "second"} {
proxy := NewDockerProxy(t.Context(), job)
@@ -243,12 +246,10 @@ func TestDockerProxy(t *testing.T) {
require.NoError(t, proxy.Close(context.Background()))
}
assert.EqualValues(t, 2, probes.Load())
dockerProxyDirectories.Range(func(key, value any) bool {
directory, _ := value.(dockerProxyDirectory)
directory.socket, _ = os.Stat(t.TempDir())
dockerProxyDirectories.Store(key, directory)
return true
})
workdir, err := os.Getwd()
require.NoError(t, err)
cached, _ := dockerProxyProbes.Load([3]string{"unix://" + daemonSocket, workdir, tempDir})
cached.(*dockerProxyProbe).directory.socket, _ = os.Stat(t.TempDir())
proxy := NewDockerProxy(t.Context(), "replaced daemon")
require.NotNil(t, proxy)
require.NoError(t, proxy.Close(context.Background()))
@@ -476,10 +477,23 @@ func BenchmarkDockerProxy(b *testing.B) {
type probeClient struct {
mobyclient.APIClient
create func(mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error)
remove func(context.Context, string, mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error)
stat func(string) error
mounts []container.MountPoint
create func(context.Context, mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error)
remove func(context.Context, string, mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error)
stat func(string) error
mounts []container.MountPoint
containers []container.Summary
}
func (c *probeClient) Close() error {
return nil
}
func (c *probeClient) ContainerList(context.Context, mobyclient.ContainerListOptions) (mobyclient.ContainerListResult, error) {
return mobyclient.ContainerListResult{Items: c.containers}, nil
}
func (c *probeClient) ContainerKill(context.Context, string, mobyclient.ContainerKillOptions) (mobyclient.ContainerKillResult, error) {
return mobyclient.ContainerKillResult{}, cerrdefs.ErrConflict
}
func (c *probeClient) ContainerInspect(context.Context, string, mobyclient.ContainerInspectOptions) (mobyclient.ContainerInspectResult, error) {
@@ -490,8 +504,8 @@ func (c *probeClient) ImageList(context.Context, mobyclient.ImageListOptions) (m
return mobyclient.ImageListResult{Items: []image.Summary{{ID: "removed-image"}, {ID: "probe-image"}}}, nil
}
func (c *probeClient) ContainerCreate(_ context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
return c.create(opts)
func (c *probeClient) ContainerCreate(ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
return c.create(ctx, opts)
}
func (c *probeClient) ContainerRemove(ctx context.Context, id string, opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) {
@@ -518,7 +532,7 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
var calls, removals atomic.Int32
visible := false
cli := &probeClient{
create: func(_opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
create: func(_ctx context.Context, _opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
calls.Add(1)
<-release
return mobyclient.ContainerCreateResult{ID: "probe"}, createErr
@@ -538,23 +552,18 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
results := make(chan result, 8)
for range cap(results) {
go func() {
_, seen, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
results <- result{seen, err}
probe, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
results <- result{probe.seen, err}
}()
}
synctest.Wait()
assert.EqualValues(t, 1, calls.Load())
ctx, cancel := context.WithCancel(t.Context())
cancel()
_, seen, err := probeDockerProxyDirectory(ctx, cli, key, socket)
require.ErrorIs(t, err, context.Canceled)
assert.False(t, seen)
replacement, err := os.Stat(t.TempDir())
require.NoError(t, err)
replaced := make(chan result, 1)
go func() {
_, seen, err := probeDockerProxyDirectory(t.Context(), cli, key, replacement)
replaced <- result{seen, err}
probe, err := probeDockerProxyDirectory(t.Context(), cli, key, replacement)
replaced <- result{probe.seen, err}
}()
synctest.Wait()
assert.EqualValues(t, 2, calls.Load())
@@ -574,11 +583,11 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
assert.Zero(t, removals.Load())
}
createErr, visible = nil, true
directory, seen, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
probe, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
require.NoError(t, err)
assert.True(t, seen)
assert.Equal(t, dir, directory.dir)
assert.Equal(t, dir, directory.daemonDir)
assert.True(t, probe.seen)
assert.Equal(t, dir, probe.directory.dir)
assert.Equal(t, dir, probe.directory.daemonDir)
assert.EqualValues(t, 3, calls.Load())
entries, err := os.ReadDir(dir)
require.NoError(t, err)
@@ -586,7 +595,7 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
})
})
}
t.Run("cancelled probe does not cancel another caller", func(t *testing.T) {
t.Run("cancelled caller leaves the probe to finish for the others", func(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
dir := t.TempDir()
socket, err := os.Stat(t.TempDir())
@@ -595,49 +604,105 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
release := make(chan struct{})
calls := 0
var calls, removals atomic.Int32
cli := &probeClient{
create: func(_opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
calls++
if calls == 1 {
<-release
create: func(ctx context.Context, _opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
calls.Add(1)
<-release
return mobyclient.ContainerCreateResult{ID: "probe"}, ctx.Err()
},
stat: func(_path string) error { return nil },
remove: func(_ctx context.Context, _id string, _opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) {
removals.Add(1)
return mobyclient.ContainerRemoveResult{}, nil
},
}
leader, follower, late := make(chan result, 1), make(chan result, 1), make(chan result, 1)
go func() {
probe, err := probeDockerProxyDirectory(ctx, cli, key, socket)
leader <- result{probe.seen, err}
}()
synctest.Wait()
go func() {
probe, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
follower <- result{probe.seen, err}
}()
synctest.Wait()
cancel()
leaderResult := <-leader
require.ErrorIs(t, leaderResult.err, context.Canceled)
assert.False(t, leaderResult.seen)
go func() {
probe, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
late <- result{probe.seen, err}
}()
synctest.Wait()
close(release)
for _, results := range []chan result{follower, late} {
result := <-results
require.NoError(t, result.err)
assert.True(t, result.seen)
}
assert.EqualValues(t, 1, calls.Load())
assert.EqualValues(t, 1, removals.Load())
_, cached := dockerProxyProbes.Load(key)
assert.True(t, cached)
})
})
t.Run("caller joining a probe that ran out of time retries", func(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
dir := t.TempDir()
socket, err := os.Stat(t.TempDir())
require.NoError(t, err)
var calls atomic.Int32
cli := &probeClient{
create: func(ctx context.Context, _opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
if calls.Add(1) == 1 {
<-ctx.Done()
return mobyclient.ContainerCreateResult{}, ctx.Err()
} else {
return mobyclient.ContainerCreateResult{ID: "probe"}, nil
}
return mobyclient.ContainerCreateResult{ID: "probe"}, nil
},
stat: func(_path string) error { return nil },
remove: func(_ctx context.Context, _id string, _opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) {
return mobyclient.ContainerRemoveResult{}, nil
},
}
leader, follower := make(chan result, 1), make(chan result, 1)
go func() {
_, seen, err := probeDockerProxyDirectory(ctx, cli, key, socket)
leader <- result{seen, err}
}()
synctest.Wait()
go func() {
_, seen, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
follower <- result{seen, err}
}()
synctest.Wait()
assert.Equal(t, 1, calls)
cancel()
close(release)
leaderResult, followerResult := <-leader, <-follower
require.ErrorIs(t, leaderResult.err, context.Canceled)
assert.False(t, leaderResult.seen)
require.NoError(t, followerResult.err)
assert.True(t, followerResult.seen)
assert.Equal(t, 2, calls)
probe, err := probeDockerProxyDirectory(t.Context(), cli, [3]string{"daemon", dir, dir}, socket)
require.NoError(t, err)
assert.True(t, probe.seen)
assert.EqualValues(t, 2, calls.Load())
})
})
}
func TestRemoveStaleProbeContainers(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
probeMount := container.MountPoint{Type: mount.TypeBind, Destination: "/gitea-runner-probe"}
writableMount := container.MountPoint{Type: mount.TypeBind, Destination: "/gitea-runner-probe", RW: true}
var removed []string
cli := &probeClient{
containers: []container.Summary{
{ID: "abandoned", State: container.StateCreated, Mounts: []container.MountPoint{probeMount}},
{ID: "foreign", State: container.StateCreated, Mounts: []container.MountPoint{writableMount}},
{ID: "started", State: container.StateExited, Mounts: []container.MountPoint{probeMount}},
},
remove: func(_ctx context.Context, id string, _opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) {
removed = append(removed, id)
return mobyclient.ContainerRemoveResult{}, nil
},
}
start := time.Now()
removeStaleProbeContainers(cli)
assert.Equal(t, []string{"abandoned"}, removed)
assert.GreaterOrEqual(t, time.Since(start), dockerProxyProbeWorkerTimeout+dockerProxyProbeTimeout)
})
}
func TestDaemonSeesDir(t *testing.T) {
dir := t.TempDir()
markers := make(map[string]bool)
for _, testCase := range []struct {
name string
daemonDir string
@@ -656,7 +721,7 @@ func TestDaemonSeesDir(t *testing.T) {
removed := false
var attemptedImages []string
cli := &probeClient{
create: func(opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
create: func(_ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
assert.Equal(t, []mount.Mount{{Type: mount.TypeBind, Source: testCase.daemonDir, Target: "/gitea-runner-probe", ReadOnly: true}}, opts.HostConfig.Mounts)
attemptedImages = append(attemptedImages, opts.Config.Image)
if opts.Config.Image == "removed-image" {
@@ -692,6 +757,7 @@ func TestDaemonSeesDir(t *testing.T) {
assert.Equal(t, []string{"removed-image", "probe-image"}, attemptedImages)
assert.Equal(t, testCase.seen, seen)
assert.Equal(t, testCase.removed, removed)
entries, err := os.ReadDir(dir)
require.NoError(t, err)
assert.Empty(t, entries)
+1 -1
View File
@@ -980,7 +980,7 @@ func TestDockerCreateKeepsInputEntrypoint(t *testing.T) {
RunnerOptions: "--entrypoint=runner-entrypoint",
WorkflowOptions: "--entrypoint=" + optionEntrypoint + " --init",
},
cli: &probeClient{create: func(opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
cli: &probeClient{create: func(_ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
if len(entrypoint) > 0 {
assert.Equal(t, entrypoint, opts.Config.Entrypoint)
} else {
+95 -10
View File
@@ -7,12 +7,15 @@ import (
"bufio"
"bytes"
"cmp"
"compress/gzip"
"context"
"crypto/rand"
"encoding/json"
"errors"
"fmt"
"io"
"maps"
"net"
"net/http"
"net/url"
"os"
@@ -23,6 +26,7 @@ import (
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"github.com/kballard/go-shellquote"
"github.com/spf13/pflag"
@@ -36,6 +40,7 @@ type KubernetesOptions struct {
MaxLifetime time.Duration
ForcePull bool
RunnerUUID string
Transfer *KubernetesTransfer
}
// KubernetesPod runs a job container and its service containers as one pod.
@@ -46,6 +51,10 @@ type KubernetesPod struct {
job *kubernetesContainer
services []*kubernetesContainer
arch, path string
ips []net.IP
transfer *transferServer
transferImage string
transferHelper string // empty when copies go through the API server
}
type kubernetesContainer struct {
@@ -55,6 +64,7 @@ type kubernetesContainer struct {
name string
input *NewContainerInput
probe map[string]any
gzip bool // the container can gunzip, so copies into it go compressed
}
type containerStatus struct {
@@ -74,6 +84,7 @@ type podStatus struct {
Spec struct{ Containers []struct{} }
Status struct {
Phase, Reason, Message string
PodIPs []struct{ IP string }
Conditions []struct{ Type, Status, Message string }
InitContainerStatuses, ContainerStatuses []containerStatus
}
@@ -129,14 +140,24 @@ func (p *KubernetesPod) create(ctx context.Context) error {
labels[runnerUUIDLabel] = p.options.RunnerUUID
}
metadata := map[string]any{"name": name, "labels": labels}
var runnerImage string
if hostname, err := os.Hostname(); p.client.inCluster && err == nil {
var runner struct{ Metadata struct{ Name, UID string } }
var runner struct {
Metadata struct{ Name, UID string }
Spec struct{ Containers []struct{ Image string } }
}
if err := p.client.do(ctx, http.MethodGet, p.client.path("pods/"+hostname), nil, &runner); err == nil {
metadata["ownerReferences"] = []any{map[string]any{"apiVersion": "v1", "kind": "Pod", "name": runner.Metadata.Name, "uid": runner.Metadata.UID}}
if len(runner.Spec.Containers) == 1 {
runnerImage = runner.Spec.Containers[0].Image
}
} else {
common.Logger(ctx).Debugf("The runner is not pod %s, job pods get no owner: %v", hostname, err)
}
}
if err := p.setupTransfer(runnerImage); err != nil {
return err
}
secretData := map[string]string{}
manifest, err := p.manifest(name, secretData, metadata)
if err != nil {
@@ -205,6 +226,18 @@ func (p *KubernetesPod) manifest(secret string, secretData map[string]string, me
}
}
spec := map[string]any{"restartPolicy": "Never", "containers": containers, "volumes": volumes}
if p.transferHelper != "" {
transfer := map[string]any{
"name": "gitea-runner-transfer", "image": p.transferImage, "terminationMessagePolicy": "FallbackToLogsOnError",
"command": []string{"gitea-runner", KubernetesTransferCommand, "install", p.transferHelper},
"volumeMounts": []any{map[string]any{"name": "act", "mountPath": p.job.GetActPath()}},
"securityContext": map[string]any{"allowPrivilegeEscalation": false, "capabilities": map[string]any{"drop": []string{"ALL"}}},
}
if p.options.ForcePull {
transfer["imagePullPolicy"] = "Always"
}
spec["initContainers"] = []any{transfer}
}
if len(hostnames) > 0 {
spec["hostAliases"] = []any{map[string]any{"ip": "127.0.0.1", "hostnames": hostnames}}
}
@@ -341,7 +374,13 @@ func (p *KubernetesPod) waitStarted(ctx context.Context) error {
return err
}
if pod.Status.Phase == "Failed" {
return fmt.Errorf("pod %s failed: %s", p.name, cmp.Or(pod.Status.Message, pod.Status.Reason))
reason := cmp.Or(pod.Status.Message, pod.Status.Reason)
for _, status := range append(pod.Status.InitContainerStatuses, pod.Status.ContainerStatuses...) {
if terminated := status.State.Terminated; reason == "" && terminated != nil && terminated.ExitCode != 0 {
reason = fmt.Sprintf("container %s exited with %d: %s", status.Name, terminated.ExitCode, strings.TrimSpace(terminated.Message))
}
}
return fmt.Errorf("pod %s failed: %s", p.name, reason)
}
started := len(pod.Status.ContainerStatuses) == len(pod.Spec.Containers)
for _, status := range append(pod.Status.InitContainerStatuses, pod.Status.ContainerStatuses...) {
@@ -354,6 +393,9 @@ func (p *KubernetesPod) waitStarted(ctx context.Context) error {
}
}
if started {
for _, ip := range pod.Status.PodIPs {
p.ips = append(p.ips, net.ParseIP(ip.IP))
}
return nil
}
for _, condition := range pod.Status.Conditions {
@@ -391,8 +433,11 @@ func (c *kubernetesContainer) Start(_ bool) common.Executor {
if err != nil {
return err
}
machine, path, _ := strings.Cut(output, "\n")
c.pod.arch, c.pod.path = goArchToActionArch(machine), path
probed := strings.SplitN(output, "\n", 3)
for len(probed) < 3 {
probed = append(probed, "")
}
c.pod.arch, c.gzip, c.pod.path = goArchToActionArch(probed[0]), probed[1] == "gzip", probed[2]
return nil
}
}
@@ -403,7 +448,7 @@ func (p *KubernetesPod) probe(ctx context.Context, container string) (string, er
deadline := time.Now().Add(time.Minute)
for delay := 100 * time.Millisecond; ; delay = min(2*delay, 2*time.Second) {
var stdout bytes.Buffer
err := p.client.exec(ctx, p.name, container, []string{"sh", "-c", `printf '%s\n%s' "$(uname -m)" "$PATH"`}, nil, &stdout, io.Discard)
err := p.client.exec(ctx, p.name, container, []string{"sh", "-c", `printf '%s\n%s\n%s' "$(uname -m)" "$(command -v gzip >/dev/null && echo gzip)" "$PATH"`}, nil, &stdout, io.Discard)
var exitErr ExitCodeError
var apiErr *apiError
if err == nil || ctx.Err() != nil || errors.As(err, &exitErr) || time.Now().After(deadline) ||
@@ -452,26 +497,66 @@ func (c *kubernetesContainer) Exec(command []string, env map[string]string, _, w
}
}
func (c *kubernetesContainer) extract(ctx context.Context, destPath string, write func(io.Writer) error) error {
// compressCopyMin is the size from which Copy compresses and uses the transfer helper: below it that saves next to nothing on the wire and costs a process in the pod.
const compressCopyMin = 64 << 10
// extract streams the tar write produces into destPath, gzipped with compress, as copied actions shrink about fivefold and API server traffic often crosses zones.
func (c *kubernetesContainer) extract(ctx context.Context, destPath string, compress bool, write func(io.Writer) error) error {
if compress && c.pod.transferHelper != "" {
request, done := c.pod.transfer.offer(c.pod.ips, write)
stdin, _ := json.Marshal(request)
var stderr bytes.Buffer
err := c.pod.client.exec(ctx, c.pod.name, c.name, []string{c.pod.transferHelper, KubernetesTransferCommand, "fetch", destPath}, bytes.NewReader(stdin), io.Discard, &stderr)
taken, writeErr := done()
if err == nil || ctx.Err() != nil {
return err
}
if taken || !c.pod.options.Transfer.Fallback {
return fmt.Errorf("copy to %s over the pod network: %w: %s", destPath, errors.Join(writeErr, err), strings.TrimSpace(stderr.String()))
}
c.pod.transferHelper = "" // a pod that cannot reach the runner once will not later
logging.Warnf("Copying into pod %s over the pod network failed, copying through the API server instead: %v: %s", c.pod.name, err, strings.TrimSpace(stderr.String()))
}
compress = compress && c.gzip
reader, writer := io.Pipe()
defer reader.Close()
go func() { writer.CloseWithError(write(writer)) }()
go func() {
if compress {
writer.CloseWithError(writeGzip(writer, write))
} else {
writer.CloseWithError(write(writer))
}
}()
script := `mkdir -p "$1" && tar -xf - -C "$1"`
if compress {
script = `mkdir -p "$1" && gzip -dc | tar -xf - -C "$1"` // gzip -dc rather than tar -z, which GNU tar runs gzip for anyway
}
var stderr bytes.Buffer
if err := c.pod.client.exec(ctx, c.pod.name, c.name, []string{"sh", "-c", `mkdir -p "$1" && tar -xf - -C "$1"`, "sh", destPath}, reader, io.Discard, &stderr); err != nil {
if err := c.pod.client.exec(ctx, c.pod.name, c.name, []string{"sh", "-c", script, "sh", destPath}, reader, io.Discard, &stderr); err != nil {
return fmt.Errorf("extract to %s: %w: %s", destPath, err, strings.TrimSpace(stderr.String()))
}
return nil
}
// writeGzip gzips what write produces into w, at a level fast enough to not slow copies down and nearly as small as the default.
func writeGzip(w io.Writer, write func(io.Writer) error) error {
compressed, _ := gzip.NewWriterLevel(w, gzip.BestSpeed)
return errors.Join(write(compressed), compressed.Close())
}
func (c *kubernetesContainer) Copy(destPath string, files ...*FileEntry) common.Executor {
return func(ctx context.Context) error {
return c.extract(ctx, destPath, func(writer io.Writer) error { return writeFilesTar(ctx, writer, 0, 0, files...) })
size := 0
for _, file := range files {
size += len(file.Body)
}
return c.extract(ctx, destPath, size >= compressCopyMin, func(writer io.Writer) error { return writeFilesTar(ctx, writer, 0, 0, files...) })
}
}
func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor {
return func(ctx context.Context) error {
return c.extract(ctx, destPath, func(writer io.Writer) error {
return c.extract(ctx, destPath, true, func(writer io.Writer) error { // directories are actions and workspaces, megabytes of compressible sources
return writeDirTar(ctx, writer, "/", srcPath, useGitIgnore, skipGitDir, 0, 0)
})
}
+50 -1
View File
@@ -260,6 +260,7 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
)
require.NoError(t, job.Start(false)(ctx))
assert.Len(t, cluster.requests["exec"], 3)
assert.True(t, pod.job.gzip)
stepEnv := map[string]string{"INPUT_WHO-TO-GREET": "it's\nme", "SECRET": "hunter2", "UID": "5"}
require.NoError(t, job.Exec([]string{"printenv", "SECRET"}, stepEnv, "", "")(ctx))
@@ -276,7 +277,9 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
assert.NotContains(t, query, "hunter2")
}
lastExec := func() string { return cluster.requests["exec"][len(cluster.requests["exec"])-1] }
require.NoError(t, job.Copy(workdir+"/act/", &FileEntry{Name: "workflow/event.json", Mode: 0o644, Body: `{"ref":"main"}`})(ctx))
assert.NotContains(t, lastExec(), "gzip")
archive, err := job.GetContainerArchive(ctx, workdir+"/act/workflow/event.json")
require.NoError(t, err)
reader := tar.NewReader(archive)
@@ -289,6 +292,12 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
require.NoError(t, archive.Close())
_, err = job.GetContainerArchive(ctx, workdir+"/missing")
require.Error(t, err)
large := strings.Repeat("a", compressCopyMin)
require.NoError(t, job.Copy(workdir+"/act/", &FileEntry{Name: "workflow/large", Mode: 0o644, Body: large})(ctx))
assert.Contains(t, lastExec(), url.QueryEscape("gzip -dc |"))
content, err = os.ReadFile(workdir + "/act/workflow/large")
require.NoError(t, err)
assert.Equal(t, large, string(content))
actionDir := filepath.Join(t.TempDir(), "action")
require.NoError(t, os.MkdirAll(actionDir, 0o755))
@@ -297,7 +306,8 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
script, err := os.ReadFile(workdir + "/act/actions/sha/index.js")
require.NoError(t, err)
assert.Equal(t, "main()", string(script))
assert.Contains(t, cluster.requests["exec"][len(cluster.requests["exec"])-1], "command="+url.QueryEscape(workdir+"/act/actions/sha/"))
assert.Contains(t, lastExec(), "command="+url.QueryEscape(workdir+"/act/actions/sha/"))
assert.Contains(t, lastExec(), url.QueryEscape("gzip -dc |"))
pathEnv := map[string]string{}
require.NoError(t, job.UpdateFromImageEnv(&pathEnv)(ctx))
@@ -321,6 +331,38 @@ func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
assert.Empty(t, cluster.requests["DELETE /api/v1/namespaces/ci/secrets/new"])
}
func TestKubernetesCopiesUncompressedWithoutGzip(t *testing.T) {
t.Parallel()
if runtime.GOOS == "windows" {
t.Skip("uses POSIX shell")
}
ctx := t.Context()
workdir, bin := t.TempDir(), t.TempDir()
for _, tool := range []string{"uname", "tar", "mkdir"} { // all the job container needs but gzip, as on a minimal image
found, err := exec.LookPath(tool)
require.NoError(t, err)
require.NoError(t, os.Symlink(found, filepath.Join(bin, tool)))
}
kubeconfig, cluster := fakeKubernetes(t, `{"spec":{"containers":[{}]},"status":{"phase":"Running","containerStatuses":[
{"name":"job","ready":true,"state":{"running":{}}}]}}`, "PATH="+bin)
pod, err := NewKubernetesPod(KubernetesOptions{Kubeconfig: kubeconfig})
require.NoError(t, err)
job := pod.JobContainer(&NewContainerInput{Image: "node", WorkingDir: workdir})
require.NoError(t, job.Create(nil, nil)(ctx))
require.NoError(t, job.Start(false)(ctx))
assert.False(t, pod.job.gzip)
assert.Equal(t, bin, pod.path)
actionDir := filepath.Join(t.TempDir(), "action")
require.NoError(t, os.MkdirAll(actionDir, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(actionDir, "index.js"), []byte("main()"), 0o644))
require.NoError(t, job.CopyDir(workdir+"/act/actions/sha/", actionDir+"/", false, false)(ctx))
script, err := os.ReadFile(workdir + "/act/actions/sha/index.js")
require.NoError(t, err)
assert.Equal(t, "main()", string(script))
assert.NotContains(t, cluster.requests["exec"][len(cluster.requests["exec"])-1], "gzip")
}
func TestKubernetesPodFailsBeforeSteps(t *testing.T) {
kubeconfig, _ := fakeKubernetes(t, `{"spec":{"containers":[{}]},"status":{"containerStatuses":[
{"name":"job","state":{"waiting":{"reason":"ErrImagePull","message":"no such image"}}}]}}`)
@@ -361,6 +403,13 @@ func TestKubernetesPodFailsBeforeSteps(t *testing.T) {
job := pod.JobContainer(&NewContainerInput{Image: "missing"})
require.NoError(t, job.Create(nil, nil)(t.Context()))
require.ErrorContains(t, job.Start(false)(t.Context()), "ErrImagePull: no such image")
failed, _ := fakeKubernetes(t, `{"spec":{"containers":[{}]},"status":{"phase":"Failed","initContainerStatuses":[
{"name":"gitea-runner-transfer","state":{"terminated":{"exitCode":1,"message":"exec format error\n"}}}]}}`)
pod, err = NewKubernetesPod(KubernetesOptions{Kubeconfig: failed})
require.NoError(t, err)
job = pod.JobContainer(&NewContainerInput{Image: "node"})
require.NoError(t, job.Create(nil, nil)(t.Context()))
require.ErrorContains(t, job.Start(false)(t.Context()), "container gitea-runner-transfer exited with 1: exec format error")
plugin := filepath.Join(t.TempDir(), "kubeconfig")
require.NoError(t, os.WriteFile(plugin, []byte(`
+248
View File
@@ -0,0 +1,248 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"cmp"
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"math/big"
"net"
"net/http"
"os"
"os/exec"
"path"
"path/filepath"
"slices"
"strconv"
"strings"
"sync"
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
)
// KubernetesTransfer has job pods fetch large copies from the runner over the pod network instead of through the API server.
type KubernetesTransfer struct {
Host string // empty uses the outbound IP
Port uint16 // 0 picks a free port
Image string // has gitea-runner on its PATH, for an init container to copy into each job pod
Fallback bool // copy through the API server when a job pod cannot reach the runner, else fail
}
// KubernetesTransferCommand is the hidden runner subcommand job pods run the transfer helper with.
const KubernetesTransferCommand = "kubernetes-transfer"
// transferRequest goes to the helper on the exec stream's stdin, keeping the token out of the command line and the audit log.
type transferRequest struct {
URL string `json:"url"`
Pin string `json:"pin"` // hex SHA-256 of the runner's certificate
}
type transferServer struct {
base string
pin string
mu sync.Mutex
pending map[string]transferOffer
}
type transferOffer struct {
clients []net.IP
write func(io.Writer) error
result chan error
}
var (
transferServersMu sync.Mutex
transferServers = map[string]*transferServer{}
)
// startTransferServer returns the runner's server for these settings, started on first use.
func startTransferServer(host string, port uint16) (*transferServer, error) {
transferServersMu.Lock()
defer transferServersMu.Unlock()
configured := net.JoinHostPort(host, strconv.Itoa(int(port)))
if server, ok := transferServers[configured]; ok {
return server, nil
}
if host == "" {
ip := common.GetOutboundIP()
if ip == nil {
return nil, errors.New("no outbound IP for job pods to reach the runner at, set kubernetes.transfer.host")
}
host = ip.String()
}
privateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return nil, err
}
template := &x509.Certificate{SerialNumber: big.NewInt(1)} // trusted by its pin alone, so no CA, names or validity
certificate, err := x509.CreateCertificate(rand.Reader, template, template, &privateKey.PublicKey, privateKey)
if err != nil {
return nil, err
}
listener, err := net.Listen("tcp", net.JoinHostPort(host, strconv.Itoa(int(port))))
if err != nil {
return nil, fmt.Errorf("listen for job pod transfers: %w", err)
}
pin := sha256.Sum256(certificate)
server := &transferServer{base: "https://" + listener.Addr().String() + "/", pin: hex.EncodeToString(pin[:]), pending: map[string]transferOffer{}}
httpServer := &http.Server{
Handler: server,
TLSConfig: &tls.Config{Certificates: []tls.Certificate{{Certificate: [][]byte{certificate}, PrivateKey: privateKey}}, MinVersion: tls.VersionTLS13},
ReadHeaderTimeout: 10 * time.Second,
IdleTimeout: 30 * time.Second,
}
go func() { _ = httpServer.ServeTLS(listener, "", "") }()
transferServers[configured] = server
return server, nil
}
// offer serves write once to one of clients until done, which reports whether a client took it and what write failed with.
func (s *transferServer) offer(clients []net.IP, write func(io.Writer) error) (request transferRequest, done func() (taken bool, err error)) {
token := rand.Text()
result := make(chan error, 1)
s.mu.Lock()
s.pending[token] = transferOffer{clients: clients, write: write, result: result}
s.mu.Unlock()
return transferRequest{URL: s.base + token, Pin: s.pin}, func() (bool, error) {
s.mu.Lock()
_, pending := s.pending[token]
delete(s.pending, token)
s.mu.Unlock()
select {
case err := <-result:
return true, err
default:
return !pending, nil
}
}
}
func (s *transferServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
host, _, _ := net.SplitHostPort(r.RemoteAddr)
token := strings.TrimPrefix(r.URL.Path, "/")
s.mu.Lock()
offer, ok := s.pending[token]
ok = ok && r.Method == http.MethodGet && slices.ContainsFunc(offer.clients, net.ParseIP(host).Equal)
if ok {
delete(s.pending, token)
}
s.mu.Unlock()
if !ok {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Encoding", "gzip")
err := writeGzip(w, offer.write)
offer.result <- err
if err != nil {
panic(http.ErrAbortHandler) // a cut-off body fails the fetch, a short tar might not
}
}
// FetchKubernetesTransfer reads a transferRequest from request and extracts the tar it points to into dest.
func FetchKubernetesTransfer(ctx context.Context, request io.Reader, dest string) error {
var transfer transferRequest
if err := json.NewDecoder(request).Decode(&transfer); err != nil {
return fmt.Errorf("read transfer request: %w", err)
}
client := &http.Client{Transport: &http.Transport{
DialContext: (&net.Dialer{Timeout: 10 * time.Second}).DialContext,
TLSHandshakeTimeout: 10 * time.Second,
ResponseHeaderTimeout: time.Minute,
TLSClientConfig: &tls.Config{
MinVersion: tls.VersionTLS13,
InsecureSkipVerify: true, // VerifyConnection checks the pin instead
VerifyConnection: func(state tls.ConnectionState) error {
if sum := sha256.Sum256(state.PeerCertificates[0].Raw); hex.EncodeToString(sum[:]) != transfer.Pin {
return errors.New("runner certificate does not match the pin")
}
return nil
},
},
}}
httpRequest, err := http.NewRequestWithContext(ctx, http.MethodGet, transfer.URL, nil)
if err != nil {
return err
}
response, err := client.Do(httpRequest)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return fmt.Errorf("fetch: %s", response.Status)
}
if err := os.MkdirAll(dest, 0o777); err != nil {
return err
}
cmd := exec.CommandContext(ctx, "tar", "-xf", "-", "-C", dest)
cmd.Stdin, cmd.Stdout, cmd.Stderr = response.Body, os.Stderr, os.Stderr
if err := cmd.Run(); err != nil {
return err
}
_, err = io.Copy(io.Discard, response.Body) // tar stops at the first zero blocks, a cut-off body after them still fails
return err
}
// InstallKubernetesTransferHelper copies the running binary to target.
func InstallKubernetesTransferHelper(target string) error {
executable, err := os.Executable()
if err != nil {
return err
}
source, err := os.Open(executable)
if err != nil {
return err
}
defer source.Close()
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return err
}
_ = os.Remove(target) // a file left in a reused act volume would keep its mode
file, err := os.OpenFile(target, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o755)
if err != nil {
return err
}
_, err = io.Copy(file, source)
return errors.Join(err, file.Close())
}
// setupTransfer turns on transfers for the pod, with the runner pod's image unless one is set.
func (p *KubernetesPod) setupTransfer(runnerImage string) error {
settings := p.options.Transfer
if settings == nil {
return nil
}
image := cmp.Or(settings.Image, runnerImage)
var err error
var server *transferServer
if image == "" {
err = errors.New("set kubernetes.transfer.image, the runner is no single-container pod in kubernetes.namespace to take it from")
} else {
server, err = startTransferServer(settings.Host, settings.Port)
}
if err != nil {
if !settings.Fallback {
return fmt.Errorf("kubernetes.transfer: %w", err)
}
logging.Warnf("Job pod copies go through the API server: %v", err)
return nil
}
p.transfer, p.transferImage = server, image
p.transferHelper = path.Join(p.job.GetActPath(), "bin", "gitea-runner")
return nil
}
+185
View File
@@ -0,0 +1,185 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"bytes"
"context"
"crypto/rand"
"crypto/tls"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const transferHelperEnv = "GITEA_RUNNER_TEST_TRANSFER_HELPER"
func TestMain(m *testing.M) {
if os.Getenv(transferHelperEnv) == "1" && len(os.Args) == 4 && os.Args[1] == KubernetesTransferCommand && os.Args[2] == "fetch" {
if err := FetchKubernetesTransfer(context.Background(), os.Stdin, os.Args[3]); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
os.Exit(0)
}
os.Exit(m.Run())
}
func fetch(t *testing.T, request transferRequest) error {
t.Helper()
body, err := json.Marshal(request)
require.NoError(t, err)
return FetchKubernetesTransfer(t.Context(), bytes.NewReader(body), t.TempDir())
}
func TestTransferServer(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("extracts with tar")
}
server, err := startTransferServer("127.0.0.1", 0)
require.NoError(t, err)
again, err := startTransferServer("127.0.0.1", 0)
require.NoError(t, err)
assert.Same(t, server, again)
loopback := []net.IP{net.ParseIP("127.0.0.1")}
nothing := func(io.Writer) error { return nil }
t.Run("serves an offer gzipped once to an IP of its job pod", func(t *testing.T) {
request, done := server.offer([]net.IP{net.ParseIP("fd00::1"), loopback[0]}, nothing)
client := &http.Client{Transport: &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, DisableCompression: true}}
response, err := client.Get(request.URL)
require.NoError(t, err)
require.NoError(t, response.Body.Close())
assert.Equal(t, "gzip", response.Header.Get("Content-Encoding"))
require.ErrorContains(t, fetch(t, request), "404")
taken, _ := done()
assert.True(t, taken)
})
t.Run("another pod can neither take nor block an offer", func(t *testing.T) {
request, done := server.offer([]net.IP{net.ParseIP("10.1.2.3")}, nothing)
require.ErrorContains(t, fetch(t, request), "404")
taken, _ := done()
assert.False(t, taken)
})
t.Run("a failed write fails the fetch even past the end blocks tar stops at, and done reports it", func(t *testing.T) {
request, done := server.offer(loopback, func(w io.Writer) error {
_, _ = w.Write(make([]byte, 1024))
_, _ = io.CopyN(w, rand.Reader, 1<<20)
return errors.New("source gone")
})
require.Error(t, fetch(t, request))
taken, err := done()
assert.True(t, taken)
require.EqualError(t, err, "source gone")
})
t.Run("the helper refuses a runner whose certificate does not match its pin", func(t *testing.T) {
request, done := server.offer(loopback, nothing)
request.Pin = strings.Repeat("0", 64)
require.ErrorContains(t, fetch(t, request), "does not match the pin")
taken, _ := done()
assert.False(t, taken)
})
}
func TestKubernetesPodTransferImage(t *testing.T) {
newPod := func(image string, fallback bool) *KubernetesPod {
pod := &KubernetesPod{options: KubernetesOptions{Transfer: &KubernetesTransfer{Host: "127.0.0.1", Image: image, Fallback: fallback}}}
pod.JobContainer(&NewContainerInput{})
return pod
}
pod := newPod("", false)
require.NoError(t, pod.setupTransfer("gitea/runner:latest"))
assert.Equal(t, "gitea/runner:latest", pod.transferImage)
pod = newPod("registry.example/runner:1", false)
require.NoError(t, pod.setupTransfer("gitea/runner:latest"))
assert.Equal(t, "registry.example/runner:1", pod.transferImage)
pod = newPod("", true)
require.NoError(t, pod.setupTransfer(""))
assert.Empty(t, pod.transferHelper)
pod = newPod("", false)
require.ErrorContains(t, pod.setupTransfer(""), "kubernetes.transfer.image")
}
func TestKubernetesPodCopiesLargeFilesOverPodNetwork(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses POSIX shell")
}
ctx := t.Context()
workdir, err := filepath.EvalSymlinks(t.TempDir())
require.NoError(t, err)
kubeconfig, cluster := fakeKubernetes(t, `{"spec":{"containers":[{}]},"status":{"phase":"Running","podIPs":[{"ip":"127.0.0.1"}],"containerStatuses":[
{"name":"job","ready":true,"state":{"running":{}}}]}}`, transferHelperEnv+"=1")
pod, err := NewKubernetesPod(KubernetesOptions{Kubeconfig: kubeconfig, ForcePull: true, Transfer: &KubernetesTransfer{Host: "127.0.0.1", Image: "gitea/runner:latest", Fallback: true}})
require.NoError(t, err)
job := pod.JobContainer(&NewContainerInput{Name: "job", Image: "node", WorkingDir: workdir, Stdout: io.Discard, Stderr: io.Discard})
require.NoError(t, job.Create(nil, nil)(ctx))
var manifest struct {
Spec struct {
InitContainers []struct {
Image, ImagePullPolicy string
Command []string
VolumeMounts []struct{ Name, MountPath string }
}
}
}
require.NoError(t, json.Unmarshal([]byte(cluster.requests["POST /api/v1/namespaces/ci/pods"][0]), &manifest))
require.Len(t, manifest.Spec.InitContainers, 1)
initContainer := manifest.Spec.InitContainers[0]
assert.Equal(t, "gitea/runner:latest", initContainer.Image)
assert.Equal(t, "Always", initContainer.ImagePullPolicy)
assert.Equal(t, []string{"gitea-runner", KubernetesTransferCommand, "install", "/var/run/act/bin/gitea-runner"}, initContainer.Command)
assert.Equal(t, []struct{ Name, MountPath string }{{"act", "/var/run/act"}}, initContainer.VolumeMounts)
require.NoError(t, job.Start(false)(ctx))
helper := filepath.Join(t.TempDir(), "bin", "gitea-runner")
require.NoError(t, InstallKubernetesTransferHelper(helper))
pod.transferHelper = helper
lastExec := func() string { return cluster.requests["exec"][len(cluster.requests["exec"])-1] }
large := &FileEntry{Name: "large", Mode: 0o644, Body: strings.Repeat("a", compressCopyMin)}
copyLarge := func(dir string) error { return job.Copy(filepath.Join(workdir, dir)+"/", large)(ctx) }
require.NoError(t, copyLarge("transferred"))
content, err := os.ReadFile(filepath.Join(workdir, "transferred", "large"))
require.NoError(t, err)
assert.Equal(t, large.Body, string(content))
assert.Contains(t, lastExec(), "command="+KubernetesTransferCommand)
assert.NotContains(t, lastExec(), "127.0.0.1")
require.NoError(t, job.Copy(workdir+"/act/", &FileEntry{Name: "event.json", Mode: 0o644, Body: `{}`})(ctx))
assert.NotContains(t, lastExec(), "command="+KubernetesTransferCommand)
require.NoError(t, os.WriteFile(filepath.Join(workdir, "file"), nil, 0o644))
require.ErrorContains(t, copyLarge("file"), "over the pod network")
assert.Equal(t, helper, pod.transferHelper)
unreachable := filepath.Join(workdir, "unreachable")
require.NoError(t, os.WriteFile(unreachable, []byte("#!/bin/sh\necho 'connection refused' >&2\nexit 1\n"), 0o755))
pod.transferHelper = unreachable
pod.options.Transfer.Fallback = false
require.ErrorContains(t, copyLarge("direct"), "connection refused")
assert.NoFileExists(t, filepath.Join(workdir, "direct", "large"))
pod.options.Transfer.Fallback = true
require.NoError(t, copyLarge("auto"))
assert.FileExists(t, filepath.Join(workdir, "auto", "large"))
assert.Empty(t, pod.transferHelper)
}
+9 -7
View File
@@ -180,9 +180,9 @@ func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction
action := step.getActionModel()
rc.withGithubEnv(ctx, step.getGithubContext(ctx), *step.getEnv())
rc.withGithubEnv(ctx, getStepGithubContext(ctx, step), *step.getEnv())
populateEnvsFromSavedState(step.getEnv(), step, rc)
if err := populateEnvsFromInput(ctx, step.getEnv(), action, rc); err != nil {
if err := populateEnvsFromInput(ctx, step, action); err != nil {
return err
}
warnUnexpectedInputs(ctx, action, stepModel.With)
@@ -283,7 +283,7 @@ func (rc *RunContext) dockerActionImage(ctx context.Context, step actionStep, ac
}
image, ok := job.dockerActionImages[step.getStepModel()]
if !ok {
image = dockerActionImageTag(step.getGithubContext(ctx).Repository, actionName, localAction) + "-" + strings.ToLower(rand.Text())
image = dockerActionImageTag(getStepGithubContext(ctx, step).Repository, actionName, localAction) + "-" + strings.ToLower(rand.Text())
if !common.Dryrun(ctx) { // a dry run builds nothing to remove
job.dockerActionImages[step.getStepModel()] = image
}
@@ -437,7 +437,7 @@ func evalDockerEnv(ctx context.Context, step step, action *model.Action) (*expre
for key := range step.getStepModel().With {
providedInputs[strings.ToLower(key)] = true
}
eval := rc.NewExpressionEvaluator(ctx)
eval := rc.NewStepExpressionEvaluator(ctx, step)
for k, input := range action.Inputs {
if providedInputs[strings.ToLower(k)] {
continue
@@ -456,6 +456,7 @@ func evalDockerEnv(ctx context.Context, step step, action *model.Action) (*expre
}
}
mergeIntoMap(step, step.getEnv(), runsEnv, maps.Clone(*step.getEnv()))
aliasGiteaEnv(*step.getEnv())
return ee, nil
}
@@ -509,8 +510,9 @@ func inputEnvKey(inputID string) string {
return "INPUT_" + inputEnvPattern.ReplaceAllString(strings.ToUpper(inputID), "_")
}
func populateEnvsFromInput(ctx context.Context, env *map[string]string, action *model.Action, rc *RunContext) error {
eval := rc.NewExpressionEvaluator(ctx)
func populateEnvsFromInput(ctx context.Context, step step, action *model.Action) error {
env := step.getEnv()
eval := step.getRunContext().NewStepExpressionEvaluator(ctx, step)
for inputID, input := range action.Inputs {
envKey := inputEnvKey(inputID)
if _, ok := (*env)[envKey]; !ok {
@@ -652,7 +654,7 @@ func runPreStep(step actionStep) common.Executor {
x := action.Runs.Using
if !x.IsComposite() {
// defaults in pre steps were missing, however provided inputs are available
if err := populateEnvsFromInput(ctx, step.getEnv(), action, rc); err != nil {
if err := populateEnvsFromInput(ctx, step, action); err != nil {
return err
}
}
+1 -3
View File
@@ -57,9 +57,6 @@ func evaluateCompositeInputAndEnv(ctx context.Context, parent *RunContext, step
env[envKey] = value
}
}
gh := step.getGithubContext(ctx)
env["GITHUB_ACTION_REPOSITORY"] = gh.ActionRepository
env["GITHUB_ACTION_REF"] = gh.ActionRef
return env, nil
}
@@ -110,6 +107,7 @@ func newCompositeRunContext(ctx context.Context, parent *RunContext, step action
kubernetes: parent.kubernetes,
jobIndex: parent.jobIndex,
jobTotal: parent.jobTotal,
runtimeToken: parent.runtimeToken,
}
compositerc.setCompositeActionEnv(env)
compositerc.ExprEval = compositerc.NewExpressionEvaluator(ctx)
+30 -1
View File
@@ -19,6 +19,35 @@ import (
)
func TestCompositeActionParity(t *testing.T) {
t.Run("input defaults use the step identity and docker env keeps the GITEA aliases", func(t *testing.T) {
for _, uses := range []string{"./action", "own/action@v2"} {
t.Run(uses, func(t *testing.T) {
rc := &RunContext{
Config: &Config{PresetGitHubContext: &model.GithubContext{}},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/composite", "GITHUB_ACTION_REF": "v1"},
}
var currentStep step = &stepActionLocal{RunContext: rc, Step: &model.Step{Uses: uses}, env: map[string]string{}}
want := "|"
if uses == "own/action@v2" {
currentStep = &stepActionRemote{RunContext: rc, Step: &model.Step{Uses: uses}, env: map[string]string{}, remoteAction: mustNewRemoteAction(t, uses)}
want = "own/action|v2"
}
action := &model.Action{
Inputs: map[string]model.Input{"identity": {Default: "${{ github.action_repository }}|${{ github.action_ref }}"}},
Runs: model.ActionRuns{Env: map[string]string{"GITHUB_WORKFLOW_SHA": "runs-sha"}},
}
require.NoError(t, setupEnv(t.Context(), currentStep))
require.NoError(t, populateEnvsFromInput(t.Context(), currentStep, action))
assert.Equal(t, want, (*currentStep.getEnv())["INPUT_IDENTITY"])
_, err := evalDockerEnv(t.Context(), currentStep, action)
require.NoError(t, err)
assert.Equal(t, want, (*currentStep.getEnv())["identity"])
assert.Equal(t, "runs-sha", (*currentStep.getEnv())["GITEA_WORKFLOW_SHA"])
})
}
})
t.Run("inherits contexts without leaking inputs", func(t *testing.T) {
hook := logtest.NewWithLevel(logging.LevelWarn)
ctx := common.WithLogger(t.Context(), logging.New(hook))
@@ -54,7 +83,7 @@ func TestCompositeActionParity(t *testing.T) {
assert.NotContains(t, composite.Env, "INPUT_EXTRA")
nestedEnv := composite.GetEnv()
require.NoError(t, populateEnvsFromInput(ctx, &nestedEnv, &model.Action{Inputs: map[string]model.Input{"shared": {Default: "inner-default"}}}, composite))
require.NoError(t, populateEnvsFromInput(ctx, &stepActionLocal{RunContext: composite, env: nestedEnv}, &model.Action{Inputs: map[string]model.Input{"shared": {Default: "inner-default"}}}))
assert.Equal(t, "inner-default", nestedEnv["INPUT_SHARED"])
})
+11 -6
View File
@@ -90,18 +90,18 @@ var hashfiles string
// NewStepExpressionEvaluator creates a new evaluator with the `inputs` of the enclosing workflow or composite action
func (rc *RunContext) NewStepExpressionEvaluator(ctx context.Context, step step) *ExpressionEvaluator {
return rc.newStepExpressionEvaluator(ctx, step, rc.actionInputs, rc.getJobContext())
return rc.newStepExpressionEvaluator(ctx, step, rc.actionInputs, rc.getJobContext(), getStepGithubContext(ctx, step))
}
// NewActionInputsExpressionEvaluator creates a new evaluator with the step's own with: values as `inputs`
func (rc *RunContext) NewActionInputsExpressionEvaluator(ctx context.Context, step step) *ExpressionEvaluator {
return rc.newStepExpressionEvaluator(ctx, step, inputsFromEnv(*step.getEnv()), rc.getJobContext())
return rc.newStepExpressionEvaluator(ctx, step, inputsFromEnv(*step.getEnv()), rc.getJobContext(), getStepGithubContext(ctx, step))
}
func (rc *RunContext) newStepExpressionEvaluator(ctx context.Context, step step, stepInputs map[string]any, jobContext *model.JobContext) *ExpressionEvaluator {
func (rc *RunContext) newStepExpressionEvaluator(ctx context.Context, step step, stepInputs map[string]any, jobContext *model.JobContext, github *model.GithubContext) *ExpressionEvaluator {
// todo: cleanup EvaluationEnvironment creation
ee := &exprparser.EvaluationEnvironment{
Github: step.getGithubContext(ctx),
Github: github,
Env: *step.getEnv(),
Job: jobContext,
Steps: rc.StepResults,
@@ -112,7 +112,7 @@ func (rc *RunContext) newStepExpressionEvaluator(ctx context.Context, step step,
Needs: exprparser.NeedsContext(rc.Run),
// todo: should be unavailable
// but required to interpolate/evaluate the inputs in actions/composite
Inputs: getEvaluatorInputs(rc, stepInputs, rc.getGithubContext(ctx)),
Inputs: getEvaluatorInputs(rc, stepInputs, github),
HashFiles: getHashFilesFunction(ctx, rc),
}
ee.Runner = rc.getRunnerContext(ctx)
@@ -332,5 +332,10 @@ func getWorkflowSecrets(rc *RunContext) map[string]string {
if rc.caller != nil {
return rc.workflowCallSecrets
}
return rc.Config.Secrets
if rc.runtimeToken == "" {
return rc.Config.Secrets
}
secrets := map[string]string{"ACTIONS_RUNTIME_TOKEN": rc.runtimeToken} // as the daemon exposes its token
maps.Copy(secrets, rc.Config.Secrets)
return secrets
}
+8 -3
View File
@@ -504,7 +504,7 @@ jobs:
assert.Equal(t, "workflow", env["INPUT_FALLBACK"])
}
func TestResolveWorkflowCallToleratesMismatchesAndPassesInheritedSecretsVerbatim(t *testing.T) {
func TestResolveWorkflowCallToleratesMismatchesAndPassesInheritedSecretsAndRuntimeToken(t *testing.T) {
callerWorkflow, err := model.ReadWorkflow(strings.NewReader(`
jobs:
call:
@@ -527,7 +527,11 @@ jobs:
require.NoError(t, err)
notCallable, err := model.ReadWorkflow(strings.NewReader("on: push\njobs:\n test: {}\n"))
require.NoError(t, err)
config := &Config{Secrets: map[string]string{"token": "${{ github.token }}"}, PresetGitHubContext: &model.GithubContext{}}
config := &Config{
Secrets: map[string]string{"token": "${{ github.token }}"},
PresetGitHubContext: &model.GithubContext{},
RuntimeTokens: map[*model.Workflow]string{callerWorkflow: "caller-run"},
}
resolve := func(callerJobID string, workflow *model.Workflow) *RunContext {
parent, err := (&runnerImpl{config: config}).newRunContext(t.Context(), &model.Run{Workflow: callerWorkflow, JobID: callerJobID}, nil)
require.NoError(t, err)
@@ -539,9 +543,10 @@ jobs:
called := resolve("call", callee)
assert.Equal(t, map[string]any{"flag": true}, called.workflowCallInputs)
assert.Equal(t, map[string]string{"undeclared": "${{ github.token }}"}, called.workflowCallSecrets)
assert.Equal(t, "caller-run", called.GetEnv()["ACTIONS_RUNTIME_TOKEN"])
inherited := resolve("inherit", notCallable)
assert.Empty(t, inherited.workflowCallInputs)
assert.Equal(t, config.Secrets, inherited.workflowCallSecrets)
assert.Equal(t, map[string]string{"token": "${{ github.token }}", "ACTIONS_RUNTIME_TOKEN": "caller-run"}, inherited.workflowCallSecrets)
}
func TestJobNameMasksSecrets(t *testing.T) {
+1 -1
View File
@@ -111,7 +111,7 @@ func printPrepareActions(rc *RunContext, preparers []actionPreparer) common.Exec
if !ok || strings.Contains(action.Step.Uses, "${{") || action.action != nil {
continue
}
github := action.getGithubContext(common.WithLogger(ctx, logging.New(logging.DiscardHandler)))
github := action.RunContext.getGithubContext(common.WithLogger(ctx, logging.New(logging.DiscardHandler)))
remote, err := newRemoteAction(action.Step.Uses, github, action.RunContext.Config.GitHubInstance)
if err != nil || action.skipsLocalCheckout(remote, github) {
continue
+13 -11
View File
@@ -1047,23 +1047,25 @@ func makeTestRC(t *testing.T, jobYAML string) *RunContext {
func TestApplyJobTimeout(t *testing.T) {
cases := []struct {
name string
yaml string
wantTimeout bool
name string
yaml string
want time.Duration
}{
{"empty", "runs-on: ubuntu-latest", false},
{"integer", "timeout-minutes: 5\nruns-on: ubuntu-latest", true},
{"zero ignored", "timeout-minutes: 0\nruns-on: ubuntu-latest", false},
{"non-numeric ignored", "timeout-minutes: abc\nruns-on: ubuntu-latest", false},
{"empty", "runs-on: ubuntu-latest", 0},
{"integer", "timeout-minutes: 5\nruns-on: ubuntu-latest", 5 * time.Minute},
{"zero ignored", "timeout-minutes: 0\nruns-on: ubuntu-latest", 0},
{"non-numeric ignored", "timeout-minutes: abc\nruns-on: ubuntu-latest", 0},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
rc := makeTestRC(t, tc.yaml)
ctx := t.Context()
newCtx, cancel := applyJobTimeout(ctx, rc, rc.Run.Job())
newCtx, cancel := applyJobTimeout(t.Context(), rc, rc.Run.Job())
defer cancel()
_, hasDeadline := newCtx.Deadline()
assert.Equal(t, tc.wantTimeout, hasDeadline)
deadline, hasDeadline := newCtx.Deadline()
assert.Equal(t, tc.want != 0, hasDeadline)
if hasDeadline {
assert.WithinDuration(t, time.Now().Add(tc.want), deadline, time.Minute)
}
})
}
}
+1 -2
View File
@@ -84,8 +84,7 @@ func (rc *RunContext) setupHookFileCommands(ctx context.Context, env map[string]
actPath := rc.JobContainer.GetActPath()
env["GITHUB_ENV"] = path.Join(actPath, hookEnvFileCommand)
env["GITHUB_PATH"] = path.Join(actPath, hookPathFileCommand)
env["GITEA_ENV"] = env["GITHUB_ENV"]
env["GITEA_PATH"] = env["GITHUB_PATH"]
aliasGiteaEnv(env)
return rc.JobContainer.Copy(actPath,
&container.FileEntry{Name: hookEnvFileCommand, Mode: 0o666},
+1
View File
@@ -121,6 +121,7 @@ func TestRunJobHook(t *testing.T) {
assert.Equal(t, "job", jobContainer.env["GITHUB_JOB"])
assert.Equal(t, "/var/run/act/workflow/hook-envs.txt", jobContainer.env["GITHUB_ENV"])
assert.Equal(t, "/var/run/act/workflow/hook-path.txt", jobContainer.env["GITHUB_PATH"])
assert.Equal(t, jobContainer.env["GITHUB_PATH"], jobContainer.env["GITEA_PATH"])
})
t.Run("each hook has its own command state", func(t *testing.T) {
+3 -2
View File
@@ -23,13 +23,14 @@ func newLocalReusableWorkflowExecutor(rc *RunContext) common.Executor {
return common.NewErrorExecutor(err)
}
if !rc.Config.NoSkipCheckout {
preset := rc.Config.PresetGitHubContext
if preset == nil {
// resolve the local workflow against the workspace root, not the process
// working directory, so it is found regardless of where the runner is invoked
return newReusableWorkflowExecutor(rc, rc.Config.Workdir, localWorkflow.Path, false)
}
uses := fmt.Sprintf("%s/%s@%s", rc.Config.PresetGitHubContext.Repository, localWorkflow.Path, rc.Config.PresetGitHubContext.Sha)
uses := fmt.Sprintf("%s/%s@%s", preset.Repository, localWorkflow.Path, preset.Sha)
remoteReusableWorkflow, err := newRemoteReusableWorkflow(rc.Config.GitHubInstance, uses)
if err != nil {
+1 -1
View File
@@ -122,7 +122,7 @@ func TestNewLocalReusableWorkflowExecutorFindsSameRepositoryPaths(t *testing.T)
for _, uses := range []string{"./.gitea/workflows/reusable.yml", "$/.gitea/workflows/reusable.yml"} {
rc := &RunContext{
Config: &Config{Workdir: workdir},
Config: &Config{Workdir: workdir, NoSkipCheckout: true},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {Uses: uses}}}},
}
err := newLocalReusableWorkflowExecutor(rc)(t.Context())
+23 -1
View File
@@ -81,6 +81,7 @@ type RunContext struct {
jobRunDefaults model.RunDefaults // defaults.run, resolved once at job setup as GitHub does
platformImage string // container.image or the runs-on pick, resolved once by isEnabled
jobIndex, jobTotal int // strategy.job-index and job-total, total 0 when unknown
runtimeToken string // ACTIONS_RUNTIME_TOKEN of the workflow run, shared with its called workflows and composite actions
jobEnv map[string]string // workflow and job env as evaluated for the container and service specs
dockerActionImages map[*model.Step]string // Dockerfile action images built for this job, removed when it ends
@@ -193,6 +194,9 @@ func (rc *RunContext) GetEnv() map[string]string {
rc.Env = mergeMaps(rc.Run.Workflow.Env, job.Environment(), rc.Config.Env)
}
}
if rc.runtimeToken != "" {
rc.Env["ACTIONS_RUNTIME_TOKEN"] = rc.runtimeToken // as the daemon's Config.Env carries its token
}
}
if !rc.Config.DisableActEnv {
rc.Env["ACT"] = "true"
@@ -1598,6 +1602,23 @@ func isLocalCheckout(ghc *model.GithubContext, step *model.Step) bool {
return true
}
// aliasGiteaEnv mirrors the runner's GITHUB_* variables as GITEA_*, call it after the last write to them.
func aliasGiteaEnv(env map[string]string) {
for _, suffix := range []string{
"ACTION", "ACTION_PATH", "ACTION_REF", "ACTION_REPOSITORY", "ACTIONS", "ACTOR", "ACTOR_ID", "API_URL", "BASE_REF",
"ENV", "EVENT_NAME", "EVENT_PATH", "GRAPHQL_URL", "HEAD_REF", "JOB", "OUTPUT", "PATH", "REF", "REF_NAME",
"REF_PROTECTED", "REF_TYPE", "REPOSITORY", "REPOSITORY_ID", "REPOSITORY_OWNER", "REPOSITORY_OWNER_ID",
"RETENTION_DAYS", "RUN_ATTEMPT", "RUN_ID", "RUN_NUMBER", "SERVER_URL", "SHA", "STATE", "STEP_SUMMARY",
"TRIGGERING_ACTOR", "WORKFLOW", "WORKFLOW_REF", "WORKFLOW_SHA", "WORKSPACE",
} {
if value, ok := env["GITHUB_"+suffix]; ok {
env["GITEA_"+suffix] = value
} else {
delete(env, "GITEA_"+suffix)
}
}
}
func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubContext, env map[string]string) {
if _, ok := env["CI"]; !ok {
env["CI"] = "true"
@@ -1650,6 +1671,7 @@ func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubCon
env["GITHUB_GRAPHQL_URL"] = ""
env["GITHUB_RUN_ATTEMPT"] = github.RunAttempt
aliasGiteaEnv(env)
env["RUNNER_NAME"] = rc.Config.RunnerName
env["RUNNER_ENVIRONMENT"] = "self-hosted"
@@ -1662,7 +1684,7 @@ func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubCon
if rc.Config.ArtifactServerPath != "" {
env["ACTIONS_RUNTIME_URL"] = cmp.Or(os.Getenv("ACTIONS_RUNTIME_URL"), "http://"+net.JoinHostPort(rc.Config.ArtifactServerAddr, rc.Config.ArtifactServerPort)+"/")
env["ACTIONS_RUNTIME_TOKEN"] = cmp.Or(os.Getenv("ACTIONS_RUNTIME_TOKEN"), "token")
env["ACTIONS_RUNTIME_TOKEN"] = cmp.Or(rc.runtimeToken, os.Getenv("ACTIONS_RUNTIME_TOKEN"), "token")
}
if imageOS := rc.imageOS(ctx); imageOS != "" {
+18 -14
View File
@@ -877,25 +877,25 @@ func TestGetGitHubContext(t *testing.T) {
RepositoryID: "34",
RepositoryOwnerID: "56",
WorkflowRef: "owner/repo/.gitea/workflows/test.yml@refs/heads/main",
WorkflowSha: "workflow-sha",
RefProtected: true,
}
ghc = rc.getGithubContext(t.Context())
assert.Equal(t, "preset-actor", ghc.Actor)
assert.Equal(t, "triggerer", ghc.TriggeringActor)
assert.Equal(t, "job1", ghc.Job)
env := map[string]string{}
env := map[string]string{"GITHUB_WORKFLOW_SHA": "workflow-sha"}
rc.withGithubEnv(t.Context(), ghc, env)
for name, value := range map[string]string{
"GITHUB_ACTOR_ID": "12",
"GITHUB_TRIGGERING_ACTOR": "triggerer",
"GITHUB_REPOSITORY_ID": "34",
"GITHUB_REPOSITORY_OWNER_ID": "56",
"GITHUB_WORKFLOW_REF": "owner/repo/.gitea/workflows/test.yml@refs/heads/main",
"GITHUB_WORKFLOW_SHA": "workflow-sha",
"GITHUB_REF_PROTECTED": "true",
for suffix, value := range map[string]string{
"ACTOR_ID": "12",
"TRIGGERING_ACTOR": "triggerer",
"REPOSITORY_ID": "34",
"REPOSITORY_OWNER_ID": "56",
"WORKFLOW_REF": "owner/repo/.gitea/workflows/test.yml@refs/heads/main",
"WORKFLOW_SHA": "workflow-sha",
"REF_PROTECTED": "true",
} {
assert.Equal(t, value, env[name], name)
assert.Equal(t, value, env["GITHUB_"+suffix], suffix)
assert.Equal(t, value, env["GITEA_"+suffix], suffix)
}
ghc.RefProtected = false
rc.withGithubEnv(t.Context(), ghc, env)
@@ -922,12 +922,12 @@ func TestGetGitHubContext(t *testing.T) {
github := rc.getGithubContext(t.Context())
assert.Equal(t, testcase.serverURL, github.ServerURL)
assert.Equal(t, testcase.apiURL, github.APIURL)
env := map[string]string{}
env := map[string]string{"GITEA_SERVER_URL": "stale", "GITEA_API_URL": "stale"}
rc.withGithubEnv(t.Context(), github, env)
_, exported := env["GITHUB_SERVER_URL"]
assert.Equal(t, testcase.serverURL != "", exported)
assert.Equal(t, testcase.serverURL, env["GITHUB_SERVER_URL"])
assert.Equal(t, testcase.apiURL, env["GITHUB_API_URL"])
assert.Equal(t, testcase.serverURL, env["GITEA_SERVER_URL"])
assert.Equal(t, testcase.apiURL, env["GITEA_API_URL"])
})
}
}
@@ -1631,4 +1631,8 @@ func TestRunContextWithGithubEnvRunnerValues(t *testing.T) {
rc.withGithubEnv(ctx, &model.GithubContext{}, env)
assert.Equal(t, ci, env["CI"])
}
rc.runtimeToken = "run-token"
newTestCompositeRunContext(t, rc).withGithubEnv(ctx, &model.GithubContext{}, env)
assert.Equal(t, "run-token", env["ACTIONS_RUNTIME_TOKEN"])
}
+6
View File
@@ -41,6 +41,7 @@ type Config struct {
Env map[string]string // env for containers
Secrets map[string]string // list of secrets
ExtraMasks []string // values to hide that are not in Secrets, such as the proxy password
RuntimeTokens map[*model.Workflow]string // ACTIONS_RUNTIME_TOKEN per workflow run, as each is its own artifact run
Vars map[string]string // list of vars
Token string // GitHub token
InsecureSecrets bool // switch hiding output when printing to terminal
@@ -355,6 +356,11 @@ func (runner *runnerImpl) newCombinationRunContext(ctx context.Context, run *mod
caller: runner.caller,
jobIndex: index,
jobTotal: total,
runtimeToken: runner.config.RuntimeTokens[run.Workflow],
}
if runner.caller != nil {
rc.runtimeToken = runner.caller.runContext.runtimeToken
}
if err := rc.resolveWorkflowCall(ctx); err != nil {
return nil, err
+18 -12
View File
@@ -27,12 +27,24 @@ type step interface {
post() common.Executor
getRunContext() *RunContext
getGithubContext(ctx context.Context) *model.GithubContext
getStepModel() *model.Step
getEnv() *map[string]string
getIfExpression(context context.Context, stage stepStage) string
}
func stepActionIdentity(step step) (repository, ref string) {
if action, ok := step.(interface{ actionIdentity() (string, string) }); ok {
return action.actionIdentity()
}
return "", ""
}
func getStepGithubContext(ctx context.Context, step step) *model.GithubContext {
github := step.getRunContext().getGithubContext(ctx)
github.ActionRepository, github.ActionRef = stepActionIdentity(step)
return github
}
type stepStage int
const (
@@ -92,6 +104,8 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
runStep, err = isStepEnabled(ctx, ifExpression, step, stage)
}
if err == nil && runStep {
env := *step.getEnv()
env["GITHUB_ACTION_REPOSITORY"], env["GITHUB_ACTION_REF"] = stepActionIdentity(step)
err = setupInputs(ctx, step)
}
if err != nil {
@@ -143,15 +157,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
stepSummaryIndex := topRC.CurrentStepIndex
summaryFileCommand := path.Join("workflow", "step-summary-"+strconv.Itoa(stepSummaryIndex)+".md")
(*step.getEnv())["GITHUB_STEP_SUMMARY"] = path.Join(actPath, summaryFileCommand)
{
// For Gitea
(*step.getEnv())["GITEA_OUTPUT"] = (*step.getEnv())["GITHUB_OUTPUT"]
(*step.getEnv())["GITEA_STATE"] = (*step.getEnv())["GITHUB_STATE"]
(*step.getEnv())["GITEA_PATH"] = (*step.getEnv())["GITHUB_PATH"]
(*step.getEnv())["GITEA_ENV"] = (*step.getEnv())["GITHUB_ENV"]
(*step.getEnv())["GITEA_STEP_SUMMARY"] = (*step.getEnv())["GITHUB_STEP_SUMMARY"]
}
aliasGiteaEnv(*step.getEnv())
// Reset the per-phase file-command files. GITHUB_STEP_SUMMARY is intentionally
// excluded here and initialized below at most once per step so writes from later
@@ -253,7 +259,7 @@ func setupEnv(ctx context.Context, step step) error {
env := step.getEnv()
// container env is the image's baseline, which job env and $GITHUB_ENV override
mergeIntoMap(step, env, rc.containerSpec.Env, rc.GetEnv())
rc.withGithubEnv(ctx, step.getGithubContext(ctx), *env)
rc.withGithubEnv(ctx, getStepGithubContext(ctx, step), *env)
var err error
exprEval := rc.NewExpressionEvaluator(ctx)
@@ -302,7 +308,7 @@ func isStepEnabled(ctx context.Context, expr string, step step, stage stepStage)
if rc.Parent != nil && stage == stepStageMain && jobContext.Status != "cancelled" {
jobContext.Status = rc.ownStatus()
}
runStep, err := EvalBool(ctx, rc.newStepExpressionEvaluator(ctx, step, rc.actionInputs, jobContext), expr, defaultStatusCheck)
runStep, err := EvalBool(ctx, rc.newStepExpressionEvaluator(ctx, step, rc.actionInputs, jobContext, rc.getGithubContext(ctx)), expr, defaultStatusCheck)
if err != nil {
return false, fmt.Errorf("if-expression %q evaluation failed: %s", expr, err)
}
+1 -5
View File
@@ -39,7 +39,7 @@ func (sab *stepActionBuiltin) main() common.Executor {
rc.ApplyExtraPath(ctx, &sab.env)
return sab.run(ctx, &action.Context{
Container: rc.JobContainer,
Github: rc.getGithubContext(ctx),
Github: getStepGithubContext(ctx, sab),
Inputs: sab.Step.With,
HostWorkdir: rc.Config.Workdir,
Workspace: rc.JobContainer.ToContainerPath(rc.Config.Workdir),
@@ -58,10 +58,6 @@ func (sab *stepActionBuiltin) post() common.Executor { return common.NewPipeline
func (sab *stepActionBuiltin) getRunContext() *RunContext { return sab.RunContext }
func (sab *stepActionBuiltin) getGithubContext(ctx context.Context) *model.GithubContext {
return sab.RunContext.getGithubContext(ctx)
}
func (sab *stepActionBuiltin) getStepModel() *model.Step { return sab.Step }
func (sab *stepActionBuiltin) getEnv() *map[string]string { return &sab.env }
-4
View File
@@ -95,10 +95,6 @@ func (sal *stepActionLocal) getRunContext() *RunContext {
return sal.RunContext
}
func (sal *stepActionLocal) getGithubContext(ctx context.Context) *model.GithubContext {
return sal.getRunContext().getGithubContext(ctx)
}
func (sal *stepActionLocal) getStepModel() *model.Step {
return sal.Step
}
+20 -2
View File
@@ -18,6 +18,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"go.yaml.in/yaml/v4"
)
type stepActionMocks struct {
@@ -53,7 +54,7 @@ func TestStepActionLocalTest(t *testing.T) {
readAction: salm.readAction,
runAction: salm.runAction,
RunContext: &RunContext{
StepResults: map[string]*model.StepResult{},
StepResults: map[string]*model.StepResult{"previous": {Outputs: map[string]string{"value": "previous"}}},
ExprEval: &expressionEvaluator{},
Config: &Config{
Workdir: "/tmp",
@@ -68,25 +69,42 @@ func TestStepActionLocalTest(t *testing.T) {
},
},
JobContainer: cm,
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/action", "GITHUB_ACTION_REF": "v1"},
Matrix: map[string]any{"value": "matrix"},
actionInputs: map[string]any{"shared": "input"},
},
Step: &model.Step{
ID: "1",
Uses: "./path/to/action",
If: yaml.Node{Value: "github.action_repository == 'other/action' && github.action_ref == 'v1' && " +
"env.STEP_VALUE == 'step' && env.GITHUB_ACTION_REPOSITORY == 'forged/repo' && env.GITHUB_ACTION_REF == 'forged-ref' && " +
"steps.previous.outputs.value == 'previous' && inputs.shared == 'input' && matrix.value == 'matrix'"},
},
}
require.NoError(t, sal.Step.Env.Encode(map[string]string{"STEP_VALUE": "step", "GITHUB_ACTION_REPOSITORY": "forged/repo", "GITHUB_ACTION_REF": "forged-ref"}))
require.NoError(t, sal.Step.RawWith.Encode(map[string]string{"identity": "${{ github.action_repository }}|${{ github.action_ref }}"}))
salm.On("readAction", sal.Step, filepath.Clean("/tmp/path/to/action"), "", mock.Anything, mock.Anything).
Return(&model.Action{}, nil)
expectStepFileCommands(t, cm)
salm.On("runAction", sal, filepath.Clean("/tmp/path/to/action"), (*remoteAction)(nil)).Return(noopExecutor)
salm.On("runAction", sal, filepath.Clean("/tmp/path/to/action"), (*remoteAction)(nil)).Return(func(ctx context.Context) error {
sal.RunContext.setOutput(ctx, map[string]string{"name": "value"}, "ran")
return nil
})
err := sal.pre()(ctx)
require.NoError(t, err)
err = sal.main()(ctx)
require.NoError(t, err)
assert.Empty(t, sal.env["GITHUB_ACTION_REPOSITORY"])
assert.Empty(t, sal.env["GITHUB_ACTION_REF"])
assert.Equal(t, "|", sal.Step.With["identity"])
assert.Equal(t, "ran", sal.RunContext.StepResults["1"].Outputs["value"])
assert.Equal(t, "other/action", sal.RunContext.Env["GITHUB_ACTION_REPOSITORY"])
assert.Equal(t, "v1", sal.RunContext.Env["GITHUB_ACTION_REF"])
cm.AssertExpectations(t)
salm.AssertExpectations(t)
+7 -13
View File
@@ -75,7 +75,7 @@ func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
}
sar.Step.Uses = uses
github := sar.getGithubContext(ctx) // read before remoteAction is set, so `$/` resolves against the enclosing action
github := sar.RunContext.getGithubContext(ctx)
if sar.remoteAction, err = newRemoteAction(sar.Step.Uses, github, sar.RunContext.Config.GitHubInstance); err != nil {
return err
}
@@ -153,7 +153,7 @@ func (sar *stepActionRemote) main() common.Executor {
return common.NewPipelineExecutor(
sar.prepareActionExecutor(),
runStepExecutor(sar, stepStageMain, func(ctx context.Context) error {
github := sar.getGithubContext(ctx)
github := getStepGithubContext(ctx, sar)
if sar.skipsLocalCheckout(sar.remoteAction, github) {
printRunActionHeader(ctx, sar.Step, nil, sar.env, sar.RunContext)
if sar.RunContext.Config.BindWorkdir {
@@ -181,17 +181,11 @@ func (sar *stepActionRemote) getRunContext() *RunContext {
return sar.RunContext
}
func (sar *stepActionRemote) getGithubContext(ctx context.Context) *model.GithubContext {
ghc := sar.getRunContext().getGithubContext(ctx)
// extend github context if we already have an initialized remoteAction
remoteAction := sar.remoteAction
if remoteAction != nil {
ghc.ActionRepository = fmt.Sprintf("%s/%s", remoteAction.Org, remoteAction.Repo)
ghc.ActionRef = remoteAction.Ref
func (sar *stepActionRemote) actionIdentity() (string, string) {
if sar.remoteAction == nil {
return "", ""
}
return ghc
return sar.remoteAction.Org + "/" + sar.remoteAction.Repo, sar.remoteAction.Ref
}
func (sar *stepActionRemote) getStepModel() *model.Step {
@@ -205,7 +199,7 @@ func (sar *stepActionRemote) getEnv() *map[string]string {
func (sar *stepActionRemote) getIfExpression(ctx context.Context, stage stepStage) string {
switch stage {
case stepStagePre:
github := sar.getGithubContext(ctx)
github := getStepGithubContext(ctx, sar)
if sar.skipsLocalCheckout(sar.remoteAction, github) {
// skip local checkout pre step
return "false"
+5 -5
View File
@@ -76,7 +76,7 @@ func TestStepActionRemote(t *testing.T) {
status model.StepStatus
runError error
}{
{name: "run-successful", status: model.StepStatusSuccess},
{name: "run-successful", condition: "github.action_repository == 'other/composite' && github.action_ref == 'v2'", status: model.StepStatusSuccess},
{name: "run-skipped", condition: "false", status: model.StepStatusSkipped},
{name: "run-error", status: model.StepStatusFailure, runError: assert.AnError},
} {
@@ -110,6 +110,7 @@ func TestStepActionRemote(t *testing.T) {
},
StepResults: map[string]*model.StepResult{},
JobContainer: cm,
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/composite", "GITHUB_ACTION_REF": "v2"},
},
Step: &model.Step{ID: "step", Uses: "remote/action@v1", If: yaml.Node{Value: tt.condition}},
readAction: sarm.readAction,
@@ -127,6 +128,8 @@ func TestStepActionRemote(t *testing.T) {
require.NoError(t, sar.pre()(ctx))
assert.Equal(t, tt.runError, sar.main()(ctx))
assert.True(t, clonedAction)
assert.Equal(t, "remote/action", sar.env["GITHUB_ACTION_REPOSITORY"])
assert.Equal(t, "v1", sar.env["GITHUB_ACTION_REF"])
assert.Equal(t, &model.StepResult{Conclusion: tt.status, Outcome: tt.status, Outputs: map[string]string{}}, sar.RunContext.StepResults["step"])
sarm.AssertExpectations(t)
@@ -182,7 +185,7 @@ func TestStepActionRemotePrepare(t *testing.T) {
require.NoError(t, err)
root, err := prepare(job, "org/repo@v1")
require.NoError(t, err)
nested, err := prepare(&RunContext{Config: job.Config, Run: job.Run, JobContainer: job.JobContainer, Parent: job}, "org/repo/save@v1")
nested, err := prepare(&RunContext{Config: job.Config, Run: job.Run, JobContainer: job.JobContainer, Parent: job, Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "org/repo", "GITHUB_ACTION_REF": "v1"}}, "$/save")
require.NoError(t, err)
other, err := prepare(job, "org/repo@v2")
require.NoError(t, err)
@@ -346,9 +349,6 @@ func Test_newRemoteActionSelfRepo(t *testing.T) {
},
},
{name: "unknown instance", action: "$/.gitea/actions/build", github: &model.GithubContext{Repository: "owner/workflow-repo", Sha: "abc123"}},
{name: "empty path", action: "$/", github: workflow},
{name: "ref suffix is not allowed", action: "$/.gitea/actions/build@v1", github: workflow},
{name: "path traversal", action: "$/../escape", github: workflow},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
-4
View File
@@ -35,10 +35,6 @@ func (sd *stepDocker) getRunContext() *RunContext {
return sd.RunContext
}
func (sd *stepDocker) getGithubContext(ctx context.Context) *model.GithubContext {
return sd.getRunContext().getGithubContext(ctx)
}
func (sd *stepDocker) getStepModel() *model.Step {
return sd.Step
}
+9
View File
@@ -14,6 +14,7 @@ import (
"gitea.dev/actionslib/pkg/model"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.yaml.in/yaml/v4"
)
func TestStepDockerMain(t *testing.T) {
@@ -57,13 +58,16 @@ func TestStepDockerMain(t *testing.T) {
},
},
JobContainer: cm,
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/composite", "GITHUB_ACTION_REF": "v1"},
},
Step: &model.Step{
ID: "1",
Uses: "docker://node:14",
WorkingDirectory: "workdir",
If: yaml.Node{Value: "github.action_repository == 'other/composite' && github.action_ref == 'v1'"},
},
}
require.NoError(t, sd.Step.RawWith.Encode(map[string]string{"args": "identity:${{ github.action_repository }}:${{ github.action_ref }}"}))
sd.RunContext.ExprEval = sd.RunContext.NewExpressionEvaluator(ctx)
cm.On("Pull", false).Return(noopExecutor)
@@ -80,6 +84,11 @@ func TestStepDockerMain(t *testing.T) {
require.NoError(t, sd.main()(ctx))
assert.Equal(t, "node:14", input.Image)
assert.Equal(t, []string{"identity::"}, input.Cmd)
assert.Contains(t, input.Env, "GITHUB_ACTION_REPOSITORY=")
assert.Contains(t, input.Env, "GITHUB_ACTION_REF=")
assert.Equal(t, "other/composite", sd.RunContext.Env["GITHUB_ACTION_REPOSITORY"])
assert.Equal(t, "v1", sd.RunContext.Env["GITHUB_ACTION_REF"])
assert.Empty(t, input.Username)
assert.Empty(t, input.Password)
-4
View File
@@ -222,10 +222,6 @@ func (sr *stepRun) getRunContext() *RunContext {
return sr.RunContext
}
func (sr *stepRun) getGithubContext(ctx context.Context) *model.GithubContext {
return sr.getRunContext().getGithubContext(ctx)
}
func (sr *stepRun) getStepModel() *model.Step {
return sr.Step
}
+16 -2
View File
@@ -19,6 +19,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"go.yaml.in/yaml/v4"
)
type shellContainerMock struct{ *containerMock }
@@ -36,7 +37,6 @@ func TestStepRun(t *testing.T) {
sr := &stepRun{
RunContext: &RunContext{
StepResults: map[string]*model.StepResult{},
ExprEval: &expressionEvaluator{},
Config: &Config{Env: testCheckoutEnv()},
Run: &model.Run{
JobID: "1",
@@ -48,19 +48,33 @@ func TestStepRun(t *testing.T) {
},
JobContainer: cm,
jobRunDefaults: model.RunDefaults{Shell: "bash"},
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/composite", "GITHUB_ACTION_REF": "v1"},
},
Step: &model.Step{
ID: "1",
Run: "cmd",
Run: "cmd${{ github.action_repository }}${{ github.action_ref }}",
WorkingDirectory: "workdir",
If: yaml.Node{Value: "github.action_repository == 'other/composite' && github.action_ref == 'v1'"},
},
}
sr.RunContext.ExprEval = sr.RunContext.NewExpressionEvaluator(t.Context())
require.NoError(t, sr.Step.Env.Encode(map[string]string{
"GITHUB_ACTION_REPOSITORY": "forged/repo", "GITHUB_ACTION_REF": "forged-ref",
"GITEA_ACTION_REPOSITORY": "forged/repo", "GITEA_ACTION_REF": "forged-ref",
}))
cm.On("Copy", "/var/run/act", []*container.FileEntry{fileEntry}).Return(noopExecutor)
cm.On("Exec", []string{"bash", "--noprofile", "--norc", "-e", "-o", "pipefail", "/var/run/act/workflow/1.sh"}, mock.AnythingOfType("map[string]string"), "", "workdir").Return(noopExecutor)
expectStepFileCommands(t, cm)
require.NoError(t, sr.main()(t.Context()))
assert.Empty(t, sr.env["GITHUB_ACTION_REPOSITORY"])
assert.Empty(t, sr.env["GITHUB_ACTION_REF"])
assert.Empty(t, sr.env["GITEA_ACTION_REPOSITORY"])
assert.Empty(t, sr.env["GITEA_ACTION_REF"])
assert.Equal(t, sr.env["GITHUB_STEP_SUMMARY"], sr.env["GITEA_STEP_SUMMARY"])
assert.Equal(t, "other/composite", sr.RunContext.Env["GITHUB_ACTION_REPOSITORY"])
assert.Equal(t, "v1", sr.RunContext.Env["GITHUB_ACTION_REF"])
cm.AssertExpectations(t)
}
+12 -10
View File
@@ -155,18 +155,14 @@ func TestSetupEnv(t *testing.T) {
require.NoError(t, setupEnv(t.Context(), testStep))
require.NoError(t, setupInputs(t.Context(), testStep))
delete(env, "GITHUB_REF")
delete(env, "GITHUB_REF_NAME")
delete(env, "GITHUB_REF_TYPE")
delete(env, "GITHUB_SHA")
delete(env, "GITHUB_WORKSPACE")
delete(env, "GITHUB_REPOSITORY")
delete(env, "GITHUB_REPOSITORY_OWNER")
delete(env, "GITHUB_ACTOR")
for _, suffix := range []string{"REF", "REF_NAME", "REF_TYPE", "SHA", "WORKSPACE", "REPOSITORY", "REPOSITORY_OWNER", "ACTOR"} {
delete(env, "GITHUB_"+suffix)
delete(env, "GITEA_"+suffix)
}
delete(env, "RUNNER_NAME")
delete(env, "RUNNER_WORKSPACE")
assert.Equal(t, map[string]string{
expected := map[string]string{
"ACT": "true",
"ACT_SKIP_CHECKOUT": "true",
"CI": "true",
@@ -193,7 +189,13 @@ func TestSetupEnv(t *testing.T) {
"RUNNER_ENVIRONMENT": "self-hosted",
"RUNNER_PERFLOG": "/dev/null",
"RUNNER_TRACKING_ID": "",
}, env)
}
for name, value := range expected {
if suffix, ok := strings.CutPrefix(name, "GITHUB_"); ok {
expected["GITEA_"+suffix] = value
}
}
assert.Equal(t, expected, env)
for _, expression := range []string{"inputs.args", "env.ARGS"} {
t.Run("deferred inputs from "+expression, func(t *testing.T) {
+1 -1
View File
@@ -1,5 +1,5 @@
# Container image that runs your code
FROM node:24-bookworm-slim
FROM node:26-bookworm-slim
# Copies your code file from your action repository to the filesystem path `/` of the container
COPY entrypoint.sh /entrypoint.sh
+1 -1
View File
@@ -1,5 +1,5 @@
# Container image that runs your code
FROM node:24-bookworm-slim
FROM node:26-bookworm-slim
# Copies your code file from your action repository to the filesystem path `/` of the container
COPY entrypoint.sh /entrypoint.sh
+1 -1
View File
@@ -8,7 +8,7 @@ inputs:
default: World
runs:
using: docker
image: docker://node:24-bookworm-slim
image: docker://node:26-bookworm-slim
entrypoint: /bin/sh -c
env:
TEST: enabled
+1 -1
View File
@@ -45,7 +45,7 @@ A hook path that does not exist inside the job environment fails the job with `N
## Environment
A hook sees the job's environment: the workflow, job and `container:` `env:`, the runner's `envs`, and the `GITHUB_*` context variables, with the same masking applied to its output as to a step's. The step-specific ones (`GITHUB_ACTION`, `GITHUB_OUTPUT`, `GITHUB_STATE`) are not set — a hook is not a step, so `::save-state::` and `::set-output::` have nowhere to go.
A hook sees the job's environment: the workflow, job and `container:` `env:`, the runner's `envs`, and the `GITHUB_*`/`GITEA_*` context variables, with the same masking applied to its output as to a step's. The step-specific ones (`GITHUB_ACTION`, `GITHUB_OUTPUT`, `GITHUB_STATE`) are not set — a hook is not a step, so `::save-state::` and `::set-output::` have nowhere to go.
Its stdout is part of the job log, after a collapsible group naming the hook and its shell, and is scanned for workflow commands. `::add-mask::` registers a value to be masked for the rest of the job. `::set-env::` and `::add-path::` apply to the steps that follow, but as in a step they need `ACTIONS_ALLOW_UNSECURE_COMMANDS: true`, and a refused one fails the hook.
+54
View File
@@ -49,6 +49,60 @@ A job with `runs-on: [linux, gpu]` gets the `linux` and then the `gpu` entry of
Job pods mount no service account token unless the template sets `automountServiceAccountToken: true`. Job pods and their Secrets are labeled `app.kubernetes.io/managed-by: gitea-runner` and `com.gitea.runner.uuid: <runner UUID>`. The idle cleanup removes those the runner did not get to remove, once they are older than `runner.workdir_cleanup_age`.
## Copying files over the pod network
The runner copies files into the job container, such as the actions a step uses, with `tar` over `pods/exec`. That data goes through the API server, which on a managed cluster often sits in other zones than the nodes, so each byte can cross zones twice. With `transfer.mode` set to `auto` or `direct`, the job container instead fetches directories, such as actions and workspaces, and other copies of 64 KiB or more gzipped from the runner pod. Smaller ones, like step scripts, stay on `pods/exec`, as do copies out of the job container and step output.
```yaml
kubernetes:
transfer:
mode: direct # api (default), auto or direct
host: "" # address the runner listens on and job pods reach it at, defaults to its outbound IP, the pod IP in a cluster
port: 0 # port it listens on, defaults to a free one
image: "" # image with gitea-runner on its PATH, defaults to the image of a single-container runner pod in kubernetes.namespace
```
Each job pod then gets an init container named `gitea-runner-transfer` from `image`, which copies its `gitea-runner` into the `act` volume for the job container to fetch with over HTTPS. The modes differ in what happens when a job pod cannot reach the runner, for example because of a NetworkPolicy, or when no `image` is set and the runner is not a single-container pod in `kubernetes.namespace`:
| Mode | Copies | When the pod network does not work |
|---|---|---|
| `api` | through the API server | not used |
| `auto` | over the pod network | the job pod gets its copies through the API server, after one failed attempt, and a warning is logged |
| `direct` | over the pod network | the job fails, so a broken setup shows instead of quietly costing API server traffic again |
In both `auto` and `direct`, a job fails when its init container does, for example when `image` cannot be pulled or has no `gitea-runner` on its PATH. A private `image` needs `imagePullSecrets` in `pod_template`, and a versioned tag keeps job pods on the runner's version. An `initContainers` entry named `gitea-runner-transfer` in `pod_template` merges into the init container, for example to set its `resources`.
The copies include secrets, such as `run` scripts with `${{ secrets.* }}` expanded, SSH keys for checkout and the runner's client key, so a runner only serves them to its own job pods:
- The runner listens with a certificate it generates at start. Its SHA-256 pin reaches the job pod over the `pods/exec` stream, which the API server protects with TLS, and the job pod trusts no other certificate, so nobody on the pod network can read or change a copy.
- Every copy has a one-time token, sent on that stream's stdin, so it shows neither in the command line nor in the API server audit log.
- The runner serves a copy only to a connection from an IP of the job pod it is for. Other pods get a 404 and leave the copy for its job pod. This needs the runner to see the job pods' own addresses, so it does not work for a runner outside the cluster or behind a proxy that rewrites source addresses.
To also keep other pods from connecting at all, allow the port only from the runner's own job pods, which carry its UUID in the `com.gitea.runner.uuid` label. The policy denies every other connection to the runner pod, so it also needs to allow `cache.port`, and both ports need fixed values:
```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: gitea-runner-transfer
spec:
podSelector:
matchLabels:
app: runner # the runner pod
policyTypes: [Ingress]
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: <kubernetes.namespace>
podSelector:
matchLabels:
com.gitea.runner.uuid: <runner UUID, from the .runner file>
ports:
- port: 9090 # kubernetes.transfer.port
- port: 9091 # cache.port
```
## Services
Services share the pod's network, so a step reaches one by its id or `localhost`, and two services cannot listen on the same port. An id that is not a valid hostname is sanitized, so `my_db` is reached as `my-db`. Pass secrets to services through `env`, as `command` is visible in the pod spec. Of `options`, only `--health-*` is supported. They become a startup probe with Docker's defaults, which stops a service that fails its retries, so the job fails as on Docker.
+11 -10
View File
@@ -28,7 +28,7 @@ func testRunCancellation(t *testing.T) {
pushWorkflow(t, api, repo, "cancel.yml")
wfRun := waitForRun(t, api, repo)
waitForRunningJobLog(t, api, repo, wfRun.ID, "e2e-live-log-marker")
waitForRunningJobLog(t, api, repo, wfRun.ID, "e2e-live-log-marker", 1)
if !cancelSupported {
requireSuccess(t, api, repo, wfRun.ID)
return
@@ -49,31 +49,32 @@ func testRunCancellation(t *testing.T) {
}
}
func waitForRunningJobLog(t *testing.T, api *GiteaAPI, repo string, runID int64, substr string) {
func waitForRunningJobLog(t *testing.T, api *GiteaAPI, repo string, runID int64, prefix string, count int) {
t.Helper()
// The budget covers scheduling and the container start, which a loaded runner makes slow.
ctx, cancel := context.WithTimeout(t.Context(), runTimeout)
defer cancel()
status := "none"
for {
jobs, err := api.Jobs(ctx, repo, runID)
if err != nil {
t.Fatalf("list jobs: %v", err)
}
if len(jobs) > 0 {
status = jobs[0].Status
if status == "in_progress" {
logs, err := api.JobLogs(ctx, repo, jobs[0].ID)
if err == nil && commandRow(logs, substr) == substr {
return
logged := 0
for _, job := range jobs {
if job.Status == "in_progress" {
if logs, err := api.JobLogs(ctx, repo, job.ID); err == nil && commandRow(logs, prefix) != "" {
logged++
}
}
}
if logged >= count {
return
}
select {
case <-ctx.Done():
dumpRunLogs(t, api, repo, runID)
t.Fatalf("job of run %d is %q and never logged %q", runID, status, substr)
t.Fatalf("%d of %d running jobs of run %d logged %q", logged, count, runID, prefix)
case <-time.After(pollInterval):
}
}
+321
View File
@@ -0,0 +1,321 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build e2e
package e2e
import (
"bytes"
"fmt"
"os"
"os/exec"
"path/filepath"
"slices"
"strings"
"sync"
"syscall"
"testing"
"time"
)
type syncBuffer struct {
mu sync.Mutex
buf bytes.Buffer
}
func (b *syncBuffer) Write(p []byte) (int, error) {
b.mu.Lock()
defer b.mu.Unlock()
return b.buf.Write(p)
}
func (b *syncBuffer) String() string {
b.mu.Lock()
defer b.mu.Unlock()
return b.buf.String()
}
type cliRunner struct {
api *GiteaAPI
repo string
bin string
cfgFile string
dir string
}
// newCLIRunner registers the built binary against a fresh repo with a single docker label.
func newCLIRunner(t *testing.T, label, extraRunnerConfig string) *cliRunner {
t.Helper()
api, repo := newScenario(t)
token, err := fixture.RegistrationToken(t.Context(), repo)
if err != nil {
t.Fatalf("get registration token: %v", err)
}
dir := t.TempDir()
bin, err := runnerBinary()
if err != nil {
t.Fatal(err)
}
cfg := fmt.Sprintf(`runner:
file: %s
fetch_interval: 250ms
fetch_interval_max: 250ms
state_report_interval: 500ms
log_report_interval: 500ms
%scache:
dir: %s
container:
network: %s
`, filepath.Join(dir, ".runner"), extraRunnerConfig, filepath.Join(dir, "cache"), fixture.network)
c := &cliRunner{api: api, repo: repo, bin: bin, cfgFile: filepath.Join(dir, "config.yaml"), dir: dir}
if err := os.WriteFile(c.cfgFile, []byte(cfg), 0o600); err != nil {
t.Fatalf("write config: %v", err)
}
out, err := exec.CommandContext(t.Context(), c.bin, "register", "--no-interactive", "-c", c.cfgFile,
"--instance", fixture.baseURL, "--token", token, "--name", repo,
"--labels", label+":docker://"+os.Getenv("E2E_JOB_IMAGE")).CombinedOutput()
if err != nil {
t.Fatalf("register: %v\n%s", err, out)
}
if _, err := os.Stat(filepath.Join(dir, ".runner")); err != nil {
t.Fatalf("register did not write the runner file: %v\n%s", err, out)
}
return c
}
type cliDaemon struct {
cmd *exec.Cmd
log *syncBuffer
exited chan error
}
func (c *cliRunner) startDaemon(t *testing.T, args ...string) *cliDaemon {
t.Helper()
d := &cliDaemon{log: &syncBuffer{}, exited: make(chan error, 1)}
d.cmd = exec.Command(c.bin, append([]string{"daemon", "-c", c.cfgFile}, args...)...)
d.cmd.Stdout, d.cmd.Stderr = d.log, d.log
if err := d.cmd.Start(); err != nil {
t.Fatalf("start daemon: %v", err)
}
go func() { d.exited <- d.cmd.Wait() }()
t.Cleanup(func() {
_ = d.cmd.Process.Kill() // no-op once the daemon has exited
})
return d
}
// wait returns the daemon's exit error, failing the test when it is still running after timeout.
func (d *cliDaemon) wait(t *testing.T, timeout time.Duration) error {
t.Helper()
select {
case err := <-d.exited:
return err
case <-time.After(timeout):
t.Fatalf("daemon still running after %s\n%s", timeout, d.log)
return nil
}
}
func (d *cliDaemon) terminate(t *testing.T) {
t.Helper()
if err := d.cmd.Process.Signal(syscall.SIGTERM); err != nil {
t.Fatalf("signal daemon: %v", err)
}
}
// requireSuccess waits for the run and dumps the daemon output next to the job logs on failure.
func (d *cliDaemon) requireSuccess(t *testing.T, c *cliRunner, runID int64) {
t.Helper()
completed, err := c.api.WaitForRunConclusion(t.Context(), c.repo, runID, runTimeout)
if err != nil || completed.Conclusion != "success" {
t.Logf("daemon output:\n%s", d.log)
requireSuccess(t, c.api, c.repo, runID)
}
}
// testCLIRegisterDaemon drives the built binary the way a user does: register, then daemon.
func testCLIRegisterDaemon(t *testing.T) {
t.Parallel()
c := newCLIRunner(t, "e2e-cli", "")
d := c.startDaemon(t)
pushWorkflow(t, c.api, c.repo, "cli.yml")
wfRun := waitForRun(t, c.api, c.repo)
d.requireSuccess(t, c, wfRun.ID)
if logs := runLogs(t, c.api, c.repo, wfRun.ID); commandRow(logs, "e2e-cli-job-ran") == "" {
t.Errorf("job logs are missing the marker:\n%s", logs)
}
d.terminate(t)
if err := d.wait(t, 30*time.Second); err != nil {
t.Errorf("daemon did not exit cleanly on SIGTERM: %v\n%s", err, d.log)
}
}
// testCLIOnce checks that --once takes a single job and then exits on its own.
func testCLIOnce(t *testing.T) {
t.Parallel()
c := newCLIRunner(t, "e2e-once", "")
d := c.startDaemon(t, "--once")
pushWorkflow(t, c.api, c.repo, "cli_once.yml")
wfRun := waitForRun(t, c.api, c.repo)
if err := d.wait(t, runTimeout); err != nil {
t.Errorf("daemon --once exited with %v\n%s", err, d.log)
}
jobs, err := c.api.Jobs(t.Context(), c.repo, wfRun.ID)
if err != nil {
t.Fatalf("list jobs: %v", err)
}
var succeeded, pending int
for _, job := range jobs {
switch {
case job.Conclusion == "success":
succeeded++
case job.Status != "in_progress" && job.Status != "completed":
pending++
}
}
if succeeded != 1 || pending != 1 {
t.Errorf("got %d succeeded and %d waiting jobs, want 1 and 1: %+v\n%s", succeeded, pending, jobs, d.log)
}
}
// testCLILabelsOverride checks that daemon --labels replaces the labels given at registration.
func testCLILabelsOverride(t *testing.T) {
t.Parallel()
c := newCLIRunner(t, "e2e-registered", "")
d := c.startDaemon(t, "--labels", "e2e-override:docker://"+os.Getenv("E2E_JOB_IMAGE"))
pushWorkflow(t, c.api, c.repo, "cli_labels.yml")
wfRun := waitForRun(t, c.api, c.repo)
d.requireSuccess(t, c, wfRun.ID)
runners, err := c.api.Runners(t.Context(), c.repo)
if err != nil {
t.Fatalf("list runners: %v", err)
}
for _, runner := range runners {
if runner.Name != c.repo {
continue
}
var labels []string
for _, label := range runner.Labels {
labels = append(labels, label.Name)
}
if !slices.Equal(labels, []string{"e2e-override"}) {
t.Errorf("runner has labels %v on the server, want only the --labels override", labels)
}
return
}
t.Errorf("runner %q not found among %+v", c.repo, runners)
}
// testCLIShutdownDrains checks that SIGTERM lets a running job finish within shutdown_timeout.
func testCLIShutdownDrains(t *testing.T) {
t.Parallel()
c := newCLIRunner(t, "e2e-shutdown", " shutdown_timeout: 1m\n")
gate, release := newGate(t)
if err := c.api.CreateVariable(t.Context(), c.repo, "E2E_GATE", gate); err != nil {
t.Fatalf("create gate variable: %v", err)
}
d := c.startDaemon(t)
pushWorkflow(t, c.api, c.repo, "cli_shutdown.yml")
wfRun := waitForRun(t, c.api, c.repo)
waitForRunningJobLog(t, c.api, c.repo, wfRun.ID, "e2e-shutdown-started", 1)
d.terminate(t)
deadline := time.After(time.Minute)
for !strings.Contains(d.log.String(), "shutdown initiated") {
select {
case <-deadline:
t.Fatalf("daemon never started shutting down\n%s", d.log)
case <-time.After(pollInterval):
}
}
release()
if err := d.wait(t, runTimeout); err != nil {
t.Errorf("daemon did not exit cleanly: %v\n%s", err, d.log)
}
if strings.Contains(d.log.String(), "cancelled in progress jobs") {
t.Errorf("the running job was cancelled instead of drained\n%s", d.log)
}
d.requireSuccess(t, c, wfRun.ID)
if logs := runLogs(t, c.api, c.repo, wfRun.ID); commandRow(logs, "e2e-shutdown-finished") == "" {
t.Errorf("job did not run to the end:\n%s", logs)
}
}
// testCLIShutdownCancels checks that a job still running at shutdown_timeout is cancelled.
func testCLIShutdownCancels(t *testing.T) {
t.Parallel()
c := newCLIRunner(t, "e2e-shutdown", " shutdown_timeout: 1s\n")
d := c.startDaemon(t)
pushWorkflow(t, c.api, c.repo, "cli_shutdown.yml")
wfRun := waitForRun(t, c.api, c.repo)
waitForRunningJobLog(t, c.api, c.repo, wfRun.ID, "e2e-shutdown-started", 1)
d.terminate(t)
if err := d.wait(t, runTimeout); err != nil {
t.Errorf("daemon did not exit cleanly after cancelling the job: %v\n%s", err, d.log)
}
if !strings.Contains(d.log.String(), "cancelled in progress jobs") {
t.Errorf("daemon did not report cancelling the running job\n%s", d.log)
}
completed, err := c.api.WaitForRunConclusion(t.Context(), c.repo, wfRun.ID, time.Minute)
if err != nil {
t.Fatalf("cancelled run was never finalized: %v", err)
}
if completed.Conclusion == "success" {
t.Errorf("run concluded success although the job was cancelled")
}
}
func testExecArtifacts(t *testing.T) {
t.Parallel()
ctx := t.Context()
bin, err := runnerBinary()
if err != nil {
t.Fatal(err)
}
workflow, err := os.ReadFile("testdata/workflows/exec_artifact.yml")
if err != nil {
t.Fatal(err)
}
workflows := t.TempDir()
for _, name := range []string{"a.yml", "b.yml"} {
if err := os.WriteFile(filepath.Join(workflows, name), workflow, 0o644); err != nil {
t.Fatal(err)
}
}
workdir := t.TempDir()
for _, args := range [][]string{
{"init", "-q"},
{"remote", "add", "origin", "https://example.com/e2e/exec.git"},
{"-c", "user.name=e2e", "-c", "[email protected]", "commit", "-q", "--allow-empty", "-m", "init"},
} {
if out, err := exec.CommandContext(ctx, "git", append([]string{"-C", workdir}, args...)...).CombinedOutput(); err != nil {
t.Fatalf("git %v: %v\n%s", args, err, out)
}
}
artifacts := t.TempDir()
args := []string{"exec", "-C", workdir, "-W", workflows, "-i", os.Getenv("E2E_JOB_IMAGE"), "--artifact-server-path", artifacts, "--network", fixture.network}
cmd := exec.CommandContext(ctx, bin, args...)
cmd.Env = slices.DeleteFunc(os.Environ(), func(kv string) bool { return strings.HasPrefix(kv, "ACTIONS_") })
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("exec: %v\n%s", err, out)
}
if blobs, _ := filepath.Glob(filepath.Join(artifacts, "v4", "*", "*", "blob")); len(blobs) != 2 {
t.Fatalf("want one v4 artifact blob per workflow run, got %v", blobs)
}
}
+14 -1
View File
@@ -28,10 +28,23 @@ func TestCompatibility(t *testing.T) {
t.Run("payloads", testPayloads)
t.Run("cache_and_artifacts", testCacheAndArtifactRoundTrips)
t.Run("cancellation_and_log_streaming", testRunCancellation)
t.Run("dispatch", testWorkflowDispatch)
t.Run("dispatch", inAllModes(testWorkflowDispatch))
t.Run("builtin_checkout", testBuiltinCheckout)
t.Run("exec_artifacts", testExecArtifacts)
t.Run("moving_action_ref", testMovingActionRef)
t.Run("ephemeral", testEphemeralRunner)
t.Run("failure_paths", inAllModes(testFailurePaths))
t.Run("job_timeout", testJobTimeout)
t.Run("cli_register_daemon", testCLIRegisterDaemon)
t.Run("cli_once", testCLIOnce)
t.Run("cli_labels_override", testCLILabelsOverride)
t.Run("cli_shutdown_drains", testCLIShutdownDrains)
t.Run("cli_shutdown_cancels", testCLIShutdownCancels)
t.Run("host_mode", testHostMode)
t.Run("reusable_workflow", inAllModes(testReusableWorkflow))
t.Run("label_routing", testLabelRouting)
t.Run("capacity", testCapacity)
t.Run("reconnect", testReconnect)
t.Run("kubernetes", testKubernetes)
}
+4 -20
View File
@@ -11,27 +11,11 @@ import (
"time"
)
func testWorkflowDispatch(t *testing.T) {
t.Parallel()
ctx := t.Context()
api, repo := newScenario(t)
pushWorkflow(t, api, repo, "dispatch.yml")
branch, err := api.DefaultBranch(ctx, repo)
if err != nil {
t.Fatalf("get default branch: %v", err)
func testWorkflowDispatch(t *testing.T, m mode) {
result := runWorkflow(t, m, workflowSpec{files: []string{"dispatch.yml"}, inputs: map[string]string{"subject": "dispatch-input-value"}})
if result.conclusion != "success" {
t.Errorf("run concluded %q, want success", result.conclusion)
}
inputs := map[string]string{"subject": "dispatch-input-value"}
if err := waitForDispatch(ctx, api, repo, "dispatch.yml", branch, inputs); err != nil {
t.Fatalf("dispatch workflow: %v", err)
}
dispatched := waitForRun(t, api, repo)
if dispatched.Event != "workflow_dispatch" {
t.Fatalf("run event is %q, want workflow_dispatch", dispatched.Event)
}
requireSuccess(t, api, repo, dispatched.ID)
}
// Gitea indexes workflows asynchronously after the Contents API commit.
+45
View File
@@ -0,0 +1,45 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build e2e
package e2e
import (
"testing"
"time"
)
func testFailurePaths(t *testing.T, m mode) {
result := runWorkflow(t, m, workflowSpec{files: []string{"failure.yml"}})
if result.conclusion != "failure" {
t.Errorf("run concluded %q, want failure", result.conclusion)
}
for job, want := range map[string]string{"fail": "failure", "tolerated": "success", "dependent": "skipped"} {
if got := result.job(job); got != want {
t.Errorf("job %q concluded %q, want %q", job, got, want)
}
}
for _, marker := range []string{"e2e-before-fail", "e2e-failure-cleanup", "e2e-tolerated-next"} {
if !result.hasLine(marker) {
t.Errorf("logs are missing %q", marker)
}
}
if result.hasLine("e2e-skipped-step") {
t.Error("a success() step ran after a failed step")
}
}
func testJobTimeout(t *testing.T) {
t.Parallel()
api, repo, _ := startIsolatedScenario(t, "timeout.yml", "e2e-timeout", runnerOptions{timeout: 15 * time.Second})
wfRun := waitForRun(t, api, repo)
completed, err := api.WaitForRunConclusion(t.Context(), repo, wfRun.ID, runTimeout)
if err != nil {
dumpRunLogs(t, api, repo, wfRun.ID)
t.Fatalf("wait for run: %v", err)
}
if logs := runLogs(t, api, repo, wfRun.ID); completed.Conclusion != "failure" || commandRow(logs, "e2e-timeout-started") == "" {
t.Errorf("run concluded %q, want failure after the job started, logs:\n%s", completed.Conclusion, logs)
}
}
+29 -6
View File
@@ -37,6 +37,14 @@ type ActionJob struct {
Name string `json:"name"`
Status string `json:"status"`
Conclusion string `json:"conclusion"`
RunnerName string `json:"runner_name"`
}
type ActionRunner struct {
Name string `json:"name"`
Labels []struct {
Name string `json:"name"`
} `json:"labels"`
}
func (a *GiteaAPI) CreateRepo(ctx context.Context, name string) error {
@@ -45,12 +53,16 @@ func (a *GiteaAPI) CreateRepo(ctx context.Context, name string) error {
}
func (a *GiteaAPI) CreateFile(ctx context.Context, repo, path, content, message string) error {
body := map[string]any{
"content": base64.StdEncoding.EncodeToString([]byte(content)),
"message": message,
return a.CreateFiles(ctx, repo, map[string]string{path: content}, message)
}
func (a *GiteaAPI) CreateFiles(ctx context.Context, repo string, files map[string]string, message string) error {
var operations []map[string]string
for path, content := range files {
operations = append(operations, map[string]string{"operation": "create", "path": path, "content": base64.StdEncoding.EncodeToString([]byte(content))})
}
url := fmt.Sprintf("/api/v1/repos/%s/%s/contents/%s", giteaAdminUser, repo, path)
return a.doJSON(ctx, http.MethodPost, url, body, nil)
body := map[string]any{"files": operations, "message": message}
return a.doJSON(ctx, http.MethodPost, fmt.Sprintf("/api/v1/repos/%s/%s/contents", giteaAdminUser, repo), body, nil)
}
func (a *GiteaAPI) CreateTag(ctx context.Context, repo, name string) error {
@@ -146,6 +158,17 @@ func (a *GiteaAPI) Jobs(ctx context.Context, repo string, runID int64) ([]Action
return resp.Jobs, nil
}
func (a *GiteaAPI) Runners(ctx context.Context, repo string) ([]ActionRunner, error) {
var resp struct {
Runners []ActionRunner `json:"runners"`
}
url := fmt.Sprintf("/api/v1/repos/%s/%s/actions/runners", giteaAdminUser, repo)
if err := a.doJSON(ctx, http.MethodGet, url, nil, &resp); err != nil {
return nil, err
}
return resp.Runners, nil
}
func (a *GiteaAPI) JobLogs(ctx context.Context, repo string, jobID int64) (string, error) {
url := fmt.Sprintf("/api/v1/repos/%s/%s/actions/jobs/%d/logs", giteaAdminUser, repo, jobID)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, a.baseURL+url, nil)
@@ -165,7 +188,7 @@ func (a *GiteaAPI) JobLogs(ctx context.Context, repo string, jobID int64) (strin
return "", err
}
if resp.StatusCode >= 300 {
return "", fmt.Errorf("GET %s: %d: %s", url, resp.StatusCode, body)
return "", statusError{method: http.MethodGet, path: url, code: resp.StatusCode, body: string(body)}
}
return string(body), nil
}
+17 -95
View File
@@ -15,9 +15,7 @@ import (
"io"
"net"
"net/http"
"net/netip"
"os"
"strconv"
"strings"
"time"
@@ -40,7 +38,8 @@ type GiteaFixture struct {
image string
version string
baseURL string
network string // set in container mode; jobs must join it
network string // the user-defined network every container of the suite joins
selfIP string // the test process's address on network, which job containers can reach
adminToken string
}
@@ -55,44 +54,10 @@ func dockerClient(ctx context.Context) (mobyclient.APIClient, error) {
return cli, nil
}
// Prefer docker bridge gateway, then LAN IP; never 127.0.0.1 (ACTIONS_RUNTIME_URL must reach the host from job containers).
func hostAddress(ctx context.Context, cli mobyclient.APIClient) netip.Addr {
if gateway, ok := bridgeGateway(ctx, cli); ok && bindable(gateway) {
return gateway
}
loopback := netip.AddrFrom4([4]byte{127, 0, 0, 1})
conn, err := net.Dial("udp", "192.0.2.1:80") // TEST-NET-1: routing table only
if err != nil {
return loopback
}
defer conn.Close()
addr, ok := conn.LocalAddr().(*net.UDPAddr)
if !ok {
return loopback
}
return addr.AddrPort().Addr().Unmap()
}
func bridgeGateway(ctx context.Context, cli mobyclient.APIClient) (netip.Addr, bool) {
inspected, err := cli.NetworkInspect(ctx, "bridge", mobyclient.NetworkInspectOptions{})
if err != nil {
return netip.Addr{}, false
}
for _, cfg := range inspected.Network.IPAM.Config {
if cfg.Gateway.Is4() {
return cfg.Gateway, true
}
}
return netip.Addr{}, false
}
// When tests run inside a container (sibling docker daemon), join that network and address by name.
func selfNetwork(ctx context.Context, cli mobyclient.APIClient) (string, bool) {
func selfNetwork(ctx context.Context, cli mobyclient.APIClient) (string, string, bool) {
if _, err := os.Stat("/.dockerenv"); err != nil {
return "", false
return "", "", false
}
id, err := os.ReadFile("/proc/sys/kernel/hostname")
@@ -100,42 +65,19 @@ func selfNetwork(ctx context.Context, cli mobyclient.APIClient) (string, bool) {
id, err = os.ReadFile("/etc/hostname")
}
if err != nil {
return "", false
return "", "", false
}
inspected, err := cli.ContainerInspect(ctx, strings.TrimSpace(string(id)), mobyclient.ContainerInspectOptions{})
if err != nil || inspected.Container.NetworkSettings == nil {
return "", false
return "", "", false
}
for name := range inspected.Container.NetworkSettings.Networks {
for name, endpoint := range inspected.Container.NetworkSettings.Networks {
if name != "host" && name != "none" && name != "bridge" { // only user-defined nets have DNS
return name, true
return name, endpoint.IPAddress.String(), true
}
}
return "", false
}
func bindable(addr netip.Addr) bool {
l, err := net.Listen("tcp", net.JoinHostPort(addr.String(), "0"))
if err != nil {
return false
}
_ = l.Close()
return true
}
func freeHostPort(host string) (int, error) {
l, err := net.Listen("tcp", net.JoinHostPort(host, "0"))
if err != nil {
return 0, err
}
defer l.Close()
addr, ok := l.Addr().(*net.TCPAddr)
if !ok {
return 0, fmt.Errorf("unexpected listener address type %T", l.Addr())
}
return addr.Port, nil
return "", "", false
}
func StartGitea(ctx context.Context, cli mobyclient.APIClient) (*GiteaFixture, error) {
@@ -151,31 +93,12 @@ func StartGitea(ctx context.Context, cli mobyclient.APIClient) (*GiteaFixture, e
return nil, errors.New("E2E_GITEA_IMAGE is not set")
}
name := fmt.Sprintf("gitea-runner-e2e-%d", time.Now().UnixNano())
containerPort := network.MustParsePort("3000/tcp")
var (
baseURL string
hostConfig = &apicontainer.HostConfig{}
netConfig *network.NetworkingConfig
sharedNet, _ = selfNetwork(ctx, cli)
)
if sharedNet != "" {
baseURL = "http://" + net.JoinHostPort(name, "3000")
netConfig = &network.NetworkingConfig{
EndpointsConfig: map[string]*network.EndpointSettings{sharedNet: {}},
}
} else {
host := hostAddress(ctx, cli)
port, err := freeHostPort(host.String())
if err != nil {
return nil, fmt.Errorf("find a free host port: %w", err)
}
baseURL = "http://" + net.JoinHostPort(host.String(), strconv.Itoa(port))
hostConfig.PortBindings = network.PortMap{
containerPort: []network.PortBinding{{HostIP: host, HostPort: strconv.Itoa(port)}},
}
sharedNet, selfIP, ok := selfNetwork(ctx, cli)
if !ok {
return nil, errors.New("e2e tests must run in a container on a user-defined network, `make test-e2e` provides one")
}
name := fmt.Sprintf("gitea-runner-e2e-%d", time.Now().UnixNano())
baseURL := "http://" + net.JoinHostPort(name, "3000")
if _, err := cli.ImageInspect(ctx, image); err != nil {
return nil, fmt.Errorf("inspect gitea image %s: %w", image, err)
@@ -191,17 +114,16 @@ func StartGitea(ctx context.Context, cli mobyclient.APIClient) (*GiteaFixture, e
"GITEA__server__LFS_START_SERVER=true",
"GITEA__server__ROOT_URL=" + baseURL + "/",
},
ExposedPorts: network.PortSet{containerPort: struct{}{}},
},
HostConfig: hostConfig,
NetworkingConfig: netConfig,
HostConfig: &apicontainer.HostConfig{Tmpfs: map[string]string{"/data": "exec"}},
NetworkingConfig: &network.NetworkingConfig{EndpointsConfig: map[string]*network.EndpointSettings{sharedNet: {}}},
Name: name,
})
if err != nil {
return nil, fmt.Errorf("create gitea container: %w", err)
}
f := &GiteaFixture{cli: cli, id: resp.ID, image: image, baseURL: baseURL, network: sharedNet}
f := &GiteaFixture{cli: cli, id: resp.ID, image: image, baseURL: baseURL, network: sharedNet, selfIP: selfIP}
closeClient = false
if _, err := cli.ContainerStart(ctx, f.id, mobyclient.ContainerStartOptions{}); err != nil {
+33
View File
@@ -0,0 +1,33 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build e2e
package e2e
import (
"os"
"testing"
)
func testHostMode(t *testing.T) {
t.Parallel()
ctx := t.Context()
api, repo := newScenario(t)
hostname, err := os.Hostname()
if err != nil {
t.Fatalf("hostname: %v", err)
}
if err := api.CreateVariable(ctx, repo, "E2E_HOSTNAME", hostname); err != nil {
t.Fatalf("create variable: %v", err)
}
startRunner(t, repo, "e2e-host", runnerOptions{host: true})
pushWorkflow(t, api, repo, "host.yml")
wfRun := waitForRun(t, api, repo)
requireSuccess(t, api, repo, wfRun.ID)
if logs := runLogs(t, api, repo, wfRun.ID); commandRow(logs, "e2e-host-job-ran") == "" {
t.Errorf("job logs are missing the marker:\n%s", logs)
}
}
+120 -78
View File
@@ -9,15 +9,16 @@ import (
"bufio"
"bytes"
"context"
"errors"
"fmt"
"io"
"net"
"net/netip"
"net/url"
"os"
"path/filepath"
"strconv"
"slices"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
@@ -32,125 +33,168 @@ import (
func testKubernetes(t *testing.T) {
t.Parallel()
ctx := t.Context()
k3sID, kubernetes, images := startK3s(t)
cluster := requireCluster(t)
api, repo := newScenario(t)
if err := api.CreateVariable(ctx, repo, "E2E_SERVICE_IMAGE", images[1]); err != nil {
if err := api.CreateVariable(ctx, repo, "E2E_SERVICE_IMAGE", cluster.serviceImage); err != nil {
t.Fatalf("create service image variable: %v", err)
}
startRunner(t, repo, "e2e-kubernetes", runnerOptions{image: images[0], kubernetes: kubernetes})
startRunner(t, repo, "e2e-kubernetes", runnerOptions{image: cluster.jobImage, kubernetes: cluster.kubernetes})
pushWorkflow(t, api, repo, "kubernetes.yml")
wfRun := waitForRun(t, api, repo)
requireSuccess(t, api, repo, wfRun.ID)
}
if out, err := containerExec(ctx, fixture.cli, k3sID, "", nil,
// checkClusterLeaks runs after every test, so a pod or secret any of them left behind fails the suite.
func checkClusterLeaks() error {
if !clusterStarted.Load() {
return nil
}
cluster, err := k3s()
if err != nil {
return nil // the tests that needed the cluster already failed
}
if out, err := containerExec(context.Background(), dockerCLI, cluster.id, "", nil,
"kubectl", "wait", "--for=delete", "pods,secrets", "--selector=app.kubernetes.io/managed-by=gitea-runner", "--timeout=1m",
); err != nil {
t.Errorf("job pods or secrets left behind: %v: %s", err, out)
return fmt.Errorf("job pods or secrets left behind: %w: %s", err, out)
}
return nil
}
type kubeCluster struct {
id string
kubernetes *config.Kubernetes
jobImage, serviceImage string
}
var (
k3s = sync.OnceValues(startK3s) // independent of Gitea, so it can boot alongside it
clusterStarted atomic.Bool
clusterCleanup []func()
sharedCluster = sync.OnceValues(func() (*kubeCluster, error) {
cluster, err := k3s()
if err != nil {
return nil, err
}
gitea, err := fixture.cli.ContainerInspect(context.Background(), fixture.id, mobyclient.ContainerInspectOptions{})
if err != nil {
return nil, fmt.Errorf("inspect gitea container: %w", err)
}
giteaURL, err := url.Parse(fixture.baseURL)
if err != nil {
return nil, fmt.Errorf("parse gitea url: %w", err)
}
cluster.kubernetes.PodTemplate = map[string]any{"spec": map[string]any{"hostAliases": []any{map[string]any{
"ip": gitea.Container.NetworkSettings.Networks[fixture.network].IPAddress.String(),
"hostnames": []any{giteaURL.Hostname()},
}}}}
return cluster, nil
})
)
func requireCluster(t *testing.T) *kubeCluster {
t.Helper()
cluster, err := sharedCluster()
if err != nil {
t.Fatal(err)
}
return cluster
}
func stopCluster() {
if !clusterStarted.Load() {
return
}
_, _ = k3s()
for _, cleanup := range slices.Backward(clusterCleanup) {
cleanup()
}
}
func startK3s(t *testing.T) (string, *config.Kubernetes, []string) {
t.Helper()
ctx := t.Context()
cli := fixture.cli
func startK3s() (*kubeCluster, error) {
clusterStarted.Store(true)
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
cli := dockerCLI
sharedNet, _, ok := selfNetwork(ctx, cli)
if !ok {
return nil, errors.New("e2e tests must run in a container on a user-defined network")
}
name := fmt.Sprintf("gitea-runner-e2e-k3s-%d", time.Now().UnixNano())
apiPort := network.MustParsePort("6443/tcp")
server := "https://" + name + ":6443"
hostConfig := &apicontainer.HostConfig{Privileged: true}
var netConfig *network.NetworkingConfig
if fixture.network != "" {
netConfig = &network.NetworkingConfig{EndpointsConfig: map[string]*network.EndpointSettings{fixture.network: {}}}
} else {
loopback := netip.MustParseAddr("127.0.0.1")
port, err := freeHostPort(loopback.String())
if err != nil {
t.Fatalf("find a free host port: %v", err)
}
server = "https://" + net.JoinHostPort(loopback.String(), strconv.Itoa(port))
hostConfig.PortBindings = network.PortMap{apiPort: []network.PortBinding{{HostIP: loopback, HostPort: strconv.Itoa(port)}}}
}
created, err := cli.ContainerCreate(ctx, mobyclient.ContainerCreateOptions{
Config: &apicontainer.Config{
Image: os.Getenv("E2E_K3S_IMAGE"),
Cmd: []string{"server", "--disable=traefik,metrics-server,servicelb,local-storage,coredns", "--tls-san=" + name, "--kubelet-arg=fail-cgroupv1=false"},
ExposedPorts: network.PortSet{apiPort: struct{}{}},
Image: os.Getenv("E2E_K3S_IMAGE"),
Cmd: []string{"server", "--disable=traefik,metrics-server,servicelb,local-storage,coredns", "--tls-san=" + name, "--kubelet-arg=fail-cgroupv1=false"},
},
HostConfig: hostConfig,
NetworkingConfig: netConfig,
HostConfig: &apicontainer.HostConfig{Privileged: true},
NetworkingConfig: &network.NetworkingConfig{EndpointsConfig: map[string]*network.EndpointSettings{sharedNet: {}}},
Name: name,
})
if err != nil {
t.Fatalf("create k3s container: %v", err)
return nil, fmt.Errorf("create k3s container: %w", err)
}
t.Cleanup(func() {
clusterCleanup = append(clusterCleanup, func() {
_, _ = cli.ContainerRemove(context.Background(), created.ID, mobyclient.ContainerRemoveOptions{Force: true, RemoveVolumes: true})
})
if _, err := cli.ContainerStart(ctx, created.ID, mobyclient.ContainerStartOptions{}); err != nil {
t.Fatalf("start k3s container: %v", err)
return nil, fmt.Errorf("start k3s container: %w", err)
}
containerdCtx, stopWaiting := context.WithTimeout(ctx, 2*time.Minute)
defer stopWaiting()
if err := waitForContainerLog(containerdCtx, created.ID, "containerd is now running"); err != nil {
t.Fatalf("wait for k3s containerd: %v", err)
if err := waitForContainerLog(ctx, created.ID, "containerd is now running"); err != nil {
return nil, fmt.Errorf("wait for k3s containerd: %w", err)
}
sandboxPulled := make(chan struct{})
go func() { // first pods would otherwise pull the sandbox image themselves
defer close(sandboxPulled)
pullCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
_, _ = containerExec(pullCtx, cli, created.ID, "", nil, "sh", "-c",
`ctr --namespace=k8s.io images pull "docker.io/$(sed -n 's/^ *sandbox = "\(.*\)"$/\1/p' /var/lib/rancher/k3s/agent/etc/containerd/config.toml)"`)
}()
images, err := preloadImages(ctx, created.ID, os.Getenv("E2E_JOB_IMAGE"), os.Getenv("SERVICE_IMAGE"))
<-sandboxPulled
if err != nil {
return nil, err
}
images := preloadImages(t, created.ID, os.Getenv("E2E_JOB_IMAGE"), os.Getenv("SERVICE_IMAGE"))
waitCtx, cancel := context.WithTimeout(ctx, 2*time.Minute)
defer cancel()
for {
_, err := containerExec(waitCtx, cli, created.ID, "", nil, "kubectl", "get", "serviceaccount", "default")
_, err := containerExec(ctx, cli, created.ID, "", nil, "kubectl", "get", "serviceaccount", "default")
if err == nil {
break
}
inspected, inspectErr := cli.ContainerInspect(ctx, created.ID, mobyclient.ContainerInspectOptions{})
if inspectErr != nil || !inspected.Container.State.Running || waitCtx.Err() != nil {
if inspectErr != nil || !inspected.Container.State.Running || ctx.Err() != nil {
var logs bytes.Buffer
if reader, logsErr := cli.ContainerLogs(ctx, created.ID, mobyclient.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Tail: "30"}); logsErr == nil {
if reader, logsErr := cli.ContainerLogs(context.Background(), created.ID, mobyclient.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Tail: "30"}); logsErr == nil {
_, _ = stdcopy.StdCopy(&logs, &logs, reader)
_ = reader.Close()
}
t.Fatalf("k3s did not start: %v, state %+v %v, last log lines:\n%s", err, inspected.Container.State, inspectErr, logs.String())
return nil, fmt.Errorf("k3s did not start: %w, state %+v %v, last log lines:\n%s", err, inspected.Container.State, inspectErr, logs.String())
}
select {
case <-waitCtx.Done():
case <-ctx.Done():
case <-time.After(pollInterval):
}
}
kubeconfig, err := containerExec(ctx, cli, created.ID, "", nil, "cat", "/etc/rancher/k3s/k3s.yaml")
if err != nil {
t.Fatalf("read kubeconfig: %v", err)
return nil, fmt.Errorf("read kubeconfig: %w", err)
}
path := filepath.Join(t.TempDir(), "kubeconfig")
path := filepath.Join(binDir, "kubeconfig")
if err := os.WriteFile(path, []byte(strings.ReplaceAll(kubeconfig, "https://127.0.0.1:6443", server)), 0o600); err != nil {
t.Fatalf("write kubeconfig: %v", err)
return nil, fmt.Errorf("write kubeconfig: %w", err)
}
kubernetes := &config.Kubernetes{Kubeconfig: path, Namespace: "default"}
if fixture.network != "" {
gitea, err := cli.ContainerInspect(ctx, fixture.id, mobyclient.ContainerInspectOptions{})
if err != nil {
t.Fatalf("inspect gitea container: %v", err)
}
giteaURL, err := url.Parse(fixture.baseURL)
if err != nil {
t.Fatalf("parse gitea url: %v", err)
}
kubernetes.PodTemplate = map[string]any{"spec": map[string]any{"hostAliases": []any{map[string]any{
"ip": gitea.Container.NetworkSettings.Networks[fixture.network].IPAddress.String(),
"hostnames": []any{giteaURL.Hostname()},
}}}}
}
return created.ID, kubernetes, images
return &kubeCluster{id: created.ID, kubernetes: &config.Kubernetes{Kubeconfig: path, Namespace: "default"}, jobImage: images[0], serviceImage: images[1]}, nil
}
func waitForContainerLog(ctx context.Context, id, message string) error {
logs, err := fixture.cli.ContainerLogs(ctx, id, mobyclient.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Follow: true})
logs, err := dockerCLI.ContainerLogs(ctx, id, mobyclient.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Follow: true})
if err != nil {
return err
}
@@ -175,25 +219,23 @@ func waitForContainerLog(ctx context.Context, id, message string) error {
return fmt.Errorf("container exited before logging %q", message)
}
func preloadImages(t *testing.T, k3sID string, images ...string) []string {
t.Helper()
ctx := t.Context()
func preloadImages(ctx context.Context, k3sID string, images ...string) ([]string, error) {
var aliases []string
for index, image := range images {
alias := fmt.Sprintf("gitea-runner-e2e/preload-%d:%d", index, time.Now().UnixNano())
if _, err := fixture.cli.ImageTag(ctx, mobyclient.ImageTagOptions{Source: image, Target: alias}); err != nil {
t.Fatalf("tag %s: %v", image, err)
if _, err := dockerCLI.ImageTag(ctx, mobyclient.ImageTagOptions{Source: image, Target: alias}); err != nil {
return nil, fmt.Errorf("tag %s: %w", image, err)
}
t.Cleanup(func() { _, _ = fixture.cli.ImageRemove(context.Background(), alias, mobyclient.ImageRemoveOptions{}) })
clusterCleanup = append(clusterCleanup, func() { _, _ = dockerCLI.ImageRemove(context.Background(), alias, mobyclient.ImageRemoveOptions{}) })
aliases = append(aliases, alias)
}
archive, err := fixture.cli.ImageSave(ctx, aliases)
archive, err := dockerCLI.ImageSave(ctx, aliases)
if err != nil {
t.Fatalf("save %v: %v", images, err)
return nil, fmt.Errorf("save %v: %w", images, err)
}
defer archive.Close()
if out, err := containerExec(ctx, fixture.cli, k3sID, "", archive, "ctr", "--namespace=k8s.io", "images", "import", "-"); err != nil {
t.Fatalf("import %v into k3s: %v: %s", images, err, out)
if out, err := containerExec(ctx, dockerCLI, k3sID, "", archive, "ctr", "--namespace=k8s.io", "images", "import", "-"); err != nil {
return nil, fmt.Errorf("import %v into k3s: %w: %s", images, err, out)
}
return aliases
return aliases, nil
}
+63 -1
View File
@@ -6,15 +6,27 @@
package e2e
import (
"cmp"
"context"
"errors"
"flag"
"fmt"
"net"
"net/http"
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"gitea.com/gitea/runner/internal/app/poll"
mobyclient "github.com/moby/moby/client"
)
const runTimeout = 3 * time.Minute
@@ -25,6 +37,18 @@ var fixture *GiteaFixture
var skipReason string
var binDir string
var dockerCLI mobyclient.APIClient
var runnerBinary = sync.OnceValues(func() (string, error) {
bin := filepath.Join(binDir, "gitea-runner")
if out, err := exec.Command(cmp.Or(os.Getenv("GO"), "go"), "build", "-o", bin, "..").CombinedOutput(); err != nil {
return "", fmt.Errorf("build runner: %w\n%s", err, out)
}
return bin, nil
})
func TestMain(m *testing.M) {
os.Exit(runSuite(m))
}
@@ -38,6 +62,17 @@ func runSuite(m *testing.M) int {
skipReason = fmt.Sprintf("docker unavailable: %v", err)
return m.Run()
}
dockerCLI = cli
if binDir, err = os.MkdirTemp("", "gitea-runner-e2e-"); err != nil {
fmt.Fprintf(os.Stderr, "create binary dir: %v\n", err)
return 1
}
defer os.RemoveAll(binDir)
defer stopCluster()
if flag.Parse(); flag.Lookup("test.run").Value.String() == "" { // a filtered run starts the cluster only if it needs it
go func() { _, _ = k3s() }()
}
f, err := StartGitea(ctx, cli)
if err != nil {
@@ -49,7 +84,12 @@ func runSuite(m *testing.M) int {
fmt.Fprintf(os.Stderr, "gitea fixture: image=%s version=%s\n", fixture.image, fixture.version)
return m.Run()
code := m.Run()
if err := checkClusterLeaks(); err != nil && code == 0 {
fmt.Fprintln(os.Stderr, err)
code = 1
}
return code
}
func newScenario(t *testing.T) (*GiteaAPI, string) {
@@ -109,6 +149,10 @@ func runLogs(t *testing.T, api *GiteaAPI, repo string, runID int64) string {
var all strings.Builder
for _, job := range jobs {
logs, err := api.JobLogs(ctx, repo, job.ID)
var response statusError
if errors.As(err, &response) && response.code == http.StatusNotFound {
continue // skipped jobs and reusable workflow callers never ran a task, so have no log
}
if err != nil {
t.Fatalf("job logs for %s: %v", job.Name, err)
}
@@ -136,6 +180,24 @@ func dumpRunLogs(t *testing.T, api *GiteaAPI, repo string, runID int64) {
}
}
// newGate serves a URL that answers 503 until release, for jobs to poll instead of sleeping.
func newGate(t *testing.T) (string, func()) {
t.Helper()
var released atomic.Bool
listener, err := net.Listen("tcp", ":0")
if err != nil {
t.Fatalf("listen for the gate: %v", err)
}
server := &http.Server{Handler: http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
if !released.Load() {
w.WriteHeader(http.StatusServiceUnavailable)
}
}), ReadHeaderTimeout: time.Second}
go func() { _ = server.Serve(listener) }()
t.Cleanup(func() { _ = server.Close() })
return fmt.Sprintf("http://%s/", net.JoinHostPort(fixture.selfIP, strconv.Itoa(listener.Addr().(*net.TCPAddr).Port))), func() { released.Store(true) }
}
func waitForRun(t *testing.T, api *GiteaAPI, repo string) *ActionRun {
t.Helper()
return waitForRunAfter(t, api, repo, 0)
+227
View File
@@ -0,0 +1,227 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build e2e
package e2e
import (
"bufio"
"bytes"
"cmp"
"encoding/json"
"errors"
"os"
"os/exec"
"path/filepath"
"slices"
"strings"
"testing"
)
type mode struct {
exec, kubernetes bool
}
var allModes = []mode{{}, {kubernetes: true}, {exec: true}, {exec: true, kubernetes: true}}
func (m mode) String() string {
driver, backend := "daemon", "docker"
if m.exec {
driver = "exec"
}
if m.kubernetes {
backend = "kubernetes"
}
return driver + "_" + backend
}
func inAllModes(scenario func(*testing.T, mode)) func(*testing.T) {
return func(t *testing.T) {
t.Parallel()
for _, m := range allModes {
t.Run(m.String(), func(t *testing.T) {
t.Parallel()
scenario(t, m)
})
}
}
}
type workflowSpec struct {
files []string
inputs map[string]string // dispatches files[0] instead of relying on push
execArgs []string
}
type workflowResult struct {
conclusion string
jobs map[string]string
lines []string
}
func (r *workflowResult) job(id string) string {
return cmp.Or(r.jobs[id], "skipped") // exec reports nothing for a job it skipped
}
func (r *workflowResult) hasLine(line string) bool {
return slices.Contains(r.lines, line)
}
func runWorkflow(t *testing.T, m mode, spec workflowSpec) *workflowResult {
t.Helper()
run := daemonWorkflow
if m.exec {
run = execWorkflow
}
result := run(t, m, spec)
t.Cleanup(func() {
if t.Failed() {
t.Logf("%s concluded %q, jobs %v, logs:\n%s", m, result.conclusion, result.jobs, strings.Join(result.lines, "\n"))
}
})
return result
}
func daemonWorkflow(t *testing.T, m mode, spec workflowSpec) *workflowResult {
t.Helper()
ctx := t.Context()
api, repo := newScenario(t)
if m.kubernetes {
cluster := requireCluster(t)
startRunner(t, repo, "e2e-kubernetes", runnerOptions{image: cluster.jobImage, kubernetes: cluster.kubernetes, capacity: 16})
}
files := map[string]string{}
for _, file := range spec.files {
content, err := os.ReadFile(filepath.Join("testdata", "workflows", file))
if err != nil {
t.Fatalf("read workflow fixture %s: %v", file, err)
}
if m.kubernetes {
content = bytes.ReplaceAll(content, []byte("runs-on: ubuntu-latest"), []byte("runs-on: e2e-kubernetes"))
}
files[".gitea/workflows/"+file] = string(content)
}
if err := api.CreateFiles(ctx, repo, files, "add workflows"); err != nil {
t.Fatalf("push workflows: %v", err)
}
if spec.inputs != nil {
branch, err := api.DefaultBranch(ctx, repo)
if err != nil {
t.Fatalf("get default branch: %v", err)
}
if err := waitForDispatch(ctx, api, repo, spec.files[0], branch, spec.inputs); err != nil {
t.Fatalf("dispatch workflow: %v", err)
}
}
wfRun := waitForRun(t, api, repo)
completed, err := api.WaitForRunConclusion(ctx, repo, wfRun.ID, runTimeout)
if err != nil {
dumpRunLogs(t, api, repo, wfRun.ID)
t.Fatalf("wait for run: %v", err)
}
jobs, err := api.Jobs(ctx, repo, wfRun.ID)
if err != nil {
t.Fatalf("list jobs: %v", err)
}
result := &workflowResult{conclusion: completed.Conclusion, jobs: map[string]string{}}
for _, job := range jobs {
result.jobs[job.Name] = job.Conclusion
}
for line := range strings.SplitSeq(runLogs(t, api, repo, wfRun.ID), "\n") {
if _, payload, found := strings.Cut(line, "Z "); found {
result.lines = append(result.lines, strings.TrimRight(payload, "\r"))
}
}
return result
}
func execWorkflow(t *testing.T, m mode, spec workflowSpec) *workflowResult {
t.Helper()
ctx := t.Context()
bin, err := runnerBinary()
if err != nil {
t.Fatal(err)
}
workdir := t.TempDir()
workflows := filepath.Join(workdir, ".gitea", "workflows")
if err := os.MkdirAll(workflows, 0o755); err != nil {
t.Fatal(err)
}
for _, file := range spec.files {
content, err := os.ReadFile(filepath.Join("testdata", "workflows", file))
if err != nil {
t.Fatalf("read workflow fixture %s: %v", file, err)
}
if err := os.WriteFile(filepath.Join(workflows, file), content, 0o644); err != nil {
t.Fatal(err)
}
}
for _, args := range [][]string{
{"init", "-q"},
{"remote", "add", "origin", "https://example.com/e2e/exec.git"},
{"-c", "user.name=e2e", "-c", "[email protected]", "commit", "-q", "--allow-empty", "-m", "init"},
} {
if out, err := exec.CommandContext(ctx, "git", append([]string{"-C", workdir}, args...)...).CombinedOutput(); err != nil {
t.Fatalf("git %v: %v\n%s", args, err, out)
}
}
args := []string{"exec", "--json", "-C", workdir, "-W", workflows, "--artifact-server-path", t.TempDir(), "--artifact-server-port", "0"}
env := slices.DeleteFunc(os.Environ(), func(kv string) bool { return strings.HasPrefix(kv, "ACTIONS_") })
if m.kubernetes {
cluster := requireCluster(t)
args = append(args, "-i", "kubernetes://"+cluster.jobImage)
env = append(env, "KUBECONFIG="+cluster.kubernetes.Kubeconfig)
} else {
args = append(args, "-i", os.Getenv("E2E_JOB_IMAGE"), "--network", fixture.network)
}
if spec.inputs != nil {
args = append(args, "-E", "workflow_dispatch")
for name, value := range spec.inputs {
args = append(args, "--input", name+"="+value)
}
}
cmd := exec.CommandContext(ctx, bin, append(args, spec.execArgs...)...)
cmd.Env = env
var stdout, stderr bytes.Buffer
cmd.Stdout, cmd.Stderr = &stdout, &stderr
runErr := cmd.Run()
var exitErr *exec.ExitError
if runErr != nil && !errors.As(runErr, &exitErr) {
t.Fatalf("exec: %v\n%s", runErr, stderr.String())
}
result := &workflowResult{conclusion: "success", jobs: map[string]string{}}
if runErr != nil {
result.conclusion = "failure"
}
scanner := bufio.NewScanner(&stdout)
scanner.Buffer(nil, 1<<20)
for scanner.Scan() {
var entry struct {
JobID string `json:"jobID"`
JobResult string `json:"jobResult"`
Msg string `json:"msg"`
RawOutput bool `json:"raw_output"`
}
if json.Unmarshal(scanner.Bytes(), &entry) != nil {
continue
}
if entry.JobResult != "" {
result.jobs[entry.JobID] = entry.JobResult
}
if entry.RawOutput {
for line := range strings.SplitSeq(strings.TrimRight(entry.Msg, "\n"), "\n") {
result.lines = append(result.lines, strings.TrimRight(line, "\r"))
}
}
}
if result.conclusion != "success" && len(result.jobs) == 0 {
t.Fatalf("exec ran no job: %v\n%s", runErr, stderr.String())
}
return result
}
+138
View File
@@ -0,0 +1,138 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build e2e
package e2e
import (
"fmt"
"io"
"net"
"net/url"
"strconv"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
)
// flakyProxy forwards TCP to Gitea and can drop every connection to simulate an outage.
type flakyProxy struct {
ln net.Listener
target string
down atomic.Bool
refused atomic.Int64
mu sync.Mutex
conns map[net.Conn]struct{}
}
func newFlakyProxy(t *testing.T, targetURL string) *flakyProxy {
t.Helper()
u, err := url.Parse(targetURL)
if err != nil {
t.Fatalf("parse %q: %v", targetURL, err)
}
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
p := &flakyProxy{ln: ln, target: u.Host, conns: map[net.Conn]struct{}{}}
go p.serve()
t.Cleanup(func() {
_ = ln.Close()
p.dropAll()
})
return p
}
func (p *flakyProxy) URL() string { return "http://" + p.ln.Addr().String() }
func (p *flakyProxy) serve() {
for {
c, err := p.ln.Accept()
if err != nil {
return
}
if p.down.Load() {
p.refused.Add(1)
_ = c.Close()
continue
}
up, err := net.Dial("tcp", p.target)
if err != nil {
_ = c.Close()
continue
}
p.mu.Lock()
p.conns[c], p.conns[up] = struct{}{}, struct{}{}
p.mu.Unlock()
go func() { _, _ = io.Copy(up, c); _ = up.Close() }()
go func() { _, _ = io.Copy(c, up); _ = c.Close() }()
}
}
func (p *flakyProxy) dropAll() {
p.mu.Lock()
defer p.mu.Unlock()
for c := range p.conns {
_ = c.Close()
}
clear(p.conns)
}
// cut starts the outage and returns once the runner has been refused at least refusals more times.
func (p *flakyProxy) cut(t *testing.T, refusals int64) {
t.Helper()
before := p.refused.Load()
p.down.Store(true)
p.dropAll()
deadline := time.Now().Add(30 * time.Second)
for p.refused.Load() < before+refusals {
if time.Now().After(deadline) {
t.Fatal("runner never retried during the outage")
}
time.Sleep(pollInterval)
}
}
func (p *flakyProxy) restore() { p.down.Store(false) }
// testReconnect checks that a runner recovers from a lost connection to Gitea, idle or mid-job.
func testReconnect(t *testing.T) {
t.Parallel()
api, repo := newScenario(t)
gate, release := newGate(t)
if err := api.CreateVariable(t.Context(), repo, "E2E_GATE", gate); err != nil {
t.Fatalf("create gate variable: %v", err)
}
proxy := newFlakyProxy(t, fixture.baseURL)
startRunner(t, repo, "e2e-reconnect", runnerOptions{host: true, address: proxy.URL()})
proxy.cut(t, 2)
pushWorkflow(t, api, repo, "reconnect_idle.yml")
idle := waitForRun(t, api, repo)
proxy.restore()
requireSuccess(t, api, repo, idle.ID)
pushWorkflow(t, api, repo, "reconnect.yml")
slow := waitForRunAfter(t, api, repo, idle.ID)
waitForRunningJobLog(t, api, repo, slow.ID, "e2e-before-outage", 1)
// the job logs a line every 200ms until released, so the outage spans several that can only arrive after it
proxy.cut(t, 8)
proxy.restore()
release()
requireSuccess(t, api, repo, slow.ID)
logs := runLogs(t, api, repo, slow.ID)
ticks, err := strconv.Atoi(strings.TrimPrefix(commandRow(logs, "ticks="), "ticks="))
if err != nil {
t.Fatalf("the job did not report its tick count: %v\n%s", err, logs)
}
for i := 1; i <= ticks; i++ {
if tick := fmt.Sprintf("tick-%d", i); commandRow(logs, tick) != tick {
t.Errorf("log line %s written during or after the outage was lost:\n%s", tick, logs)
break
}
}
}
+19
View File
@@ -0,0 +1,19 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build e2e
package e2e
import "testing"
// testReusableWorkflow runs a local workflow_call workflow and consumes its output.
func testReusableWorkflow(t *testing.T, m mode) {
result := runWorkflow(t, m, workflowSpec{
files: []string{"reusable_callee.yml", "reusable_caller.yml"},
execArgs: []string{"--no-skip-checkout"},
})
if result.conclusion != "success" || !result.hasLine("e2e-reusable-ran") {
t.Errorf("run concluded %q, want success with the called workflow's marker", result.conclusion)
}
}
+17 -9
View File
@@ -30,11 +30,16 @@ type runnerOptions struct {
cacheHost string
image string
kubernetes *config.Kubernetes
host bool // register a host-mode label instead of a docker one
name string // suffix that keeps runner names unique when a test starts several
address string // Gitea address the runner talks to, default the fixture
timeout time.Duration
}
func startRunner(t *testing.T, repo, labelName string, options runnerOptions) *poll.Poller {
t.Helper()
ctx := t.Context()
address := cmp.Or(options.address, fixture.baseURL)
token, err := fixture.RegistrationToken(ctx, repo)
if err != nil {
t.Fatalf("get registration token: %v", err)
@@ -56,28 +61,31 @@ func startRunner(t *testing.T, repo, labelName string, options runnerOptions) *p
cfg.Runner.StateReportInterval = 500 * time.Millisecond // so cancel reaches the job quickly
cfg.Runner.LogReportInterval = 500 * time.Millisecond
cfg.Runner.Capacity = max(options.capacity, 1)
if fixture.network != "" {
cfg.Container.Network = fixture.network
}
cfg.Runner.Timeout = cmp.Or(options.timeout, cfg.Runner.Timeout)
cfg.Container.Network = fixture.network
scheme := labels.SchemeDocker
if options.kubernetes != nil {
scheme, cfg.Kubernetes = labels.SchemeKubernetes, *options.kubernetes
}
rawLabel := labelName + ":" + scheme + "://" + cmp.Or(options.image, os.Getenv("E2E_JOB_IMAGE"))
if options.host {
rawLabel = labelName + ":" + labels.SchemeHost
cfg.Host.WorkdirParent = t.TempDir()
}
label, err := labels.Parse(rawLabel)
if err != nil {
t.Fatalf("parse label %q: %v", rawLabel, err)
}
labelNames := []string{label.Name}
pingCli := client.New(fixture.baseURL, cfg.Runner.TLSConfig(), "", "", config.RequestTimeout, cfg.Runner.ExtraHeaders)
if _, err := pingCli.Ping(ctx, &pingv1.PingRequest{Data: t.Name()}); err != nil {
t.Fatalf("ping %s: %v", fixture.baseURL, err)
pingCli := client.New(address, cfg.Runner.TLSConfig(), "", "", config.RequestTimeout, cfg.Runner.ExtraHeaders)
if _, err := pingCli.Ping(ctx, &pingv1.PingRequest{Data: t.Name() + options.name}); err != nil {
t.Fatalf("ping %s: %v", address, err)
}
regResp, err := pingCli.Register(ctx, &runnerv1.RegisterRequest{
Name: t.Name(),
Name: t.Name() + options.name,
Token: token,
Version: "e2e",
Labels: labelNames,
@@ -96,12 +104,12 @@ func startRunner(t *testing.T, repo, labelName string, options runnerOptions) *p
UUID: regResp.Runner.Uuid,
Name: regResp.Runner.Name,
Token: regResp.Runner.Token,
Address: fixture.baseURL,
Address: address,
Labels: []string{rawLabel},
Ephemeral: regResp.Runner.Ephemeral,
}
cli := client.New(fixture.baseURL, cfg.Runner.TLSConfig(), reg.UUID, reg.Token, config.RequestTimeout, cfg.Runner.ExtraHeaders)
cli := client.New(address, cfg.Runner.TLSConfig(), reg.UUID, reg.Token, config.RequestTimeout, cfg.Runner.ExtraHeaders)
runner := run.NewRunner(cfg, reg, cli)
if _, err := runner.Declare(ctx, labelNames); err != nil {
+119
View File
@@ -0,0 +1,119 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build e2e
package e2e
import (
"context"
"strconv"
"strings"
"testing"
"time"
)
// testLabelRouting checks that each job reaches the runner owning its label and that a
// job with no matching runner stays queued.
func testLabelRouting(t *testing.T) {
t.Parallel()
ctx := t.Context()
api, repo := newScenario(t)
startRunner(t, repo, "e2e-route-a", runnerOptions{host: true, name: "-a"})
startRunner(t, repo, "e2e-route-b", runnerOptions{host: true, name: "-b"})
pushWorkflow(t, api, repo, "route.yml")
wfRun := waitForRun(t, api, repo)
ctx, cancel := context.WithTimeout(ctx, runTimeout)
defer cancel()
var jobs map[string]ActionJob
for {
list, err := api.Jobs(ctx, repo, wfRun.ID)
if err != nil {
t.Fatalf("list jobs: %v", err)
}
jobs = map[string]ActionJob{}
for _, job := range list {
jobs[job.Name] = job
}
if jobs["a"].Status == "completed" && jobs["b"].Status == "completed" {
break
}
select {
case <-ctx.Done():
t.Fatalf("jobs a and b did not complete: %+v", list)
case <-time.After(pollInterval):
}
}
for name, suffix := range map[string]string{"a": "-a", "b": "-b"} {
if got := jobs[name].RunnerName; !strings.HasSuffix(got, suffix) {
t.Errorf("job %s ran on runner %q, want the one named *%s", name, got, suffix)
}
if jobs[name].Conclusion != "success" {
t.Errorf("job %s concluded %q", name, jobs[name].Conclusion)
}
}
// a and b were picked up after unmatched was queued, so a runner that could take it would have by now
if status := jobs["unmatched"].Status; status == "in_progress" || status == "completed" {
t.Errorf("job with no matching runner is %q, want it queued", status)
}
}
// testCapacity checks that runner.capacity bounds how many jobs run at once.
func testCapacity(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
capacity int
concurrent bool
}{
{"serial", 1, false},
{"parallel", 2, true},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
api, repo := newScenario(t)
gate, release := newGate(t)
if err := api.CreateVariable(t.Context(), repo, "E2E_GATE", gate); err != nil {
t.Fatalf("create gate variable: %v", err)
}
startRunner(t, repo, "e2e-capacity", runnerOptions{capacity: tc.capacity})
pushWorkflow(t, api, repo, "capacity.yml")
wfRun := waitForRun(t, api, repo)
waitForRunningJobLog(t, api, repo, wfRun.ID, "span-start=", tc.capacity)
release()
requireSuccess(t, api, repo, wfRun.ID)
jobs, err := api.Jobs(t.Context(), repo, wfRun.ID)
if err != nil || len(jobs) != 2 {
t.Fatalf("list jobs: %v (%d jobs)", err, len(jobs))
}
var start, end [2]int64
for i, job := range jobs {
logs, err := api.JobLogs(t.Context(), repo, job.ID)
if err != nil {
t.Fatalf("job logs: %v", err)
}
start[i] = spanValue(t, logs, "span-start=")
end[i] = spanValue(t, logs, "span-end=")
}
overlap := max(start[0], start[1]) < min(end[0], end[1])
if overlap != tc.concurrent {
t.Errorf("capacity %d: jobs overlapped=%v, want %v (starts %v ends %v)", tc.capacity, overlap, tc.concurrent, start, end)
}
})
}
}
func spanValue(t *testing.T, logs, prefix string) int64 {
t.Helper()
row := commandRow(logs, prefix)
n, err := strconv.ParseInt(strings.TrimPrefix(row, prefix), 10, 64)
if err != nil {
t.Fatalf("no %q value in logs:\n%s", prefix, logs)
}
return n
}
+17
View File
@@ -0,0 +1,17 @@
name: capacity
on: push
jobs:
one:
runs-on: e2e-capacity
steps:
- run: |
echo "span-start=$(date +%s%N)"
until curl -fs "${{ vars.E2E_GATE }}" > /dev/null; do sleep 0.1; done
echo "span-end=$(date +%s%N)"
two:
runs-on: e2e-capacity
steps:
- run: |
echo "span-start=$(date +%s%N)"
until curl -fs "${{ vars.E2E_GATE }}" > /dev/null; do sleep 0.1; done
echo "span-end=$(date +%s%N)"
+7
View File
@@ -0,0 +1,7 @@
name: cli
on: push
jobs:
hello:
runs-on: e2e-cli
steps:
- run: echo e2e-cli-job-ran
+7
View File
@@ -0,0 +1,7 @@
name: cli-labels
on: push
jobs:
hello:
runs-on: e2e-override
steps:
- run: echo e2e-override-ran
+11
View File
@@ -0,0 +1,11 @@
name: cli-once
on: push
jobs:
first:
runs-on: e2e-once
steps:
- run: echo e2e-once-first
second:
runs-on: e2e-once
steps:
- run: echo e2e-once-second
+10
View File
@@ -0,0 +1,10 @@
name: cli-shutdown
on: push
jobs:
slow:
runs-on: e2e-shutdown
steps:
- run: |
echo e2e-shutdown-started
until curl -fs "${{ vars.E2E_GATE }}" > /dev/null; do sleep 0.1; done
echo e2e-shutdown-finished
+15
View File
@@ -0,0 +1,15 @@
on: push
jobs:
roundtrip:
runs-on: ubuntu-latest
steps:
- run: mkdir -p out && echo exec-payload > out/data.txt
- uses: actions/upload-artifact@v7
with:
name: exec-artifact
path: out/data.txt
- uses: actions/download-artifact@v8
with:
name: exec-artifact
path: got
- run: grep -q exec-payload got/data.txt
+23
View File
@@ -0,0 +1,23 @@
name: failure
on: push
jobs:
fail:
runs-on: ubuntu-latest
steps:
- run: echo e2e-before-fail
- run: exit 3
- if: success()
run: echo e2e-skipped-step
- if: failure()
run: echo e2e-failure-cleanup
tolerated:
runs-on: ubuntu-latest
steps:
- continue-on-error: true
run: exit 1
- run: echo e2e-tolerated-next
dependent:
needs: fail
runs-on: ubuntu-latest
steps:
- run: echo dependent
+10
View File
@@ -0,0 +1,10 @@
name: host
on: push
jobs:
host:
runs-on: e2e-host
steps:
- run: |
test "$(hostname)" = "${{ vars.E2E_HOSTNAME }}"
test -n "$GITHUB_WORKSPACE"
echo e2e-host-job-ran
+18
View File
@@ -0,0 +1,18 @@
name: reconnect
on:
push:
paths:
- .gitea/workflows/reconnect.yml
jobs:
slow:
runs-on: e2e-reconnect
steps:
- run: |
echo e2e-before-outage
i=0
until curl -fs "${{ vars.E2E_GATE }}" > /dev/null; do
i=$((i + 1))
echo "tick-$i"
sleep 0.2
done
echo "ticks=$i"
+10
View File
@@ -0,0 +1,10 @@
name: reconnect-idle
on:
push:
paths:
- .gitea/workflows/reconnect_idle.yml
jobs:
hello:
runs-on: e2e-reconnect
steps:
- run: echo e2e-idle-recovered
+20
View File
@@ -0,0 +1,20 @@
name: reusable-callee
on:
workflow_call:
inputs:
who:
type: string
required: true
outputs:
greeting:
value: ${{ jobs.say.outputs.greeting }}
jobs:
say:
runs-on: ubuntu-latest
outputs:
greeting: ${{ steps.greet.outputs.greeting }}
steps:
- id: greet
run: |
echo "greeting=hello-${{ inputs.who }}" >> "$GITHUB_OUTPUT"
echo e2e-reusable-ran
+12
View File
@@ -0,0 +1,12 @@
name: reusable-caller
on: push
jobs:
call:
uses: ./.gitea/workflows/reusable_callee.yml
with:
who: e2e
verify:
needs: call
runs-on: ubuntu-latest
steps:
- run: test "${{ needs.call.outputs.greeting }}" = "hello-e2e"
+15
View File
@@ -0,0 +1,15 @@
name: route
on: push
jobs:
a:
runs-on: e2e-route-a
steps:
- run: echo route-a
b:
runs-on: e2e-route-b
steps:
- run: echo route-b
unmatched:
runs-on: e2e-route-none
steps:
- run: echo route-none
+9
View File
@@ -0,0 +1,9 @@
name: timeout
on: push
jobs:
timeout:
runs-on: e2e-timeout
steps:
- run: |
echo e2e-timeout-started
sleep 300
+4 -4
View File
@@ -5,9 +5,9 @@ go 1.27
toolchain go1.27.3
require (
connectrpc.com/connect/v2 v2.0.1
gitea.dev/actionslib v1.5.0
github.com/bmatcuk/doublestar/v4 v4.10.3
connectrpc.com/connect/v2 v2.0.2
gitea.dev/actionslib v1.5.1
github.com/bmatcuk/doublestar/v4 v4.10.4
github.com/coder/websocket v1.8.15
github.com/containerd/errdefs v1.0.0
github.com/creack/pty v1.1.24
@@ -33,7 +33,7 @@ require (
go.opentelemetry.io/otel/trace v1.47.0
go.opentelemetry.io/proto/otlp v1.11.1
go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525
golang.org/x/net v0.61.0
golang.org/x/net v0.62.0
golang.org/x/sync v0.24.0
golang.org/x/sys v0.49.0
golang.org/x/term v0.47.0
+8 -8
View File
@@ -1,13 +1,13 @@
connectrpc.com/connect/v2 v2.0.1 h1:WTAycxe87UqxSM5oxS30d1dalZ+C5xz8Ec8O6zVYkHQ=
connectrpc.com/connect/v2 v2.0.1/go.mod h1:N32OLrstcPUzkykFynnefFro3+xzSdOrZlK2jHmS/xo=
gitea.dev/actionslib v1.5.0 h1:dC3v2HTWbbvyNBSdg2W3kd4WNC/p+FnZaKpl1MdzM00=
gitea.dev/actionslib v1.5.0/go.mod h1:C9wmOFgDBCXyRvg4A3KTX8Vszs8v9+elH4QLGVDolqU=
connectrpc.com/connect/v2 v2.0.2 h1:PoPDmOlhc46elsuCdKCKTxjNBb3HJlKF+s/sP2oucPw=
connectrpc.com/connect/v2 v2.0.2/go.mod h1:N32OLrstcPUzkykFynnefFro3+xzSdOrZlK2jHmS/xo=
gitea.dev/actionslib v1.5.1 h1:QVOaENNWx4EFRlZvrWmMTKaKLbaqxdlhWLtsI4kf2v8=
gitea.dev/actionslib v1.5.1/go.mod h1:C9wmOFgDBCXyRvg4A3KTX8Vszs8v9+elH4QLGVDolqU=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bmatcuk/doublestar/v4 v4.10.3 h1:1UIv1+fqH3++ViVkpnaPTB+Pi1UHycnPFAq14zgK+gI=
github.com/bmatcuk/doublestar/v4 v4.10.3/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/bmatcuk/doublestar/v4 v4.10.4 h1:oTKkRmRuOsKMqFe1AQazd9gGBFNEpOmgKp/xzB97dnA=
github.com/bmatcuk/doublestar/v4 v4.10.4/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/coder/websocket v1.8.15 h1:6B2JPeOGlpff2Uz6vOEH1Vzpi0iUz20A+lPVhPHtNUA=
@@ -111,8 +111,8 @@ go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525 h1:FtzHH3s777Nx50YcPAcodAPm1mNakc228nMYbr3f+90=
go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw=
golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/net v0.62.0 h1:18ELalfM5+XNRwzM2nYqXsZQ9ByS1uhj2O/U7+5jqRY=
golang.org/x/net v0.62.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA=
golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
+3
View File
@@ -58,6 +58,9 @@ func Execute(ctx context.Context) {
// ./gitea-runner bug-report
rootCmd.AddCommand(loadBugReportCmd())
// ./gitea-runner kubernetes-transfer, run inside Kubernetes job pods
rootCmd.AddCommand(loadKubernetesTransferCmd())
// ./gitea-runner config
rootCmd.AddCommand(loadConfigCmd(&configFile))
+19 -22
View File
@@ -7,12 +7,11 @@ package cmd
import (
"cmp"
"context"
"crypto/rand"
"encoding/hex"
"encoding/json/v2"
"errors"
"fmt"
"maps"
"net"
"os"
"path/filepath"
"slices"
@@ -481,24 +480,7 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
execArgs.artifactServerPath = tempDir
}
// Register ACTIONS_RUNTIME_TOKEN against local cache server
env := execArgs.LoadEnvs()
const actionsRuntimeTokenEnvName = "ACTIONS_RUNTIME_TOKEN"
actionsRuntimeToken := env[actionsRuntimeTokenEnvName]
if actionsRuntimeToken == "" {
actionsRuntimeToken = os.Getenv(actionsRuntimeTokenEnvName)
}
if actionsRuntimeToken == "" {
tmpBranch := make([]byte, 12)
if _, err := rand.Read(tmpBranch); err != nil {
actionsRuntimeToken = "token"
} else {
actionsRuntimeToken = hex.EncodeToString(tmpBranch)
}
env[actionsRuntimeTokenEnvName] = actionsRuntimeToken
os.Setenv(actionsRuntimeTokenEnvName, actionsRuntimeToken)
}
handler.RegisterJob(actionsRuntimeToken, artifactcache.JobCredential{Repo: "__local/__exec"})
// no service aliases: exec builds one config for the whole plan
run.BypassProxyForDockerHost(os.Getenv("DOCKER_HOST"))
@@ -517,7 +499,22 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
}
config.Env["ACT_EXEC"] = "true"
config.Secrets[actionsRuntimeTokenEnvName] = actionsRuntimeToken
inheritedToken := cmp.Or(env["ACTIONS_RUNTIME_TOKEN"], os.Getenv("ACTIONS_RUNTIME_TOKEN"))
config.RuntimeTokens = map[*model.Workflow]string{}
sameFile := map[string]int{}
for _, stage := range plan.Stages {
for _, run := range stage.Runs {
if config.RuntimeTokens[run.Workflow] != "" {
continue
}
sameFile[run.Workflow.File]++
workflowRun := fmt.Sprintf("%s/%s/%d", cmp.Or(env["GITHUB_RUN_ID"], "1"), run.Workflow.File, sameFile[run.Workflow.File]) // each workflow its own run as on GitHub, also when file names repeat across directories
token := cmp.Or(inheritedToken, artifacts.LocalRuntimeToken(workflowRun, maxLifetime))
config.RuntimeTokens[run.Workflow] = token
config.ExtraMasks = append(config.ExtraMasks, token)
handler.RegisterJob(token, artifactcache.JobCredential{Repo: "__local/__exec", Results: "http://" + net.JoinHostPort(execArgs.artifactServerAddr, execArgs.artifactServerPort)})
}
}
config.Token = cmp.Or(config.Secrets["GITEA_TOKEN"], config.Secrets["GITHUB_TOKEN"])
@@ -580,9 +577,9 @@ func loadExecCmd(ctx context.Context) *cobra.Command {
execCmd.Flags().StringArrayVarP(&execArg.containerCapAdd, "container-cap-add", "", []string{}, "kernel capabilities to add to the workflow containers (e.g. --container-cap-add SYS_PTRACE)")
execCmd.Flags().StringArrayVarP(&execArg.containerCapDrop, "container-cap-drop", "", []string{}, "kernel capabilities to remove from the workflow containers (e.g. --container-cap-drop SYS_PTRACE)")
execCmd.Flags().StringVarP(&execArg.containerOptions, "container-opts", "", "", "container options")
execCmd.PersistentFlags().StringVarP(&execArg.artifactServerPath, "artifact-server-path", "", ".", "Defines the path where the artifact server stores uploads and retrieves downloads from. If not specified the artifact server will not start.")
execCmd.PersistentFlags().StringVarP(&execArg.artifactServerPath, "artifact-server-path", "", "", "Defines the path where the artifact server stores uploads and retrieves downloads from, outside the workspace that jobs copy. Defaults to a temporary directory removed on exit.")
execCmd.PersistentFlags().StringVarP(&execArg.artifactServerAddr, "artifact-server-addr", "", "", "Defines the address where the artifact server listens")
execCmd.PersistentFlags().StringVarP(&execArg.artifactServerPort, "artifact-server-port", "", "34567", "Defines the port where the artifact server listens (will only bind to localhost).")
execCmd.PersistentFlags().StringVarP(&execArg.artifactServerPort, "artifact-server-port", "", "34567", "Defines the port where the artifact server listens")
execCmd.PersistentFlags().StringVarP(&execArg.defaultActionsURL, "default-actions-url", "", "https://github.com", "Defines the default url of action instance.")
execCmd.PersistentFlags().BoolVarP(&execArg.noSkipCheckout, "no-skip-checkout", "", false, "Do not skip actions/checkout")
execCmd.PersistentFlags().BoolVarP(&execArg.debug, "debug", "d", false, "enable debug log")
+35
View File
@@ -0,0 +1,35 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"gitea.com/gitea/runner/act/container"
"github.com/spf13/cobra"
)
// loadKubernetesTransferCmd is what job pods run when kubernetes.transfer is on: the init container installs the binary, the job container fetches with it.
func loadKubernetesTransferCmd() *cobra.Command {
transferCmd := &cobra.Command{
Use: container.KubernetesTransferCommand,
Short: "Copy files from the runner into a Kubernetes job pod",
Hidden: true,
}
transferCmd.AddCommand(&cobra.Command{
Use: "install <path>",
Short: "Copy this binary to path",
Args: cobra.ExactArgs(1),
RunE: func(_cmd *cobra.Command, args []string) error {
return container.InstallKubernetesTransferHelper(args[0])
},
}, &cobra.Command{
Use: "fetch <dir>",
Short: "Extract the tar stream the transfer request on stdin points to into dir",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
return container.FetchKubernetesTransfer(cmd.Context(), cmd.InOrStdin(), args[0])
},
})
return transferCmd
}
+39 -14
View File
@@ -415,6 +415,11 @@ func (r *Runner) kubernetesOptions(runsOn []string) container.KubernetesOptions
PodTemplates: []map[string]any{r.cfg.Kubernetes.PodTemplate},
RunnerUUID: r.uuid,
}
if transfer := r.cfg.Kubernetes.Transfer; transfer.Mode == config.KubernetesTransferModeAuto || transfer.Mode == config.KubernetesTransferModeDirect {
options.Transfer = &container.KubernetesTransfer{
Host: transfer.Host, Port: transfer.Port, Image: transfer.Image, Fallback: transfer.Mode == config.KubernetesTransferModeAuto,
}
}
names := r.labels.Names()
for index, name := range runsOn {
if template, ok := r.cfg.Kubernetes.PodTemplates[name]; ok && slices.Contains(names, name) && !slices.Contains(runsOn[:index], name) {
@@ -533,7 +538,7 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
// is removed when the task finishes, so a leaked token stops working as
// soon as the job ends rather than remaining valid for the runner's
// lifetime.
revokeCache, resultsURL := r.registerCacheForTask(giteaRuntimeToken, preset.Repository, cacheURL, reporter)
revokeCache, resultsURL := r.registerCacheForTask(giteaRuntimeToken, cacheCredential(preset, cacheURL), reporter)
defer revokeCache()
eventJSON, err := json.Marshal(preset.Event)
@@ -690,34 +695,54 @@ func (r *Runner) instanceOutOfReach(instance string) bool {
//
// Safe with an empty token (older Gitea did not issue one).
// It also returns the URL to advertise as ACTIONS_RESULTS_URL.
func (r *Runner) registerCacheForTask(token, repo, publicURL string, reporter *report.Reporter) (func(), string) {
func (r *Runner) registerCacheForTask(token string, cred artifactcache.JobCredential, reporter *report.Reporter) (func(), string) {
if token == "" || r.cacheHandler == nil {
return func() {}, ""
}
cred := artifactcache.JobCredential{
Repo: repo,
PublicURL: publicURL,
Results: r.envs["ACTIONS_RESULTS_URL"], // the instance as the job would reach it
InsecureTLS: r.cfg.Runner.Insecure,
}
cred.Results = r.envs["ACTIONS_RESULTS_URL"] // the instance as the job would reach it
cred.InsecureTLS = r.cfg.Runner.Insecure
revoke := r.cacheHandler.RegisterJob(token, cred)
if r.cfg.Cache.ExternalServer != "" {
revokeLocal, revokeExternal := revoke, r.registerExternalCacheJob(token, repo, publicURL, reporter)
revokeLocal, revokeExternal := revoke, r.registerExternalCacheJob(token, cred, reporter)
revoke = func() { revokeLocal(); revokeExternal() }
}
return revoke, r.cacheHandler.ResultsURL(cred)
}
// cacheCredential scopes the job's cache entries by ref, as GitHub does.
func cacheCredential(preset *model.GithubContext, publicURL string) artifactcache.JobCredential {
ref := preset.Ref
if strings.HasPrefix(preset.EventName, "pull_request") && preset.EventName != "pull_request_target" {
number, _ := preset.Event["number"].(float64)
ref = fmt.Sprintf("refs/pull/%d/head", int64(number)) // review events take github.ref from the reviewer's commit_id
}
return artifactcache.JobCredential{
Repo: preset.Repository,
Ref: ref,
BaseRef: branchRef(preset.BaseRef),
DefaultRef: branchRef(model.AsString(model.NestedMapLookup(preset.Event, "repository", "default_branch"))),
PublicURL: publicURL,
}
}
func branchRef(branch string) string {
if branch == "" {
return ""
}
return "refs/heads/" + branch
}
// registerExternalCacheJob POSTs to the remote cache-server's control-plane.
// Failures are logged but not fatal: if registration fails, the cache will
// 401 the job's requests — better than failing the whole task for a cache
// outage. The warning is mirrored to the job log so users can see why their
// cache calls 401, instead of having to read the runner daemon's stderr.
func (r *Runner) registerExternalCacheJob(token, repo, publicURL string, reporter *report.Reporter) func() {
func (r *Runner) registerExternalCacheJob(token string, cred artifactcache.JobCredential, reporter *report.Reporter) func() {
base := strings.TrimRight(r.cfg.Cache.ExternalServer, "/")
if err := postInternalCache(base+"/_internal/register", r.cfg.Cache.ExternalSecret, map[string]any{
"token": token, "repo": repo, "public_url": publicURL,
}); err != nil {
if err := postInternalCache(base+"/_internal/register", r.cfg.Cache.ExternalSecret, struct {
Token string `json:"token"`
artifactcache.JobCredential
}{token, cred}); err != nil {
logging.Warnf("cache external_server register failed (%s): %v", base, err)
if reporter != nil {
reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf(
@@ -736,7 +761,7 @@ func (r *Runner) registerExternalCacheJob(token, repo, publicURL string, reporte
}
}
func postInternalCache(url, secret string, body map[string]any) error {
func postInternalCache(url, secret string, body any) error {
buf, err := json.Marshal(body)
if err != nil {
return err
+28 -3
View File
@@ -17,6 +17,7 @@ import (
clientmocks "gitea.com/gitea/runner/internal/pkg/client/mocks"
"gitea.com/gitea/runner/internal/pkg/config"
"gitea.dev/actionslib/pkg/model"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -24,7 +25,7 @@ import (
func TestRunner_registerCacheForTask_NoOps(t *testing.T) {
for token, handler := range map[string]*artifactcache.Handler{"tok": nil, "": {}} {
r := &Runner{cacheHandler: handler}
unregister, _ := r.registerCacheForTask(token, "owner/repo", "", nil)
unregister, _ := r.registerCacheForTask(token, artifactcache.JobCredential{Repo: "owner/repo"}, nil)
require.NotNil(t, unregister, token)
unregister()
}
@@ -39,7 +40,7 @@ func TestRunner_CacheFullFlow_MatchesToolkit(t *testing.T) {
r := &Runner{cfg: &config.Config{}, cacheHandler: handler, envs: map[string]string{"ACTIONS_RESULTS_URL": "https://gitea.example"}}
const publicURL = "http://a1b2c3d4e5f6:8088"
token := "full-flow-token"
unregister, resultsURL := r.registerCacheForTask(token, "owner/repo", publicURL, nil)
unregister, resultsURL := r.registerCacheForTask(token, artifactcache.JobCredential{Repo: "owner/repo", PublicURL: publicURL}, nil)
assert.Equal(t, publicURL, resultsURL)
base := handler.ExternalURL() + "/_apis/artifactcache"
@@ -112,6 +113,24 @@ func decodeJSON(resp *http.Response, v any) error {
return json.UnmarshalRead(resp.Body, v)
}
func TestCacheCredential(t *testing.T) {
repository := map[string]any{"default_branch": "main"}
review := &model.GithubContext{
Repository: "owner/repo", EventName: "pull_request_review", Ref: "refs/heads/main", BaseRef: "release",
Event: map[string]any{"repository": repository, "number": float64(3)},
}
assert.Equal(t, artifactcache.JobCredential{
Repo: "owner/repo", Ref: "refs/pull/3/head", BaseRef: "refs/heads/release", DefaultRef: "refs/heads/main", PublicURL: "http://cache",
}, cacheCredential(review, "http://cache"))
review.EventName, review.Ref = "pull_request_target", "refs/heads/release"
assert.Equal(t, "refs/heads/release", cacheCredential(review, "").Ref)
assert.Equal(t, artifactcache.JobCredential{
Repo: "owner/repo", Ref: "refs/heads/feature", DefaultRef: "refs/heads/main",
}, cacheCredential(&model.GithubContext{Repository: "owner/repo", Ref: "refs/heads/feature", Event: map[string]any{"repository": repository}}, ""))
}
func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
const secret = "shared-secret-for-tests"
remote, err := artifactcache.StartHandler(artifactcache.Options{Dir: t.TempDir(), OutboundIP: "127.0.0.1", InternalSecret: secret})
@@ -151,7 +170,7 @@ func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
require.Equal(t, http.StatusUnauthorized, probe(),
"token must be unknown to the remote server before registration")
unregister, resultsURL := r.registerCacheForTask(token, repo, publicURL, nil)
unregister, resultsURL := r.registerCacheForTask(token, artifactcache.JobCredential{Repo: repo, Ref: "refs/heads/main", PublicURL: publicURL}, nil)
require.NotEqual(t, http.StatusUnauthorized, probe(),
"token must be accepted after registerCacheForTask")
@@ -194,6 +213,12 @@ func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
require.NoError(t, decodeJSON(resp, &hit))
require.True(t, strings.HasPrefix(hit.ArchiveLocation, publicURL+"/"), hit.ArchiveLocation)
unregisterOther, _ := r.registerCacheForTask("external-other-token", artifactcache.JobCredential{Repo: repo, Ref: "refs/heads/other"}, nil)
defer unregisterOther()
resp = cacheRequest(t, "external-other-token", http.MethodGet, base+"/cache?keys=ext-key&version=v", "", "", "")
resp.Body.Close()
assert.Equal(t, http.StatusNoContent, resp.StatusCode, "the remote server must scope the entry by the ref the runner sent")
dl, err := http.Get(strings.Replace(hit.ArchiveLocation, publicURL, r.cacheHandler.ExternalURL(), 1))
require.NoError(t, err)
defer dl.Body.Close()
+14 -1
View File
@@ -18,8 +18,10 @@ import (
"testing"
"time"
"gitea.com/gitea/runner/act/artifactcache"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/container"
"gitea.com/gitea/runner/act/runner"
clientmocks "gitea.com/gitea/runner/internal/pkg/client/mocks"
"gitea.com/gitea/runner/internal/pkg/config"
@@ -275,6 +277,17 @@ func TestRunnerKubernetesOptionsLayerTheLabelPodTemplatesInRunsOnOrder(t *testin
require.Equal(t, []map[string]any{{"base": true}, {"large": true}, {"gpu": true}}, r.kubernetesOptions([]string{"linux", "large", "other", "gpu", "large"}).PodTemplates)
}
func TestRunnerKubernetesOptionsTransfer(t *testing.T) {
kubernetesRunner := &Runner{cfg: &config.Config{}}
kubernetesRunner.cfg.Kubernetes.Transfer = config.KubernetesTransfer{Mode: config.KubernetesTransferModeAPI, Host: "10.0.0.5", Port: 4000, Image: "gitea/runner:latest"}
require.Nil(t, kubernetesRunner.kubernetesOptions(nil).Transfer)
kubernetesRunner.cfg.Kubernetes.Transfer.Mode = config.KubernetesTransferModeAuto
require.Equal(t, &container.KubernetesTransfer{Host: "10.0.0.5", Port: 4000, Image: "gitea/runner:latest", Fallback: true}, kubernetesRunner.kubernetesOptions(nil).Transfer)
kubernetesRunner.cfg.Kubernetes.Transfer.Mode = config.KubernetesTransferModeDirect
require.Equal(t, &container.KubernetesTransfer{Host: "10.0.0.5", Port: 4000, Image: "gitea/runner:latest"}, kubernetesRunner.kubernetesOptions(nil).Transfer)
}
// Proxy variables are assembled per task, because a job's service containers have to be
// reached directly and they are only known once the workflow is parsed.
func TestNewRunnerLeavesProxyToTheTask(t *testing.T) {
@@ -325,7 +338,7 @@ func TestNewRunnerCacheServiceV2(t *testing.T) {
// The registration is what makes it true: the cache server takes the results service over,
// having been told which instance to forward the artifact half to.
revoke, resultsURL := r.registerCacheForTask(token, "owner/repo", r.cacheHandler.ExternalURL(), nil)
revoke, resultsURL := r.registerCacheForTask(token, artifactcache.JobCredential{Repo: "owner/repo", PublicURL: r.cacheHandler.ExternalURL()}, nil)
defer revoke()
require.Equal(t, r.cacheHandler.ExternalURL(), resultsURL)
+12
View File
@@ -306,6 +306,18 @@ kubernetes:
#pod_template: {}
# Pod templates merged over pod_template, in runs-on order, for jobs whose runs-on has the label they are keyed by.
#pod_templates: {}
# Have job pods fetch directories, such as actions, and copies of 64 KiB or more gzipped from the runner over the pod network,
# instead of the runner streaming them through the API server. An init container copies gitea-runner from image into each job pod.
# api: through the API server (default). auto: over the pod network, through the API server for a job pod that
# cannot reach the runner. direct: over the pod network only, so a broken setup fails jobs instead of quietly
# costing API server traffic again.
#transfer:
# mode: api
# # Address and port the runner listens on and job pods reach it at, the outbound IP and a free port when empty.
# host: ""
# port: 0
# # Image with gitea-runner on its PATH, the runner pod's image when it has a single container and runs in namespace.
# image: ""
# Optional local task-admission checks. Disabled by default. When enabled, low
# disk space or a failing script pauses new task fetching; existing jobs continue.
+25
View File
@@ -210,6 +210,25 @@ type Kubernetes struct {
Namespace string `yaml:"namespace"` // Namespace of job pods, empty uses the service account's, else the kubeconfig context's, else default.
PodTemplate map[string]any `yaml:"pod_template"` // PodTemplate is a pod manifest the runner merges its own onto.
PodTemplates map[string]map[string]any `yaml:"pod_templates"` // PodTemplates are merged over PodTemplate, in runs-on order, for jobs whose runs-on has the label they are keyed by.
Transfer KubernetesTransfer `yaml:"transfer"` // Transfer has job pods fetch copied files from the runner over the pod network instead of through the API server.
}
// Kubernetes transfer modes: how files the runner copies reach job pods.
const (
KubernetesTransferModeAPI = "api" // through the API server
KubernetesTransferModeAuto = "auto" // over the pod network, through the API server for a job pod that cannot reach the runner
KubernetesTransferModeDirect = "direct" // over the pod network only, a copy that cannot reach the runner fails
)
// KubernetesTransferModes are the valid kubernetes.transfer.mode values.
var KubernetesTransferModes = []string{KubernetesTransferModeAPI, KubernetesTransferModeAuto, KubernetesTransferModeDirect}
// KubernetesTransfer configures copying files into job pods over the pod network.
type KubernetesTransfer struct {
Mode string `yaml:"mode"` // Mode is one of KubernetesTransferModes, api when empty.
Host string `yaml:"host"` // Host is the address the runner listens on and job pods reach it at, empty uses the outbound IP.
Port uint16 `yaml:"port"` // Port the runner listens on for job pods, 0 picks a free one.
Image string `yaml:"image"` // Image has gitea-runner on its PATH, empty uses the image of a single-container runner pod.
}
// Metrics represents the configuration for the Prometheus metrics endpoint.
@@ -346,6 +365,12 @@ func LoadDefault(file string) (*Config, error) {
if !slices.Contains(ToolCacheModes, cfg.Runner.ToolCacheMode) {
return nil, fmt.Errorf("invalid runner.tool_cache_mode %q: must be one of %q", cfg.Runner.ToolCacheMode, ToolCacheModes)
}
if cfg.Kubernetes.Transfer.Mode == "" {
cfg.Kubernetes.Transfer.Mode = KubernetesTransferModeAPI
}
if !slices.Contains(KubernetesTransferModes, cfg.Kubernetes.Transfer.Mode) {
return nil, fmt.Errorf("invalid kubernetes.transfer.mode %q: must be one of %q", cfg.Kubernetes.Transfer.Mode, KubernetesTransferModes)
}
if cfg.Runner.LogReportLevel != "" {
if _, err := logging.ParseLevel(cfg.Runner.LogReportLevel); err != nil {
return nil, fmt.Errorf("invalid runner.log_report_level: %w", err)
+13
View File
@@ -140,6 +140,19 @@ func TestLoadDefault_ToolCacheMode(t *testing.T) {
assert.Contains(t, err.Error(), "tool_cache_mode")
}
func TestLoadDefault_KubernetesTransferMode(t *testing.T) {
cfg, err := LoadDefault(write(t, "kubernetes:\n transfer:\n host: 10.0.0.5\n"))
require.NoError(t, err)
assert.Equal(t, KubernetesTransferModeAPI, cfg.Kubernetes.Transfer.Mode, "copies stay on the API server unless asked")
cfg, err = LoadDefault(write(t, "kubernetes:\n transfer:\n mode: direct\n"))
require.NoError(t, err)
assert.Equal(t, KubernetesTransferModeDirect, cfg.Kubernetes.Transfer.Mode)
_, err = LoadDefault(write(t, "kubernetes:\n transfer:\n mode: fast\n"))
require.ErrorContains(t, err, "kubernetes.transfer.mode")
}
func TestLoadDefault_RejectsInvalidLogReportLevel(t *testing.T) {
_, err := LoadDefault(write(t, "runner:\n log_report_level: loud\n"))
require.ErrorContains(t, err, "runner.log_report_level")
+1 -1
View File
@@ -104,7 +104,7 @@ func TestTreeKillCancelBeforeCaptureKillsTree(t *testing.T) {
require.NoError(t, err)
t.Cleanup(func() { _ = stdin.Close() })
t.Cleanup(func() { _ = input.Close() })
cmd := exec.CommandContext(t.Context(), "/bin/sh", "-c", "cat <&0 & echo $!; read value")
cmd := exec.CommandContext(t.Context(), "/bin/sh", "-c", "exec 3<&0; cat <&3 & echo $!; read value")
cmd.Stdin = stdin
cmd.SysProcAttr = SysProcAttr("", false)
output, err := cmd.StdoutPipe()
+1 -1
View File
@@ -1,3 +1,3 @@
#!/usr/bin/env bash
exec s6-notifyoncheck -n 0 dockerd-entrypoint.sh
exec s6-notifyoncheck -d -n 0 dockerd-entrypoint.sh
+13 -1
View File
@@ -23,4 +23,16 @@ for pull in "${pulls[@]}"; do
done
((result == 0)) || exit "$result"
exec "${GO:-go}" test -tags e2e -count=1 -parallel "$E2E_CONCURRENCY" -timeout 20m -v ./e2e/...
args=(test -tags e2e -count=1 -parallel "$E2E_CONCURRENCY" -timeout 20m -v "$@" ./e2e/...)
if [[ -f /.dockerenv ]]; then
exec "${GO:-go}" "${args[@]}"
fi
# Run like CI: in a Linux container on a user-defined network, which every container of the suite joins.
network="gitea-runner-e2e-$$"
docker network create "$network" > /dev/null
trap 'docker network rm "$network" > /dev/null || true' EXIT
docker run --rm --network "$network" -w "$PWD" -v "$PWD:$PWD" -v /var/run/docker.sock:/var/run/docker.sock \
-v gitea-runner-e2e-gomod:/go/pkg/mod -v gitea-runner-e2e-gocache:/root/.cache/go-build \
-e E2E_CONCURRENCY -e E2E_JOB_IMAGE -e E2E_GITEA_IMAGE -e E2E_K3S_IMAGE -e SERVICE_IMAGE \
"golang:$(sed -n 's/^toolchain go//p' go.mod)" go "${args[@]}"