Compare commits

...
6 Commits
Author SHA1 Message Date
261ab374ce enhance(cache): scope cache entries by ref like GitHub (#1361)
Cache entries were only scoped by repository, so an entry a pull request saved could be restored on the default branch, for example through a `restore-keys` prefix.

Entries now carry the ref of the job that saved them. As on GitHub ([Restrictions for accessing a cache](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#restrictions-for-accessing-a-cache)), a job restores entries of its own ref first, then of its pull request's base branch, then of the default branch. Pull request events are scoped by their pull request number.

Entries saved before this change are no longer restored and expire through `cache.retention`, so caches start cold once after the upgrade. Runners sharing a cache, and an external `cache-server`, should be upgraded together.

Co-authored-by: silverwind <[email protected]>
Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1361
Reviewed-by: silverwind <[email protected]>
Co-authored-by: JorgeDLMV <[email protected]>
2026-10-30 22:34:45 +00:00
KenanZhuandsilverwind ca8e9c3c64 enhance: add GITEA_* aliases for built-in workflow environment variables (#1358)
Closes https://gitea.com/gitea/runner/issues/1317, which makes https://github.com/go-gitea/gitea/issues/39813 redundant as the runner sets these variables.

Every built-in `GITHUB_*` environment variable the runner sets is now also available as `GITEA_*` with the same value, so Gitea-native workflows don't have to rely on GitHub naming. The `GITEA_*` names mirror the `GITHUB_*` ones after their last write, so they stay equal and are unset together.

Workflow, job and runner `envs` values for these `GITEA_*` names are now overwritten, the same as for `GITHUB_*`.

---------

Co-authored-by: silverwind <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1358
Reviewed-by: silverwind <[email protected]>
Co-authored-by: KenanZhu <[email protected]>
2026-10-30 21:21:02 +00:00
silverwindandbircni d8d4a4657d fix: scope the action identity to the step executing an action (#1359)
`github.action_repository`/`github.action_ref` and their env vars now belong to the step executing an action, like on GitHub: set for remote actions, empty for `run`, `docker://` and local steps. Before, a local action nested in a remote composite resolved `$/` against the remote action instead of the workflow's repository.

Updates actionslib to v1.5.1, which accepts `$/` as the repository root action.

Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1359
Reviewed-by: bircni <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-29 17:41:27 +00:00
silverwind 5fd2f82735 test: fix race in TestTreeKillCancelBeforeCaptureKillsTree (#1360)
`TestTreeKillCancelBeforeCaptureKillsTree` fails intermittently on CI, e.g. https://gitea.com/gitea/runner/actions/runs/1155856.

Dash, the Ubuntu `/bin/sh`, points a background job's stdin at `/dev/null` before applying its redirections, so `cat <&0 &` read EOF and exited at once and the liveness check raced it. The pipe is now handed over on fd 3, which dash leaves alone.

Introduced in https://gitea.com/gitea/runner/pulls/1338

Reviewed-on: https://gitea.com/gitea/runner/pulls/1360
Reviewed-by: Zettat123 <[email protected]>
2026-10-29 16:46:53 +00:00
silverwind c6567296a0 fix: stop the docker proxy probe from leaking its container (#1357)
When the probe's deadline expired during `ContainerCreate`, dockerd still created the container but the probe never got its ID, leaving it in `Created` with its image's anonymous volumes. A slow daemon leaked one per job.

The probe now runs detached with its own timeout, so the create is never cancelled and the container is always removed. Callers still fall back after their own deadline, and the runner waits for pending probes before exiting.

Fixes: https://gitea.com/gitea/runner/issues/1353
Reviewed-on: https://gitea.com/gitea/runner/pulls/1357
Reviewed-by: Zettat123 <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-29 15:53:32 +00:00
Renovate Bot e3607b576e fix(deps): update module golang.org/x/net to v0.62.0 [security] (#1356)
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [golang.org/x/net](https://pkg.go.dev/golang.org/x/net) | [`v0.61.0` → `v0.62.0`](https://cs.opensource.google/go/x/net/+/refs/tags/v0.61.0...refs/tags/v0.62.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fnet/v0.62.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fnet/v0.61.0/v0.62.0?slim=true) |

---

### Avoid quadratic complexity when coalescing text nodes in html in golang.org/x/net
[CVE-2026-78661](https://nvd.nist.gov/vuln/detail/CVE-2026-78661) / [GO-2026-6860](https://pkg.go.dev/vuln/GO-2026-6860)

<details>
<summary>More information</summary>

#### Details
When coalescing adjacent text nodes in addText and fosterParent, appending to an existing TextNode's Data via string concatenation (n.Data += text) reallocated and copied the accumulated string on every token. Inputs containing many short text runs separated by ignored end tags (such as "a</x>") caused O(n^2) allocations and byte copies.

Accumulate coalesced text in per-node byte slices during parsing and materialize Data strings once at the end of parse.

#### Severity
Unknown

#### References
- [https://go.dev/cl/849365](https://go.dev/cl/849365)
- [https://go.dev/issue/82224](https://go.dev/issue/82224)
- [https://groups.google.com/g/golang-announce/c/sqrs2vTx_Xs](https://groups.google.com/g/golang-announce/c/sqrs2vTx_Xs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6860) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Check all ancestor namespace transitions when rendering raw text in html in golang.org/x/net
[CVE-2026-97045](https://nvd.nist.gov/vuln/detail/CVE-2026-97045) / [GO-2026-6866](https://pkg.go.dev/vuln/GO-2026-6866)

<details>
<summary>More information</summary>

#### Details
In childTextNodesAreLiteral, walking up the tree previously stopped at the nearest foreign ancestor once an HTML integration point was found. However, that integration point may itself be a descendant of an element that was foster-parented into a foreign namespace (such as <mglyph> or <malignmark> fostered under <mtext>), causing a parser to resolve the subtree in a different foreign namespace upon re-parsing the rendered output.

Continue walking the full ancestor chain and verify that every transition from a non-empty namespace occurs at an HTML integration point before emitting literal text.

#### Severity
Unknown

#### References
- [https://go.dev/cl/855121](https://go.dev/cl/855121)
- [https://go.dev/issue/82622](https://go.dev/issue/82622)
- [https://groups.google.com/g/golang-announce/c/sqrs2vTx_Xs](https://groups.google.com/g/golang-announce/c/sqrs2vTx_Xs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6866) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xODEuNyIsInVwZGF0ZWRJblZlciI6IjQ0LjE4MS43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1356
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
2026-10-28 06:33:58 +00:00
37 changed files with 596 additions and 314 deletions

No files matched your search

+2
View File
@@ -303,6 +303,8 @@ Use `uses: builtin:checkout` to check out a repository without downloading `acti
Each runner starts its own cache server, so runners do not share cached entries. A runner in a container on the jobs' Docker daemon attaches itself to each job network, so jobs reach its cache without extra setup. Each runner starts its own cache server, so runners do not share cached entries. A runner in a container on the jobs' Docker daemon attaches itself to each job network, so jobs reach its cache without extra setup.
As on GitHub, a job restores caches saved for its own ref, its pull request's base branch and the default branch, so a cache saved by a pull request is only restored by that pull request.
**Sharing a cache between runners** **Sharing a cache between runners**
Run one `gitea-runner cache-server` and point every runner at it, using the same secret everywhere, for example from `openssl rand -hex 32`: Run one `gitea-runner cache-server` and point every runner at it, using the same secret everywhere, for example from `openssl rand -hex 32`:
-2
View File
@@ -6,7 +6,5 @@
// //
// Inspired by https://github.com/sp-ricard-valverde/github-act-cache-server // Inspired by https://github.com/sp-ricard-valverde/github-act-cache-server
// //
// TODO: Authorization
// TODO: Restrictions for accessing a cache, see https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache
// TODO: Force deleting cache entries, see https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries // TODO: Force deleting cache entries, see https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries
package artifactcache package artifactcache
+65 -47
View File
@@ -59,6 +59,10 @@ type credKey struct{}
type JobCredential struct { type JobCredential struct {
Repo string `json:"repo"` Repo string `json:"repo"`
Ref string `json:"ref"`
BaseRef string `json:"base_ref"`
DefaultRef string `json:"default_ref"`
// Results is the instance whose artifact service this server forwards for the job, and // Results is the instance whose artifact service this server forwards for the job, and
// InsecureTLS how the runner reaches it; see results.go. // InsecureTLS how the runner reaches it; see results.go.
Results string `json:"-"` Results string `json:"-"`
@@ -68,6 +72,10 @@ type JobCredential struct {
PublicURL string `json:"public_url"` PublicURL string `json:"public_url"`
} }
func (cred JobCredential) owns(cache *Cache) bool {
return cache.Repo == cred.Repo && cache.Ref == cred.Ref
}
// credEntry holds a registered job's credential along with an active // credEntry holds a registered job's credential along with an active
// registration count. RegisterJob is reference-counted so that if two tasks // registration count. RegisterJob is reference-counted so that if two tasks
// briefly share an ACTIONS_RUNTIME_TOKEN — e.g. a runner that retries a task // briefly share an ACTIONS_RUNTIME_TOKEN — e.g. a runner that retries a task
@@ -653,7 +661,7 @@ func (h *Handler) find(w http.ResponseWriter, r *http.Request) {
} }
defer db.Close() defer db.Close()
cache, err := h.lookupCache(db, cred.Repo, keys, version) cache, err := h.lookupCache(db, cred, keys, version)
if err != nil { if err != nil {
h.responseJSON(w, r, 500, err) h.responseJSON(w, r, 500, err)
return return
@@ -669,26 +677,27 @@ func (h *Handler) find(w http.ResponseWriter, r *http.Request) {
}) })
} }
// lookupCache returns the entry to restore for these keys, or (nil, nil) when there is none: // lookupCache returns the entry to restore or (nil, nil), dropping matches whose blob was pruned.
// either nothing matched, or the match had lost its blob to a prune, in which case the dangling func (h *Handler) lookupCache(db *cacheDB, cred JobCredential, keys []string, version string) (*Cache, error) {
// entry is dropped on the way out. for {
func (h *Handler) lookupCache(db *cacheDB, repo string, keys []string, version string) (*Cache, error) { cache, err := findCache(db, cred, keys, version)
cache, err := findCache(db, repo, keys, version) if err != nil || cache == nil {
if err != nil || cache == nil { return nil, err
return nil, err }
ok, err := h.storage.Exist(cache.ID)
if err != nil {
return nil, err
}
if ok {
// Handing out a download URL counts as access, or eviction could drop the entry between
// this call and the GET that follows it.
h.touch(db, cache)
return cache, nil
}
if !h.deleteCache(db, cache) {
return nil, nil //nolint:nilnil // absence is not an error here
}
} }
ok, err := h.storage.Exist(cache.ID)
if err != nil {
return nil, err
}
if !ok {
h.deleteCache(db, cache)
return nil, nil //nolint:nilnil // absence is not an error here
}
// Handing out a download URL counts as access, or eviction could drop the entry between
// this call and the GET that follows it.
h.touch(db, cache)
return cache, nil
} }
// POST /_apis/artifactcache/caches // POST /_apis/artifactcache/caches
@@ -700,7 +709,7 @@ func (h *Handler) reserve(w http.ResponseWriter, r *http.Request) {
return return
} }
cache := &Cache{Repo: cred.Repo, Key: api.Key, Version: api.Version, Size: api.Size} cache := &Cache{Repo: cred.Repo, Ref: cred.Ref, Key: api.Key, Version: api.Version, Size: api.Size}
if cache.Size == 0 { if cache.Size == 0 {
cache.Size = -1 cache.Size = -1
} }
@@ -749,7 +758,7 @@ func (h *Handler) upload(w http.ResponseWriter, r *http.Request) {
return return
} }
if cache.Repo != cred.Repo { if !cred.owns(cache) {
h.responseJSON(w, r, 403, fmt.Errorf("cache %d: forbidden", id)) h.responseJSON(w, r, 403, fmt.Errorf("cache %d: forbidden", id))
return return
} }
@@ -808,7 +817,7 @@ func (h *Handler) commit(w http.ResponseWriter, r *http.Request) {
return return
} }
if cache.Repo != cred.Repo { if !cred.owns(cache) {
h.responseJSON(w, r, 403, fmt.Errorf("cache %d: forbidden", id)) h.responseJSON(w, r, 403, fmt.Errorf("cache %d: forbidden", id))
return return
} }
@@ -1071,25 +1080,35 @@ func (h *Handler) signedArtifactURL(cred JobCredential, cacheID uint64, exp time
return h.signedURL(cred, apiPath+"/artifacts", "", cacheID, exp) return h.signedURL(cred, apiPath+"/artifacts", "", cacheID, exp)
} }
// if not found, return (nil, nil) instead of an error. // findCache tries every key in one ref before the next ref, and returns (nil, nil) on a miss.
func findCache(db *cacheDB, repo string, keys []string, version string) (*Cache, error) { func findCache(db *cacheDB, cred JobCredential, keys []string, version string) (*Cache, error) {
for _, prefix := range keys { refs := []string{cred.Ref}
// if a key in the list matches exactly, don't return partial matches for _, ref := range []string{cred.BaseRef, cred.DefaultRef} {
exact, err := findExactCache(db, repo, prefix, version, true) if ref != "" && !slices.Contains(refs, ref) {
if err != nil { refs = append(refs, ref)
return nil, err
} }
if exact != nil { }
return exact, nil caches, err := db.find(func(cache *Cache) bool {
} return cache.Repo == cred.Repo && cache.Version == version && cache.Complete && slices.Contains(refs, cache.Ref) &&
caches, err := db.find(func(cache *Cache) bool { slices.ContainsFunc(keys, func(prefix string) bool { return strings.HasPrefix(cache.Key, prefix) })
return cache.Repo == repo && strings.HasPrefix(cache.Key, prefix) && cache.Version == version && cache.Complete })
}) if err != nil {
if err != nil { return nil, fmt.Errorf("find cache: %w", err)
return nil, fmt.Errorf("find cache: %w", err) }
} for _, ref := range refs {
if cache := latest(caches, (*Cache).createdAtNano); cache != nil { for _, prefix := range keys {
return cache, nil var exact, partial []*Cache
for _, cache := range caches {
if cache.Ref == ref && cache.Key == prefix {
exact = append(exact, cache)
} else if cache.Ref == ref && strings.HasPrefix(cache.Key, prefix) {
partial = append(partial, cache)
}
}
// if a key in the list matches exactly, don't return partial matches
if cache := cmp.Or(latest(exact, (*Cache).createdAtNano), latest(partial, (*Cache).createdAtNano)); cache != nil {
return cache, nil
}
} }
} }
return nil, nil //nolint:nilnil // pre-existing issue from nektos/act return nil, nil //nolint:nilnil // pre-existing issue from nektos/act
@@ -1111,9 +1130,9 @@ func latest(caches []*Cache, stamp func(*Cache) int64) *Cache {
// A completed entry is the one to restore, sorted by when it was written. An incomplete one is a // A completed entry is the one to restore, sorted by when it was written. An incomplete one is a
// reservation being uploaded to, sorted by when it was last written to, because the upload route // reservation being uploaded to, sorted by when it was last written to, because the upload route
// touches UsedAt on every part. // touches UsedAt on every part.
func findExactCache(db *cacheDB, repo, key, version string, complete bool) (*Cache, error) { func findExactCache(db *cacheDB, cred JobCredential, key, version string, complete bool) (*Cache, error) {
caches, err := db.find(func(cache *Cache) bool { caches, err := db.find(func(cache *Cache) bool {
return cache.Key == key && cache.Repo == repo && cache.Version == version && cache.Complete == complete return cred.owns(cache) && cache.Key == key && cache.Version == version && cache.Complete == complete
}) })
if err != nil { if err != nil {
return nil, fmt.Errorf("find cache: %w", err) return nil, fmt.Errorf("find cache: %w", err)
@@ -1280,13 +1299,12 @@ func (h *Handler) evictExpired(db *cacheDB) {
h.sweep(db, func(cache *Cache) bool { return cache.UsedAt < expired }) h.sweep(db, func(cache *Cache) bool { return cache.UsedAt < expired })
} }
// evictSuperseded removes entries a newer one with the same key and version replaced. The // evictSuperseded removes entries a newer one with the same repo, ref, key and version replaced.
// grouping includes Repo so two repos sharing a (key, version) do not evict each other.
func (h *Handler) evictSuperseded(db *cacheDB) { func (h *Handler) evictSuperseded(db *cacheDB) {
type entry struct{ repo, key, version string } type entry struct{ repo, ref, key, version string }
groups := make(map[entry][]*Cache) groups := make(map[entry][]*Cache)
for _, cache := range h.caches(db, func(cache *Cache) bool { return cache.Complete }) { for _, cache := range h.caches(db, func(cache *Cache) bool { return cache.Complete }) {
group := entry{cache.Repo, cache.Key, cache.Version} group := entry{cache.Repo, cache.Ref, cache.Key, cache.Version}
groups[group] = append(groups[group], cache) groups[group] = append(groups[group], cache)
} }
for _, caches := range groups { for _, caches := range groups {
+70 -47
View File
@@ -759,11 +759,55 @@ func TestHandler_PrefixUsesNewestReservationWithinSecond(t *testing.T) {
db, err := handler.openDB() db, err := handler.openDB()
require.NoError(t, err) require.NoError(t, err)
defer db.Close() defer db.Close()
cache, err := findCache(db, testRepo, []string{"deps-"}, "v") cache, err := findCache(db, JobCredential{Repo: testRepo}, []string{"deps-"}, "v")
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, newer, cache) require.Equal(t, newer, cache)
} }
func TestHandler_RestoreRefs(t *testing.T) {
t.Parallel()
const main, release, pull = "refs/heads/main", "refs/heads/release", "refs/pull/1/head"
now := time.Now().Unix()
entry := func(ref, key string, age int64) *Cache {
return &Cache{Repo: testRepo, Ref: ref, Key: key, Version: "v", Complete: true, CreatedAt: now - 3600 + age, UsedAt: now}
}
exactOnMain := entry(main, "deps-exact", -1)
onRelease := entry(release, "deps-release", 0)
onMain := entry(main, "deps-main", 1)
onPull := entry(pull, "deps-pull", 2)
unscoped := entry("", "deps-unscoped", 3)
handler := newTestHandler(t, Policy{}, exactOnMain, onRelease, onMain, onPull, unscoped, entry("refs/pull/2/head", "deps-lost", 4))
for _, cache := range []*Cache{exactOnMain, onRelease, onMain, onPull, unscoped} {
writeBlob(t, handler, cache.ID)
}
db, err := handler.openDB()
require.NoError(t, err)
defer db.Close()
for _, tc := range []struct {
name string
cred JobCredential
keys []string
want *Cache
}{
{"the default branch never restores what a pull request wrote", JobCredential{Ref: main, DefaultRef: main}, []string{"deps-"}, onMain},
{"a pull request restores its own entry first", JobCredential{Ref: pull, BaseRef: release, DefaultRef: main}, []string{"deps-"}, onPull},
{"a pull request falls back past a lost blob to its base branch before the default branch", JobCredential{Ref: "refs/pull/2/head", BaseRef: release, DefaultRef: main}, []string{"deps-"}, onRelease},
{"a branch does not restore a sibling branch", JobCredential{Ref: "refs/heads/other", DefaultRef: main}, []string{"deps-"}, onMain},
{"a prefix in the own ref wins over an exact key in the default branch", JobCredential{Ref: release, DefaultRef: main}, []string{"deps-exact", "deps-"}, onRelease},
{"a job with a ref never restores entries saved without one", JobCredential{Ref: "refs/heads/other"}, []string{"deps-"}, nil},
{"a job without a ref only restores entries saved without one", JobCredential{}, []string{"deps-"}, unscoped},
} {
t.Run(tc.name, func(t *testing.T) {
tc.cred.Repo = testRepo
cache, err := handler.lookupCache(db, tc.cred, tc.keys, "v")
require.NoError(t, err)
assert.Equal(t, tc.want, cache)
})
}
}
func TestHandler_evictOnCommit(t *testing.T) { func TestHandler_evictOnCommit(t *testing.T) {
t.Parallel() t.Parallel()
full := &Cache{Repo: testRepo, Key: "full", Version: "v", Complete: true, Size: 4 * miB, UsedAt: time.Now().Add(-time.Hour).Unix()} full := &Cache{Repo: testRepo, Key: "full", Version: "v", Complete: true, Size: 4 * miB, UsedAt: time.Now().Add(-time.Hour).Unix()}
@@ -938,6 +982,7 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
defer handler.Close() defer handler.Close()
handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"}) handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"}) handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"})
handler.RegisterJob("token-a-pull", JobCredential{Repo: "owner/repoA", Ref: "refs/pull/1/head"})
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
key := "shared-key" key := "shared-key"
@@ -972,13 +1017,20 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
resp.Body.Close() resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode) assert.Equal(t, http.StatusOK, resp.StatusCode)
req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison"))) for _, client := range []*http.Client{clientB, {Transport: &bearerTransport{token: "token-a-pull"}}} {
require.NoError(t, err) req, err = http.NewRequest(http.MethodPatch, fmt.Sprintf("%s/caches/%d", base, id), bytes.NewReader([]byte("poison")))
req.Header.Set("Content-Range", "bytes 0-5/*") require.NoError(t, err)
resp, err = clientB.Do(req) req.Header.Set("Content-Range", "bytes 0-5/*")
require.NoError(t, err) resp, err = client.Do(req)
resp.Body.Close() require.NoError(t, err)
assert.Equal(t, http.StatusForbidden, resp.StatusCode) resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
resp, err = client.Post(fmt.Sprintf("%s/caches/%d", base, id), "", nil)
require.NoError(t, err)
resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
}
} }
func TestHandler_ArtifactSignature(t *testing.T) { func TestHandler_ArtifactSignature(t *testing.T) {
@@ -1083,49 +1135,20 @@ func TestHandler_ArtifactSignatureDownload(t *testing.T) {
assert.Equal(t, content, body) assert.Equal(t, content, body)
} }
func TestHandler_GC_PerRepoDedup(t *testing.T) { func TestHandler_GC_DedupPerRepoAndRef(t *testing.T) {
t.Parallel() t.Parallel()
dir := filepath.Join(t.TempDir(), "artifactcache")
handler, err := StartHandler(Options{Dir: dir})
require.NoError(t, err)
defer handler.Close()
handler.RegisterJob("tok-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("tok-b", JobCredential{Repo: "owner/repoB"})
key := "shared-dedup-key"
version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20"
db, err := handler.openDB()
require.NoError(t, err)
now := time.Now().Unix() now := time.Now().Unix()
stale := time.Now().Add(-inUseGrace - time.Minute).Unix() stale := time.Now().Add(-inUseGrace - time.Minute).Unix()
a := &Cache{Repo: "owner/repoA", Key: key, Version: version, Complete: true, CreatedAt: stale, UsedAt: stale, Size: 1} entries := []*Cache{
b := &Cache{Repo: "owner/repoB", Key: key, Version: version, Complete: true, CreatedAt: now, UsedAt: now, Size: 1} {Repo: "owner/repoA", Key: "shared", Version: "v", Complete: true, CreatedAt: stale, UsedAt: stale, Size: 1},
require.NoError(t, handler.insertCache(db, a)) {Repo: "owner/repoB", Key: "shared", Version: "v", Complete: true, CreatedAt: now, UsedAt: now, Size: 1},
require.NoError(t, handler.insertCache(db, b)) {Repo: "owner/repoA", Ref: "refs/pull/1/head", Key: "shared", Version: "v", Complete: true, CreatedAt: now, UsedAt: now, Size: 1},
require.NoError(t, handler.storage.Write(a.ID, 0, strings.NewReader("a"))) }
_, err = handler.storage.Commit(a.ID, 1) handler := newTestHandler(t, Policy{}, entries...)
require.NoError(t, err)
require.NoError(t, handler.storage.Write(b.ID, 0, strings.NewReader("b")))
_, err = handler.storage.Commit(b.ID, 1)
require.NoError(t, err)
require.NoError(t, db.Close())
handler.gcAt = time.Time{} handler.gcAt = time.Time{}
handler.gcCache() handler.gcCache()
db, err = handler.openDB() assert.Len(t, keptKeys(t, handler, entries), len(entries))
require.NoError(t, err)
defer db.Close()
after, err := db.find(func(cache *Cache) bool { return cache.Key == key && cache.Version == version })
require.NoError(t, err)
repos := make(map[string]bool)
for _, c := range after {
repos[c.Repo] = true
}
assert.True(t, repos["owner/repoA"], "repoA's cache must survive dedup against repoB")
assert.True(t, repos["owner/repoB"], "repoB's cache must survive dedup against repoA")
} }
func TestHandler_InternalAPI_Disabled(t *testing.T) { func TestHandler_InternalAPI_Disabled(t *testing.T) {
@@ -1190,10 +1213,10 @@ func TestHandler_InternalAPI_AuthAndUsage(t *testing.T) {
} }
assert.Equal(t, http.StatusUnauthorized, probe("via-internal-api")) assert.Equal(t, http.StatusUnauthorized, probe("via-internal-api"))
assert.Equal(t, http.StatusOK, post("/_internal/register", secret, `{"token":"via-internal-api","repo":"owner/repo","public_url":"http://runner.example","results":"https://gitea.example","insecure_tls":true}`)) assert.Equal(t, http.StatusOK, post("/_internal/register", secret, `{"token":"via-internal-api","repo":"owner/repo","ref":"refs/pull/1/head","base_ref":"refs/heads/release","default_ref":"refs/heads/main","public_url":"http://runner.example","results":"https://gitea.example","insecure_tls":true}`))
cred, ok := handler.lookupCredential("via-internal-api") cred, ok := handler.lookupCredential("via-internal-api")
require.True(t, ok) require.True(t, ok)
assert.Equal(t, JobCredential{Repo: testRepo, PublicURL: "http://runner.example"}, cred) assert.Equal(t, JobCredential{Repo: testRepo, Ref: "refs/pull/1/head", BaseRef: "refs/heads/release", DefaultRef: "refs/heads/main", PublicURL: "http://runner.example"}, cred)
assert.NotEqual(t, http.StatusUnauthorized, probe("via-internal-api")) assert.NotEqual(t, http.StatusUnauthorized, probe("via-internal-api"))
assert.Equal(t, http.StatusOK, post("/_internal/revoke", secret, `{"token":"via-internal-api"}`)) assert.Equal(t, http.StatusOK, post("/_internal/revoke", secret, `{"token":"via-internal-api"}`))
assert.Equal(t, http.StatusUnauthorized, probe("via-internal-api")) assert.Equal(t, http.StatusUnauthorized, probe("via-internal-api"))
+6 -5
View File
@@ -73,7 +73,7 @@ func (h *Handler) v2CreateCacheEntry(w http.ResponseWriter, r *http.Request) {
// An exact (key, version) match means the entry is already cached; the client then skips // An exact (key, version) match means the entry is already cached; the client then skips
// the upload. A prefix match must not count here, or a shorter key would be reported as // the upload. A prefix match must not count here, or a shorter key would be reported as
// existing and silently never saved. // existing and silently never saved.
if existing, err := findExactCache(db, cred.Repo, req.Key, req.Version, true); err != nil { if existing, err := findExactCache(db, cred, req.Key, req.Version, true); err != nil {
h.twirpError(w, r, twirpInternal, err) h.twirpError(w, r, twirpInternal, err)
return return
} else if existing != nil { } else if existing != nil {
@@ -84,7 +84,7 @@ func (h *Handler) v2CreateCacheEntry(w http.ResponseWriter, r *http.Request) {
// A second live reservation is finalized by whichever job calls last, against the other's upload. // A second live reservation is finalized by whichever job calls last, against the other's upload.
owner := hashedToken(bearerToken(r)) owner := hashedToken(bearerToken(r))
if pending, err := findExactCache(db, cred.Repo, req.Key, req.Version, false); err != nil { if pending, err := findExactCache(db, cred, req.Key, req.Version, false); err != nil {
h.twirpError(w, r, twirpInternal, err) h.twirpError(w, r, twirpInternal, err)
return return
} else if pending != nil && pending.UsedAt > time.Now().Add(-uploadStallTimeout).Unix() { } else if pending != nil && pending.UsedAt > time.Now().Add(-uploadStallTimeout).Unix() {
@@ -103,6 +103,7 @@ func (h *Handler) v2CreateCacheEntry(w http.ResponseWriter, r *http.Request) {
now := time.Now() now := time.Now()
cache := &Cache{ cache := &Cache{
Repo: cred.Repo, Repo: cred.Repo,
Ref: cred.Ref,
Key: req.Key, Key: req.Key,
Version: req.Version, Version: req.Version,
Owner: owner, Owner: owner,
@@ -137,13 +138,13 @@ func (h *Handler) v2FinalizeCacheEntryUpload(w http.ResponseWriter, r *http.Requ
} }
defer db.Close() defer db.Close()
cache, err := findExactCache(db, cred.Repo, req.Key, req.Version, false) cache, err := findExactCache(db, cred, req.Key, req.Version, false)
if err == nil && cache != nil && cache.Owner != hashedToken(bearerToken(r)) { if err == nil && cache != nil && cache.Owner != hashedToken(bearerToken(r)) {
cache = nil // not the reservation this job made, so not this job's to commit cache = nil // not the reservation this job made, so not this job's to commit
} }
if err == nil && cache == nil { if err == nil && cache == nil {
// A retry whose first response was lost finds the entry already committed. // A retry whose first response was lost finds the entry already committed.
cache, err = findExactCache(db, cred.Repo, req.Key, req.Version, true) cache, err = findExactCache(db, cred, req.Key, req.Version, true)
} }
if err != nil { if err != nil {
h.twirpError(w, r, twirpInternal, err) h.twirpError(w, r, twirpInternal, err)
@@ -186,7 +187,7 @@ func (h *Handler) v2GetCacheEntryDownloadURL(w http.ResponseWriter, r *http.Requ
} }
defer db.Close() defer db.Close()
cache, err := h.lookupCache(db, cred.Repo, req.keys(), req.Version) cache, err := h.lookupCache(db, cred, req.keys(), req.Version)
if err != nil { if err != nil {
h.twirpError(w, r, twirpInternal, err) h.twirpError(w, r, twirpInternal, err)
return return
+18
View File
@@ -231,6 +231,24 @@ func TestCacheServiceV2Lookups(t *testing.T) {
got := v2Call(t, handler, otherClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"}) got := v2Call(t, handler, otherClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})
assert.Equal(t, false, got["ok"]) assert.Equal(t, false, got["ok"])
}) })
t.Run("each ref saves and restores its own entry for the same key", func(t *testing.T) {
clients := map[string]*http.Client{"x": testClient}
for _, ref := range []string{"y", "z"} {
handler.RegisterJob(ref, JobCredential{Repo: testRepo, Ref: "refs/pull/" + ref + "/head"})
clients[ref] = &http.Client{Transport: &bearerTransport{token: ref}}
created := v2Call(t, handler, clients[ref], "CreateCacheEntry", map[string]any{"key": "deps-abc", "version": "v1"})
require.Equal(t, true, created["ok"])
require.Equal(t, http.StatusCreated, putBlob(t, created["signed_upload_url"].(string), []byte(ref)))
}
for _, ref := range []string{"y", "z"} {
require.Equal(t, true, v2Call(t, handler, clients[ref], "FinalizeCacheEntryUpload", map[string]any{"key": "deps-abc", "version": "v1", "size_bytes": 1})["ok"])
}
for content, client := range clients {
got := v2Call(t, handler, client, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})
assert.Equal(t, content, string(getURL(t, got["signed_download_url"].(string))))
}
})
} }
func TestCacheServiceV2RefusalsAreTwirp(t *testing.T) { func TestCacheServiceV2RefusalsAreTwirp(t *testing.T) {
+1
View File
@@ -15,6 +15,7 @@ type Request struct {
type Cache struct { type Cache struct {
ID uint64 `json:"id"` ID uint64 `json:"id"`
Repo string `json:"repo"` Repo string `json:"repo"`
Ref string `json:"ref,omitzero"`
Key string `json:"key"` Key string `json:"key"`
Version string `json:"version"` Version string `json:"version"`
Size int64 `json:"cacheSize"` Size int64 `json:"cacheSize"`
+72 -36
View File
@@ -40,7 +40,9 @@ const (
jobLabel = "com.gitea.runner.job." jobLabel = "com.gitea.runner.job."
maxCreateBody = 8 << 20 maxCreateBody = 8 << 20
dockerProxyProbeTimeout = 5 * time.Second dockerProxyProbeTimeout = 5 * time.Second
dockerProxyProbeWorkerTimeout = time.Minute
dockerProxyProbeTarget = "/gitea-runner-probe"
) )
var ( var (
@@ -48,7 +50,7 @@ var (
rawStreamPath = regexp.MustCompile(`^(/v[0-9.]+)?/(containers/[^/]+/attach|exec/[^/]+/start)$`) rawStreamPath = regexp.MustCompile(`^(/v[0-9.]+)?/(containers/[^/]+/attach|exec/[^/]+/start)$`)
) )
var dockerProxyDirectories, dockerProxyProbes sync.Map var dockerProxyProbes sync.Map
type dockerProxyDirectory struct { type dockerProxyDirectory struct {
dir, daemonDir string dir, daemonDir string
@@ -68,7 +70,7 @@ func NewDockerProxy(ctx context.Context, job string) *DockerProxy {
} }
probeCtx, cancel := context.WithTimeout(ctx, dockerProxyProbeTimeout) probeCtx, cancel := context.WithTimeout(ctx, dockerProxyProbeTimeout)
defer cancel() defer cancel()
cli, err := GetDockerClient(probeCtx) cli, err := client.New(client.FromEnv) // negotiates on first request, so a cached probe costs no round trip
if err != nil { if err != nil {
return nil return nil
} }
@@ -83,65 +85,69 @@ func NewDockerProxy(ctx context.Context, job string) *DockerProxy {
} }
workdir, _ := os.Getwd() workdir, _ := os.Getwd()
key := [3]string{cli.DaemonHost(), workdir, os.TempDir()} key := [3]string{cli.DaemonHost(), workdir, os.TempDir()}
cached, _ := dockerProxyDirectories.Load(key) probe, err := probeDockerProxyDirectory(probeCtx, cli, key, socket)
directory, seen := cached.(dockerProxyDirectory)
seen = seen && os.SameFile(directory.socket, socket) // daemon replacement invalidates cached paths
if !seen {
directory, seen, err = probeDockerProxyDirectory(probeCtx, cli, key, socket)
}
if err != nil { if err != nil {
common.Logger(ctx).Infof("docker proxy probe failed, jobs get the daemon socket directly: %v", err) common.Logger(ctx).Infof("docker proxy probe failed, jobs get the daemon socket directly: %v", err)
return nil return nil
} }
if !seen { if !probe.seen {
common.Logger(ctx).Infof("the docker daemon cannot reach the runner's temporary or working directory, jobs get the daemon socket directly") common.Logger(ctx).Infof("the docker daemon cannot reach the runner's temporary or working directory, jobs get the daemon socket directly")
return nil return nil
} }
if ctx.Err() != nil { if ctx.Err() != nil {
return nil return nil
} }
proxy, err := StartDockerProxy(daemonSocket, directory.dir, job) proxy, err := StartDockerProxy(daemonSocket, probe.directory.dir, job)
if err != nil { if err != nil {
dockerProxyDirectories.Delete(key) dockerProxyProbes.CompareAndDelete(key, probe)
common.Logger(ctx).Warnf("docker proxy not started, the job gets the daemon socket directly: %v", err) common.Logger(ctx).Warnf("docker proxy not started, the job gets the daemon socket directly: %v", err)
return nil return nil
} }
dockerProxyDirectories.Store(key, directory) proxy.Socket = probe.directory.daemonDir + strings.TrimPrefix(proxy.Socket, probe.directory.dir)
proxy.Socket = directory.daemonDir + strings.TrimPrefix(proxy.Socket, directory.dir)
return proxy return proxy
} }
func probeDockerProxyDirectory(ctx context.Context, cli client.APIClient, key [3]string, socket os.FileInfo) (dockerProxyDirectory, bool, error) { func probeDockerProxyDirectory(ctx context.Context, cli client.APIClient, key [3]string, socket os.FileInfo) (*dockerProxyProbe, error) {
for { for {
probe := &dockerProxyProbe{directory: dockerProxyDirectory{socket: socket}, done: make(chan struct{})} probe := &dockerProxyProbe{directory: dockerProxyDirectory{socket: socket}, done: make(chan struct{})}
actual, loaded := dockerProxyProbes.LoadOrStore(key, probe) actual, loaded := dockerProxyProbes.LoadOrStore(key, probe)
if loaded { if loaded {
probe, _ = actual.(*dockerProxyProbe) probe, _ = actual.(*dockerProxyProbe)
if !os.SameFile(probe.directory.socket, socket) { if !os.SameFile(probe.directory.socket, socket) { // daemon replacement invalidates cached paths
dockerProxyProbes.CompareAndDelete(key, probe) dockerProxyProbes.CompareAndDelete(key, probe)
continue continue
} }
select {
case <-ctx.Done():
return dockerProxyDirectory{}, false, ctx.Err()
case <-probe.done:
}
if ctx.Err() == nil && (errors.Is(probe.err, context.Canceled) || errors.Is(probe.err, context.DeadlineExceeded)) {
continue
}
} else { } else {
probe.directory.dir, probe.directory.daemonDir = runnerContainerWorkdir(ctx, cli) go func() { // outlives callers' deadlines so a slow daemon still gets its result cached
probe.seen = probe.directory.daemonDir != "" defer cli.Close()
if !probe.seen { ctx, cancel := context.WithTimeout(context.Background(), dockerProxyProbeWorkerTimeout)
if probe.directory.dir, probe.err = filepath.Abs(key[2]); probe.err == nil { defer cancel()
probe.directory.daemonDir = probe.directory.dir probe.directory.dir, probe.directory.daemonDir = runnerContainerWorkdir(ctx, cli)
probe.seen, probe.err = daemonSeesDir(ctx, cli, probe.directory.dir, probe.directory.daemonDir) probe.seen = probe.directory.daemonDir != ""
if !probe.seen {
if probe.directory.dir, probe.err = filepath.Abs(key[2]); probe.err == nil {
probe.directory.daemonDir = probe.directory.dir
probe.seen, probe.err = daemonSeesDir(ctx, cli, probe.directory.dir, probe.directory.daemonDir)
}
} }
} if probe.seen {
dockerProxyProbes.CompareAndDelete(key, probe) dockerProxyProbes.LoadOrStore(key, probe)
close(probe.done) } else {
dockerProxyProbes.CompareAndDelete(key, probe)
}
close(probe.done)
removeStaleProbeContainers(cli)
}()
} }
return probe.directory, probe.seen, probe.err select {
case <-ctx.Done():
return &dockerProxyProbe{}, ctx.Err()
case <-probe.done:
}
if ctx.Err() == nil && errors.Is(probe.err, context.DeadlineExceeded) {
continue
}
return probe, probe.err
} }
} }
@@ -200,7 +206,7 @@ func daemonSeesDir(ctx context.Context, cli client.APIClient, dir, daemonDir str
created, err = cli.ContainerCreate(ctx, client.ContainerCreateOptions{ created, err = cli.ContainerCreate(ctx, client.ContainerCreateOptions{
Config: &container.Config{Image: image.ID, Cmd: []string{"true"}}, Config: &container.Config{Image: image.ID, Cmd: []string{"true"}},
HostConfig: &container.HostConfig{Mounts: []mount.Mount{ HostConfig: &container.HostConfig{Mounts: []mount.Mount{
{Type: mount.TypeBind, Source: daemonDir, Target: "/gitea-runner-probe", ReadOnly: true}, // not the marker itself, podman creates a missing bind source where docker rejects it {Type: mount.TypeBind, Source: daemonDir, Target: dockerProxyProbeTarget, ReadOnly: true}, // not the marker itself, podman creates a missing bind source where docker rejects it
}}, }},
}) })
if !cerrdefs.IsNotFound(err) { if !cerrdefs.IsNotFound(err) {
@@ -213,7 +219,7 @@ func daemonSeesDir(ctx context.Context, cli client.APIClient, dir, daemonDir str
if err != nil { if err != nil {
return false, err return false, err
} }
seen, err := containerSeesMarker(ctx, cli, dir, created.ID, "/gitea-runner-probe") seen, err := containerSeesMarker(ctx, cli, dir, created.ID, dockerProxyProbeTarget)
cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), dockerProxyProbeTimeout) cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), dockerProxyProbeTimeout)
defer cancel() defer cancel()
if _, removeErr := cli.ContainerRemove(cleanupCtx, created.ID, client.ContainerRemoveOptions{Force: true, RemoveVolumes: true}); removeErr != nil { if _, removeErr := cli.ContainerRemove(cleanupCtx, created.ID, client.ContainerRemoveOptions{Force: true, RemoveVolumes: true}); removeErr != nil {
@@ -222,6 +228,36 @@ func daemonSeesDir(ctx context.Context, cli client.APIClient, dir, daemonDir str
return seen, err return seen, err
} }
// removeStaleProbeContainers removes probe containers left by abandoned creates once any probe in flight when listed has ended.
func removeStaleProbeContainers(cli client.APIClient) {
ctx, cancel := context.WithTimeout(context.Background(), dockerProxyProbeTimeout)
defer cancel()
listed, err := cli.ContainerList(ctx, client.ContainerListOptions{
All: true,
Filters: make(client.Filters).Add("status", string(container.StateCreated)).Add("volume", dockerProxyProbeTarget),
})
if err != nil {
common.Logger(ctx).Warnf("listing stale docker proxy probe containers failed: %v", err)
return
}
listed.Items = slices.DeleteFunc(listed.Items, func(stale container.Summary) bool {
return stale.State != container.StateCreated || !slices.ContainsFunc(stale.Mounts, func(mountPoint container.MountPoint) bool {
return mountPoint.Type == mount.TypeBind && !mountPoint.RW && mountPoint.Destination == dockerProxyProbeTarget
})
})
if len(listed.Items) == 0 {
return
}
time.Sleep(dockerProxyProbeWorkerTimeout + dockerProxyProbeTimeout)
for _, stale := range listed.Items {
ctx, cancel := context.WithTimeout(context.Background(), dockerProxyProbeTimeout)
if err := (&containerReference{cli: cli, id: stale.ID}).remove()(ctx); err != nil {
common.Logger(ctx).Warnf("removing stale docker proxy probe container failed: %v", err)
}
cancel()
}
}
// StartDockerProxy serves a job's docker socket in dir, labelling what the job creates through it. // StartDockerProxy serves a job's docker socket in dir, labelling what the job creates through it.
func StartDockerProxy(daemonSocket, dir, job string) (*DockerProxy, error) { func StartDockerProxy(daemonSocket, dir, job string) (*DockerProxy, error) {
info, err := os.Stat(daemonSocket) info, err := os.Stat(daemonSocket)
+122 -56
View File
@@ -94,6 +94,8 @@ func TestDockerProxy(t *testing.T) {
return return
} }
_, _ = w.Write([]byte(`[{"Id":"probe-image"}]`)) _, _ = w.Write([]byte(`[{"Id":"probe-image"}]`))
case strings.HasSuffix(r.URL.Path, "/containers/json"):
_, _ = w.Write([]byte(`[]`))
case strings.HasSuffix(r.URL.Path, "/json"): case strings.HasSuffix(r.URL.Path, "/json"):
http.NotFound(w, r) http.NotFound(w, r)
case strings.HasSuffix(r.URL.Path, "/archive"): case strings.HasSuffix(r.URL.Path, "/archive"):
@@ -235,7 +237,8 @@ func TestDockerProxy(t *testing.T) {
t.Skip("automatic proxy discovery requires Linux") t.Skip("automatic proxy discovery requires Linux")
} }
t.Setenv("DOCKER_HOST", "unix://"+daemonSocket) t.Setenv("DOCKER_HOST", "unix://"+daemonSocket)
t.Setenv("TMPDIR", shortTempDir(t)) tempDir := shortTempDir(t)
t.Setenv("TMPDIR", tempDir)
require.Nil(t, NewDockerProxy(t.Context(), "failed")) require.Nil(t, NewDockerProxy(t.Context(), "failed"))
for _, job := range []string{"first", "second"} { for _, job := range []string{"first", "second"} {
proxy := NewDockerProxy(t.Context(), job) proxy := NewDockerProxy(t.Context(), job)
@@ -243,12 +246,10 @@ func TestDockerProxy(t *testing.T) {
require.NoError(t, proxy.Close(context.Background())) require.NoError(t, proxy.Close(context.Background()))
} }
assert.EqualValues(t, 2, probes.Load()) assert.EqualValues(t, 2, probes.Load())
dockerProxyDirectories.Range(func(key, value any) bool { workdir, err := os.Getwd()
directory, _ := value.(dockerProxyDirectory) require.NoError(t, err)
directory.socket, _ = os.Stat(t.TempDir()) cached, _ := dockerProxyProbes.Load([3]string{"unix://" + daemonSocket, workdir, tempDir})
dockerProxyDirectories.Store(key, directory) cached.(*dockerProxyProbe).directory.socket, _ = os.Stat(t.TempDir())
return true
})
proxy := NewDockerProxy(t.Context(), "replaced daemon") proxy := NewDockerProxy(t.Context(), "replaced daemon")
require.NotNil(t, proxy) require.NotNil(t, proxy)
require.NoError(t, proxy.Close(context.Background())) require.NoError(t, proxy.Close(context.Background()))
@@ -476,10 +477,23 @@ func BenchmarkDockerProxy(b *testing.B) {
type probeClient struct { type probeClient struct {
mobyclient.APIClient mobyclient.APIClient
create func(mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) create func(context.Context, mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error)
remove func(context.Context, string, mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) remove func(context.Context, string, mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error)
stat func(string) error stat func(string) error
mounts []container.MountPoint mounts []container.MountPoint
containers []container.Summary
}
func (c *probeClient) Close() error {
return nil
}
func (c *probeClient) ContainerList(context.Context, mobyclient.ContainerListOptions) (mobyclient.ContainerListResult, error) {
return mobyclient.ContainerListResult{Items: c.containers}, nil
}
func (c *probeClient) ContainerKill(context.Context, string, mobyclient.ContainerKillOptions) (mobyclient.ContainerKillResult, error) {
return mobyclient.ContainerKillResult{}, cerrdefs.ErrConflict
} }
func (c *probeClient) ContainerInspect(context.Context, string, mobyclient.ContainerInspectOptions) (mobyclient.ContainerInspectResult, error) { func (c *probeClient) ContainerInspect(context.Context, string, mobyclient.ContainerInspectOptions) (mobyclient.ContainerInspectResult, error) {
@@ -490,8 +504,8 @@ func (c *probeClient) ImageList(context.Context, mobyclient.ImageListOptions) (m
return mobyclient.ImageListResult{Items: []image.Summary{{ID: "removed-image"}, {ID: "probe-image"}}}, nil return mobyclient.ImageListResult{Items: []image.Summary{{ID: "removed-image"}, {ID: "probe-image"}}}, nil
} }
func (c *probeClient) ContainerCreate(_ context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) { func (c *probeClient) ContainerCreate(ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
return c.create(opts) return c.create(ctx, opts)
} }
func (c *probeClient) ContainerRemove(ctx context.Context, id string, opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) { func (c *probeClient) ContainerRemove(ctx context.Context, id string, opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) {
@@ -518,7 +532,7 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
var calls, removals atomic.Int32 var calls, removals atomic.Int32
visible := false visible := false
cli := &probeClient{ cli := &probeClient{
create: func(_opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) { create: func(_ctx context.Context, _opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
calls.Add(1) calls.Add(1)
<-release <-release
return mobyclient.ContainerCreateResult{ID: "probe"}, createErr return mobyclient.ContainerCreateResult{ID: "probe"}, createErr
@@ -538,23 +552,18 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
results := make(chan result, 8) results := make(chan result, 8)
for range cap(results) { for range cap(results) {
go func() { go func() {
_, seen, err := probeDockerProxyDirectory(t.Context(), cli, key, socket) probe, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
results <- result{seen, err} results <- result{probe.seen, err}
}() }()
} }
synctest.Wait() synctest.Wait()
assert.EqualValues(t, 1, calls.Load()) assert.EqualValues(t, 1, calls.Load())
ctx, cancel := context.WithCancel(t.Context())
cancel()
_, seen, err := probeDockerProxyDirectory(ctx, cli, key, socket)
require.ErrorIs(t, err, context.Canceled)
assert.False(t, seen)
replacement, err := os.Stat(t.TempDir()) replacement, err := os.Stat(t.TempDir())
require.NoError(t, err) require.NoError(t, err)
replaced := make(chan result, 1) replaced := make(chan result, 1)
go func() { go func() {
_, seen, err := probeDockerProxyDirectory(t.Context(), cli, key, replacement) probe, err := probeDockerProxyDirectory(t.Context(), cli, key, replacement)
replaced <- result{seen, err} replaced <- result{probe.seen, err}
}() }()
synctest.Wait() synctest.Wait()
assert.EqualValues(t, 2, calls.Load()) assert.EqualValues(t, 2, calls.Load())
@@ -574,11 +583,11 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
assert.Zero(t, removals.Load()) assert.Zero(t, removals.Load())
} }
createErr, visible = nil, true createErr, visible = nil, true
directory, seen, err := probeDockerProxyDirectory(t.Context(), cli, key, socket) probe, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
require.NoError(t, err) require.NoError(t, err)
assert.True(t, seen) assert.True(t, probe.seen)
assert.Equal(t, dir, directory.dir) assert.Equal(t, dir, probe.directory.dir)
assert.Equal(t, dir, directory.daemonDir) assert.Equal(t, dir, probe.directory.daemonDir)
assert.EqualValues(t, 3, calls.Load()) assert.EqualValues(t, 3, calls.Load())
entries, err := os.ReadDir(dir) entries, err := os.ReadDir(dir)
require.NoError(t, err) require.NoError(t, err)
@@ -586,7 +595,7 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
}) })
}) })
} }
t.Run("cancelled probe does not cancel another caller", func(t *testing.T) { t.Run("cancelled caller leaves the probe to finish for the others", func(t *testing.T) {
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
socket, err := os.Stat(t.TempDir()) socket, err := os.Stat(t.TempDir())
@@ -595,49 +604,105 @@ func TestDockerProxyDirectoryProbes(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context()) ctx, cancel := context.WithCancel(t.Context())
defer cancel() defer cancel()
release := make(chan struct{}) release := make(chan struct{})
calls := 0 var calls, removals atomic.Int32
cli := &probeClient{ cli := &probeClient{
create: func(_opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) { create: func(ctx context.Context, _opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
calls++ calls.Add(1)
if calls == 1 { <-release
<-release return mobyclient.ContainerCreateResult{ID: "probe"}, ctx.Err()
},
stat: func(_path string) error { return nil },
remove: func(_ctx context.Context, _id string, _opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) {
removals.Add(1)
return mobyclient.ContainerRemoveResult{}, nil
},
}
leader, follower, late := make(chan result, 1), make(chan result, 1), make(chan result, 1)
go func() {
probe, err := probeDockerProxyDirectory(ctx, cli, key, socket)
leader <- result{probe.seen, err}
}()
synctest.Wait()
go func() {
probe, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
follower <- result{probe.seen, err}
}()
synctest.Wait()
cancel()
leaderResult := <-leader
require.ErrorIs(t, leaderResult.err, context.Canceled)
assert.False(t, leaderResult.seen)
go func() {
probe, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
late <- result{probe.seen, err}
}()
synctest.Wait()
close(release)
for _, results := range []chan result{follower, late} {
result := <-results
require.NoError(t, result.err)
assert.True(t, result.seen)
}
assert.EqualValues(t, 1, calls.Load())
assert.EqualValues(t, 1, removals.Load())
_, cached := dockerProxyProbes.Load(key)
assert.True(t, cached)
})
})
t.Run("caller joining a probe that ran out of time retries", func(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
dir := t.TempDir()
socket, err := os.Stat(t.TempDir())
require.NoError(t, err)
var calls atomic.Int32
cli := &probeClient{
create: func(ctx context.Context, _opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
if calls.Add(1) == 1 {
<-ctx.Done()
return mobyclient.ContainerCreateResult{}, ctx.Err() return mobyclient.ContainerCreateResult{}, ctx.Err()
} else {
return mobyclient.ContainerCreateResult{ID: "probe"}, nil
} }
return mobyclient.ContainerCreateResult{ID: "probe"}, nil
}, },
stat: func(_path string) error { return nil }, stat: func(_path string) error { return nil },
remove: func(_ctx context.Context, _id string, _opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) { remove: func(_ctx context.Context, _id string, _opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) {
return mobyclient.ContainerRemoveResult{}, nil return mobyclient.ContainerRemoveResult{}, nil
}, },
} }
leader, follower := make(chan result, 1), make(chan result, 1) probe, err := probeDockerProxyDirectory(t.Context(), cli, [3]string{"daemon", dir, dir}, socket)
go func() { require.NoError(t, err)
_, seen, err := probeDockerProxyDirectory(ctx, cli, key, socket) assert.True(t, probe.seen)
leader <- result{seen, err} assert.EqualValues(t, 2, calls.Load())
}()
synctest.Wait()
go func() {
_, seen, err := probeDockerProxyDirectory(t.Context(), cli, key, socket)
follower <- result{seen, err}
}()
synctest.Wait()
assert.Equal(t, 1, calls)
cancel()
close(release)
leaderResult, followerResult := <-leader, <-follower
require.ErrorIs(t, leaderResult.err, context.Canceled)
assert.False(t, leaderResult.seen)
require.NoError(t, followerResult.err)
assert.True(t, followerResult.seen)
assert.Equal(t, 2, calls)
}) })
}) })
} }
func TestRemoveStaleProbeContainers(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
probeMount := container.MountPoint{Type: mount.TypeBind, Destination: "/gitea-runner-probe"}
writableMount := container.MountPoint{Type: mount.TypeBind, Destination: "/gitea-runner-probe", RW: true}
var removed []string
cli := &probeClient{
containers: []container.Summary{
{ID: "abandoned", State: container.StateCreated, Mounts: []container.MountPoint{probeMount}},
{ID: "foreign", State: container.StateCreated, Mounts: []container.MountPoint{writableMount}},
{ID: "started", State: container.StateExited, Mounts: []container.MountPoint{probeMount}},
},
remove: func(_ctx context.Context, id string, _opts mobyclient.ContainerRemoveOptions) (mobyclient.ContainerRemoveResult, error) {
removed = append(removed, id)
return mobyclient.ContainerRemoveResult{}, nil
},
}
start := time.Now()
removeStaleProbeContainers(cli)
assert.Equal(t, []string{"abandoned"}, removed)
assert.GreaterOrEqual(t, time.Since(start), dockerProxyProbeWorkerTimeout+dockerProxyProbeTimeout)
})
}
func TestDaemonSeesDir(t *testing.T) { func TestDaemonSeesDir(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
markers := make(map[string]bool) markers := make(map[string]bool)
for _, testCase := range []struct { for _, testCase := range []struct {
name string name string
daemonDir string daemonDir string
@@ -656,7 +721,7 @@ func TestDaemonSeesDir(t *testing.T) {
removed := false removed := false
var attemptedImages []string var attemptedImages []string
cli := &probeClient{ cli := &probeClient{
create: func(opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) { create: func(_ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
assert.Equal(t, []mount.Mount{{Type: mount.TypeBind, Source: testCase.daemonDir, Target: "/gitea-runner-probe", ReadOnly: true}}, opts.HostConfig.Mounts) assert.Equal(t, []mount.Mount{{Type: mount.TypeBind, Source: testCase.daemonDir, Target: "/gitea-runner-probe", ReadOnly: true}}, opts.HostConfig.Mounts)
attemptedImages = append(attemptedImages, opts.Config.Image) attemptedImages = append(attemptedImages, opts.Config.Image)
if opts.Config.Image == "removed-image" { if opts.Config.Image == "removed-image" {
@@ -692,6 +757,7 @@ func TestDaemonSeesDir(t *testing.T) {
assert.Equal(t, []string{"removed-image", "probe-image"}, attemptedImages) assert.Equal(t, []string{"removed-image", "probe-image"}, attemptedImages)
assert.Equal(t, testCase.seen, seen) assert.Equal(t, testCase.seen, seen)
assert.Equal(t, testCase.removed, removed) assert.Equal(t, testCase.removed, removed)
entries, err := os.ReadDir(dir) entries, err := os.ReadDir(dir)
require.NoError(t, err) require.NoError(t, err)
assert.Empty(t, entries) assert.Empty(t, entries)
+1 -1
View File
@@ -980,7 +980,7 @@ func TestDockerCreateKeepsInputEntrypoint(t *testing.T) {
RunnerOptions: "--entrypoint=runner-entrypoint", RunnerOptions: "--entrypoint=runner-entrypoint",
WorkflowOptions: "--entrypoint=" + optionEntrypoint + " --init", WorkflowOptions: "--entrypoint=" + optionEntrypoint + " --init",
}, },
cli: &probeClient{create: func(opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) { cli: &probeClient{create: func(_ctx context.Context, opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
if len(entrypoint) > 0 { if len(entrypoint) > 0 {
assert.Equal(t, entrypoint, opts.Config.Entrypoint) assert.Equal(t, entrypoint, opts.Config.Entrypoint)
} else { } else {
+9 -7
View File
@@ -180,9 +180,9 @@ func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction
action := step.getActionModel() action := step.getActionModel()
rc.withGithubEnv(ctx, step.getGithubContext(ctx), *step.getEnv()) rc.withGithubEnv(ctx, getStepGithubContext(ctx, step), *step.getEnv())
populateEnvsFromSavedState(step.getEnv(), step, rc) populateEnvsFromSavedState(step.getEnv(), step, rc)
if err := populateEnvsFromInput(ctx, step.getEnv(), action, rc); err != nil { if err := populateEnvsFromInput(ctx, step, action); err != nil {
return err return err
} }
warnUnexpectedInputs(ctx, action, stepModel.With) warnUnexpectedInputs(ctx, action, stepModel.With)
@@ -283,7 +283,7 @@ func (rc *RunContext) dockerActionImage(ctx context.Context, step actionStep, ac
} }
image, ok := job.dockerActionImages[step.getStepModel()] image, ok := job.dockerActionImages[step.getStepModel()]
if !ok { if !ok {
image = dockerActionImageTag(step.getGithubContext(ctx).Repository, actionName, localAction) + "-" + strings.ToLower(rand.Text()) image = dockerActionImageTag(getStepGithubContext(ctx, step).Repository, actionName, localAction) + "-" + strings.ToLower(rand.Text())
if !common.Dryrun(ctx) { // a dry run builds nothing to remove if !common.Dryrun(ctx) { // a dry run builds nothing to remove
job.dockerActionImages[step.getStepModel()] = image job.dockerActionImages[step.getStepModel()] = image
} }
@@ -437,7 +437,7 @@ func evalDockerEnv(ctx context.Context, step step, action *model.Action) (*expre
for key := range step.getStepModel().With { for key := range step.getStepModel().With {
providedInputs[strings.ToLower(key)] = true providedInputs[strings.ToLower(key)] = true
} }
eval := rc.NewExpressionEvaluator(ctx) eval := rc.NewStepExpressionEvaluator(ctx, step)
for k, input := range action.Inputs { for k, input := range action.Inputs {
if providedInputs[strings.ToLower(k)] { if providedInputs[strings.ToLower(k)] {
continue continue
@@ -456,6 +456,7 @@ func evalDockerEnv(ctx context.Context, step step, action *model.Action) (*expre
} }
} }
mergeIntoMap(step, step.getEnv(), runsEnv, maps.Clone(*step.getEnv())) mergeIntoMap(step, step.getEnv(), runsEnv, maps.Clone(*step.getEnv()))
aliasGiteaEnv(*step.getEnv())
return ee, nil return ee, nil
} }
@@ -509,8 +510,9 @@ func inputEnvKey(inputID string) string {
return "INPUT_" + inputEnvPattern.ReplaceAllString(strings.ToUpper(inputID), "_") return "INPUT_" + inputEnvPattern.ReplaceAllString(strings.ToUpper(inputID), "_")
} }
func populateEnvsFromInput(ctx context.Context, env *map[string]string, action *model.Action, rc *RunContext) error { func populateEnvsFromInput(ctx context.Context, step step, action *model.Action) error {
eval := rc.NewExpressionEvaluator(ctx) env := step.getEnv()
eval := step.getRunContext().NewStepExpressionEvaluator(ctx, step)
for inputID, input := range action.Inputs { for inputID, input := range action.Inputs {
envKey := inputEnvKey(inputID) envKey := inputEnvKey(inputID)
if _, ok := (*env)[envKey]; !ok { if _, ok := (*env)[envKey]; !ok {
@@ -652,7 +654,7 @@ func runPreStep(step actionStep) common.Executor {
x := action.Runs.Using x := action.Runs.Using
if !x.IsComposite() { if !x.IsComposite() {
// defaults in pre steps were missing, however provided inputs are available // defaults in pre steps were missing, however provided inputs are available
if err := populateEnvsFromInput(ctx, step.getEnv(), action, rc); err != nil { if err := populateEnvsFromInput(ctx, step, action); err != nil {
return err return err
} }
} }
-3
View File
@@ -57,9 +57,6 @@ func evaluateCompositeInputAndEnv(ctx context.Context, parent *RunContext, step
env[envKey] = value env[envKey] = value
} }
} }
gh := step.getGithubContext(ctx)
env["GITHUB_ACTION_REPOSITORY"] = gh.ActionRepository
env["GITHUB_ACTION_REF"] = gh.ActionRef
return env, nil return env, nil
} }
+30 -1
View File
@@ -19,6 +19,35 @@ import (
) )
func TestCompositeActionParity(t *testing.T) { func TestCompositeActionParity(t *testing.T) {
t.Run("input defaults use the step identity and docker env keeps the GITEA aliases", func(t *testing.T) {
for _, uses := range []string{"./action", "own/action@v2"} {
t.Run(uses, func(t *testing.T) {
rc := &RunContext{
Config: &Config{PresetGitHubContext: &model.GithubContext{}},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/composite", "GITHUB_ACTION_REF": "v1"},
}
var currentStep step = &stepActionLocal{RunContext: rc, Step: &model.Step{Uses: uses}, env: map[string]string{}}
want := "|"
if uses == "own/action@v2" {
currentStep = &stepActionRemote{RunContext: rc, Step: &model.Step{Uses: uses}, env: map[string]string{}, remoteAction: mustNewRemoteAction(t, uses)}
want = "own/action|v2"
}
action := &model.Action{
Inputs: map[string]model.Input{"identity": {Default: "${{ github.action_repository }}|${{ github.action_ref }}"}},
Runs: model.ActionRuns{Env: map[string]string{"GITHUB_WORKFLOW_SHA": "runs-sha"}},
}
require.NoError(t, setupEnv(t.Context(), currentStep))
require.NoError(t, populateEnvsFromInput(t.Context(), currentStep, action))
assert.Equal(t, want, (*currentStep.getEnv())["INPUT_IDENTITY"])
_, err := evalDockerEnv(t.Context(), currentStep, action)
require.NoError(t, err)
assert.Equal(t, want, (*currentStep.getEnv())["identity"])
assert.Equal(t, "runs-sha", (*currentStep.getEnv())["GITEA_WORKFLOW_SHA"])
})
}
})
t.Run("inherits contexts without leaking inputs", func(t *testing.T) { t.Run("inherits contexts without leaking inputs", func(t *testing.T) {
hook := logtest.NewWithLevel(logging.LevelWarn) hook := logtest.NewWithLevel(logging.LevelWarn)
ctx := common.WithLogger(t.Context(), logging.New(hook)) ctx := common.WithLogger(t.Context(), logging.New(hook))
@@ -54,7 +83,7 @@ func TestCompositeActionParity(t *testing.T) {
assert.NotContains(t, composite.Env, "INPUT_EXTRA") assert.NotContains(t, composite.Env, "INPUT_EXTRA")
nestedEnv := composite.GetEnv() nestedEnv := composite.GetEnv()
require.NoError(t, populateEnvsFromInput(ctx, &nestedEnv, &model.Action{Inputs: map[string]model.Input{"shared": {Default: "inner-default"}}}, composite)) require.NoError(t, populateEnvsFromInput(ctx, &stepActionLocal{RunContext: composite, env: nestedEnv}, &model.Action{Inputs: map[string]model.Input{"shared": {Default: "inner-default"}}}))
assert.Equal(t, "inner-default", nestedEnv["INPUT_SHARED"]) assert.Equal(t, "inner-default", nestedEnv["INPUT_SHARED"])
}) })
+5 -5
View File
@@ -90,18 +90,18 @@ var hashfiles string
// NewStepExpressionEvaluator creates a new evaluator with the `inputs` of the enclosing workflow or composite action // NewStepExpressionEvaluator creates a new evaluator with the `inputs` of the enclosing workflow or composite action
func (rc *RunContext) NewStepExpressionEvaluator(ctx context.Context, step step) *ExpressionEvaluator { func (rc *RunContext) NewStepExpressionEvaluator(ctx context.Context, step step) *ExpressionEvaluator {
return rc.newStepExpressionEvaluator(ctx, step, rc.actionInputs, rc.getJobContext()) return rc.newStepExpressionEvaluator(ctx, step, rc.actionInputs, rc.getJobContext(), getStepGithubContext(ctx, step))
} }
// NewActionInputsExpressionEvaluator creates a new evaluator with the step's own with: values as `inputs` // NewActionInputsExpressionEvaluator creates a new evaluator with the step's own with: values as `inputs`
func (rc *RunContext) NewActionInputsExpressionEvaluator(ctx context.Context, step step) *ExpressionEvaluator { func (rc *RunContext) NewActionInputsExpressionEvaluator(ctx context.Context, step step) *ExpressionEvaluator {
return rc.newStepExpressionEvaluator(ctx, step, inputsFromEnv(*step.getEnv()), rc.getJobContext()) return rc.newStepExpressionEvaluator(ctx, step, inputsFromEnv(*step.getEnv()), rc.getJobContext(), getStepGithubContext(ctx, step))
} }
func (rc *RunContext) newStepExpressionEvaluator(ctx context.Context, step step, stepInputs map[string]any, jobContext *model.JobContext) *ExpressionEvaluator { func (rc *RunContext) newStepExpressionEvaluator(ctx context.Context, step step, stepInputs map[string]any, jobContext *model.JobContext, github *model.GithubContext) *ExpressionEvaluator {
// todo: cleanup EvaluationEnvironment creation // todo: cleanup EvaluationEnvironment creation
ee := &exprparser.EvaluationEnvironment{ ee := &exprparser.EvaluationEnvironment{
Github: step.getGithubContext(ctx), Github: github,
Env: *step.getEnv(), Env: *step.getEnv(),
Job: jobContext, Job: jobContext,
Steps: rc.StepResults, Steps: rc.StepResults,
@@ -112,7 +112,7 @@ func (rc *RunContext) newStepExpressionEvaluator(ctx context.Context, step step,
Needs: exprparser.NeedsContext(rc.Run), Needs: exprparser.NeedsContext(rc.Run),
// todo: should be unavailable // todo: should be unavailable
// but required to interpolate/evaluate the inputs in actions/composite // but required to interpolate/evaluate the inputs in actions/composite
Inputs: getEvaluatorInputs(rc, stepInputs, rc.getGithubContext(ctx)), Inputs: getEvaluatorInputs(rc, stepInputs, github),
HashFiles: getHashFilesFunction(ctx, rc), HashFiles: getHashFilesFunction(ctx, rc),
} }
ee.Runner = rc.getRunnerContext(ctx) ee.Runner = rc.getRunnerContext(ctx)
+1 -1
View File
@@ -111,7 +111,7 @@ func printPrepareActions(rc *RunContext, preparers []actionPreparer) common.Exec
if !ok || strings.Contains(action.Step.Uses, "${{") || action.action != nil { if !ok || strings.Contains(action.Step.Uses, "${{") || action.action != nil {
continue continue
} }
github := action.getGithubContext(common.WithLogger(ctx, logging.New(logging.DiscardHandler))) github := action.RunContext.getGithubContext(common.WithLogger(ctx, logging.New(logging.DiscardHandler)))
remote, err := newRemoteAction(action.Step.Uses, github, action.RunContext.Config.GitHubInstance) remote, err := newRemoteAction(action.Step.Uses, github, action.RunContext.Config.GitHubInstance)
if err != nil || action.skipsLocalCheckout(remote, github) { if err != nil || action.skipsLocalCheckout(remote, github) {
continue continue
+1 -2
View File
@@ -84,8 +84,7 @@ func (rc *RunContext) setupHookFileCommands(ctx context.Context, env map[string]
actPath := rc.JobContainer.GetActPath() actPath := rc.JobContainer.GetActPath()
env["GITHUB_ENV"] = path.Join(actPath, hookEnvFileCommand) env["GITHUB_ENV"] = path.Join(actPath, hookEnvFileCommand)
env["GITHUB_PATH"] = path.Join(actPath, hookPathFileCommand) env["GITHUB_PATH"] = path.Join(actPath, hookPathFileCommand)
env["GITEA_ENV"] = env["GITHUB_ENV"] aliasGiteaEnv(env)
env["GITEA_PATH"] = env["GITHUB_PATH"]
return rc.JobContainer.Copy(actPath, return rc.JobContainer.Copy(actPath,
&container.FileEntry{Name: hookEnvFileCommand, Mode: 0o666}, &container.FileEntry{Name: hookEnvFileCommand, Mode: 0o666},
+1
View File
@@ -121,6 +121,7 @@ func TestRunJobHook(t *testing.T) {
assert.Equal(t, "job", jobContainer.env["GITHUB_JOB"]) assert.Equal(t, "job", jobContainer.env["GITHUB_JOB"])
assert.Equal(t, "/var/run/act/workflow/hook-envs.txt", jobContainer.env["GITHUB_ENV"]) assert.Equal(t, "/var/run/act/workflow/hook-envs.txt", jobContainer.env["GITHUB_ENV"])
assert.Equal(t, "/var/run/act/workflow/hook-path.txt", jobContainer.env["GITHUB_PATH"]) assert.Equal(t, "/var/run/act/workflow/hook-path.txt", jobContainer.env["GITHUB_PATH"])
assert.Equal(t, jobContainer.env["GITHUB_PATH"], jobContainer.env["GITEA_PATH"])
}) })
t.Run("each hook has its own command state", func(t *testing.T) { t.Run("each hook has its own command state", func(t *testing.T) {
+18
View File
@@ -1602,6 +1602,23 @@ func isLocalCheckout(ghc *model.GithubContext, step *model.Step) bool {
return true return true
} }
// aliasGiteaEnv mirrors the runner's GITHUB_* variables as GITEA_*, call it after the last write to them.
func aliasGiteaEnv(env map[string]string) {
for _, suffix := range []string{
"ACTION", "ACTION_PATH", "ACTION_REF", "ACTION_REPOSITORY", "ACTIONS", "ACTOR", "ACTOR_ID", "API_URL", "BASE_REF",
"ENV", "EVENT_NAME", "EVENT_PATH", "GRAPHQL_URL", "HEAD_REF", "JOB", "OUTPUT", "PATH", "REF", "REF_NAME",
"REF_PROTECTED", "REF_TYPE", "REPOSITORY", "REPOSITORY_ID", "REPOSITORY_OWNER", "REPOSITORY_OWNER_ID",
"RETENTION_DAYS", "RUN_ATTEMPT", "RUN_ID", "RUN_NUMBER", "SERVER_URL", "SHA", "STATE", "STEP_SUMMARY",
"TRIGGERING_ACTOR", "WORKFLOW", "WORKFLOW_REF", "WORKFLOW_SHA", "WORKSPACE",
} {
if value, ok := env["GITHUB_"+suffix]; ok {
env["GITEA_"+suffix] = value
} else {
delete(env, "GITEA_"+suffix)
}
}
}
func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubContext, env map[string]string) { func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubContext, env map[string]string) {
if _, ok := env["CI"]; !ok { if _, ok := env["CI"]; !ok {
env["CI"] = "true" env["CI"] = "true"
@@ -1654,6 +1671,7 @@ func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubCon
env["GITHUB_GRAPHQL_URL"] = "" env["GITHUB_GRAPHQL_URL"] = ""
env["GITHUB_RUN_ATTEMPT"] = github.RunAttempt env["GITHUB_RUN_ATTEMPT"] = github.RunAttempt
aliasGiteaEnv(env)
env["RUNNER_NAME"] = rc.Config.RunnerName env["RUNNER_NAME"] = rc.Config.RunnerName
env["RUNNER_ENVIRONMENT"] = "self-hosted" env["RUNNER_ENVIRONMENT"] = "self-hosted"
+14 -14
View File
@@ -877,25 +877,25 @@ func TestGetGitHubContext(t *testing.T) {
RepositoryID: "34", RepositoryID: "34",
RepositoryOwnerID: "56", RepositoryOwnerID: "56",
WorkflowRef: "owner/repo/.gitea/workflows/test.yml@refs/heads/main", WorkflowRef: "owner/repo/.gitea/workflows/test.yml@refs/heads/main",
WorkflowSha: "workflow-sha",
RefProtected: true, RefProtected: true,
} }
ghc = rc.getGithubContext(t.Context()) ghc = rc.getGithubContext(t.Context())
assert.Equal(t, "preset-actor", ghc.Actor) assert.Equal(t, "preset-actor", ghc.Actor)
assert.Equal(t, "triggerer", ghc.TriggeringActor) assert.Equal(t, "triggerer", ghc.TriggeringActor)
assert.Equal(t, "job1", ghc.Job) assert.Equal(t, "job1", ghc.Job)
env := map[string]string{} env := map[string]string{"GITHUB_WORKFLOW_SHA": "workflow-sha"}
rc.withGithubEnv(t.Context(), ghc, env) rc.withGithubEnv(t.Context(), ghc, env)
for name, value := range map[string]string{ for suffix, value := range map[string]string{
"GITHUB_ACTOR_ID": "12", "ACTOR_ID": "12",
"GITHUB_TRIGGERING_ACTOR": "triggerer", "TRIGGERING_ACTOR": "triggerer",
"GITHUB_REPOSITORY_ID": "34", "REPOSITORY_ID": "34",
"GITHUB_REPOSITORY_OWNER_ID": "56", "REPOSITORY_OWNER_ID": "56",
"GITHUB_WORKFLOW_REF": "owner/repo/.gitea/workflows/test.yml@refs/heads/main", "WORKFLOW_REF": "owner/repo/.gitea/workflows/test.yml@refs/heads/main",
"GITHUB_WORKFLOW_SHA": "workflow-sha", "WORKFLOW_SHA": "workflow-sha",
"GITHUB_REF_PROTECTED": "true", "REF_PROTECTED": "true",
} { } {
assert.Equal(t, value, env[name], name) assert.Equal(t, value, env["GITHUB_"+suffix], suffix)
assert.Equal(t, value, env["GITEA_"+suffix], suffix)
} }
ghc.RefProtected = false ghc.RefProtected = false
rc.withGithubEnv(t.Context(), ghc, env) rc.withGithubEnv(t.Context(), ghc, env)
@@ -922,12 +922,12 @@ func TestGetGitHubContext(t *testing.T) {
github := rc.getGithubContext(t.Context()) github := rc.getGithubContext(t.Context())
assert.Equal(t, testcase.serverURL, github.ServerURL) assert.Equal(t, testcase.serverURL, github.ServerURL)
assert.Equal(t, testcase.apiURL, github.APIURL) assert.Equal(t, testcase.apiURL, github.APIURL)
env := map[string]string{} env := map[string]string{"GITEA_SERVER_URL": "stale", "GITEA_API_URL": "stale"}
rc.withGithubEnv(t.Context(), github, env) rc.withGithubEnv(t.Context(), github, env)
_, exported := env["GITHUB_SERVER_URL"] _, exported := env["GITHUB_SERVER_URL"]
assert.Equal(t, testcase.serverURL != "", exported) assert.Equal(t, testcase.serverURL != "", exported)
assert.Equal(t, testcase.serverURL, env["GITHUB_SERVER_URL"]) assert.Equal(t, testcase.serverURL, env["GITEA_SERVER_URL"])
assert.Equal(t, testcase.apiURL, env["GITHUB_API_URL"]) assert.Equal(t, testcase.apiURL, env["GITEA_API_URL"])
}) })
} }
} }
+18 -12
View File
@@ -27,12 +27,24 @@ type step interface {
post() common.Executor post() common.Executor
getRunContext() *RunContext getRunContext() *RunContext
getGithubContext(ctx context.Context) *model.GithubContext
getStepModel() *model.Step getStepModel() *model.Step
getEnv() *map[string]string getEnv() *map[string]string
getIfExpression(context context.Context, stage stepStage) string getIfExpression(context context.Context, stage stepStage) string
} }
func stepActionIdentity(step step) (repository, ref string) {
if action, ok := step.(interface{ actionIdentity() (string, string) }); ok {
return action.actionIdentity()
}
return "", ""
}
func getStepGithubContext(ctx context.Context, step step) *model.GithubContext {
github := step.getRunContext().getGithubContext(ctx)
github.ActionRepository, github.ActionRef = stepActionIdentity(step)
return github
}
type stepStage int type stepStage int
const ( const (
@@ -92,6 +104,8 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
runStep, err = isStepEnabled(ctx, ifExpression, step, stage) runStep, err = isStepEnabled(ctx, ifExpression, step, stage)
} }
if err == nil && runStep { if err == nil && runStep {
env := *step.getEnv()
env["GITHUB_ACTION_REPOSITORY"], env["GITHUB_ACTION_REF"] = stepActionIdentity(step)
err = setupInputs(ctx, step) err = setupInputs(ctx, step)
} }
if err != nil { if err != nil {
@@ -143,15 +157,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
stepSummaryIndex := topRC.CurrentStepIndex stepSummaryIndex := topRC.CurrentStepIndex
summaryFileCommand := path.Join("workflow", "step-summary-"+strconv.Itoa(stepSummaryIndex)+".md") summaryFileCommand := path.Join("workflow", "step-summary-"+strconv.Itoa(stepSummaryIndex)+".md")
(*step.getEnv())["GITHUB_STEP_SUMMARY"] = path.Join(actPath, summaryFileCommand) (*step.getEnv())["GITHUB_STEP_SUMMARY"] = path.Join(actPath, summaryFileCommand)
aliasGiteaEnv(*step.getEnv())
{
// For Gitea
(*step.getEnv())["GITEA_OUTPUT"] = (*step.getEnv())["GITHUB_OUTPUT"]
(*step.getEnv())["GITEA_STATE"] = (*step.getEnv())["GITHUB_STATE"]
(*step.getEnv())["GITEA_PATH"] = (*step.getEnv())["GITHUB_PATH"]
(*step.getEnv())["GITEA_ENV"] = (*step.getEnv())["GITHUB_ENV"]
(*step.getEnv())["GITEA_STEP_SUMMARY"] = (*step.getEnv())["GITHUB_STEP_SUMMARY"]
}
// Reset the per-phase file-command files. GITHUB_STEP_SUMMARY is intentionally // Reset the per-phase file-command files. GITHUB_STEP_SUMMARY is intentionally
// excluded here and initialized below at most once per step so writes from later // excluded here and initialized below at most once per step so writes from later
@@ -253,7 +259,7 @@ func setupEnv(ctx context.Context, step step) error {
env := step.getEnv() env := step.getEnv()
// container env is the image's baseline, which job env and $GITHUB_ENV override // container env is the image's baseline, which job env and $GITHUB_ENV override
mergeIntoMap(step, env, rc.containerSpec.Env, rc.GetEnv()) mergeIntoMap(step, env, rc.containerSpec.Env, rc.GetEnv())
rc.withGithubEnv(ctx, step.getGithubContext(ctx), *env) rc.withGithubEnv(ctx, getStepGithubContext(ctx, step), *env)
var err error var err error
exprEval := rc.NewExpressionEvaluator(ctx) exprEval := rc.NewExpressionEvaluator(ctx)
@@ -302,7 +308,7 @@ func isStepEnabled(ctx context.Context, expr string, step step, stage stepStage)
if rc.Parent != nil && stage == stepStageMain && jobContext.Status != "cancelled" { if rc.Parent != nil && stage == stepStageMain && jobContext.Status != "cancelled" {
jobContext.Status = rc.ownStatus() jobContext.Status = rc.ownStatus()
} }
runStep, err := EvalBool(ctx, rc.newStepExpressionEvaluator(ctx, step, rc.actionInputs, jobContext), expr, defaultStatusCheck) runStep, err := EvalBool(ctx, rc.newStepExpressionEvaluator(ctx, step, rc.actionInputs, jobContext, rc.getGithubContext(ctx)), expr, defaultStatusCheck)
if err != nil { if err != nil {
return false, fmt.Errorf("if-expression %q evaluation failed: %s", expr, err) return false, fmt.Errorf("if-expression %q evaluation failed: %s", expr, err)
} }
+1 -5
View File
@@ -39,7 +39,7 @@ func (sab *stepActionBuiltin) main() common.Executor {
rc.ApplyExtraPath(ctx, &sab.env) rc.ApplyExtraPath(ctx, &sab.env)
return sab.run(ctx, &action.Context{ return sab.run(ctx, &action.Context{
Container: rc.JobContainer, Container: rc.JobContainer,
Github: rc.getGithubContext(ctx), Github: getStepGithubContext(ctx, sab),
Inputs: sab.Step.With, Inputs: sab.Step.With,
HostWorkdir: rc.Config.Workdir, HostWorkdir: rc.Config.Workdir,
Workspace: rc.JobContainer.ToContainerPath(rc.Config.Workdir), Workspace: rc.JobContainer.ToContainerPath(rc.Config.Workdir),
@@ -58,10 +58,6 @@ func (sab *stepActionBuiltin) post() common.Executor { return common.NewPipeline
func (sab *stepActionBuiltin) getRunContext() *RunContext { return sab.RunContext } func (sab *stepActionBuiltin) getRunContext() *RunContext { return sab.RunContext }
func (sab *stepActionBuiltin) getGithubContext(ctx context.Context) *model.GithubContext {
return sab.RunContext.getGithubContext(ctx)
}
func (sab *stepActionBuiltin) getStepModel() *model.Step { return sab.Step } func (sab *stepActionBuiltin) getStepModel() *model.Step { return sab.Step }
func (sab *stepActionBuiltin) getEnv() *map[string]string { return &sab.env } func (sab *stepActionBuiltin) getEnv() *map[string]string { return &sab.env }
-4
View File
@@ -95,10 +95,6 @@ func (sal *stepActionLocal) getRunContext() *RunContext {
return sal.RunContext return sal.RunContext
} }
func (sal *stepActionLocal) getGithubContext(ctx context.Context) *model.GithubContext {
return sal.getRunContext().getGithubContext(ctx)
}
func (sal *stepActionLocal) getStepModel() *model.Step { func (sal *stepActionLocal) getStepModel() *model.Step {
return sal.Step return sal.Step
} }
+20 -2
View File
@@ -18,6 +18,7 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.yaml.in/yaml/v4"
) )
type stepActionMocks struct { type stepActionMocks struct {
@@ -53,7 +54,7 @@ func TestStepActionLocalTest(t *testing.T) {
readAction: salm.readAction, readAction: salm.readAction,
runAction: salm.runAction, runAction: salm.runAction,
RunContext: &RunContext{ RunContext: &RunContext{
StepResults: map[string]*model.StepResult{}, StepResults: map[string]*model.StepResult{"previous": {Outputs: map[string]string{"value": "previous"}}},
ExprEval: &expressionEvaluator{}, ExprEval: &expressionEvaluator{},
Config: &Config{ Config: &Config{
Workdir: "/tmp", Workdir: "/tmp",
@@ -68,25 +69,42 @@ func TestStepActionLocalTest(t *testing.T) {
}, },
}, },
JobContainer: cm, JobContainer: cm,
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/action", "GITHUB_ACTION_REF": "v1"},
Matrix: map[string]any{"value": "matrix"},
actionInputs: map[string]any{"shared": "input"},
}, },
Step: &model.Step{ Step: &model.Step{
ID: "1", ID: "1",
Uses: "./path/to/action", Uses: "./path/to/action",
If: yaml.Node{Value: "github.action_repository == 'other/action' && github.action_ref == 'v1' && " +
"env.STEP_VALUE == 'step' && env.GITHUB_ACTION_REPOSITORY == 'forged/repo' && env.GITHUB_ACTION_REF == 'forged-ref' && " +
"steps.previous.outputs.value == 'previous' && inputs.shared == 'input' && matrix.value == 'matrix'"},
}, },
} }
require.NoError(t, sal.Step.Env.Encode(map[string]string{"STEP_VALUE": "step", "GITHUB_ACTION_REPOSITORY": "forged/repo", "GITHUB_ACTION_REF": "forged-ref"}))
require.NoError(t, sal.Step.RawWith.Encode(map[string]string{"identity": "${{ github.action_repository }}|${{ github.action_ref }}"}))
salm.On("readAction", sal.Step, filepath.Clean("/tmp/path/to/action"), "", mock.Anything, mock.Anything). salm.On("readAction", sal.Step, filepath.Clean("/tmp/path/to/action"), "", mock.Anything, mock.Anything).
Return(&model.Action{}, nil) Return(&model.Action{}, nil)
expectStepFileCommands(t, cm) expectStepFileCommands(t, cm)
salm.On("runAction", sal, filepath.Clean("/tmp/path/to/action"), (*remoteAction)(nil)).Return(noopExecutor) salm.On("runAction", sal, filepath.Clean("/tmp/path/to/action"), (*remoteAction)(nil)).Return(func(ctx context.Context) error {
sal.RunContext.setOutput(ctx, map[string]string{"name": "value"}, "ran")
return nil
})
err := sal.pre()(ctx) err := sal.pre()(ctx)
require.NoError(t, err) require.NoError(t, err)
err = sal.main()(ctx) err = sal.main()(ctx)
require.NoError(t, err) require.NoError(t, err)
assert.Empty(t, sal.env["GITHUB_ACTION_REPOSITORY"])
assert.Empty(t, sal.env["GITHUB_ACTION_REF"])
assert.Equal(t, "|", sal.Step.With["identity"])
assert.Equal(t, "ran", sal.RunContext.StepResults["1"].Outputs["value"])
assert.Equal(t, "other/action", sal.RunContext.Env["GITHUB_ACTION_REPOSITORY"])
assert.Equal(t, "v1", sal.RunContext.Env["GITHUB_ACTION_REF"])
cm.AssertExpectations(t) cm.AssertExpectations(t)
salm.AssertExpectations(t) salm.AssertExpectations(t)
+7 -13
View File
@@ -75,7 +75,7 @@ func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
} }
sar.Step.Uses = uses sar.Step.Uses = uses
github := sar.getGithubContext(ctx) // read before remoteAction is set, so `$/` resolves against the enclosing action github := sar.RunContext.getGithubContext(ctx)
if sar.remoteAction, err = newRemoteAction(sar.Step.Uses, github, sar.RunContext.Config.GitHubInstance); err != nil { if sar.remoteAction, err = newRemoteAction(sar.Step.Uses, github, sar.RunContext.Config.GitHubInstance); err != nil {
return err return err
} }
@@ -153,7 +153,7 @@ func (sar *stepActionRemote) main() common.Executor {
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
sar.prepareActionExecutor(), sar.prepareActionExecutor(),
runStepExecutor(sar, stepStageMain, func(ctx context.Context) error { runStepExecutor(sar, stepStageMain, func(ctx context.Context) error {
github := sar.getGithubContext(ctx) github := getStepGithubContext(ctx, sar)
if sar.skipsLocalCheckout(sar.remoteAction, github) { if sar.skipsLocalCheckout(sar.remoteAction, github) {
printRunActionHeader(ctx, sar.Step, nil, sar.env, sar.RunContext) printRunActionHeader(ctx, sar.Step, nil, sar.env, sar.RunContext)
if sar.RunContext.Config.BindWorkdir { if sar.RunContext.Config.BindWorkdir {
@@ -181,17 +181,11 @@ func (sar *stepActionRemote) getRunContext() *RunContext {
return sar.RunContext return sar.RunContext
} }
func (sar *stepActionRemote) getGithubContext(ctx context.Context) *model.GithubContext { func (sar *stepActionRemote) actionIdentity() (string, string) {
ghc := sar.getRunContext().getGithubContext(ctx) if sar.remoteAction == nil {
return "", ""
// extend github context if we already have an initialized remoteAction
remoteAction := sar.remoteAction
if remoteAction != nil {
ghc.ActionRepository = fmt.Sprintf("%s/%s", remoteAction.Org, remoteAction.Repo)
ghc.ActionRef = remoteAction.Ref
} }
return sar.remoteAction.Org + "/" + sar.remoteAction.Repo, sar.remoteAction.Ref
return ghc
} }
func (sar *stepActionRemote) getStepModel() *model.Step { func (sar *stepActionRemote) getStepModel() *model.Step {
@@ -205,7 +199,7 @@ func (sar *stepActionRemote) getEnv() *map[string]string {
func (sar *stepActionRemote) getIfExpression(ctx context.Context, stage stepStage) string { func (sar *stepActionRemote) getIfExpression(ctx context.Context, stage stepStage) string {
switch stage { switch stage {
case stepStagePre: case stepStagePre:
github := sar.getGithubContext(ctx) github := getStepGithubContext(ctx, sar)
if sar.skipsLocalCheckout(sar.remoteAction, github) { if sar.skipsLocalCheckout(sar.remoteAction, github) {
// skip local checkout pre step // skip local checkout pre step
return "false" return "false"
+5 -5
View File
@@ -76,7 +76,7 @@ func TestStepActionRemote(t *testing.T) {
status model.StepStatus status model.StepStatus
runError error runError error
}{ }{
{name: "run-successful", status: model.StepStatusSuccess}, {name: "run-successful", condition: "github.action_repository == 'other/composite' && github.action_ref == 'v2'", status: model.StepStatusSuccess},
{name: "run-skipped", condition: "false", status: model.StepStatusSkipped}, {name: "run-skipped", condition: "false", status: model.StepStatusSkipped},
{name: "run-error", status: model.StepStatusFailure, runError: assert.AnError}, {name: "run-error", status: model.StepStatusFailure, runError: assert.AnError},
} { } {
@@ -110,6 +110,7 @@ func TestStepActionRemote(t *testing.T) {
}, },
StepResults: map[string]*model.StepResult{}, StepResults: map[string]*model.StepResult{},
JobContainer: cm, JobContainer: cm,
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/composite", "GITHUB_ACTION_REF": "v2"},
}, },
Step: &model.Step{ID: "step", Uses: "remote/action@v1", If: yaml.Node{Value: tt.condition}}, Step: &model.Step{ID: "step", Uses: "remote/action@v1", If: yaml.Node{Value: tt.condition}},
readAction: sarm.readAction, readAction: sarm.readAction,
@@ -127,6 +128,8 @@ func TestStepActionRemote(t *testing.T) {
require.NoError(t, sar.pre()(ctx)) require.NoError(t, sar.pre()(ctx))
assert.Equal(t, tt.runError, sar.main()(ctx)) assert.Equal(t, tt.runError, sar.main()(ctx))
assert.True(t, clonedAction) assert.True(t, clonedAction)
assert.Equal(t, "remote/action", sar.env["GITHUB_ACTION_REPOSITORY"])
assert.Equal(t, "v1", sar.env["GITHUB_ACTION_REF"])
assert.Equal(t, &model.StepResult{Conclusion: tt.status, Outcome: tt.status, Outputs: map[string]string{}}, sar.RunContext.StepResults["step"]) assert.Equal(t, &model.StepResult{Conclusion: tt.status, Outcome: tt.status, Outputs: map[string]string{}}, sar.RunContext.StepResults["step"])
sarm.AssertExpectations(t) sarm.AssertExpectations(t)
@@ -182,7 +185,7 @@ func TestStepActionRemotePrepare(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
root, err := prepare(job, "org/repo@v1") root, err := prepare(job, "org/repo@v1")
require.NoError(t, err) require.NoError(t, err)
nested, err := prepare(&RunContext{Config: job.Config, Run: job.Run, JobContainer: job.JobContainer, Parent: job}, "org/repo/save@v1") nested, err := prepare(&RunContext{Config: job.Config, Run: job.Run, JobContainer: job.JobContainer, Parent: job, Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "org/repo", "GITHUB_ACTION_REF": "v1"}}, "$/save")
require.NoError(t, err) require.NoError(t, err)
other, err := prepare(job, "org/repo@v2") other, err := prepare(job, "org/repo@v2")
require.NoError(t, err) require.NoError(t, err)
@@ -346,9 +349,6 @@ func Test_newRemoteActionSelfRepo(t *testing.T) {
}, },
}, },
{name: "unknown instance", action: "$/.gitea/actions/build", github: &model.GithubContext{Repository: "owner/workflow-repo", Sha: "abc123"}}, {name: "unknown instance", action: "$/.gitea/actions/build", github: &model.GithubContext{Repository: "owner/workflow-repo", Sha: "abc123"}},
{name: "empty path", action: "$/", github: workflow},
{name: "ref suffix is not allowed", action: "$/.gitea/actions/build@v1", github: workflow},
{name: "path traversal", action: "$/../escape", github: workflow},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
-4
View File
@@ -35,10 +35,6 @@ func (sd *stepDocker) getRunContext() *RunContext {
return sd.RunContext return sd.RunContext
} }
func (sd *stepDocker) getGithubContext(ctx context.Context) *model.GithubContext {
return sd.getRunContext().getGithubContext(ctx)
}
func (sd *stepDocker) getStepModel() *model.Step { func (sd *stepDocker) getStepModel() *model.Step {
return sd.Step return sd.Step
} }
+9
View File
@@ -14,6 +14,7 @@ import (
"gitea.dev/actionslib/pkg/model" "gitea.dev/actionslib/pkg/model"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.yaml.in/yaml/v4"
) )
func TestStepDockerMain(t *testing.T) { func TestStepDockerMain(t *testing.T) {
@@ -57,13 +58,16 @@ func TestStepDockerMain(t *testing.T) {
}, },
}, },
JobContainer: cm, JobContainer: cm,
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/composite", "GITHUB_ACTION_REF": "v1"},
}, },
Step: &model.Step{ Step: &model.Step{
ID: "1", ID: "1",
Uses: "docker://node:14", Uses: "docker://node:14",
WorkingDirectory: "workdir", WorkingDirectory: "workdir",
If: yaml.Node{Value: "github.action_repository == 'other/composite' && github.action_ref == 'v1'"},
}, },
} }
require.NoError(t, sd.Step.RawWith.Encode(map[string]string{"args": "identity:${{ github.action_repository }}:${{ github.action_ref }}"}))
sd.RunContext.ExprEval = sd.RunContext.NewExpressionEvaluator(ctx) sd.RunContext.ExprEval = sd.RunContext.NewExpressionEvaluator(ctx)
cm.On("Pull", false).Return(noopExecutor) cm.On("Pull", false).Return(noopExecutor)
@@ -80,6 +84,11 @@ func TestStepDockerMain(t *testing.T) {
require.NoError(t, sd.main()(ctx)) require.NoError(t, sd.main()(ctx))
assert.Equal(t, "node:14", input.Image) assert.Equal(t, "node:14", input.Image)
assert.Equal(t, []string{"identity::"}, input.Cmd)
assert.Contains(t, input.Env, "GITHUB_ACTION_REPOSITORY=")
assert.Contains(t, input.Env, "GITHUB_ACTION_REF=")
assert.Equal(t, "other/composite", sd.RunContext.Env["GITHUB_ACTION_REPOSITORY"])
assert.Equal(t, "v1", sd.RunContext.Env["GITHUB_ACTION_REF"])
assert.Empty(t, input.Username) assert.Empty(t, input.Username)
assert.Empty(t, input.Password) assert.Empty(t, input.Password)
-4
View File
@@ -222,10 +222,6 @@ func (sr *stepRun) getRunContext() *RunContext {
return sr.RunContext return sr.RunContext
} }
func (sr *stepRun) getGithubContext(ctx context.Context) *model.GithubContext {
return sr.getRunContext().getGithubContext(ctx)
}
func (sr *stepRun) getStepModel() *model.Step { func (sr *stepRun) getStepModel() *model.Step {
return sr.Step return sr.Step
} }
+16 -2
View File
@@ -19,6 +19,7 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.yaml.in/yaml/v4"
) )
type shellContainerMock struct{ *containerMock } type shellContainerMock struct{ *containerMock }
@@ -36,7 +37,6 @@ func TestStepRun(t *testing.T) {
sr := &stepRun{ sr := &stepRun{
RunContext: &RunContext{ RunContext: &RunContext{
StepResults: map[string]*model.StepResult{}, StepResults: map[string]*model.StepResult{},
ExprEval: &expressionEvaluator{},
Config: &Config{Env: testCheckoutEnv()}, Config: &Config{Env: testCheckoutEnv()},
Run: &model.Run{ Run: &model.Run{
JobID: "1", JobID: "1",
@@ -48,19 +48,33 @@ func TestStepRun(t *testing.T) {
}, },
JobContainer: cm, JobContainer: cm,
jobRunDefaults: model.RunDefaults{Shell: "bash"}, jobRunDefaults: model.RunDefaults{Shell: "bash"},
Env: map[string]string{"GITHUB_ACTION_REPOSITORY": "other/composite", "GITHUB_ACTION_REF": "v1"},
}, },
Step: &model.Step{ Step: &model.Step{
ID: "1", ID: "1",
Run: "cmd", Run: "cmd${{ github.action_repository }}${{ github.action_ref }}",
WorkingDirectory: "workdir", WorkingDirectory: "workdir",
If: yaml.Node{Value: "github.action_repository == 'other/composite' && github.action_ref == 'v1'"},
}, },
} }
sr.RunContext.ExprEval = sr.RunContext.NewExpressionEvaluator(t.Context())
require.NoError(t, sr.Step.Env.Encode(map[string]string{
"GITHUB_ACTION_REPOSITORY": "forged/repo", "GITHUB_ACTION_REF": "forged-ref",
"GITEA_ACTION_REPOSITORY": "forged/repo", "GITEA_ACTION_REF": "forged-ref",
}))
cm.On("Copy", "/var/run/act", []*container.FileEntry{fileEntry}).Return(noopExecutor) cm.On("Copy", "/var/run/act", []*container.FileEntry{fileEntry}).Return(noopExecutor)
cm.On("Exec", []string{"bash", "--noprofile", "--norc", "-e", "-o", "pipefail", "/var/run/act/workflow/1.sh"}, mock.AnythingOfType("map[string]string"), "", "workdir").Return(noopExecutor) cm.On("Exec", []string{"bash", "--noprofile", "--norc", "-e", "-o", "pipefail", "/var/run/act/workflow/1.sh"}, mock.AnythingOfType("map[string]string"), "", "workdir").Return(noopExecutor)
expectStepFileCommands(t, cm) expectStepFileCommands(t, cm)
require.NoError(t, sr.main()(t.Context())) require.NoError(t, sr.main()(t.Context()))
assert.Empty(t, sr.env["GITHUB_ACTION_REPOSITORY"])
assert.Empty(t, sr.env["GITHUB_ACTION_REF"])
assert.Empty(t, sr.env["GITEA_ACTION_REPOSITORY"])
assert.Empty(t, sr.env["GITEA_ACTION_REF"])
assert.Equal(t, sr.env["GITHUB_STEP_SUMMARY"], sr.env["GITEA_STEP_SUMMARY"])
assert.Equal(t, "other/composite", sr.RunContext.Env["GITHUB_ACTION_REPOSITORY"])
assert.Equal(t, "v1", sr.RunContext.Env["GITHUB_ACTION_REF"])
cm.AssertExpectations(t) cm.AssertExpectations(t)
} }
+12 -10
View File
@@ -155,18 +155,14 @@ func TestSetupEnv(t *testing.T) {
require.NoError(t, setupEnv(t.Context(), testStep)) require.NoError(t, setupEnv(t.Context(), testStep))
require.NoError(t, setupInputs(t.Context(), testStep)) require.NoError(t, setupInputs(t.Context(), testStep))
delete(env, "GITHUB_REF") for _, suffix := range []string{"REF", "REF_NAME", "REF_TYPE", "SHA", "WORKSPACE", "REPOSITORY", "REPOSITORY_OWNER", "ACTOR"} {
delete(env, "GITHUB_REF_NAME") delete(env, "GITHUB_"+suffix)
delete(env, "GITHUB_REF_TYPE") delete(env, "GITEA_"+suffix)
delete(env, "GITHUB_SHA") }
delete(env, "GITHUB_WORKSPACE")
delete(env, "GITHUB_REPOSITORY")
delete(env, "GITHUB_REPOSITORY_OWNER")
delete(env, "GITHUB_ACTOR")
delete(env, "RUNNER_NAME") delete(env, "RUNNER_NAME")
delete(env, "RUNNER_WORKSPACE") delete(env, "RUNNER_WORKSPACE")
assert.Equal(t, map[string]string{ expected := map[string]string{
"ACT": "true", "ACT": "true",
"ACT_SKIP_CHECKOUT": "true", "ACT_SKIP_CHECKOUT": "true",
"CI": "true", "CI": "true",
@@ -193,7 +189,13 @@ func TestSetupEnv(t *testing.T) {
"RUNNER_ENVIRONMENT": "self-hosted", "RUNNER_ENVIRONMENT": "self-hosted",
"RUNNER_PERFLOG": "/dev/null", "RUNNER_PERFLOG": "/dev/null",
"RUNNER_TRACKING_ID": "", "RUNNER_TRACKING_ID": "",
}, env) }
for name, value := range expected {
if suffix, ok := strings.CutPrefix(name, "GITHUB_"); ok {
expected["GITEA_"+suffix] = value
}
}
assert.Equal(t, expected, env)
for _, expression := range []string{"inputs.args", "env.ARGS"} { for _, expression := range []string{"inputs.args", "env.ARGS"} {
t.Run("deferred inputs from "+expression, func(t *testing.T) { t.Run("deferred inputs from "+expression, func(t *testing.T) {
+1 -1
View File
@@ -45,7 +45,7 @@ A hook path that does not exist inside the job environment fails the job with `N
## Environment ## Environment
A hook sees the job's environment: the workflow, job and `container:` `env:`, the runner's `envs`, and the `GITHUB_*` context variables, with the same masking applied to its output as to a step's. The step-specific ones (`GITHUB_ACTION`, `GITHUB_OUTPUT`, `GITHUB_STATE`) are not set — a hook is not a step, so `::save-state::` and `::set-output::` have nowhere to go. A hook sees the job's environment: the workflow, job and `container:` `env:`, the runner's `envs`, and the `GITHUB_*`/`GITEA_*` context variables, with the same masking applied to its output as to a step's. The step-specific ones (`GITHUB_ACTION`, `GITHUB_OUTPUT`, `GITHUB_STATE`) are not set — a hook is not a step, so `::save-state::` and `::set-output::` have nowhere to go.
Its stdout is part of the job log, after a collapsible group naming the hook and its shell, and is scanned for workflow commands. `::add-mask::` registers a value to be masked for the rest of the job. `::set-env::` and `::add-path::` apply to the steps that follow, but as in a step they need `ACTIONS_ALLOW_UNSECURE_COMMANDS: true`, and a refused one fails the hook. Its stdout is part of the job log, after a collapsible group naming the hook and its shell, and is scanned for workflow commands. `::add-mask::` registers a value to be masked for the rest of the job. `::set-env::` and `::add-path::` apply to the steps that follow, but as in a step they need `ACTIONS_ALLOW_UNSECURE_COMMANDS: true`, and a refused one fails the hook.
+2 -2
View File
@@ -6,7 +6,7 @@ toolchain go1.27.3
require ( require (
connectrpc.com/connect/v2 v2.0.1 connectrpc.com/connect/v2 v2.0.1
gitea.dev/actionslib v1.5.0 gitea.dev/actionslib v1.5.1
github.com/bmatcuk/doublestar/v4 v4.10.3 github.com/bmatcuk/doublestar/v4 v4.10.3
github.com/coder/websocket v1.8.15 github.com/coder/websocket v1.8.15
github.com/containerd/errdefs v1.0.0 github.com/containerd/errdefs v1.0.0
@@ -33,7 +33,7 @@ require (
go.opentelemetry.io/otel/trace v1.47.0 go.opentelemetry.io/otel/trace v1.47.0
go.opentelemetry.io/proto/otlp v1.11.1 go.opentelemetry.io/proto/otlp v1.11.1
go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525 go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525
golang.org/x/net v0.61.0 golang.org/x/net v0.62.0
golang.org/x/sync v0.24.0 golang.org/x/sync v0.24.0
golang.org/x/sys v0.49.0 golang.org/x/sys v0.49.0
golang.org/x/term v0.47.0 golang.org/x/term v0.47.0
+4 -4
View File
@@ -1,7 +1,7 @@
connectrpc.com/connect/v2 v2.0.1 h1:WTAycxe87UqxSM5oxS30d1dalZ+C5xz8Ec8O6zVYkHQ= connectrpc.com/connect/v2 v2.0.1 h1:WTAycxe87UqxSM5oxS30d1dalZ+C5xz8Ec8O6zVYkHQ=
connectrpc.com/connect/v2 v2.0.1/go.mod h1:N32OLrstcPUzkykFynnefFro3+xzSdOrZlK2jHmS/xo= connectrpc.com/connect/v2 v2.0.1/go.mod h1:N32OLrstcPUzkykFynnefFro3+xzSdOrZlK2jHmS/xo=
gitea.dev/actionslib v1.5.0 h1:dC3v2HTWbbvyNBSdg2W3kd4WNC/p+FnZaKpl1MdzM00= gitea.dev/actionslib v1.5.1 h1:QVOaENNWx4EFRlZvrWmMTKaKLbaqxdlhWLtsI4kf2v8=
gitea.dev/actionslib v1.5.0/go.mod h1:C9wmOFgDBCXyRvg4A3KTX8Vszs8v9+elH4QLGVDolqU= gitea.dev/actionslib v1.5.1/go.mod h1:C9wmOFgDBCXyRvg4A3KTX8Vszs8v9+elH4QLGVDolqU=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
@@ -111,8 +111,8 @@ go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg= go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525 h1:FtzHH3s777Nx50YcPAcodAPm1mNakc228nMYbr3f+90= go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525 h1:FtzHH3s777Nx50YcPAcodAPm1mNakc228nMYbr3f+90=
go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0= go.yaml.in/yaml/v4 v4.0.0-rc.6.0.20261013184558-04dd63131525/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw= golang.org/x/net v0.62.0 h1:18ELalfM5+XNRwzM2nYqXsZQ9ByS1uhj2O/U7+5jqRY=
golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI= golang.org/x/net v0.62.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA= golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA=
golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0= golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA= golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
+34 -14
View File
@@ -533,7 +533,7 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
// is removed when the task finishes, so a leaked token stops working as // is removed when the task finishes, so a leaked token stops working as
// soon as the job ends rather than remaining valid for the runner's // soon as the job ends rather than remaining valid for the runner's
// lifetime. // lifetime.
revokeCache, resultsURL := r.registerCacheForTask(giteaRuntimeToken, preset.Repository, cacheURL, reporter) revokeCache, resultsURL := r.registerCacheForTask(giteaRuntimeToken, cacheCredential(preset, cacheURL), reporter)
defer revokeCache() defer revokeCache()
eventJSON, err := json.Marshal(preset.Event) eventJSON, err := json.Marshal(preset.Event)
@@ -690,34 +690,54 @@ func (r *Runner) instanceOutOfReach(instance string) bool {
// //
// Safe with an empty token (older Gitea did not issue one). // Safe with an empty token (older Gitea did not issue one).
// It also returns the URL to advertise as ACTIONS_RESULTS_URL. // It also returns the URL to advertise as ACTIONS_RESULTS_URL.
func (r *Runner) registerCacheForTask(token, repo, publicURL string, reporter *report.Reporter) (func(), string) { func (r *Runner) registerCacheForTask(token string, cred artifactcache.JobCredential, reporter *report.Reporter) (func(), string) {
if token == "" || r.cacheHandler == nil { if token == "" || r.cacheHandler == nil {
return func() {}, "" return func() {}, ""
} }
cred := artifactcache.JobCredential{ cred.Results = r.envs["ACTIONS_RESULTS_URL"] // the instance as the job would reach it
Repo: repo, cred.InsecureTLS = r.cfg.Runner.Insecure
PublicURL: publicURL,
Results: r.envs["ACTIONS_RESULTS_URL"], // the instance as the job would reach it
InsecureTLS: r.cfg.Runner.Insecure,
}
revoke := r.cacheHandler.RegisterJob(token, cred) revoke := r.cacheHandler.RegisterJob(token, cred)
if r.cfg.Cache.ExternalServer != "" { if r.cfg.Cache.ExternalServer != "" {
revokeLocal, revokeExternal := revoke, r.registerExternalCacheJob(token, repo, publicURL, reporter) revokeLocal, revokeExternal := revoke, r.registerExternalCacheJob(token, cred, reporter)
revoke = func() { revokeLocal(); revokeExternal() } revoke = func() { revokeLocal(); revokeExternal() }
} }
return revoke, r.cacheHandler.ResultsURL(cred) return revoke, r.cacheHandler.ResultsURL(cred)
} }
// cacheCredential scopes the job's cache entries by ref, as GitHub does.
func cacheCredential(preset *model.GithubContext, publicURL string) artifactcache.JobCredential {
ref := preset.Ref
if strings.HasPrefix(preset.EventName, "pull_request") && preset.EventName != "pull_request_target" {
number, _ := preset.Event["number"].(float64)
ref = fmt.Sprintf("refs/pull/%d/head", int64(number)) // review events take github.ref from the reviewer's commit_id
}
return artifactcache.JobCredential{
Repo: preset.Repository,
Ref: ref,
BaseRef: branchRef(preset.BaseRef),
DefaultRef: branchRef(model.AsString(model.NestedMapLookup(preset.Event, "repository", "default_branch"))),
PublicURL: publicURL,
}
}
func branchRef(branch string) string {
if branch == "" {
return ""
}
return "refs/heads/" + branch
}
// registerExternalCacheJob POSTs to the remote cache-server's control-plane. // registerExternalCacheJob POSTs to the remote cache-server's control-plane.
// Failures are logged but not fatal: if registration fails, the cache will // Failures are logged but not fatal: if registration fails, the cache will
// 401 the job's requests — better than failing the whole task for a cache // 401 the job's requests — better than failing the whole task for a cache
// outage. The warning is mirrored to the job log so users can see why their // outage. The warning is mirrored to the job log so users can see why their
// cache calls 401, instead of having to read the runner daemon's stderr. // cache calls 401, instead of having to read the runner daemon's stderr.
func (r *Runner) registerExternalCacheJob(token, repo, publicURL string, reporter *report.Reporter) func() { func (r *Runner) registerExternalCacheJob(token string, cred artifactcache.JobCredential, reporter *report.Reporter) func() {
base := strings.TrimRight(r.cfg.Cache.ExternalServer, "/") base := strings.TrimRight(r.cfg.Cache.ExternalServer, "/")
if err := postInternalCache(base+"/_internal/register", r.cfg.Cache.ExternalSecret, map[string]any{ if err := postInternalCache(base+"/_internal/register", r.cfg.Cache.ExternalSecret, struct {
"token": token, "repo": repo, "public_url": publicURL, Token string `json:"token"`
}); err != nil { artifactcache.JobCredential
}{token, cred}); err != nil {
logging.Warnf("cache external_server register failed (%s): %v", base, err) logging.Warnf("cache external_server register failed (%s): %v", base, err)
if reporter != nil { if reporter != nil {
reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf( reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf(
@@ -736,7 +756,7 @@ func (r *Runner) registerExternalCacheJob(token, repo, publicURL string, reporte
} }
} }
func postInternalCache(url, secret string, body map[string]any) error { func postInternalCache(url, secret string, body any) error {
buf, err := json.Marshal(body) buf, err := json.Marshal(body)
if err != nil { if err != nil {
return err return err
+28 -3
View File
@@ -17,6 +17,7 @@ import (
clientmocks "gitea.com/gitea/runner/internal/pkg/client/mocks" clientmocks "gitea.com/gitea/runner/internal/pkg/client/mocks"
"gitea.com/gitea/runner/internal/pkg/config" "gitea.com/gitea/runner/internal/pkg/config"
"gitea.dev/actionslib/pkg/model"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@@ -24,7 +25,7 @@ import (
func TestRunner_registerCacheForTask_NoOps(t *testing.T) { func TestRunner_registerCacheForTask_NoOps(t *testing.T) {
for token, handler := range map[string]*artifactcache.Handler{"tok": nil, "": {}} { for token, handler := range map[string]*artifactcache.Handler{"tok": nil, "": {}} {
r := &Runner{cacheHandler: handler} r := &Runner{cacheHandler: handler}
unregister, _ := r.registerCacheForTask(token, "owner/repo", "", nil) unregister, _ := r.registerCacheForTask(token, artifactcache.JobCredential{Repo: "owner/repo"}, nil)
require.NotNil(t, unregister, token) require.NotNil(t, unregister, token)
unregister() unregister()
} }
@@ -39,7 +40,7 @@ func TestRunner_CacheFullFlow_MatchesToolkit(t *testing.T) {
r := &Runner{cfg: &config.Config{}, cacheHandler: handler, envs: map[string]string{"ACTIONS_RESULTS_URL": "https://gitea.example"}} r := &Runner{cfg: &config.Config{}, cacheHandler: handler, envs: map[string]string{"ACTIONS_RESULTS_URL": "https://gitea.example"}}
const publicURL = "http://a1b2c3d4e5f6:8088" const publicURL = "http://a1b2c3d4e5f6:8088"
token := "full-flow-token" token := "full-flow-token"
unregister, resultsURL := r.registerCacheForTask(token, "owner/repo", publicURL, nil) unregister, resultsURL := r.registerCacheForTask(token, artifactcache.JobCredential{Repo: "owner/repo", PublicURL: publicURL}, nil)
assert.Equal(t, publicURL, resultsURL) assert.Equal(t, publicURL, resultsURL)
base := handler.ExternalURL() + "/_apis/artifactcache" base := handler.ExternalURL() + "/_apis/artifactcache"
@@ -112,6 +113,24 @@ func decodeJSON(resp *http.Response, v any) error {
return json.UnmarshalRead(resp.Body, v) return json.UnmarshalRead(resp.Body, v)
} }
func TestCacheCredential(t *testing.T) {
repository := map[string]any{"default_branch": "main"}
review := &model.GithubContext{
Repository: "owner/repo", EventName: "pull_request_review", Ref: "refs/heads/main", BaseRef: "release",
Event: map[string]any{"repository": repository, "number": float64(3)},
}
assert.Equal(t, artifactcache.JobCredential{
Repo: "owner/repo", Ref: "refs/pull/3/head", BaseRef: "refs/heads/release", DefaultRef: "refs/heads/main", PublicURL: "http://cache",
}, cacheCredential(review, "http://cache"))
review.EventName, review.Ref = "pull_request_target", "refs/heads/release"
assert.Equal(t, "refs/heads/release", cacheCredential(review, "").Ref)
assert.Equal(t, artifactcache.JobCredential{
Repo: "owner/repo", Ref: "refs/heads/feature", DefaultRef: "refs/heads/main",
}, cacheCredential(&model.GithubContext{Repository: "owner/repo", Ref: "refs/heads/feature", Event: map[string]any{"repository": repository}}, ""))
}
func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) { func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
const secret = "shared-secret-for-tests" const secret = "shared-secret-for-tests"
remote, err := artifactcache.StartHandler(artifactcache.Options{Dir: t.TempDir(), OutboundIP: "127.0.0.1", InternalSecret: secret}) remote, err := artifactcache.StartHandler(artifactcache.Options{Dir: t.TempDir(), OutboundIP: "127.0.0.1", InternalSecret: secret})
@@ -151,7 +170,7 @@ func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
require.Equal(t, http.StatusUnauthorized, probe(), require.Equal(t, http.StatusUnauthorized, probe(),
"token must be unknown to the remote server before registration") "token must be unknown to the remote server before registration")
unregister, resultsURL := r.registerCacheForTask(token, repo, publicURL, nil) unregister, resultsURL := r.registerCacheForTask(token, artifactcache.JobCredential{Repo: repo, Ref: "refs/heads/main", PublicURL: publicURL}, nil)
require.NotEqual(t, http.StatusUnauthorized, probe(), require.NotEqual(t, http.StatusUnauthorized, probe(),
"token must be accepted after registerCacheForTask") "token must be accepted after registerCacheForTask")
@@ -194,6 +213,12 @@ func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
require.NoError(t, decodeJSON(resp, &hit)) require.NoError(t, decodeJSON(resp, &hit))
require.True(t, strings.HasPrefix(hit.ArchiveLocation, publicURL+"/"), hit.ArchiveLocation) require.True(t, strings.HasPrefix(hit.ArchiveLocation, publicURL+"/"), hit.ArchiveLocation)
unregisterOther, _ := r.registerCacheForTask("external-other-token", artifactcache.JobCredential{Repo: repo, Ref: "refs/heads/other"}, nil)
defer unregisterOther()
resp = cacheRequest(t, "external-other-token", http.MethodGet, base+"/cache?keys=ext-key&version=v", "", "", "")
resp.Body.Close()
assert.Equal(t, http.StatusNoContent, resp.StatusCode, "the remote server must scope the entry by the ref the runner sent")
dl, err := http.Get(strings.Replace(hit.ArchiveLocation, publicURL, r.cacheHandler.ExternalURL(), 1)) dl, err := http.Get(strings.Replace(hit.ArchiveLocation, publicURL, r.cacheHandler.ExternalURL(), 1))
require.NoError(t, err) require.NoError(t, err)
defer dl.Body.Close() defer dl.Body.Close()
+2 -1
View File
@@ -18,6 +18,7 @@ import (
"testing" "testing"
"time" "time"
"gitea.com/gitea/runner/act/artifactcache"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging" "gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/runner" "gitea.com/gitea/runner/act/runner"
@@ -325,7 +326,7 @@ func TestNewRunnerCacheServiceV2(t *testing.T) {
// The registration is what makes it true: the cache server takes the results service over, // The registration is what makes it true: the cache server takes the results service over,
// having been told which instance to forward the artifact half to. // having been told which instance to forward the artifact half to.
revoke, resultsURL := r.registerCacheForTask(token, "owner/repo", r.cacheHandler.ExternalURL(), nil) revoke, resultsURL := r.registerCacheForTask(token, artifactcache.JobCredential{Repo: "owner/repo", PublicURL: r.cacheHandler.ExternalURL()}, nil)
defer revoke() defer revoke()
require.Equal(t, r.cacheHandler.ExternalURL(), resultsURL) require.Equal(t, r.cacheHandler.ExternalURL(), resultsURL)
+1 -1
View File
@@ -104,7 +104,7 @@ func TestTreeKillCancelBeforeCaptureKillsTree(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { _ = stdin.Close() }) t.Cleanup(func() { _ = stdin.Close() })
t.Cleanup(func() { _ = input.Close() }) t.Cleanup(func() { _ = input.Close() })
cmd := exec.CommandContext(t.Context(), "/bin/sh", "-c", "cat <&0 & echo $!; read value") cmd := exec.CommandContext(t.Context(), "/bin/sh", "-c", "exec 3<&0; cat <&3 & echo $!; read value")
cmd.Stdin = stdin cmd.Stdin = stdin
cmd.SysProcAttr = SysProcAttr("", false) cmd.SysProcAttr = SysProcAttr("", false)
output, err := cmd.StdoutPipe() output, err := cmd.StdoutPipe()