Files
runner/DEVELOPMENT.md
T
silverwind 95ef17d022 fix: prevent masking gaps in workflow command logs (#1384)
Mask partially escaped workflow command output and complete one-character values while preserving literal output and stream line endings.

Fixes: https://gitea.com/gitea/runner/issues/300
Reviewed-on: https://gitea.com/gitea/runner/pulls/1384
Reviewed-by: bircni <[email protected]>
2026-11-05 08:01:44 +00:00

4.9 KiB

Development

Job log line format

Gitea stores one log row per line and its web UI decodes the payload, so getting the encoding wrong never fails a test here, it only shows up in the browser.

A row cannot contain a real newline. FormatLog rewrites \n to a literal backslash-n and truncates at 64 KiB on a byte boundary.

The payload of a line starting with a recognised prefix is decoded, with the escape set depending on the prefix:

prefix decodes
##[error] ##[warning] ##[notice] ##[debug] ##[group] ##[endgroup] ##[add-matcher] %25 %0D %0A %3B %5D
::error:: ::warning:: ::notice:: ::debug:: (with or without key=value properties), ::group:: ::endgroup:: ::add-matcher:: %25 %0D %0A
##[command] [command], or no recognised prefix nothing

Rules

  • Emitting a command line? Escape the payload with runner.EscapeCommandData. One escaper covers both forms: it escapes % first, so a literal %3B becomes %253B that the extra ##[…] rules cannot match, and a raw ; or ] is never decoded. It is also what makes multi-line work, \n becomes %0A and the UI turns it back into a line break.
  • Forwarding a command from step output? Preserve its escaping unless masking changes it. Decoding for display happens in the UI.
  • No prefix? Do not escape, and split multi-line values into one row each.
  • Interpolating a secret? Masking runs after escaping. Recognised command payloads are normalised before masking to cover partially escaped values. Their original escaping is preserved when no replacement occurs. AppendSecretMasker also registers encoded forms for other output. Complete nonblank one-character values are masked, while one-byte fragments of multiline values are skipped.
  • Command properties also escape %3A and %2C, which the UI never decodes, so the command handler folds an annotation's location into its decoded message and escapes the result again.

Logging

Log through act/common/logging; the linter rejects importing log/slog elsewhere. A job's logger comes from common.Logger(ctx); code with no job uses the package functions (logging.Infof, logging.With). Both take printf-style messages, and With("key", value) adds fields.

  • The process-wide logger masks the secrets of every live task, the job logger masks its own. Going around either leaks them.
  • The job logger's output can reach the job log workflow authors see, the process-wide logger's stays on the runner host, so keep operator-only detail (host paths, internal addresses) off the job logger.
  • Libraries that log through slog's global logger are forwarded to the process-wide one, so they follow its level and masking too. Avoid dependencies that log through logrus, nothing forwards it.
  • Tests read what was logged with logtest.NewLogger(), or logtest.SetDefault(t) for the package functions.

Built-in actions

Each lives in internal/pkg/action/<name> and is registered in builtinActions in act/runner/step_action_builtin.go.

End-to-end compatibility tests

make test-e2e runs the runner against E2E_GITEA_IMAGE. It defaults to the nightly image. It requires Docker and is excluded from make test. CI runs stable and nightly variants in parallel. Outside a container it runs go test in a Linux container on its own network, as CI does, so both take the same paths. E2E_ARGS passes extra flags, e.g. make test-e2e E2E_ARGS='-run TestCompatibility/dispatch'.

The cli_* scenarios build the binary and drive register and daemon as a user would, including --once, --labels and shutdown with a running job. failure_paths covers failed, skipped and tolerated jobs, job_timeout a job killed at the runner's deadline. label_routing and capacity cover scheduling, reconnect drops the connection to Gitea through a proxy, and host_mode and reusable_workflow cover host labels and workflow_call. dispatch, failure_paths and reusable_workflow run in every mode: through the daemon and exec, each on Docker and Kubernetes.

The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated repository runners. A run path is <workflow>@<ref> and a log row is <timestamp>Z <payload>.

Windows tests

make test-windows cross-compiles the test binaries, installs Windows Server 2025 Core in a dockur/windows QEMU VM on the docker host and runs them there. The host must be amd64 and allow privileged containers. WINDOWS_TEST_TIMEOUT sets the minutes allowed for installing Windows and running the tests, 165 by default. The VM uses KVM when the host has /dev/kvm, otherwise Windows is emulated and much slower. Docker-backed tests skip in the VM and the race detector is off, it needs cgo. WINDOWS_TEST_CACHE names a directory to keep the installed Windows in, so later runs skip the install. CI runs this on every pull request.