Mask partially escaped workflow command output and complete one-character values while preserving literal output and stream line endings. Fixes: https://gitea.com/gitea/runner/issues/300 Reviewed-on: https://gitea.com/gitea/runner/pulls/1384 Reviewed-by: bircni <[email protected]>
4.9 KiB
Development
Job log line format
Gitea stores one log row per line and its web UI decodes the payload, so getting the encoding wrong never fails a test here, it only shows up in the browser.
A row cannot contain a real newline. FormatLog rewrites \n to a literal backslash-n and
truncates at 64 KiB on a byte boundary.
The payload of a line starting with a recognised prefix is decoded, with the escape set depending on the prefix:
| prefix | decodes |
|---|---|
##[error] ##[warning] ##[notice] ##[debug] ##[group] ##[endgroup] ##[add-matcher] |
%25 %0D %0A %3B %5D |
::error:: ::warning:: ::notice:: ::debug:: (with or without key=value properties), ::group:: ::endgroup:: ::add-matcher:: |
%25 %0D %0A |
##[command] [command], or no recognised prefix |
nothing |
Rules
- Emitting a command line? Escape the payload with
runner.EscapeCommandData. One escaper covers both forms: it escapes%first, so a literal%3Bbecomes%253Bthat the extra##[…]rules cannot match, and a raw;or]is never decoded. It is also what makes multi-line work,\nbecomes%0Aand the UI turns it back into a line break. - Forwarding a command from step output? Preserve its escaping unless masking changes it. Decoding for display happens in the UI.
- No prefix? Do not escape, and split multi-line values into one row each.
- Interpolating a secret? Masking runs after escaping. Recognised command payloads are
normalised before masking to cover partially escaped values. Their original escaping is
preserved when no replacement occurs.
AppendSecretMaskeralso registers encoded forms for other output. Complete nonblank one-character values are masked, while one-byte fragments of multiline values are skipped. - Command properties also escape
%3Aand%2C, which the UI never decodes, so the command handler folds an annotation's location into its decoded message and escapes the result again.
Logging
Log through act/common/logging; the linter rejects importing log/slog elsewhere.
A job's logger comes from common.Logger(ctx); code with no job uses the package functions
(logging.Infof, logging.With). Both take printf-style messages, and With("key", value) adds
fields.
- The process-wide logger masks the secrets of every live task, the job logger masks its own. Going around either leaks them.
- The job logger's output can reach the job log workflow authors see, the process-wide logger's stays on the runner host, so keep operator-only detail (host paths, internal addresses) off the job logger.
- Libraries that log through slog's global logger are forwarded to the process-wide one, so they follow its level and masking too. Avoid dependencies that log through logrus, nothing forwards it.
- Tests read what was logged with
logtest.NewLogger(), orlogtest.SetDefault(t)for the package functions.
Built-in actions
Each lives in internal/pkg/action/<name> and is registered in builtinActions in
act/runner/step_action_builtin.go.
End-to-end compatibility tests
make test-e2e runs the runner against E2E_GITEA_IMAGE. It defaults to the nightly image.
It requires Docker and is excluded from make test. CI runs stable and nightly variants in
parallel. Outside a container it runs go test in a Linux container on its own network, as CI does,
so both take the same paths. E2E_ARGS passes extra flags, e.g. make test-e2e E2E_ARGS='-run TestCompatibility/dispatch'.
The cli_* scenarios build the binary and drive register and daemon as a user would, including --once,
--labels and shutdown with a running job. failure_paths covers failed, skipped and tolerated jobs, job_timeout
a job killed at the runner's deadline. label_routing and capacity cover scheduling, reconnect drops the
connection to Gitea through a proxy, and host_mode and reusable_workflow cover host labels and workflow_call.
dispatch, failure_paths and reusable_workflow run in every mode: through the daemon and exec, each on
Docker and Kubernetes.
The suite shares one Gitea, regular runner and k3s cluster. Cache and ephemeral scenarios use isolated
repository runners. A run path is <workflow>@<ref> and a log row is <timestamp>Z <payload>.
Windows tests
make test-windows cross-compiles the test binaries, installs Windows Server 2025 Core in a
dockur/windows QEMU VM on the docker host and runs them there.
The host must be amd64 and allow privileged containers. WINDOWS_TEST_TIMEOUT sets the minutes
allowed for installing Windows and running the tests, 165 by default. The VM uses KVM when the host
has /dev/kvm, otherwise Windows is emulated and much slower. Docker-backed tests skip in the VM
and the race detector is off, it needs cgo. WINDOWS_TEST_CACHE names a directory to keep the
installed Windows in, so later runs skip the install. CI runs this on every pull request.