Files
docs/runner-docs_versioned_docs/version-5/monitoring.md
T
silverwind be6aefc7e8 Simplify the runner docs and fix their outdated facts (#600)
The runner pages had drifted from the runner. They claimed Podman is unsupported, a registration retry limit of 10, a `container.env` proxy override that has no effect, and they used the deprecated `generate-config` command, while the actual instructions were buried under implementation details.

The develop and 5.x pages are rewritten from the current runner sources (`main` is `v5.2.0`). They are shorter and aimed at operators, with a new page covering what jobs get. The reference generator now documents the `config` command, and the quickstart pages in every language use the `gitea-runner` binary name.

Reviewed-on: https://gitea.com/gitea/docs/pulls/600
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: silverwind <[email protected]>
2026-10-21 05:25:45 +00:00

3.6 KiB

sidebar_position
sidebar_position
8

Monitoring

Metrics

metrics:
  enabled: true
  addr: "127.0.0.1:9101"

With metrics enabled, the runner serves these endpoints:

Endpoint Purpose
/metrics Prometheus metrics
/healthz Liveness, always 200 while the process runs
/readyz Readiness, 503 when a health check fails, the runner cannot reach Gitea for longer than metrics.readiness_grace (30s), or the runner is shutting down

The listener has no authentication. If you expose it beyond localhost, for example with addr: ":9101", protect it with a firewall.

The runner exports these metrics in addition to the standard Go and process metrics:

Metric Meaning
gitea_runner_state 1 for the current state: busy, idle or unavailable
gitea_runner_job_running, gitea_runner_capacity Running jobs and configured capacity
gitea_runner_job_capacity_utilization_ratio Running jobs divided by capacity
gitea_runner_job_total Finished jobs by status: success, failure, cancelled, skipped, timeout or unknown
gitea_runner_job_duration_seconds Job duration histogram
gitea_runner_poll_fetch_total Job fetches by result: task, empty or error
gitea_runner_client_errors_total Failed requests to Gitea by method and code

Consider alerts for gitea_runner_state{state="unavailable"}, a rising gitea_runner_poll_fetch_total{result="error"}, and a utilization ratio that stays at 1, which means jobs may be waiting for a free slot.

OpenTelemetry

Set OTEL_EXPORTER_OTLP_ENDPOINT in the runner's environment to send traces and the metrics above to an OpenTelemetry collector. Each job has a trace with a span for each step:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

Jobs receive TRACEPARENT, so instrumented tools can add spans to the job's trace. See telemetry.md for all variables and security notes.

Health checks

Health checks stop a runner from accepting new jobs while its machine has a problem, such as a full disk:

health_check:
  enabled: true
  min_free_disk_space_mb: 1024
  script: /usr/local/bin/check-runner.sh
  • The disk check measures free space on the filesystem containing host.workdir_parent, or container.workdir_parent when container.bind_workdir is enabled.
  • script is optional. The runner becomes unavailable if the script exits with a non-zero status, fails to start, or runs longer than health_check.timeout (10s). The runner reuses the script's result for health_check.interval (30s).
  • Checks only run while the runner is idle. Running jobs are never interrupted.
  • The runner resumes accepting jobs once the checks pass again.

Logs

The runner writes logs to stderr. View them in the systemd journal or with docker logs for a runner container. Set log.level to control verbosity, from trace to error.

The runner streams step output to Gitea without including it in its own log. To also keep a local copy of every job log, set a directory:

log:
  job:
    dir: /var/log/gitea-runner

The runner writes each job log to <start time>-task-<id>.log in this directory and masks secrets. It keeps the files for log.job.retention (168h). Only the user running the runner can read them.

Reporting a problem

gitea-runner bug-report prints the runner version, Go version, platform and CPU count. Include its output when you open an issue at gitea/runner.