Closes#37.
Gitea generates an OpenAPI 3.0 document since 1.27 (`templates/swagger/v1_openapi3_json.tmpl`, renamed to `templates/swagger/v1-openapi3.generated.json` on main, served by an instance at `/openapi3.v1.json`), but this site never picked it up and still published Swagger 2.0 only. That is what the issue is about: code generators such as `openapi-python-client` reject the document with "You may be trying to use a Swagger document; this is not supported by this project".
**What this does**
- `update_api_docs.sh` downloads the OpenAPI 3.0 document next to the Swagger 2.0 one, rewrites the same placeholders (they sit in `servers` instead of `basePath`) and writes `static/openapi3-latest.json` and `static/openapi3-<minor>.json`. Versions that do not have one (1.26 and older) are skipped with a message and keep their Swagger 2.0 document.
- The OpenAPI 3.0 document is rendered where it exists (`/api/` and `/api/next/`), the older versions keep rendering Swagger 2.0. Both documents describe the same api and carry the same operation ids, so no url of an operation page changes; the `basePath` normalization is now only applied to the Swagger 2.0 documents, since a full url in `servers` is valid.
- The overview page of each version links both documents for download, which is what the issue actually needs: `https://docs.gitea.com/openapi3-27.json` for the current release, `https://docs.gitea.com/openapi3-latest.json` for gitea main.
- The scheduled workflow now refreshes and proposes both documents of gitea main.
**Verification**
- `pnpm dlx @redocly/cli lint static/openapi3-latest.json`: valid, only content warnings that come from upstream (missing 4xx responses, ambiguous paths).
- `openapi-python-client generate --path static/openapi3-latest.json`, the tool from the issue, generates a client. The only warning left is `POST /markdown/raw`, whose request body is `text/plain`.
- Operation ids of `static/swagger-27.json` and `static/openapi3-27.json` are identical (481 on both sides), so the operation pages keep their urls.
- Built the api product and compared a rendered operation page against the Swagger 2.0 one: same sections, the request body is now marked required where the document says so.
The released Swagger 2.0 documents are kept: they are the only description 1.26 and older ever produced, and some tooling still expects them.
<!-- cloudflare-preview --> Preview: https://pr-514.docs-gitea-com.pages.dev
Reviewed-on: https://gitea.com/gitea/docs/pulls/514
Reviewed-by: bircni <[email protected]>
Replace [#307](https://gitea.com/gitea/docs/issues/307)
Closes [#237](https://gitea.com/gitea/docs/issues/237)
## Summary
Rebuilds docs.gitea.com with [Astro](https://astro.build) and [Starlight](https://starlight.astro.build), replacing Docusaurus. Every
published url keeps working and no content file was moved: the site is a new rendering layer over the existing `docs/`, `versioned_docs/`, `i18n/`, `runner-docs/` and `static/swagger-*.json` trees.
For the first time the API reference is part of the site rather than a Redoc
bundle: all seven swagger documents are rendered into real pages, one per
operation, so they are linkable, crawlable and searchable.
## Why
- the Docusaurus build needs an 8 GB heap and about 2 minutes for 2478 pages;
the Astro build produces 5751 pages, API reference included, in about 100
seconds
- the API reference was a single client rendered Redoc page per version, absent
from the site search and from search engines
- sidebars, version lists and language lists were configured in three different
places and drifted apart
## What is in here
- **`packages/content-loader`** — the product × version × language matrix
(`products.ts`) and an Astro content loader that reads the existing markdown
trees directly. The matrix is the single source of truth: content loading,
sidebars, the version and language pickers, the version banner, the search
facets and the API schemas are all derived from it.
- **Content compatibility** — frontmatter `slug`, `sidebar_position` and
`sidebar_label` keep working, `_category_.json` still drives sidebar labels,
order and the generated category pages, `@version@` style release variables are
still substituted, `:::note` admonitions become Starlight asides and the 2778
relative `*.md` links are rewritten to urls while the site is built.
- **API reference** — `starlight-openapi` renders the seven swagger documents.
Operation urls are hyphenated (`/api/operations/list-admin-workflow-jobs/`),
the sidebar shows the operations of the version being read with a coloured
HTTP method badge, and the overview page links to one page per tag instead of
repeating all 484 operations.
- **Navigation** — a version picker that follows the reader to the same page in
another version, a language picker that only offers the languages the current
product is published in, and product links for Docs, API, Runner and
Enterprise.
- **Theme** — the light and dark palettes of about.gitea.com, mapped onto the
Starlight variables.
- **Search** — Pagefind by default; Algolia DocSearch takes over when
`PUBLIC_DOCSEARCH_APP_ID` and `PUBLIC_DOCSEARCH_API_KEY` are set. Every page
carries `docsearch:product`, `docsearch:version` and `docsearch:language` meta
tags, so a search stays inside what is being read.
`cloudflare/docsearch-crawler.json` holds the crawler configuration.
## Bugs fixed along the way
- `GET /user/applications/oauth2` and `GET /user/applications/oauth2/{id}` have
operation ids that only differ in case, so they collapsed onto the same url
and one of the two pages was silently dropped. They are now
`user-get-oauth2-application` and `user-get-oauth2-application-by-id`.
- The API overview and every tag page were all titled "Overview".
## Url compatibility
The routes of both builds were compared page by page during the migration with
`scripts/url-diff.mjs`:
```
identical: 2368, missing: 0, accepted: 110, added: 3292
```
The 110 accepted ones are all redirected in `cloudflare/_redirects`: the
localized copies of the English-only API and Runner docs, the Docusaurus search
page, and `/1.27/`, `/runner/3/`, `/api/1.27/` which are aliases of the versions
served at the product root. The added ones are the API operation and tag pages
plus the routes the Starlight language fallback serves in English when a
translation is missing — Docusaurus answered those with a 404.
## Workflows
- `checks` builds the site and type checks it on every pull request
- `Build and Publish Docs site` publishes `sites/docs/dist` to S3/CloudFront and
to Cloudflare Pages, and copies `cloudflare/_headers` and
`cloudflare/_redirects` into the deployment
- `update swagger files` and `update runner reference` are unchanged, they only
touch content
- `make cut-version PRODUCT=docs VERSION=1.28` replaces
`docusaurus docs:version`
## Removed
`docusaurus.config.js`, the swizzled theme under `src/`, and the Docusaurus UI
translations (`i18n/*/code.json`, `i18n/*/docusaurus-theme-classic/`). The
strings those carried — the product names, the footer column titles and the
outdated translation notice — were ported to
`sites/docs/src/config/strings.ts`; Starlight ships the rest of its interface in
both Chinese locales. The documentation content itself is untouched.
## Follow ups
- nine relative links are broken in the sources and reported by every build, the
same ones Docusaurus warned about; `GITEA_DOCS_STRICT_LINKS=true` turns them
into an error once they are fixed
- `cloudflare/worker.js` has to be deployed for `/enterprise/` to keep resolving
- the Algolia index has to be created and crawled before the search credentials
are set
## Testing
```shell
make install
make serve-fast # english, the version served at the root
make serve # the whole matrix
make build # 5751 pages
make check # 0 errors
make serve-built # build and serve, the only way to try the search locally
```
## Screenshots
<img width="1371" alt="image.png" src="attachments/69acdd77-cc89-4635-a8bd-1163a34afa86">
<img width="1810" alt="image.png" src="attachments/76212416-4825-4b4d-bf59-3e549900c96f">
<img width="1789" alt="image.png" src="attachments/7be84cda-5abe-48bd-8839-7ae1ee7806e7">
---------
Co-authored-by: bircni <[email protected]>
Reviewed-on: https://gitea.com/gitea/docs/pulls/496
Reviewed-by: bircni <[email protected]>
The scheduled `update swagger files` job has failed on **every** run since it was added (I checked the run list on gitea.com, ~250 consecutive failures). Two reasons:
1. Upstream moved the file: `go-gitea/gitea` main no longer has `templates/swagger/v1_json.tmpl`, it is now `templates/swagger/v1-swagger.generated.json`, and the placeholders changed from `{{.SwaggerAppVer}}` / `{{.SwaggerAppSubUrl}}` to `0.0.0+GITEA-API-APP-VERSION` / `/GITEA-API-APP-SUBURL/api/v1`. Because the script used `curl --silent` without `--fail`, the 404 body was written into `static/swagger-latest.json`.
2. `main` is protected (`required_approvals=1`), so the job's `git push` to main could never succeed.
Changes:
- `update_api_docs.sh`: `set -euo pipefail`, download with `--fail` (a broken download now aborts instead of committing garbage), try the new generated json first and fall back to the old template for released tags, handle the 1.28+/1.24+/<1.24 placeholder variants, and add a `--latest-only` flag.
- `Makefile`: new `update-api-docs-latest` target for the cron job; `clean` no longer deletes the committed `static/swagger-*.json` files (deleting them breaks `make build`).
- `.gitea/workflows/update_swagger.yaml`: every 12h (plus `workflow_dispatch`) it regenerates only `static/swagger-latest.json`, verifies no placeholder is left and the file is valid JSON, force pushes to `bot/update-swagger-latest` and opens a pull request via the API (HTTP 409 = a PR is already open, it just now points at the new commit). It skips entirely when the bot branch already carries the same file, so no PR churn.
- `README.md`: document the two make targets and the automation.
`secrets.DEPLOY_TOKEN` must belong to a real user with `write:repository` so the PR checks are triggered; the job fails with a clear message if it is empty.
Tested locally in a scratch clone: `make update-api-docs-latest` produces a valid `dev` spec with no placeholders left, `make update-api-docs` still regenerates all released versions, and the workflow shell logic was simulated against a local bare repo for all three paths (open PR / bot branch already up to date / main already up to date), including a shallow clone to confirm `fetch-depth: 1` force pushes work.
Fixes#454
Reviewed-on: https://gitea.com/gitea/docs/pulls/481
Reviewed-by: techknowlogick <[email protected]>
The command order of prepare-docs is incorrect.
We should prepare same version with different locale at once, otherwise the doc files for `zh-cn` in `i18n` folder will be incorrect.
Before: (files in current is as same as version 1.19)
After:
Co-authored-by: Lunny Xiao <[email protected]>
Reviewed-on: https://gitea.com/gitea/gitea-docusaurus/pulls/92
Reviewed-by: Lunny Xiao <[email protected]>
Reviewed-by: silverwind <[email protected]>
Co-authored-by: yp05327 <[email protected]>
Co-committed-by: yp05327 <[email protected]>
These are some things I changed to make local development easier, especially when having to update multiple versions of docs at once like the docusaurus v3 update.
i.e.
```makefile
GITEA_REMOTE := ~/code/code.gitea.io/gitea
GITEA_LATEST_BRANCH := docs-v3
GITEA_VERSION_BRANCH_PREFIX := docs-v3-
...
```
to run the Makefile against my local repo.
Reviewed-on: https://gitea.com/gitea/gitea-docusaurus/pulls/89
Co-authored-by: jolheiser <[email protected]>
Co-committed-by: jolheiser <[email protected]>
Closes#30Closes#31Closes#42
Rather than partially cloning, instead we clone once and then `clean` (to remove generated files), `reset` (to reset changed files), and `checkout` branches as needed. This should allow the outdated translation check to continue to work while also reducing build times significantly.
Before:
After:
Reviewed-on: https://gitea.com/gitea/gitea-docusaurus/pulls/52
Co-authored-by: John Olheiser <[email protected]>
Co-committed-by: John Olheiser <[email protected]>
Close#46
Same as those in [blog](https://gitea.com/gitea/blog/pulls/272), but for docs and api
Ejected DocPage (unsafe to eject) and ApiDoc (safe to eject) for layout changes
Screenshots
Co-authored-by: techknowlogick <[email protected]>
Co-authored-by: techknowlogick <[email protected]>
Reviewed-on: https://gitea.com/gitea/gitea-docusaurus/pulls/47
Co-authored-by: HesterG <[email protected]>
Co-committed-by: HesterG <[email protected]>
# Changes
- Added [redocusaurus](https://github.com/rohit-gohri/redocusaurus) package to display swagger api with reference to [woodpecker](https://github.com/woodpecker-ci/woodpecker/tree/master/docs)
- Change `sed` for swagger.json for proper version and baseurl.
- Add a dropdown for different api versions. (screenshot below)
- On API pages, only API version dropdown will show(hide locale dropdown, version dropdown for doc, and doc search)
- Allow build CSR API pages for development so it saves building time.
# Screenshots
Search:
Reviewed-on: https://gitea.com/gitea/gitea-docusaurus/pulls/32
Co-authored-by: HesterG <[email protected]>
Co-committed-by: HesterG <[email protected]>
Modified `loop_docs.sh` to take `version` and `locale` and params so do not need other `loop_docs*` scripts.
Need confirmation with version information for 1.20 and latest.
Like version in `static/20-swagger.json` and `static/latest-swagger.json`, and version used to replace `{{< version >}}`
Will wait for a stable version for 1.20
Reviewed-on: https://gitea.com/gitea/gitea-docusaurus/pulls/28
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: HesterG <[email protected]>
Co-committed-by: HesterG <[email protected]>
- `git log` needs all commit history to compare latest commit of files, so changed `git clone --depth=1` to `git clone`. This will make the step take longer, so might be improved if there is a better way.
- `src/theme/MDXContent/index.js` is from [ejecting of the component](https://docusaurus.io/docs/swizzling#ejecting) inside `@docusaurus/theme-classic` plugin, and this one is safe to eject according to docusaurus
- [Outdated component style reference](https://mui.com/material-ui/react-alert/)
- Added [`Translate` component](https://docusaurus.io/docs/next/docusaurus-core#translate) to `Outdated` so it can be localized. [reference](https://docusaurus.io/docs/next/i18n/tutorial#translate-your-react-code)
- One way to check for the specific outdated documents:
search for `lastest commit timestamp` in [prepare nightly docs and prepare 1.19 docs steps](https://gitea.com/gitea/gitea-docusaurus/actions/runs/74)
# After
The Chinese documents that are outdated (latest commit is ealier than laster commit of English version):
Reviewed-on: https://gitea.com/gitea/gitea-docusaurus/pulls/25
Reviewed-by: Lunny Xiao <[email protected]>
Co-authored-by: HesterG <[email protected]>
Co-committed-by: HesterG <[email protected]>
# Main changes and Some notices
- Chines Docs should be put into corresponding versions' folders under `i18n/zh-cn/docusaurus-plugin-content-docs`. See [translate-markdown-files](https://docusaurus.io/docs/i18n/tutorial#translate-markdown-files) for reference. And also [srs-docs](https://github.com/ossrs/srs-docs) is a project that can be used as a reference. So changed the related paths in `loop_docs*` scripts.
- The markdown files under `docs/<version>` and `i18n/zh-cn/docusaurus-plugin-content-docs/<version>` must have the same name to be referred as translated. Files inside docs are corresponding to `current` version.
For example, `docs/administration/mail-templates.md` and `i18n/zh-cn/docusaurus-plugin-content-docs/current/administration/mail-templates.md`. Filename of `mail-templates.md` must be the same (cannot be `mail-templates.zh-cn.md` and `mail-templates.us-en.md`)
So trim the `.en-us.md` and `.zh-cn.md` to `.md` to ensure the same names inside `loop_docs*` scripts.
- Used `npx docusaurus write-translations --locale zh-cn` to do sidebar, footer, and header translations, so no need to keep `sideBarCN` anymore. [reference](https://docusaurus.io/docs/cli#docusaurus-write-translations-sitedir)
- Local Tests for `en` and `zh-cn` are separeted, run the following on local respectly to test them
```bash
# test en version
npm run start
# test zh-cn version
npm run start -- --locale zh-cn
```
- If wants to test `en` and `zh-cn` both at the same time, run build and serve
```
npm run build
npm run serve
```
- Added some indexed page, for example:
Reviewed-on: https://gitea.com/gitea/gitea-docusaurus/pulls/13
Co-authored-by: HesterG <[email protected]>
Co-committed-by: HesterG <[email protected]>
Lunny Xiao