Merge pull request #4929 from owncloud/test_antora_3.1_upgrade

Antora 3.1 upgrade
owncloud · Jan 30, 2024 · 5bc7a8a · 5bc7a8a
2 parents 7342d68 + 1192420
commit 5bc7a8a
Show file tree

Hide file tree

Showing 17 changed files with 1,281 additions and 917 deletions.
diff --git a/.drone.star b/.drone.star
@@ -1,3 +1,6 @@
+# Environment variables needed to generate the search index are only provided here in the docs repo in .drone.star
+# Also see the documentation for more details.
+
 def main(ctx):
     # Config
 
@@ -70,33 +73,24 @@ def build(ctx, deployment_branch):
             {
                 "name": "docs-deps",
                 "pull": "always",
-                "image": "owncloudci/nodejs:16",
+                "image": "owncloudci/nodejs:18",
                 "commands": [
                     "yarn install",
                 ],
             },
-            {
-                "name": "docs-validate",
-                "pull": "always",
-                "image": "owncloudci/nodejs:16",
-                "commands": [
-                    "yarn validate --fetch",
-                ],
-            },
             {
                 "name": "docs-build",
                 "pull": "always",
-                "image": "owncloudci/nodejs:16",
+                "image": "owncloudci/nodejs:18",
                 "environment": {
-                    "BUILD_SEARCH_INDEX": ctx.build.branch == deployment_branch,
                     "UPDATE_SEARCH_INDEX": ctx.build.branch == deployment_branch,
                     "ELASTICSEARCH_NODE": from_secret("elasticsearch_node"),
                     "ELASTICSEARCH_INDEX": from_secret("elasticsearch_index"),
                     "ELASTICSEARCH_READ_AUTH": from_secret("elasticsearch_read_auth"),
                     "ELASTICSEARCH_WRITE_AUTH": from_secret("elasticsearch_write_auth"),
                 },
                 "commands": [
-                    "yarn antora --fetch --attribute format=html",
+                    "yarn antora",
                     "bin/optimize_crawl -x",
                 ],
             },

diff --git a/README.md b/README.md
@@ -35,6 +35,9 @@ With regard to language and style issues, consult the [Style Guide](./docs/style
 
 ## Generating the Documentation
 
+**IMPORTANT**  
+We recently have upgraded to `node 18.19.0`. In case you used a lower node version for your local doc repos, you must upgrade them **all**. See the link below for details. 
+
 To generate and view the documentation locally or planning major changes, refer to the [Building the Documentation guide](./docs/build-the-docs.md).
 
 ## Common Content and Styling the Documentation
@@ -73,4 +76,4 @@ Please refer to [Create a New Version Branch for Docs](./docs/new-version-branch
 
 ## HTML to PDF
 
-At the moment, creating a pdf from a component via Antora is broken and will be fixed when updating to Antora 3. In the meanwhile a workaround is provided, see the [HTML to PDF](./docs/html-to-pdf.md) description.
+At the moment, creating a pdf from a component via Antora is broken and will be fixed past updating to Antora 3. In the meanwhile a workaround is provided, see the [HTML to PDF](./docs/html-to-pdf.md) description.
diff --git a/docs/antora-site-structure.md b/docs/antora-site-structure.md
@@ -21,40 +21,43 @@
 
 ## Why Antora
 
-The main reasons for using [Antora][link-antora] is the following:
+The main reasons for using [Antora][link-antora] are the following:
 
-1. It uses and extends the [asciidoc][link-asciidoc] text writing format
+1. It uses and extends the [asciidoc][link-asciidoc] text writing format.
 
-2. It extends asciidoc with [multi-repo][link-playbook] capabilities. Thus, the writer does not need to care about repos anymore as this is virtualized by Antora and content can be accessed in a standardized way.
+2. It extends asciidoc with [multi-repo][link-playbook] capabilities.
 
-3. Antora separates between writing the documentation and the [UI-Template][link-ui] defining how the content is presented.
+3. The writer does not need to care about repos, or content locations, etc. anymore as this is virtualized by Antora and content can be accessed in a standardized way via [Resource ID Coordinates](https://docs.antora.org/antora/latest/page/resource-id-coordinates/).
+
+4. Antora distincts writing the documentation and the [UI-Template][link-ui] defining how the content is presented.
 
 ## Scope of Documentation Repositories
 
-The ownCloud documentation consists of a main repo named `docs` and additional product repos like the clients or others which are included as content sources defined in `site.yml`.
+The ownCloud documentation consists of a building repo named `docs` which has NO content and additional repos like ocis or the clients which are included as content sources defined in `site.yml`.
+
+Our setup is made in a way, where each repo can build it's own content individually for local testing and checking validity. Only the build of the `docs` repo creates documentation which is pushed to the web. There is one exception, when reenabled, PDF's are created AND pushed via the CI from the content source where it applies.
 
-Our setup is made in a way, where each repo can build it's own content individually for testing and checking validity. Only the build of main repo `docs` creates documentation which is pushed to the web. There is one exception, PDF's are created AND pushed via the CI from _each_ content source.
+Note that the `docs-main` repo only contains the entry page but no detailed product description. 
 
 
 ```
-main        content source
+repo        content source
 
-docs   --> docs-server
+docs   -->  docs-main
+            docs-ocis
             docs-client-desktop
             docs-client-ios-app 
             docs-client-android
             ...
 
 ```
-Note that the arrow from main to content source is intentionally unidirectional in our setup and should be respected. See more details about the reason below. 
+Note that the arrow from repo to content source is intentionally unidirectional in our setup and should be respected. See more details about the reason below. 
 
 ### Scope of the playbook files (site.yml)
 
-In general, only one `site.yml` file is neccessary for the whole environment and its definitions are **available to the whole site**. This `site.yml` file is located in the main repo `docs`. We have additionally for each content source its own `site.yml` for testing purposes only. The scope of these local `site.yml` files is restricted to the respective content source and any definitions made are not availabe outside. 
+In general, only one `site.yml` file is neccessary for the whole environment and its definitions are **available to the whole site**. This `site.yml` file is located in the `docs` repo. We have additionally for each content source its own `site.yml` for testing purposes only. The scope of these local `site.yml` files is restricted to the respective content source and any definitions made are not availabe outside. 
 
-Due to this fact, you need to re-add all relevant attributes of the main `site.yml` file in the `site.yml` file of the content source which accesses it, else a local build will return warnings about unresolved attributes.
-
-If you have added an attribute in a content source `site.yml` file, you must add this attibute to the main `site.yml` file to avoid a build warning (unresolved attribute) during a build of the entire documentation.
+Due to this fact, you need to re-add all relevant attributes to the `site.yml` file as defined in the sourced `site.yml` file which accesses it, else a local build will return warnings about unresolved attributes. This is also valid for the building repo.
 
 Note that this behaviour is relevant for the playbook `site.yml` files only and does not apply to the component descriptor files `antora.yml`.
 
@@ -110,26 +113,26 @@ The navigation file `nav.adoc` is under the `partials` directory and not at the
 
 Beside the necessary directories for node, other important directories are:
 ```
-bin/              helper scripts to maintain the documentation
-book_templates/   template file(s) to create the pdf file
-generator/        scripts needed by antora for the build process
-lib/              extension for antora not delivered by node like tabs or remote-include-processor
-pdf_web/          output directory of generated pdf files, only used locally!
-public/           output directory of generated html files, only used locally!
-resources/        themes necessary for creating pdf files
-tmp/              temp directory used for htmltest (broken link checking)
+bin/              Helper scripts to maintain the documentation
+book_templates/   Template file(s) to create the pdf file
+ext-antora/       Antora extensions necessary for the build process
+ext-asciidoc/     Asciidoc extensions necessary for the build process
+pdf_web/          Output directory of generated pdf files, only used locally!
+public/           Output directory of generated html files, only used locally!
+resources/        Themes necessary for creating pdf files
+tmp/              Temp directory used for htmltest (broken link checking)
 ```
 ### Important files
 
 The following files are important to run a build properly; note that node related stuff is not mentioned explicitly:
 
 ```
-.drone.star       define the build process steps when triggered by a PR
+.drone.star       Define the build process steps when triggered by a PR
                   necessary for the creation of the pdf file 
-antora.yml        contains source files and attributes that only belong
+package.json      Define the antora environment und scripts to run at the cli
+antora.yml        Contains source files and attributes that only belong
                   to the component (version dependent!)
-package.json      define the antora environment und scripts to run at the cli
-site.yml          global site definitions including attributes (version independent!)
+site.yml          Global site definitions including attributes (version independent!)
 
 ```
 
@@ -146,10 +149,13 @@ To manage versions in docs, we use branches. This means that any content based o
 1. The scope of attributes defined in a page is limited to that page only.
 2. The scope of attributes defined in `antora.yml` is limited to the branch and component where it is defined. This is also true for attributes that should be accessed from the UI-Template.
 3. The scope of attributes defined in `site.yml` is _global_. The term global has two flavours in our setup:
-    1. When used in the main repo `docs`, it becomes available to all sources at any time.
-    2. When used in a content source, it is only availabe to the content source during a test build.
+    1. When used in the building repo `docs`, it becomes available to all sources at any time.
+    2. When used in a content source, it is only availabe to the content source during a local test build.
 4. Attributes starting with `page-` are also available to the UI-Template when running a build. The rules above apply. This is important when defining UI content based on attributes. To access these attributes in the UI-Template use `page.attribute.name` where `name` is without leading `page-` For details see [AsciiDoc Attributes in Antora][custom-attrib-link].
 
+**IMPORTANT**  
+If used attributes are not defined during build time, Antora hard stops and files an error. You must fix those issues BEFORE creating the PR - else it never will get green. Also see the CI log for details.
+
 ## The Antora UI Template
 
 As described on top, Antora separates the writing of text and the [presentation design][antora-ui-link]. Our presentation design is defined in the [docs-ui][docs-ui-link] repository. Any change made in the UI affects the complete documentation, careful tests are mandatory.
diff --git a/docs/build-the-docs.md b/docs/build-the-docs.md
@@ -88,13 +88,13 @@ nvm ls-remote | grep "Latest LTS"
       v12.22.12   (Latest LTS: Erbium)
        v14.21.3   (Latest LTS: Fermium)
        v16.20.2   (Latest LTS: Gallium)
-       v18.18.2   (Latest LTS: Hydrogen)
+       v18.19.2   (Latest LTS: Hydrogen)
        v20.10.0   (Latest LTS: Iron)
 ```
 Then install a suitable LTS version. You can install as many versions as you like or need, see example below.
 
 ```
-nvm install 16.13.2
+nvm install 18.19.2
 ```
 
 List the installed versions
@@ -105,27 +105,36 @@ nvm ls
        v12.18.2
        v14.18.3
         v15.5.1
-->     v16.13.2
+       v16.13.2
+->     v18.19.0
          system
-default -> 16.13.2 (-> v16.13.2)
+
+default -> 18.19.0 (-> v18.19.0)
 ...
 ```
 
-**Important:** For docs, DO NOT use a version _above_ v10.23.0 and _below_ v14.17.0 as it may later conflict with other dependencies especially with the `yarn serve` command where you will get warnings and it may not work as expected.
+**Important:**  
+For docs, DO NOT use a version _above_ v10.23.0 and _below_ v14.17.0 as it may later conflict with other dependencies especially with the `yarn serve` command where you will get warnings and it may not work as expected.
+
+**Info:**  
+The backend to push to the web uses node v18, see the `.drone.star` file. It is recommended to stay with the same release if possible.
+
+**Important:**  
+If you have used a lower node version like 16.13.2, you now must upgrade **ALL** your doc repos to use v18. See the next steps to do so.
 
-**Info:** The backend to push to the web also uses node v16, see the `.drone.star` file. It is recommended to stay with the same release if possible.
+#### Use a Particular Node Version
 
 Switch to a specific installed version of Node at any time, use the following command:
 
 ```
-nvm use 16.13.2
+nvm use 18.19.0
 ```
 **Important:** If you have additional concurrent terminals open, you must close these terminals first and reopen them to use the new setup.
 
 To make a particular Node version default in new terminals, type:
 
 ```
-nvm alias default 16.13.2
+nvm alias default 18.19.0
 ```
 
 #### Yarn

diff --git a/docs/doc-guidelines.md b/docs/doc-guidelines.md
@@ -37,7 +37,7 @@ Commit all changed files and push the branch to GitHub. Now, log in to Github an
 
 3. Include Support
 
-   If there is a change that might have effects on support, add `@cdamken` either as reviewer or in the PR description.
+   If there is a change that might have effects on support, add a note in the PR description.
 
 4. The main branch for changes is `master`. Any changes start there. Exception:
 

diff --git a/docs/getting-started.md b/docs/getting-started.md
@@ -199,7 +199,7 @@ $ git push {username} :<branchname>
 
 ## The Branching Workflow
 
-Please refer to [the branching workflow](./the-branching-workflow.md) to learn about how git branches are used to manage and build the documentation.
+The branching workflow is only necessary for versioned repos (documentations). Please refer to the [branching workflow](./the-branching-workflow.md) to learn about how git branches are used to manage and build the documentation.
 
 ## Backporting
 

diff --git a/docs/the-branching-workflow.md b/docs/the-branching-workflow.md
@@ -2,4 +2,4 @@
 
 The Docs repo itself is not versioned as it assembles other doc repos and other doc repos might use versioning. For any repo that uses versioning, changes in that repo must be made in master and backported to the corresponding branch if applicable. Usually only 2 working branches and master named `next` are maintained. Any documentation that is versioned can use `latest` in the URL instead of a branch number which points automatically to the latest stable branch of the corresponding documentation.
 
-See the "Create a New Version Branch documentation" link in each doc repo readme at the bottom for details how to do versioning if applicable.
+See the **Create a New Version Branch** documentation link in each doc repo readme for details how to do versioning if applicable.
diff --git a/docs/the-build-pipeline.md b/docs/the-build-pipeline.md
@@ -3,8 +3,7 @@
 ownCloud uses <a href="https://drone.io/">Drone</a> for building and deploying the documentation.
 Drone is a [Continuous Delivery](https://www.continuousdelivery.com/) platform that helps optimize and automate software delivery.
 
-The build pipeline is configured in [.drone.yml](https://github.com/owncloud/docs/blob/master/.drone.yml), located in the root directory of the docs repository.
-You can view the current build status of the docs at https://drone.owncloud.com/owncloud/docs.
+The build pipeline is configured in [.drone.yml](https://github.com/owncloud/docs/blob/master/.drone.yml), located in the root directory of the docs repository. You can view the current build status of the docs at https://drone.owncloud.com/owncloud/docs.
 
 Every push to the docs master branch triggers a drone build. At the end of the build process, the docs are deployed to production.
 
@@ -15,15 +14,14 @@ However, in essence, here is how the build pipeline works:
 2. All required dependencies are installed.
 3. The documentation is validated.
 4. The documentation is built.
-5. The documentation PDF manuals are built.
+5. The documentation PDF manuals are built - if enabled.
 6. The PDF manuals are deployed.
 7. The HTML version of the documentation is deployed.
 8. A build notification is sent.
 
 ## Build Notifications
 
-Build notifications are sent to the `#documentation` channel in https://talk.owncloud.com when a build of the master, 10.0, and 10.1 branches fails.
-Notifications are not sent for successful builds.
+Build notifications are sent to the `#documentation` channel in https://talk.owncloud.com when a full docs build fails. Notifications are not sent for successful builds.
 
 No Pull Request notifications are sent to the `#documentation` channel, as this information is visible within the Pull Request details.