Skip to content

Commit

Permalink
Merge pull request #887 from opengisch/docx
Browse files Browse the repository at this point in the history
translate MkDocs navigation content on Transifex
  • Loading branch information
signedav authored Feb 28, 2024
2 parents cb819b8 + 6d78685 commit ecd4b50
Show file tree
Hide file tree
Showing 6 changed files with 278 additions and 26 deletions.
24 changes: 17 additions & 7 deletions .github/workflows/doc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,7 @@ on:
- 'docs/**'
- '.github/workflows/doc.yml'
workflow_dispatch: # useful for testing tx pushes
workflow_call:

permissions:
contents: write

defaults:
run:
Expand All @@ -37,32 +34,45 @@ jobs:
python-version: '3.10'

- name: Install Python requirements
run: pip install -r requirements.txt
run: |
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Install Transifex client
run: |
curl -OL https://github.com/transifex/cli/releases/download/v1.6.10/tx-linux-amd64.tar.gz
tar -xvzf tx-linux-amd64.tar.gz
- name: Extract translatable content from mkdocs.yml config
run: ./scripts/mkdocs_tx.py create_source

- name: Configure Transifex
run: scripts/transifex_utils.py
env:
TX_TOKEN: ${{ secrets.TX_TOKEN }}

- name: Push source files to Transifex
if: ${{ github.event_name == 'push' }}
if: ${{ github.event_name != 'pull_request' }}
run: ./tx push
env:
TX_TOKEN: ${{ secrets.TX_TOKEN }}

- name: Pull translations from Transifex
if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name == 'opengisch/QgisModelBaker' && github.actor != 'dependabot[bot]' }}
if: contains(fromJSON('["push", "workflow_dispatch"]'), github.event_name) || ${{ github.event_name == 'pull_request' && github.repository == 'opengisch/QgisModelBaker' }}
run: |
./tx pull --translations --all --minimum-perc 10
./tx status
env:
TX_TOKEN: ${{ secrets.TX_TOKEN }}

- name: Translate Mkdocs config
if: contains(fromJSON('["push", "workflow_dispatch"]'), github.event_name) || ${{ github.event_name == 'pull_request' && github.repository == 'opengisch/QgisModelBaker' }}
run: |
./scripts/mkdocs_tx.py -s en update_config
./scripts/mkdocs_tx_commit.sh
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Build documentation
run: mkdocs build

Expand All @@ -74,5 +84,5 @@ jobs:
if-no-files-found: error

- name: Deploy to GitHub Pages
if: ${{ github.event_name == 'push' }}
if: contains(fromJSON('["push", "workflow_dispatch"]'), github.event_name)
run: mkdocs gh-deploy --force
54 changes: 35 additions & 19 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Project information
site_name: QGIS Model Baker Documentation
site_description: >-
This site contains documentation about QGIS Model Baker
site_url: https://opengisch.github.io/QgisModelBaker/

docs_dir: docs
Expand Down Expand Up @@ -39,23 +37,23 @@ theme:
nav:
- Home: index.md
- User Guide:
- Get Started: user_guide/get_started.md
- Model and Data Import Workflow: user_guide/import_workflow.md
- Export Data Workflow: user_guide/export_workflow.md
- Validate Data: user_guide/validation.md
- Plugin Configuration: user_guide/plugin_configuration.md
- Get Started: user_guide/get_started.md
- Model and Data Import Workflow: user_guide/import_workflow.md
- Export Data Workflow: user_guide/export_workflow.md
- Validate Data: user_guide/validation.md
- Plugin Configuration: user_guide/plugin_configuration.md
- Tipps & Tricks:
- Repositories: background_info/repositories.md
- Basket and Dataset Handling: background_info/basket_handling.md
- OID Generator: background_info/oid_tid_generator.md
- UsabILIty Hub:
- Model Baker Integration: background_info/usabilityhub/modelbaker_integration.md
- Technical Concept: background_info/usabilityhub/technical_concept.md
- User Guide: background_info/usabilityhub/user_guide.md
- Optimized Projects for Extended Models: background_info/extended_models_optimization.md
- Catalogues and their special cases: background_info/catalogues.md
- Meta Attributes: background_info/meta_attributes.md
- Migrate from ili2db 3 to 4: background_info/upgrade_3_to_4.md
- Repositories: background_info/repositories.md
- Basket and Dataset Handling: background_info/basket_handling.md
- OID Generator: background_info/oid_tid_generator.md
- UsabILIty Hub:
- Model Baker Integration: background_info/usabilityhub/modelbaker_integration.md
- Technical Concept: background_info/usabilityhub/technical_concept.md
- User Guide: background_info/usabilityhub/user_guide.md
- Optimized Projects for Extended Models: background_info/extended_models_optimization.md
- Catalogues and their special cases: background_info/catalogues.md
- Meta Attributes: background_info/meta_attributes.md
- Migrate from ili2db 3 to 4: background_info/upgrade_3_to_4.md
#- Relations in QGIS: maybe from here https://github.com/signedav/interlis_relations_in_qgis
#- INTERLIS Syntax in 10 Minutes: maybe stuff from here https://github.com/signedav/talk_iliuniverse
- Development: development.md
Expand All @@ -78,19 +76,37 @@ plugins:
name: Deutsch
site_name: QGIS Model Baker Dokumentation
nav_translations:
Home: Home
User Guide: Benutzerhandbuch
Get Started: Loslegen
Model and Data Import Workflow: Modell und Daten Import Workflow
Export Data Workflow: Daten Export Workflow
Validate Data: Daten Validierung
Plugin Configuration: Plugin Konfiguration
Tipps & Tricks: Tipps & Tricks
Repositories: Repositories
Basket and Dataset Handling: Dataset und Basket Handling
OID Generator: OID Generator
Optimized Projects for Extended Models : Optimierte Projekte für erweiterte Modelle
UsabILIty Hub: UsabILIty Hub
Model Baker Integration: Model Baker Integration
Technical Concept: Technisches Konzept
Optimized Projects for Extended Models: Optimierte Projekte für erweiterte
Modelle
Catalogues and their special cases: Kataloge und ihre Spezialfälle
Meta Attributes: Metaattribute
Migrate from ili2db 3 to 4: Migration von ili2db 3 zu 4
Development: Enwicklung
palette:
- scheme: default
primary: blue grey
toggle:
icon: material/weather-night
name: Switch to dark mode
- scheme: slate
primary: blue grey
toggle:
icon: material/weather-sunny
name: Switch to light mode
primary: white

# Page tree
2 changes: 2 additions & 0 deletions docs/requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ruamel.yaml==0.18.6
pre-commit==3.6.2
182 changes: 182 additions & 0 deletions docs/scripts/mkdocs_tx.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
#!/usr/bin/env python

import argparse
import copy

from ruamel.yaml import YAML

# This scripts helps with translatable content from mkdocs.yml
# It provides commands:
# * to create the YAML translatable file
# * to update the mkdocs.yml with the translated content


def read_config(file_path: str):
yaml = YAML(typ="rt")
yaml.preserve_quotes = True
with open(file_path) as f:
return yaml.load(f)


def nav_config(config) -> dict:
_nav_config = {}

def add_nav_entry(title, content):
if title:
_nav_config[title] = title
for _entry in content:
if type(_entry) == str:
# this is pointing to a page directly, skipping
continue
for _title, _content in _entry.items():
add_nav_entry(_title, _content)

add_nav_entry(None, config["nav"])

return _nav_config


def site_description(config, source_language) -> str:
_site_description = None
found = 0
try:
_site_description = config["site_description"]
found += 1
except KeyError:
pass
try:
for plugin in config["plugins"]:
if not isinstance(plugin, str) and "i18n" in plugin:
for lang_info in plugin["i18n"]["languages"]:
lang = lang_info["locale"]
if lang == source_language:
_site_description = lang_info["site_description"]
found += 1
except KeyError:
pass
if not found:
print("No site description found")
elif found > 1 and _site_description != config["site_description"]:
print("ERROR: site description found twice and different")
assert False

return _site_description


def create_translation_source(config_path, source_path, source_language):
config = read_config(config_path)

tx_cfg = {
"nav": nav_config(config),
"site_description": site_description(config, source_language),
}

try:
tx_cfg["theme"] = {"palette": []}
for palette in config["theme"]["palette"]:
tx_cfg["theme"]["palette"].append(
{"toggle": {"name": palette["toggle"]["name"]}}
)
except KeyError:
print("No theme/palette/toggle/name to translate")

with open(source_path, "w") as f:
yaml = YAML()
yaml.dump(tx_cfg, f)


def update_config(config_path, source_path, source_language):
config = read_config(config_path)

found = False
for plugin in config["plugins"]:
if type(plugin) != str and "i18n" in plugin:
found = True
for lang_info in plugin["i18n"]["languages"]:
lang = lang_info["locale"]
print(f"language found: '{lang}'")

if lang == source_language:
print("skipping source language")
continue

tx_file = f'{source_path.removesuffix(".yml")}.{lang}.yml'
with open(tx_file) as f:
yaml = YAML()
tx = yaml.load(f)

for _title, _translation in tx["nav"].items():
if not _translation:
tx["nav"][_title] = _title
lang_info["nav_translations"] = tx["nav"]

try:
lang_info["site_description"] = tx[
"site_description"
] or site_description(config, source_language)
except KeyError:
print("No site description in translation")

try:
lang_info["palette"] = copy.deepcopy(config["theme"]["palette"])
i = 0
for palette in tx["theme"]["palette"]:
_name = (
palette["toggle"]["name"]
or config["theme"]["palette"][i]["toggle"]["name"]
)
lang_info["palette"][i]["toggle"]["name"] = _name
i += 1
except KeyError:
print("No theme/palette/toggle/name in translation")

assert found

with open(config_path, "w") as f:
yaml = YAML()
yaml.indent(mapping=2, sequence=4, offset=2)
yaml.dump(config, f)


if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"-c", "--config_path", default="mkdocs.yml", help="mkdocs.yml complete path"
)
parser.add_argument(
"-s", "--source_language", default="en", help="source language of the config"
)
parser.add_argument(
"-t",
"--translation_file_path",
default="mkdocs_tx.yml",
help="Translation file to create and translate",
)

subparsers = parser.add_subparsers(title="command", dest="command")

# create the parser for the create_source command
parser_source = subparsers.add_parser(
"create_source", help="Creates the source file to be translated"
)

# create the parser for the update_config command
parser_update = subparsers.add_parser(
"update_config",
help="Updates the mkdocs.yml config file from the downloaded translated files",
)

args = parser.parse_args()

if args.command == "create_source":
create_translation_source(
args.config_path, args.translation_file_path, args.source_language
)

elif args.command == "update_config":
update_config(
args.config_path, args.translation_file_path, args.source_language
)

else:
raise ValueError
31 changes: 31 additions & 0 deletions docs/scripts/mkdocs_tx_commit.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env bash

set -e

pre-commit install
pre-commit run --files mkdocs.yml || true

if [[ $(git diff --exit-code mkdocs.yml) ]]; then
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "github-actions[bot]"

echo "detected changes in mkdocs.yml"
if [[ ${GITHUB_EVENT_NAME} == "pull_request" ]]; then
# on PR push to the same branch
gh pr checkout $(echo "${GITHUB_REF_NAME}" | cut -d/ -f1)
git add mkdocs.yml
git commit -m "Update mkdocs.yml translation" --no-verify
git push
else
# on push/workflow_dispatch create a pull request
git checkout ${GITHUB_REF_NAME}
BRANCH="update-mkdocs-tx-$RANDOM"
git checkout -b ${BRANCH}
git add mkdocs.yml
git commit -m "Update mkdocs.yml translation" --no-verify
git push -u origin $BRANCH
gh pr create -B ${GITHUB_REF_NAME} -H ${BRANCH} --title 'Update mkdocs translations' --body 'run from mkdocs_tx'
fi
else
echo "no change mkdocs.yml"
fi
11 changes: 11 additions & 0 deletions docs/scripts/transifex_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,17 @@ def create_transifex_config():
f.write("[main]\n")
f.write("host = https://www.transifex.com\n\n")

if os.path.isfile(f"{root}/mkdocs_tx.yml"):
print(f"Found mkdocs config translated content")
f.write(f"[o:{TX_ORGANIZATION}:p:{TX_PROJECT}:r:site_navigation]\n")
f.write("resource_name = site navigation\n")
f.write("file_filter = mkdocs_tx.<lang>.yml\n")
f.write(f"source_file = mkdocs_tx.yml\n")
f.write(f"source_lang = {TX_SOURCE_LANG}\n")
f.write(f"type = YAML_GENERIC\n\n")
else:
print("No translation of mkdocs config found")

for file in glob.iglob(current_dir + "/../docs/**/*.md", recursive=True):
# Get relative path of file
relative_path = os.path.relpath(file, start=root)
Expand Down

0 comments on commit ecd4b50

Please sign in to comment.