Skip to content

Commit

Permalink
fixed md linting issue
Browse files Browse the repository at this point in the history
  • Loading branch information
@JHalbauer
JHalbauer committed Mar 20, 2024
1 parent 4812257 commit 3527844
Showing 1 changed file with 128 additions and 113 deletions.
241 changes: 128 additions & 113 deletions How-to-create-a-GRASS-GIS-addon.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,124 +26,133 @@ are needed as a minimum.
## How to create a GRASS GIS addon

A GRASS python module consists of
* Makefile
* html documentation file
* either a python script (most easy with one script as whole addon)
* or C code

#### Reuse & Recycle:
* it helps if GRASS GIS source code is there to look at
* best to look at existing GRASS addons
* [r.mapcalc.simple](https://github.com/OSGeo/grass/tree/master/scripts/r.mapcalc.simple)
* [v.example](https://github.com/mundialis/v.example)
* ...
* inside the GRASS GIS source code all python modules are in folder 'scripts'
* or see https://grasswiki.osgeo.org/wiki/Category:Python, especially https://grasswiki.osgeo.org/wiki/GRASS_Python_Scripting_Library
* use ```r.blend --script``` and it will generate how it would look like as addon (like a template, not a copy of source code!) (will always create generic long version)
* use of predefinded functions ```import grass.script as gscript``` (e.g. gscript.run_command, gscript.message, gscript.fatal, gscript.warning, gscript.read_command)
* located in 'python/script', please read to avoid to reinvent the wheel
* TODO add docs (but better to read source code because it might be more up-to-date)
* use of [grass-gis-helpers](https://github.com/mundialis/grass-gis-helpers) library
* use of [github-workflows](https://github.com/mundialis/github-workflows)
* add linting workflow
* add workflow to run tests
* Submitting rules:
* https://github.com/OSGeo/grass/blob/master/CONTRIBUTING.md
- Makefile
- html documentation file
- either a python script (most easy with one script as whole addon)
- or C code

#### Reuse & Recycle:

- it helps if GRASS GIS source code is there to look at
- best to look at existing GRASS addons
- [r.mapcalc.simple](https://github.com/OSGeo/grass/tree/master/scripts/r.mapcalc.simple)
- [v.example](https://github.com/mundialis/v.example)
- ...
- inside the GRASS GIS source code all python modules are in folder 'scripts'
- or see https://grasswiki.osgeo.org/wiki/Category:Python, especially https://grasswiki.osgeo.org/wiki/GRASS_Python_Scripting_Library
- use `r.blend --script` and it will generate how it would look like as addon (like a template, not a copy of source code!) (will always create generic long version)
- use of predefinded functions `import grass.script as gscript` (e.g. gscript.run_command, gscript.message, gscript.fatal, gscript.warning, gscript.read_command)
- located in 'python/script', please read to avoid to reinvent the wheel
- TODO add docs (but better to read source code because it might be more up-to-date)
- use of [grass-gis-helpers](https://github.com/mundialis/grass-gis-helpers) library
- use of [github-workflows](https://github.com/mundialis/github-workflows)
- add linting workflow
- add workflow to run tests
- Submitting rules:
- https://github.com/OSGeo/grass/blob/master/CONTRIBUTING.md

#### Structure (here `r.blend` as example)

1. shebang (first line)
2. header (author, purpose, license)
3. ```# % ``` comments are important (ignored by python but important for parser)
* See https://grass.osgeo.org/grass-devel/manuals/g.parser.html

```
r.blend -c first=aspect second=elevation output=elev_shade_blend
```
###### ```# % module```
###### ```# % options```
* (e.g. 'input', 'output', 'first'), some are predefined (predefined or custom, predefined is more convenient but also needs more knowledge to use)
* key is key in command line (e.g. 'first')
* answer is default values
* access them in main function like ```options['first']```

###### ```# % flag```

4. ```def main():``` reads in all variables (```options['first']```)
* a main function is required
5. indefinite additional functions are possible
6. include parser at the end before calling main function:

```
if __name__ == "__main__":
options, flags = gscript.parser()
main()
```
or optionally clean temporary stuff in 'cleanup' function and call it on exit
```
if __name__ == "__main__":
options, flags = gscript.parser()
atexit.register(cleanup)
main()
```

1. header (author, purpose, license)

1. `# % ` comments are important (ignored by python but important for parser)

- See https://grass.osgeo.org/grass-devel/manuals/g.parser.html

```
r.blend -c first=aspect second=elevation output=elev_shade_blend
```

###### `# % module`

###### `# % options`

- (e.g. 'input', 'output', 'first'), some are predefined (predefined or custom, predefined is more convenient but also needs more knowledge to use)
- key is key in command line (e.g. 'first')
- answer is default values
- access them in main function like `options['first']`

###### `# % flag`

1. `def main():` reads in all variables (`options['first']`)

- a main function is required

1. indefinite additional functions are possible

1. include parser at the end before calling main function:

```
if __name__ == "__main__":
options, flags = gscript.parser()
main()
```

or optionally clean temporary stuff in 'cleanup' function and call it on exit

```
if __name__ == "__main__":
options, flags = gscript.parser()
atexit.register(cleanup)
main()
```

## Best practises

* python style guide
* [PEP 8 – Style Guide for Python Code](https://peps.python.org/pep-0008/])
* https://trac.osgeo.org/grass/wiki/Submitting
* https://trac.osgeo.org/grass/wiki/Submitting/Python
* html documentation (no full html please, parts are auto-generated at compile time)
* https://trac.osgeo.org/grass/wiki/Submitting/Docs
* to support i18n, import following module and use macro '_' before strings. The text after will be replaced with existing lookup tables for other languages
* use existing functions: esp. from PyGRASS
* API description: https://grass.osgeo.org/grass-devel/manuals/libpython/pygrass_index.html
* PyGRASS paper: An Object Oriented Python Application Programming Interface (API) for GRASS: https://www.mdpi.com/2220-9964/2/1/201/htm
- python style guide
- [PEP 8 – Style Guide for Python Code](https://peps.python.org/pep-0008/%5D)
- https://trac.osgeo.org/grass/wiki/Submitting
- https://trac.osgeo.org/grass/wiki/Submitting/Python
- html documentation (no full html please, parts are auto-generated at compile time)
- https://trac.osgeo.org/grass/wiki/Submitting/Docs
- to support i18n, import following module and use macro '\_' before strings. The text after will be replaced with existing lookup tables for other languages
- use existing functions: esp. from PyGRASS
- API description: https://grass.osgeo.org/grass-devel/manuals/libpython/pygrass_index.html
- PyGRASS paper: An Object Oriented Python Application Programming Interface (API) for GRASS: https://www.mdpi.com/2220-9964/2/1/201/htm

```
# i18N
51 import gettext
52 gettext.install('grassmods', os.path.join(os.getenv("GISBASE"), 'locale'))
```

* for this to work use message standardisation (look at locale)
* how to name it
* start with 'v.' if it is a vector module
* start with 'r.' if it is a raster module
* start with 'db.' if it is a database module
* try to stick to existing convention but no strickt rules
* do not name it too long

- for this to work use message standardisation (look at locale)
- how to name it
- start with 'v.' if it is a vector module
- start with 'r.' if it is a raster module
- start with 'db.' if it is a database module
- try to stick to existing convention but no strickt rules
- do not name it too long

#### mundialis / actinia specific

* How to handle dependencies (for installation within actinia)
* use `requirements.txt` for python packages
* How can I log to actinia output? (with `gscript.message`)
* GRASS GIS addons need to be installed globally (see `$HOME`)
* Which things are tricky
* `db.login`, `db.connect -d`
* $HOME directory might not be what you think (`/root`)
* ...

- How to handle dependencies (for installation within actinia)
- use `requirements.txt` for python packages
- How can I log to actinia output? (with `gscript.message`)
- GRASS GIS addons need to be installed globally (see `$HOME`)
- Which things are tricky
- `db.login`, `db.connect -d`
- $HOME directory might not be what you think (`/root`)
- ...

## Steps to make the GRASS GIS addon public + open source

### General code-related steps

#### General steps

* Decide whether it should be a single addon or __multi-module / toolbox__. Example for multi-module is here: [t.sentinel](https://github.com/mundialis/t.sentinel)
- Decide whether it should be a single addon or __multi-module / toolbox__. Example for multi-module is here: [t.sentinel](https://github.com/mundialis/t.sentinel)

* Check for __sensitive information__. If included, remove them and publish without git history (or rewrite if needed).
- Check for __sensitive information__. If included, remove them and publish without git history (or rewrite if needed).

#### License + Copyright
* License is GPL3. If no LICENSE.md exists, create it later directly with GitHub (see section below).
* Copyright: no single person (only as author), `mundialis -> mundialis GmbH & Co. KG`
* Add license information to `*.py` file header:

- License is GPL3. If no LICENSE.md exists, create it later directly with GitHub (see section below).
- Copyright: no single person (only as author), `mundialis -> mundialis GmbH & Co. KG`
- Add license information to `*.py` file header:

```
# COPYRIGHT: (C) 2021-2024 by mundialis GmbH & Co. KG and the GRASS Development Team
Expand All @@ -158,10 +167,12 @@ A GRASS python module consists of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
```
* Adjust year

- Adjust year

#### Linting
* Add reusable linting workflow as described [here](https://github.com/mundialis/github-workflows?tab=readme-ov-file#python-linting)

- Add reusable linting workflow as described [here](https://github.com/mundialis/github-workflows?tab=readme-ov-file#python-linting)
```
name: Python Flake8, black and pylint code quality check
Expand All @@ -171,48 +182,52 @@ A GRASS python module consists of
lint:
uses: mundialis/github-workflows/.github/workflows/linting.yml@main
```
* Run locally. Mind that there might be differences due to version mismatches.
- Run locally. Mind that there might be differences due to version mismatches.
To overcome this, pre-commit hooks from the same repository can be used
```
black --check --diff --line-length 79 .
flake8 --config=.flake8 --count --statistics --show-source .
pylint .
```
* Fix lint errors which black couldn't fix or add exception to `.flake8` file
* Black does not fix GRASS GIS parameter `#%`. This can be batch changed with `sed -i 's|^#%|# %|g' foo.py`
* There may not be any linting issues left as the pipeline would fail
- Fix lint errors which black couldn't fix or add exception to `.flake8` file
- Black does not fix GRASS GIS parameter `#%`. This can be batch changed with `sed -i 's|^#%|# %|g' foo.py`
- There may not be any linting issues left as the pipeline would fail

#### General steps Part 2

* For GRASS GIS parameters, pay attention to label and description, so that the first word after the `:` is written in capital letters
* For GRASS GIS module description the sentence should be ended with a point. No point for parameter description or any other parameter value
* for GRASS messages like `grass.message()`, `grass.fatal()` and `grass.warning()`, use the format with underscore (in order to support [message translations](https://grasswiki.osgeo.org/wiki/GRASS_messages_translation)), e.g.: `grass.message(_("Message text"))`.
* NOTE that underscores in `grass.debug()` messages are not desired.
* NOTE that variables should not go into the macro, e.g. `grass.message(_("Created output group <%s>") % output)` instead of `grass.message(_("Created output group <%s>" % output))`
* There should be no space between the text and suspension points, e.g. `Reading raster map...` instead of `Reading raster map ...`
- For GRASS GIS parameters, pay attention to label and description, so that the first word after the `:` is written in capital letters
- For GRASS GIS module description the sentence should be ended with a point. No point for parameter description or any other parameter value
- for GRASS messages like `grass.message()`, `grass.fatal()` and `grass.warning()`, use the format with underscore (in order to support [message translations](https://grasswiki.osgeo.org/wiki/GRASS_messages_translation)), e.g.: `grass.message(_("Message text"))`.
- NOTE that underscores in `grass.debug()` messages are not desired.
- NOTE that variables should not go into the macro, e.g. `grass.message(_("Created output group <%s>") % output)` instead of `grass.message(_("Created output group <%s>" % output))`
- There should be no space between the text and suspension points, e.g. `Reading raster map...` instead of `Reading raster map ...`

For more information on standardized messages see [here](https://trac.osgeo.org/grass/wiki/MessageStandardization).

#### Tests
* If tests exist, the header should look like in the actual module.
* If tests exist, the test workflow should be included. [See here](https://github.com/mundialis/github-workflows?tab=readme-ov-file#grass-gis-addon-tests) for instructions.

- If tests exist, the header should look like in the actual module.
- If tests exist, the test workflow should be included. [See here](https://github.com/mundialis/github-workflows?tab=readme-ov-file#grass-gis-addon-tests) for instructions.

#### In the end
* Create a merge request in the old repository with your changes, so it can be reviewed there and moved to GitHub when cleanup is done

- Create a merge request in the old repository with your changes, so it can be reviewed there and moved to GitHub when cleanup is done

### Steps on GitHub
* Every addon gets its own GitHub repo
* default location is at https://github.com/mundialis. If needed they can be integrated somewhere else as submodule.
* Choose "GPL3" as license
* Copy code to new repository (not via `git remote` if sensitive information are included)
* Release new addon in GitHub with 1.0.0
* Add project label if applicable (e.g. `vale`, `hermosa-earth`, `incora`)
* Add / update README.md in project repositories which use this addon.

- Every addon gets its own GitHub repo
- default location is at https://github.com/mundialis. If needed they can be integrated somewhere else as submodule.
- Choose "GPL3" as license
- Copy code to new repository (not via `git remote` if sensitive information are included)
- Release new addon in GitHub with 1.0.0
- Add project label if applicable (e.g. `vale`, `hermosa-earth`, `incora`)
- Add / update README.md in project repositories which use this addon.

### Cleanup
* Delete "old" code from internal repository
* Add hint to internal repository README that the addon was moved and where to find it
* Adjust README.md in actinia-assets/grass-gis-addons
* Adjust deployments which use the addon, if applicable

- Delete "old" code from internal repository
- Add hint to internal repository README that the addon was moved and where to find it
- Adjust README.md in actinia-assets/grass-gis-addons
- Adjust deployments which use the addon, if applicable

​

0 comments on commit 3527844

Please sign in to comment.