Skip to content

Commit

Permalink
Allow processing files in parallel via new jobs option (#186)
Browse files Browse the repository at this point in the history
* Allow processing files in parallel via new jobs option

Resolves #184

* Update details config
  • Loading branch information
facelessuser authored Jan 13, 2024
1 parent 0fed0ee commit ba2650e
Show file tree
Hide file tree
Showing 10 changed files with 353 additions and 163 deletions.
1 change: 1 addition & 0 deletions .pyspelling.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
spellchecker: aspell
jobs: 8

matrix:
- name: mkdocs
Expand Down
1 change: 1 addition & 0 deletions docs/src/dictionary/en-custom.txt
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ chainable
charset
checkable
chunking
config
could've
dicts
distro
Expand Down
5 changes: 5 additions & 0 deletions docs/src/markdown/about/changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## 2.10

- **NEW**: Allow specifying parallel processes to speed up spell checking. Number of jobs can be specified either
by command line or via the config. Command line overrides the config.

## 2.9

- **NEW**: Officially support Python 3.11 and 3.12.
Expand Down
11 changes: 11 additions & 0 deletions docs/src/markdown/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,17 @@ This can be overridden on the command line.
spellchecker: hunspell
```
You can specify the number of parallel jobs to use by setting the global option `jobs`. This create parallel jobs to
process files in a given task.

```yaml
jobs: 4
```

/// new | New 2.10
Parallel processing is new in 2.10.
///

All of the spelling tasks are contained under the keyword `matrix` and are organized in a list:

```yaml
Expand Down
23 changes: 19 additions & 4 deletions docs/src/markdown/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,20 +53,24 @@ If you want to manually install it, run `#!bash python setup.py build` and `#!ba
## Command Line Usage

```
usage: pyspelling [-h] [--version] [--verbose] [--name NAME] [--binary BINARY]
[--config CONFIG] [--spellchecker SPELLCHECKER]
usage: pyspelling [-h] [--version] [--verbose] [--name NAME | --group GROUP] [--binary BINARY] [--jobs JOBS] [--config CONFIG] [--source SOURCE] [--spellchecker SPELLCHECKER]
Spell checking tool.
optional arguments:
options:
-h, --help show this help message and exit
--version show program's version number and exit
--verbose, -v Verbosity level.
--name NAME, -n NAME Specific spelling task by name to run.
--group GROUP, -g GROUP
Specific spelling task group to run.
--binary BINARY, -b BINARY
Provide path to spell checker's binary.
--jobs JOBS, -j JOBS Specify the number of spell checker processes to run in parallel.
--config CONFIG, -c CONFIG
Spelling config.
--source SOURCE, -S SOURCE
Specify override file pattern. Only applicable when specifying exactly one --name.
--spellchecker SPELLCHECKER, -s SPELLCHECKER
Choose between aspell and hunspell
```
Expand All @@ -82,7 +86,7 @@ If you have multiple Python versions, you can run the PySpelling associated with
Python major and minor version:

```shell-session
$ pyspelling3.7
$ pyspelling3.11
```

To specify a specific configuration other than the default, or even point to a different location:
Expand Down Expand Up @@ -132,6 +136,17 @@ You can specify the spell checker type by specifying it on the command line. PyS
$ pyspelling -s hunspell
```

To run multiple jobs in parallel, you can use the `--job` or `-j` option. Processing files in parallel can speed up
processing time. Specifying jobs on the command line will override the `jobs` setting in the configuration file.

```console
$ pyspelling -n my_task -j 4
```

/// new | New 2.10
Parallel processing is new in 2.10.
///

## Supported Spell Check Versions

PySpelling is tested with Hunspell 1.6+, and recommends using only 1.6 and above. Some lower versions might work, but
Expand Down
29 changes: 29 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,35 @@ markdown_extensions:
- example
- quote
- pymdownx.blocks.details:
types:
- name: details-new
class: new
- name: details-settings
class: settings
- name: details-note
class: note
- name: details-abstract
class: abstract
- name: details-info
class: info
- name: details-tip
class: tip
- name: details-success
class: success
- name: details-question
class: question
- name: details-warning
class: warning
- name: details-failure
class: failure
- name: details-danger
class: danger
- name: details-bug
class: bug
- name: details-example
class: example
- name: details-quote
class: quote
- pymdownx.blocks.html:
- pymdownx.blocks.definition:
- pymdownx.blocks.tab:
Expand Down
Loading

0 comments on commit ba2650e

Please sign in to comment.