From ffe183e653c3d30bf3a53a9f14965717783c5ead Mon Sep 17 00:00:00 2001 From: Paul Pogonyshev Date: Tue, 30 Apr 2024 21:40:04 +0200 Subject: [PATCH 1/3] Document Doctest support. --- doc/README.adoc | 2 ++ doc/continuous-integration.adoc | 2 +- doc/extending-eldev.adoc | 10 +++--- doc/testing.adoc | 55 ++++++++++++++++++++++++--------- 4 files changed, 48 insertions(+), 21 deletions(-) diff --git a/doc/README.adoc b/doc/README.adoc index 6f688b3..eb0b66b 100644 --- a/doc/README.adoc +++ b/doc/README.adoc @@ -18,6 +18,7 @@ :uri-ert-sel: https://www.gnu.org/software/emacs/manual/html_node/ert/Test-Selectors.html :uri-buttercup: https://github.com/jorgenschaefer/emacs-buttercup :uri-buttercup-rt: https://github.com/jorgenschaefer/emacs-buttercup/blob/master/docs/running-tests.md +:uri-doctest: https://github.com/ag91/doctest :uri-ecukes: https://github.com/ecukes/ecukes :uri-cask: https://github.com/cask/cask :uri-makem-sh: https://github.com/alphapapa/makem.sh @@ -72,6 +73,7 @@ :since-1-7: image:https://img.shields.io/badge/since-1.7-8be[Since 1.7,float=right] :since-1-8: image:https://img.shields.io/badge/since-1.8-8be[Since 1.8,float=right] :since-1-9: image:https://img.shields.io/badge/since-1.9-8be[Since 1.9,float=right] +:since-1-10: image:https://img.shields.io/badge/since-1.10-8be[Since 1.10,float=right] toc::[] diff --git a/doc/continuous-integration.adoc b/doc/continuous-integration.adoc index d7d1a34..b448f88 100644 --- a/doc/continuous-integration.adoc +++ b/doc/continuous-integration.adoc @@ -5,7 +5,7 @@ Because of Eldev’s trivial installation and built-in support for testing, it is a suitable tool for use on continuous integration servers. But of course this only applies if the test framework your project uses is already supported (currently <>, -<> and <>). +<>, <> and <>). Eldev will even try to make your CI runs <>. diff --git a/doc/extending-eldev.adoc b/doc/extending-eldev.adoc index 8b9d5b6..ba5c2a1 100644 --- a/doc/extending-eldev.adoc +++ b/doc/extending-eldev.adoc @@ -56,11 +56,11 @@ Functions for many of the hooks listed below do receive arguments. {since-0-2} `eldev-test-FRAMEWORK-hook (SELECTORS)`:: Called immediately before executing tests with given framework - (<>, <>, <>). - Functions on the hook get passed `SELECTORS` as the only argument. - At this point project dependencies and additional set `test` will - have been loaded already, so functions can `require` features from - the project. + (<>, <>, <>, + <>). Functions on the hook get passed `SELECTORS` + as the only argument. At this point project dependencies and + additional set `test` will have been loaded already, so functions + can `require` features from the project. === Writing builders diff --git a/doc/testing.adoc b/doc/testing.adoc index 979a88c..69e3507 100644 --- a/doc/testing.adoc +++ b/doc/testing.adoc @@ -3,9 +3,9 @@ Eldev has built-in support for running regression/unit tests of your project. Currently, Eldev supports {uri-ert}[ERT], -{uri-buttercup}[Buttercup] and {uri-ecukes}[Ecukes] testing -frameworks. Leave a feature request in the issue tracker if you are -interested in a different library. +{uri-buttercup}[Buttercup], {uri-doctest}[Doctest] and +{uri-ecukes}[Ecukes] testing frameworks. Leave a feature request in +the issue tracker if you are interested in a different library. Simply executing @@ -73,9 +73,9 @@ command `test` work. `simple`:: Like `standard` with a few Eldev-specific tweaks that I feel are useful (see `--list-runners` output for details). This is the default runner. -`concise`:: {since-1-6} Like `simple`, but progress output for ERT and -Buttercup is replaced by a single dot per passing test. Give it a try -to decide if you like that or not. +`concise`:: {since-1-6} Like `simple`, but progress output for ERT, +Buttercup and Doctest is replaced by a single dot per passing test. +Give it a try to decide if you like that or not. If you always use a different test runner, it is a good idea to set it as the default in file `~/.config/eldev/config`. Finally, you can @@ -85,15 +85,16 @@ even write your own runner. === Frameworks As stated above, Eldev supports {uri-ert}[ERT] (Emacs built-in), -{uri-buttercup}[Buttercup] and {uri-ecukes}[Ecukes] testing -frameworks. Normally, you don’t need to specify which framework the -project uses, as the tool can autodetect that. But in rare cases you -may need to set variable `eldev-test-framework` to either `'ert`, -`'buttercup` or `'ecukes`, as appropriate. It is also possible to use -more than one framework in a project, <>. You don’t need to declare testing package(s) as -<>: Eldev will install -them itself when needed. +{uri-buttercup}[Buttercup], {uri-doctest}[Doctest] and +{uri-ecukes}[Ecukes] testing frameworks. With the exception of +Doctest, you don’t need to specify which framework the project uses, +as the tool can autodetect that. But in some cases (for Doctest — +always) you may need to set variable `eldev-test-framework` to either +`'ert`, `'buttercup`, `'doctest` or `'ecukes`, as appropriate. It is +also possible to use more than one framework in a project, +<>. You don’t need to declare testing +package(s) as <>: Eldev +will install them itself when needed. Eldev tries to provide uniform command line interface to the supported frameworks, but of course there are many differences between them. @@ -132,6 +133,30 @@ Things that won’t work with Buttercup at the moment: Unlike <>, Buttercup also has no special selectors that base on the previous run’s results. +[#doctest] +==== Doctest + +{since-1-10} {uri-doctest}[Doctest] is a framework that allows you to +embed test expressions into source code, e.g. function documentation. +As a result, your unit tests turn into documentation that your users +can read. + +Because the tests are located directly in the source code rather than +in separate files, there is no robust way to autodetect their +presence. For this reason, you must _always_ <> using variable `eldev-test-framework`. +You can also freely mix it with other tests by employing +<> in the same project. + +Support for Doctest in Eldev is quite limited and many things won’t +work currently: + +* the framework has no notion comparable to selectors, so those will + be ignored; +* Doctest doesn’t print failure backtraces, so relevant options have + no effect; +* option `--stop-on-unexpected` is not supported. + [#ecukes] ==== Ecukes From 054c4eba7cdbd76c7131aa492573985edad291d2 Mon Sep 17 00:00:00 2001 From: Paul Pogonyshev Date: Thu, 2 May 2024 17:49:38 +0200 Subject: [PATCH 2/3] Document `eldev-display-indirect-build-stdout`. --- doc/loading-modes.adoc | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/doc/loading-modes.adoc b/doc/loading-modes.adoc index 0ffc4dd..9e5b8bb 100644 --- a/doc/loading-modes.adoc +++ b/doc/loading-modes.adoc @@ -94,6 +94,38 @@ You can always ask Eldev for a full list: $ eldev --list-modes +[#indirect-builds] +=== Indirect build information + +When a loading mode require Eldev to do something in order to prepare +the project or its local dependencies for loading, it tries to do so +silently in that only stderr is normally displayed. The purpose is to +prevent secondary and partially unpredictable (more precisely, +depending on previous builds) output from interfering with normal +output. For example, if you run + + $ eldev eval "(some-project-function)" + +it might be confusing if the first line of output is instead + + ELC some-file.el + +if the loading mode is `byte-compiled` and `some-file.elc` doesn’t +exist or is out-of-date. In particular, if such output is then parsed +using some automated tool, this could lead to unexpected errors. + +{since-1-10} However, if this is not a concern in your case, you may +want to set variable `eldev-display-indirect-build-stdout` to t. This +is especially useful if your project’s loading mode is `built` and it +involves some custom non-trivial build steps, like e.g. compilation of +a helper non-Elisp program. + +Unlike with some other settings, the main project ignores values in +local dependencies. Instead, `eldev-display-indirect-build-stdout` as +defined in the main project affects both the project itself and all +local dependencies at once: only the main project “knows” if it is +important to avoid indirect build output for it or not. + [#source-directory] === Project source directory From 9285ecb58ab066496553dddb93f8248dfd074588 Mon Sep 17 00:00:00 2001 From: Paul Pogonyshev Date: Thu, 2 May 2024 17:58:14 +0200 Subject: [PATCH 3/3] Document new Podman support, which is basically the same as Docker for Eldev's purposes. --- README.adoc | 3 ++- doc/README.adoc | 1 + doc/emacs-versions.adoc | 34 +++++++++++++++++++++------------- doc/overview.adoc | 3 ++- 4 files changed, 26 insertions(+), 15 deletions(-) diff --git a/README.adoc b/README.adoc index ce896e4..c975d5d 100644 --- a/README.adoc +++ b/README.adoc @@ -51,7 +51,8 @@ provides only a brief overview. setups. * Can run on {uri-documentation}#different-emacs-versions[different Emacs version] even on the same machine; can also use - {uri-documentation}#docker[Docker] for that. + {uri-documentation}#docker[Docker] or + {uri-documentation}#podman[Podman] for that. * There are {uri-documentation}#setup-procedure[_four_ levels of configuration] — you can customize most aspects of Eldev for your project needs and personal preferences. diff --git a/doc/README.adoc b/doc/README.adoc index eb0b66b..8cf0256 100644 --- a/doc/README.adoc +++ b/doc/README.adoc @@ -38,6 +38,7 @@ :uri-travis: https://travis-ci.org/ :uri-circle: https://circleci.com/ :uri-docker: https://www.docker.com/ +:uri-podman: https://podman.io/ :uri-coveralls: https://coveralls.io/ :uri-setup-emacs: https://github.com/jcs090218/setup-emacs :uri-setup-eldev: https://github.com/emacs-eldev/setup-eldev diff --git a/doc/emacs-versions.adoc b/doc/emacs-versions.adoc index 3e8b9f7..9980a70 100644 --- a/doc/emacs-versions.adoc +++ b/doc/emacs-versions.adoc @@ -38,21 +38,28 @@ Therefore, if you use byte-compilation and switch Emacs versions, don’t forget to clean the directory. [#docker] -=== Using Docker +=== Using Docker or Podman Alternatively, if you are on a Linux or macOS system and have -{uri-docker}[Docker] installed, you can run arbitrary Eldev commands -within containers based on the images distributed by -{uri-docker-emacs}[docker-emacs]. For example: +{uri-docker}[Docker] or (since 1.10) {uri-podman}[Podman] installed, +you can run arbitrary Eldev commands within containers based on the +images distributed by {uri-docker-emacs}[docker-emacs]. For example: $ eldev docker 27.2 emacs --eval '(insert (format "Emacs version: %s" emacs-version))' will start an Emacs 27.2 container and run `eldev emacs --eval '(insert (format "Emacs version: %s" emacs-version))'` in it. +[#podman] +{since-1-10} Podman, from Eldev’s point of view, works just the +same. There is a special command for it, but it is basically +indistinguishable (except for the name): + + $ eldev podman 27.2 emacs --eval '(insert (format "Emacs version: %s" emacs-version))' + NOTE: You may have to run `xhost +local:root` which allows the Docker -container to make connections to the host X server. However, this -does come with some security considerations, see `man xhost`. +(Podman) container to make connections to the host X server. However, +this does come with some security considerations, see `man xhost`. This command can be used not only to start Emacs of given version, but to run _any_ Eldev command. For example, run project’s tests on an @@ -64,12 +71,12 @@ or evaluate something using project’s functions: $ eldev docker 26.3 eval "(my-project-read-data \"foo.bin\")" -Docker’s output is forwarded to normal Eldev output, however, because -of {uri-emacs-forwarding}[Elisp limitations], it all ends up on -Eldev’s stdout! There might also be unwieldy delays, so that output -doesn’t come smoothly as generated by the process inside Docker, but -instead in larger chunks. Before Eldev 1.2 the output would instead -only appear when Docker has exited. +Docker’s (Podman’s) output is forwarded to normal Eldev output, +however, because of {uri-emacs-forwarding}[Elisp limitations], it all +ends up on Eldev’s stdout! There might also be unwieldy delays, so +that output doesn’t come smoothly as generated by the process inside +Docker, but instead in larger chunks. Before Eldev 1.2 the output +would instead only appear when Docker has exited. It is also possible to use a custom image. For this, replace Emacs version argument (26.3 in the last example above) with the full image @@ -77,7 +84,8 @@ name. The image must contain a preinstalled Emacs of a version supported by Eldev (i.e. 24.4 and up), but not Eldev itself. Additionally, `docker run` arguments are customisable via the variable -`eldev-docker-run-extra-args`. For example, adding the following to +`eldev-docker-run-extra-args` (and likewise for Podman: +`eldev-podman-run-extra-args`). For example, adding the following to your project’s `Eldev`: .... diff --git a/doc/overview.adoc b/doc/overview.adoc index 07dbfa7..3dfecfc 100644 --- a/doc/overview.adoc +++ b/doc/overview.adoc @@ -9,7 +9,8 @@ Eldev features: * Blends nicely into <> setups. * Can run on <> even - on the same machine; can also use <> for that. + on the same machine; can also use <> or + <> for that. * There are <> — you can customize most aspects of Eldev for your project needs and personal preferences.