Use copier instead of cookiecutter #190

GenevieveBuckley · 2024-06-13T07:28:43Z

This PR switches the plugin template repo to using copier instead of cookiecutter

The biggest advantage of using copier is that users can run copier update to check for any changes in the template.

This will be very useful for ongoing code maintenance (eg: updating versions of dependencies, any changes required to keep the test_and_deploy github action working smoothly, etc.)

But it also has some smaller advantages, like being able to conditionally generate files, depending on the user's answers to the prompt questions (cookiecutter does not do this, were were previously using a hack in the hooks/post_gen.py file to delete the unnecessary files)

Steps required (ideally all steps should be close to simultaneous):

Merge this PR
rename repository napari-plugin-template instead of cookiecutter-napari-plugin (and remove the other test napari-plugin-template repo)
Merge corresponding documentation PR, replacing references to cookiecutter with copier: Replace cookiecutter references with napari-plugin-template docs#427

Previously:

jni · 2024-06-13T08:09:44Z

Thanks for reopening @GenevieveBuckley! 🚀

Tests are failing currently 🤔 Also when I pull this down to try it out, I got an error:

$ copier copy --trust cookiecutter-napari-plugin algospatial
No git tags found in template; using HEAD as ref
🎤 The name of your plugin
   algospatial
🎤 Display name for your plugin
   AlgoSpatial
🎤 Plugin module name
   algospatial
🎤 Short description of what your plugin does
   Spatial algorithms in napari
🎤 Developer name
   Juan Nunez-Iglesias
🎤 Email address
   juan.nunez-iglesias@monash.edu
🎤 Github user or organisation name
   jni
🎤 Github repository URL
   provide later
🎤 Include reader plugin?
   Yes
🎤 Include writer plugin?
   No
🎤 Include sample data plugin?
   No
🎤 Include widget plugin?
   Yes
🎤 Use git tags for versioning?
   Yes
🎤 Install pre-commit? (Code formatting checks)
   No
🎤 Install dependabot? (Automatic security updates of dependency versions)
   No
🎤 Which licence do you want your plugin code to have?
   Mozilla Public License 2.0

Copying from template version 0.0.0.post109.dev0+134ce29
    create  .
    create  .napari-hub
    create  .napari-hub/DESCRIPTION.md
    create  .napari-hub/config-yml
    create  .gitignore
    create  .github
    create  .github/workflows
    create  .github/workflows/test_and_deploy
    create  src
    create  src/algospatial
    create  src/algospatial/_tests
    create  src/algospatial/_tests/__init__.py

followed by this crash:

Traceback (most recent call last):
  File "/Users/jni/micromamba/envs/copier/bin/copier", line 10, in <module>
    sys.exit(CopierApp.run())
             ^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/plumbum/cli/application.py", line 638, in run
    inst, retcode = subapp.run(argv, exit=False)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/plumbum/cli/application.py", line 633, in run
    retcode = inst.main(*tailargs)
              ^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/cli.py", line 280, in main
    return _handle_exceptions(inner)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/cli.py", line 70, in _handle_exceptions
    method()
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/cli.py", line 271, in inner
    with self._worker(
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 217, in __exit__
    raise value
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/cli.py", line 278, in inner
    worker.run_copy()
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 769, in run_copy
    self._render_folder(src_abspath)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 659, in _render_folder
    self._render_folder(file)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 659, in _render_folder
    self._render_folder(file)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 659, in _render_folder
    self._render_folder(file)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 661, in _render_folder
    self._render_file(file)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 582, in _render_file
    new_content = tpl.render(**self._render_context()).encode()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/private/var/folders/z7/mv1jznhx59q5vc3bp09gj69m0000gn/T/copier.vcs.clone.y8mvpar1/template/src/{{module_name}}/_tests/{% if include_widget_plugin %}test_widget.py{% endif %}.jinja", line 3, in top-level template code
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/jinja2/sandbox.py", line 327, in getattr
    value = getattr(obj, attribute)
            ^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'cookiecutter' is undefined

maybe something missing from carrying the PR over? (Since there shouldn't be any reference to cookiecutter left...?)

Also, the plugin name is a default rather than a placeholder — I think it should also be a placeholder?

GenevieveBuckley · 2024-06-13T08:45:44Z

Also, the plugin name is a default rather than a placeholder — I think it should also be a placeholder?

I'm starting to feel like placeholders are most suitable for optional variables. Otherwise, if someone just hits enter the whole way through the prompts, that will give us empty prompt values and can crash the operation. Opinions?

You must have a non-empty value for three questions: plugin_name, display_name, and module_name. If any of those are left blank, because of how we've set the template repository up, you can't create a new repo from it. There's a returned non-zero exit status 1 error message.

Details (click to expand)

🎤 The name of your plugin
   napari-foobar
🎤 Display name for your plugin

🎤 Plugin module name
   napari_foobar
🎤 Short description of what your plugin does
   A simple plugin to use FooBar segmentation within napari
🎤 Developer name
   Napari Developer
🎤 Email address
   yourname@example.com
🎤 Github user or organisation name
   githubuser
🎤 Github repository URL
   provide later
🎤 Include reader plugin?
   Yes
🎤 Include writer plugin?
   Yes
🎤 Include sample data plugin?
   Yes
🎤 Include widget plugin?
   Yes
🎤 Use git tags for versioning?
   No
🎤 Install pre-commit? (Code formatting checks)
   No
🎤 Install dependabot? (Automatic security updates of dependency versions)
   No
🎤 Which licence do you want your plugin code to have?
   BSD-3

Copying from template version 0.0.0.post113.dev0+3c0a5eb
    create  .
    create  .napari-hub
    create  .napari-hub/DESCRIPTION.md
    create  .napari-hub/config-yml
    create  LICENSE
    create  .gitignore
    create  .github
    create  .github/workflows
    create  .github/workflows/test_and_deploy
    create  src
    create  src/napari_foobar
    create  src/napari_foobar/_writer.py
    create  src/napari_foobar/_tests
    create  src/napari_foobar/_tests/test_sample_data.py
    create  src/napari_foobar/_tests/test_writer.py
    create  src/napari_foobar/_tests/__init__.py
    create  src/napari_foobar/_tests/test_widget.py
    create  src/napari_foobar/_tests/test_reader.py
    create  src/napari_foobar/_sample_data.py
    create  src/napari_foobar/napari.yaml
    create  src/napari_foobar/__init__.py
    create  src/napari_foobar/_reader.py
    create  src/napari_foobar/_widget.py
    create  src/README.md
    create  src/pyproject.toml
    create  src/MANIFEST.in
    create  src/tox.ini

 > Running task 1 of 1: ['/Users/genevieb/mambaforge/envs/plugin-template-dev/bin/python3.11', '/private/var/folders/82/phlx_f6j5518qtvpm4x0_0fm0000gq/T/copier.vcs.clone.i6_fg585/_tasks.py', '--plugin_name=napari-foobar', '--module_name=napari_foobar', '--project_directory=plugin-test4', '--install_precommit=False', '--github_repository_url=provide later', '--github_username_or_organization=']
ERROR:pre_gen_project:b'\xf0\x9f\x85\x87 Invalid! 4 validation errors for PluginManifest\ndisplay_name\n  none is not an allowed value (type=type_error.none.not_allowed)\ncontributions -> sample_data -> 0 -> display_name\n  none is not an allowed value (type=type_error.none.not_allowed)\ncontributions -> sample_data -> 0 -> display_name\n  none is not an allowed value (type=type_error.none.not_allowed)\ncontributions -> sample_data -> 0 -> uri\n  field required (type=value_error.missing)'
Traceback (most recent call last):
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/bin/copier", line 10, in <module>
    sys.exit(CopierApp.run())
             ^^^^^^^^^^^^^^^
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/plumbum/cli/application.py", line 638, in run
    inst, retcode = subapp.run(argv, exit=False)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/plumbum/cli/application.py", line 633, in run
    retcode = inst.main(*tailargs)
              ^^^^^^^^^^^^^^^^^^^^
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/cli.py", line 280, in main
    return _handle_exceptions(inner)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/cli.py", line 70, in _handle_exceptions
    method()
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/cli.py", line 271, in inner
    with self._worker(
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/main.py", line 217, in __exit__
    raise value
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/cli.py", line 278, in inner
    worker.run_copy()
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/main.py", line 774, in run_copy
    self._execute_tasks(self.template.tasks)
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/main.py", line 296, in _execute_tasks
    subprocess.run(task_cmd, shell=use_shell, check=True, env=local.env)
  File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/Users/genevieb/mambaforge/envs/plugin-template-dev/bin/python3.11', '/private/var/folders/82/phlx_f6j5518qtvpm4x0_0fm0000gq/T/copier.vcs.clone.i6_fg585/_tasks.py', '--plugin_name=napari-foobar', '--module_name=napari_foobar', '--project_directory=plugin-test4', '--install_precommit=False', '--github_repository_url=provide later', '--github_username_or_organization=']' returned non-zero exit status 1.

jni · 2024-06-13T08:50:04Z

I'm fine with an error. I personally find it jarring to have to erase text to type what I want. It would be better if copier would give you the option of setting something as not-empty so that the error could be caught early, but I think that's a thing for copier to worry about, not us. (Maybe the feature exists, or maybe we can make an issue in the repo.)

jni · 2024-06-13T08:52:25Z

https://copier.readthedocs.io/en/stable/reference/user_data/#copier.user_data.Question

Looks like you can put in a "validator":

Jinja template with which to validate the user input. This template will be rendered with the combined answers as variables; it should render nothing if the value is valid, and an error message to show to the user otherwise.

I'm not familiar enough with template language here but I presume we can implement exactly the check above...

GenevieveBuckley · 2024-06-13T08:55:05Z

Oh good. There's one more check that needs to happen - a valid email address also appears to be required (you can generate a new repo from the template without it, but then tox will fail on that new repo)

jni · 2024-06-13T08:55:44Z

Here's an example of using validators:

https://copier.readthedocs.io/en/stable/configuring/#advanced-prompt-formatting

project_name:
    type: str # Any value will be treated raw as a string
    help: An awesome project needs an awesome name. Tell me yours.
    default: paradox-specifier
    validator: >-
        {% if not (project_name | regex_search('^[a-z][a-z0-9\-]+$')) %}
        project_name must start with a letter, followed one or more letters, digits or dashes all lowercase.
        {% endif %}

GenevieveBuckley · 2024-06-13T09:01:15Z

Ooh, maybe we can also use the validators to check module_name first the PEP specs. If we can get rid of _tasks.py, then we don't need to use the copier --trust command anymore, which would be nice.

jni · 2024-06-13T09:02:55Z

The above validator seems to do just that, so I think that's a yes! And indeed it would be amazing!

…nt to provide an email

GenevieveBuckley · 2024-06-14T02:05:30Z

Two points:

Sadly we can't get rid of _tasks.py and --trust. It does more than just plugin and module name validation (also git init, optional pre-commit install, and validating the manifest with npe2). I'd forgotten about those extra tasks in my excitement.
I'm choosing not to validate the input email address, because then users cannot leave it blank. That seems a bit privacy violating to me. The jinja regex engine will not match an empty string, no matter how I write that in the regex, and I can't figure out if there's an 'allow_empty' kwarg I can somehow pass it through copier.

jni · 2024-07-11T07:19:49Z

Closing in favour of napari/napari-plugin-template#7

# References and relevant issues Pair PR to ~~napari/cookiecutter-napari-plugin#190~~ napari/napari-plugin-template#7 # Description This PR removes all references to cookiecutter (except in past release notes), and replaces them with references to the `napari-plugin-template` repository. This is so that when we merge PR napari/napari-plugin-template#7 switching the template engine from cookiecutter to copier, the napari docs will match.

Use copier instead of cookiecutter

134ce29

GenevieveBuckley requested review from DragaDoncila and jni June 13, 2024 07:29

GenevieveBuckley added 2 commits June 13, 2024 18:11

Remove references to cookiecutter and pytest-cookies

80adc47

Fix typo project_path -> project_dir

796a0e4

GenevieveBuckley mentioned this pull request Jun 13, 2024

Replace cookiecutter references with napari-plugin-template napari/docs#427

Merged

Fix paths to license files

6d7a2bd

Shift files to correct location, update copier.yaml defaults

e12e6f9

Move tox.ini in child repo to the proper location

df7e578

GenevieveBuckley added 5 commits June 13, 2024 19:30

Add copier validators to user inputs

130d0f9

Fix typo in tests

1be8418

Fix __init__.py.jinja formatting

311a013

Add mandatory argument 'module_name' to tests

a41d484

Remove email address validation, since this would make it a requireme…

8a7ec95

…nt to provide an email

GenevieveBuckley mentioned this pull request Jul 11, 2024

Replace cookiecutter with Copier napari/napari-plugin-template#7

Merged

jni closed this Jul 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use copier instead of cookiecutter #190

Use copier instead of cookiecutter #190

GenevieveBuckley commented Jun 13, 2024 •

edited

Loading

jni commented Jun 13, 2024

GenevieveBuckley commented Jun 13, 2024

jni commented Jun 13, 2024

jni commented Jun 13, 2024

GenevieveBuckley commented Jun 13, 2024

jni commented Jun 13, 2024

GenevieveBuckley commented Jun 13, 2024

jni commented Jun 13, 2024

GenevieveBuckley commented Jun 14, 2024

jni commented Jul 11, 2024

Use copier instead of cookiecutter #190

Use copier instead of cookiecutter #190

Conversation

GenevieveBuckley commented Jun 13, 2024 • edited Loading

jni commented Jun 13, 2024

GenevieveBuckley commented Jun 13, 2024

jni commented Jun 13, 2024

jni commented Jun 13, 2024

GenevieveBuckley commented Jun 13, 2024

jni commented Jun 13, 2024

GenevieveBuckley commented Jun 13, 2024

jni commented Jun 13, 2024

GenevieveBuckley commented Jun 14, 2024

jni commented Jul 11, 2024

GenevieveBuckley commented Jun 13, 2024 •

edited

Loading