Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improvements to the check command. #18

Merged
merged 4 commits into from
Sep 13, 2024

Conversation

peterhuene
Copy link
Collaborator

This commit contains the following improvements:

  • use wdl-analysis for the check command that will perform a full static-analysis on the specified file or directory.
  • add a progress bar to display for the check command.
  • use clap-verbosity-flag for the quiet and verbose flags; this has the advantage of being error level by default and each -v specified drops the log level down by one; this gives a little more control over the previous implementation that supported either info (default), debug (-v), or error (-q).
  • we now colorize the error part of handling errors from command execution, displaying the full context of the error, and exiting with status code 1.

Before submitting this PR, please make sure:

  • You have added a few sentences describing the PR here.
  • You have added yourself or the appropriate individual as the assignee.
  • You have added at least one relevant code reviewer to the PR.
  • Your code builds clean without any errors or warnings.
  • You have added tests (when appropriate).
  • You have updated the README or other documentation to account for these changes (when appropriate).

This commit contains the following improvements:

* use `wdl-analysis` for the `check` command that will perform a full
  static-analysis on the specified file or directory.
* add a progress bar to display for the `check` command.
* use `clap-verbosity-flag` for the `quiet` and `verbose` flags; this has the
  advantage of being error level by default and each `-v` specified drops the
  log level down by one; this gives a little more control over the previous
  implementation that supported either info (default), debug (-v), or error
  (-q).
* we now colorize the `error` part of handling errors from command execution,
  displaying the full context of the error, and exiting with status code 1.
@peterhuene peterhuene requested a review from a-frantz September 3, 2024 19:42
Copy link
Member

@a-frantz a-frantz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty awesome. Going to checkout the branch and play around, kick some tires, then will come back and approve or share any found issues.

Copy link
Member

@a-frantz a-frantz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few thoughts from playing around with these changes.

  1. sprocket lint is now "behind" sprocket check, in that it doesn't have all the shnazzy new features you added. Can we move that logic so that it's shared by both commands?
  2. I do have an issue with the progress bar, and it's that sprocket is too dang fast! The progress bar is up for like less than a second even when I'm processing my largest test repos. I wonder what the progress bar is actually adding to the experience.
  3. Can we get a --deny-notes in addition to --deny-warnings? This is useful for CI where you want the strictest setting possible

@a-frantz
Copy link
Member

a-frantz commented Sep 4, 2024

Ah another tweak I'd like to see, but can maybe wait till another PR. IMO a screen full of diagnostics would be easier to make sense of if there was some additional break between files. Maybe just like 3 blank lines in a row? Or maybe a row of ==== or something?

@peterhuene
Copy link
Collaborator Author

peterhuene commented Sep 5, 2024

sprocket lint is now "behind" sprocket check, in that it doesn't have all the shnazzy new features you added. Can we move that logic so that it's shared by both commands?

I intentionally didn't migrate the lint command because I wanted the command to remain focused on linting a single document without having it perform a full resolution of the document and its dependencies. That is to say, lint is a lightweight command that lints only the specified file, whereas check does full static analysis on the file (with optional linting diagnostics at the same time).

I do have an issue with the progress bar, and it's that sprocket is too dang fast! The progress bar is up for like less than a second even when I'm processing my largest test repos. I wonder what the progress bar is actually adding to the experience.

Indeed, when I run it on stjudecloud/workflows, it disappears in a second and the individual progress indications are too fast to comprehend; however, as analysis does download URL-referenced imports, I think it's a good idea to have something that might indicate the tool is doing something while waiting on network I/O or processing a truly huge document graph.

We could perhaps only display the progress bar if there's been some elapsed time (say, a few seconds), that way we don't always show it in a blink such that it's not adding anything to the UX.

Can we get a --deny-notes in addition to --deny-warnings? This is useful for CI where you want the strictest setting possible

That seems reasonable, I can add that.

Ah another tweak I'd like to see, but can maybe wait till another PR. IMO a screen full of diagnostics would be easier to make sense of if there was some additional break between files. Maybe just like 3 blank lines in a row? Or maybe a row of ==== or something?

Adding some additional line breaks or other marker seems like an easy enough thing to add to the UX; I'll tinker with that.

@a-frantz
Copy link
Member

a-frantz commented Sep 5, 2024

I intentionally didn't migrate the lint command because I wanted the command to remain focused on linting a single document without having it perform a full resolution of the document and its dependencies. That is to say, lint is a lightweight command that lints only the specified file, whereas check does full static analysis on the file (with optional linting diagnostics at the same time).

Ah that makes much more sense! I have a few thoughts though:

  1. if we decide to go forward with this division of purpose, docs need to be updated to reflect this. As it was initially written, lint isn't much more than an alias for check --lint and a new direction warrants some better communication.
  2. But I'm not entirely sold on the idea yet. I think this would make much more sense if there were significant performance differences in the tasks. As it stands both complete in ~under a second even with our heaviest repos. What's the purpose of a lightweight option when the "heavy duty" version provides more information for the same time/cost???

Indeed, when I run it on stjudecloud/workflows, it disappears in a second and the individual progress indications are too fast to comprehend; however, as analysis does download URL-referenced imports, I think it's a good idea to have something that might indicate the tool is doing something while waiting on network I/O or processing a truly huge document graph.

We could perhaps only display the progress bar if there's been some elapsed time (say, a few seconds), that way we don't always show it in a blink such that it's not adding anything to the UX.

Good idea. I think gating the progress bar behind a 2 second delay sounds pretty perfect.

Copy link
Member

@claymcleod claymcleod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@peterhuene peterhuene requested a review from a-frantz September 13, 2024 17:07
@peterhuene peterhuene merged commit b266a4a into stjude-rust-labs:main Sep 13, 2024
6 checks passed
@peterhuene peterhuene deleted the check-command branch September 13, 2024 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants