All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning.
3.6.2 - 2024-09-10
This is a small release to fix a bug in the release automation for 3.6.1
.
For more information on the key content for this release, check out 3.6.1
.
Huge thanks to Ashley and the rest of the Axo team for their quick action to figure out this bug and get a workaround implemented!
- temporarily shift mac sys deps to custom build setup by @ashleygwilliams in #367
- @ashleygwilliams made their first contribution in #367
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.6.1...hipcheck-v3.6.2
3.6.1 - 2024-09-09
3.6.1
is a patch release, but it features one exciting new,
still-experimental feature: support for plugins! Previously, if you wanted
to add a new source of data or a new form of analsis to Hipcheck, you needed
to modify Hipcheck itself. This placed some limitations on our ability to grow
Hipcheck, and more importantly it conflicted with a core design philosophy
of Hipcheck: that it should empower uses to express their policies about
using open source software (this is one of our Product Values, as expressed in
RFD #2).
With 3.6.1
users can now define custom plugins to provide new sources of
data and new analyses. There's a lot more for us to do with this, including
a lot of user experience polish, releasing our first SDK to make developing
plugins easier, creating and publishing documentation on how to create,
distribute, and use plugins, and more! For now though, this launch is our
official starting point where users can create, distribute, and run plugins.
For more details on the design of the new plugin system, check out RFD #4. If you have questions on how to work with plugins, you can always ask us in our GitHub Discussions forum!
- introduce plugin-based query infrastructure to scoring by @j-lanson in #327
- Adds structures and functions to parse policy files by @mchernicoff in #330
- implement PolicyFile --> AnalysisTree conversion, remove use of WeightTree in scoring by @j-lanson in #334
- Converts a provided config TOML file to a policy file struct if no policy file is provided by @mchernicoff in #336
- Fix formatting for Rust 1.81.0, rustfmt 1.7.1-stable by @cstepanian in #337
- Add JSON Pointer Preprocessor by @cstepanian in #315
- Deprecates the config arg by @mchernicoff in #339
- Implement large portion of the plugin system by @j-lanson in #349
- Fixup
cargo xtask check
issues by @alilleybrinker - Reduce dead code "allow"s by @alilleybrinker
- Impl PluginContext::explain_default_query by @alilleybrinker
- Cleanup and add comments to PluginContext by @alilleybrinker
- Group imports at crate level by @alilleybrinker
- Move "Context" under "error" module by @alilleybrinker in #351
- update report to use investigate policy expression by @j-lanson in #352
- Move "command_util" to "util/command" by @alilleybrinker
- Move "kdl_helper" to "util/kdl" by @alilleybrinker
- Fix warnings in plugin code by @alilleybrinker
- Add
Analysis::Plugin
variant by @alilleybrinker - Added TODOs in ReportBuilder by @alilleybrinker in #353
- implement suggesting investigation if certain analyses fail by @j-lanson in #355
- Add support for reporting plugin results by @alilleybrinker in #361
- Adds weight field to policy file categories by @mchernicoff in #333
- Update hardcoded policy expressions with new JSON pointer syntax by @cstepanian in #338
- make HcEngine plugin lookup use {publisher}/{plugin} as key by @j-lanson in #362
- Install the protobuf compiler in release CI by @alilleybrinker in #328
- Copy plugins into containerfile build step by @alilleybrinker in #331
- More CI jobs by @alilleybrinker in #332
- Bump prost from 0.13.1 to 0.13.2 by @dependabot[bot] in #323
- Bump tonic from 0.12.1 to 0.12.2 by @dependabot[bot] in #322
- Bump tokio from 1.39.3 to 1.40.0 by @dependabot[bot] in #321
- Bump ureq from 2.10.0 to 2.10.1 by @dependabot[bot] in #319
- Bump xml-rs from 0.8.21 to 0.8.22 by @dependabot[bot] in #356
- Bump dashmap from 6.0.1 to 6.1.0 by @dependabot[bot] in #357
- Bump tonic-build from 0.12.1 to 0.12.2 by @dependabot[bot] in #358
- Bump rustls-native-certs from 0.7.1 to 0.8.0 by @dependabot[bot] in #360
- Bump anyhow from 1.0.86 to 1.0.87 by @dependabot[bot] in #359
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.6.0...hipcheck-v3.6.1
3.6.0 - 2024-08-30
This is a relatively small release, as we work on the initial implementation of our new plugin system as described in RFD #4.
There are two fixes in this release:
hc cache
now applies pattern toall
target by @j-lanson in #258- Fix quiet verbosity regression introduced in 3.5.0 by @patrickjcasey in #299
Plus lots of work on implementing plugins (not yet ready for use):
- add proto-file based code gen by @j-lanson in #277
- added parsing logic for download manifests by @patrickjcasey in #282
- add plugin initialization over gRPC and plugin engine struct by @j-lanson in #281
- added parsing logic for plugin manifests by @patrickjcasey in #293
- add dummy plugin and hidden cli command for invoking it by @j-lanson in #290
- added salsa integration to plugin core by @j-lanson in #298
- inital handling of plugin startup and context management by @j-lanson in #278
- plugin comms interface can handle multiple active sessions by @j-lanson in #297
- Integrated policy expressions code by @alilleybrinker in #265
- update dummy plugin to handle concurrent queries by @j-lanson in #307
- create plugin path alongside repo cache by @patrickjcasey in #301
- Update proto def to pass 'buf lint' by @alilleybrinker in #308
- dummy plugin refactored to use "Session" tracking so to overcome hanging by @j-lanson in #309
Some improvements to the website and to CI testing:
- Added README for dockerhub by @patrickjcasey in #255
- Fix website, announcement 3.5.0 by @alilleybrinker in #263
- CI improvements by @alilleybrinker in #264
- Always run tests on main to create reusable caches by @cstepanian in #279
- Fix broken containerfile by @alilleybrinker in #267
- Fix broken website deploy CI job by @alilleybrinker in #271
Refactoring of the Hipcheck source code:
- Moved test_util.rs to util/test.rs by @alilleybrinker
- Renamed util/hidden.rs to util/redacted.rs by @alilleybrinker
- Move 'http' module under 'util' module by @alilleybrinker in #270
- Use consistent module folder structure by @alilleybrinker in #310
- No more "module-inception" by @alilleybrinker in #311
- Remove unused version handling by @alilleybrinker in #312
- Create new 'init' top-level module by @alilleybrinker in #313
- Simplify .gitignore by @alilleybrinker in #314
Updates to our release infrastructure:
- Update cargo-dist to 0.21.0 by @mchernicoff in #291
And finally, many dependency version bumps:
- Bump serde from 1.0.204 to 1.0.206 by @dependabot[bot] in #276
- Bump clap from 4.5.11 to 4.5.13 by @dependabot[bot] in #275
- Bump syn from 2.0.72 to 2.0.74 by @dependabot[bot] in #273
- Bump tempfile from 3.11.0 to 3.12.0 by @dependabot[bot] in #272
- Bump cyclonedx-bom from 0.6.2 to 0.7.0 by @dependabot[bot] in #274
- Bump syn from 2.0.74 to 2.0.75 by @dependabot[bot] in #285
- Bump regex from 1.10.5 to 1.10.6 by @dependabot[bot] in #286
- Bump tokio from 1.39.2 to 1.39.3 by @dependabot[bot] in #287
- Bump indextree from 4.6.1 to 4.7.2 by @dependabot[bot] in #288
- Bump xml-rs from 0.8.20 to 0.8.21 by @dependabot[bot] in #289
- Bump quote from 1.0.36 to 1.0.37 by @dependabot[bot] in #302
- Bump which from 6.0.1 to 6.0.3 by @dependabot[bot] in #303
- Bump tabled from 0.15.0 to 0.16.0 by @dependabot[bot] in #304
- Bump indexmap from 2.2.6 to 2.4.0 by @dependabot[bot] in #305
- Bump clap from 4.5.15 to 4.5.16 by @dependabot[bot] in #306
- @patrickjcasey made their first contribution in #301
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.5.0...hipcheck-v3.6.0
3.5.0 - 2024-08-05
Hipcheck 3.5.0 continues our focus on user experience improvements with three major additions:
- Improved target selection
- The
hc cache
subcommand - Better progress reporting
In Hipcheck 3.5.0 we've enhanced the ability to specify your target of analysis. This is Hipcheck's term for "the thing you want to analyze." With this new version, you can now provide:
- Package URLs (pURLs): pURLs are a specification for a unified way of
referencing packages on a variety of popular package hosts. For example,
the pURL
pkg:npm/foobar@12.3.1
refers to the packagefoobar
version12.3.1
on NPM. - VCS URLs: VCS URLs come from the Python ecoystem, where they're used
by PIP to refer to Git repositories with additional metadata embedded in
the URL. They're also used by the SPDX Software Bill of Materials standard.
A VCS URL looks like
git+ssh://git@git.example.com/MyProject
, with the VCS being used as a prefix to the rest of the protocol, separated by a+
. - Git References: References are how Git refers to specific things you
can checkout in a Git repository, including specific commits, branches,
and tags. By default, Hipcheck checks out the latest (
HEAD
) commit of a repository, but you can now use the--ref
flag to specify a different ref to check out. - CycloneDX SBOMs: Hipcheck can now accept CycloneDX Software Bills of Material in addition to SPDX ones, in either JSON or XML format.
Note that some of the methods above allow embedding some target-type
information directly, in ways that can contradict the --type
flag when
running hc check
. If the --type
flag and the target specifier metadata
ever conflict, Hipcheck will produce an error rather than picking one option
over the other automatically.
The following commits were for this work:
- remove use of Source, SourceRepo, Remote structs in favor of Target by @j-lanson in #229
- added types for new target repo spec and resolution by @j-lanson in #210
- add more robust 'target resolution' systemD by @mchernicoff in #227
- add --ref support to remaining target seed kinds by @j-lanson in #244
- CLI can infer target type from target pURL by @mchernicoff in #205
- CLI can infer target type from target VCS URL by @mchernicoff in #209
- Hipcheck can process SBOMs using CycloneDX (currently JSON only) standard by @mchernicoff in #237
- Hipcheck can process SBOMs using CycloneDX XML files by @mchernicoff in #239
- update remote-repo updating to support git refs by @j-lanson in #240
- update local-repo copying to support git refs by @j-lanson in #242
- distinguish TargetSeed and TargetSeedKind to transfer ref info from cli by @j-lanson in #243
We've also added a new subcommand, hc cache
, for managing the repository
cache that Hipcheck maintains. When Hipcheck runs, it creates a local clone
of the repository being analyzed (if the repository is already local, Hipcheck
still clones it into the repository cache to avoid mutating the original). Over
time, this repository cache will grow, and some repositories may be quite
large. The new hc cache
subcommand lets the user see what's in the cache
with hc cache list
, and delete elements in the cache with hc cache delete
.
The following commits were for this work:
- add
hc cache
subcommand for manually listing/deleting elements of the cache by @j-lanson in #224 - add commit info to hc cache display and index file by @j-lanson in #245
- local repos are also cloned to HC_CACHE by @j-lanson in #208
- shortcut cache entry size calculation with cache index file by @j-lanson in #236
Hipcheck now shows a progress bar during execution, especially during repository cloning, which can be quite slow for larger repositories. This is intended to make it clearer how Hipcheck's time is being spent, and went it's continuing to make progress vs. being stuck.
The following commits were for this work:
- major refactor to enable better control of phases and progress bars by @vcfxb in #198
- Variety of tweaks to the shell and phase infrastructure by @vcfxb in #213
- Add progress bars when cloning repos (powered by the
git2
crate). Integratelibgit2
trace messages into our logging by @vcfxb in #222
As always, there are more changes in a single version that can be adequately described here. View the full changelog to see the rest:
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.4.0...hipcheck-v3.5.0
3.4.0 - 2024-07-04
Hipcheck 3.4.0 is an exciting release featuring 3 new subcommands!
hc setup
: When you install thehc
binary, whether through an install script with a release, withcargo binstall
, or by building it yourself, you still need to get the configuration and data files Hipcheck requires. This new command gets those files for you, so you can start using Hipcheck quickly and easily!hc update
: This lets Hipcheck update itself to newer versions! Under the hood, we're using the self-updater built and provided bycargo-dist
, the tool we use for cutting new releases with prebuilt binaries. Thehc update
command is a wrapper around that updater. This command does require that you use our install script to get the self-updater.hc scoring
: This tells you how Hipcheck is scoring results based on your current configuration. While you can see the weights for each analysis in your configuration file, it can be tedious to do the math yourself to find out exactly how much each analysis contributes to the overall score. This command does that math for you to make it easier.
Hipcheck also now has an official website! Feel free to check it out, and let us know if you encounter any issues with it.
- Add new
hc setup
command:- Add
hc setup
installation command by @j-lanson - Slightly modified arguments for
hc setup
by @alilleybrinker in #155
- Add
- Add new
hc update
command:- Adds
hc update
command to run the self-updater (if installed) to automatically bring Hipcheck up to date with a released version. Includes optional flags to update to versions other than the latest by @mchernicoff in #176
- Adds
- Add new
hc scoring
command:
- Refactor analyses and scoring to prepare for future migration to a plugin system:
- Add weight tree and replace old scoring by @j-lanson in #160
- Replace prior AnalysisResults obj with AltAnalysisResults impl by @j-lanson in #159
- Demonstrate use of hashmap with new result objects for analysis storage by @j-lanson in #130
- refactor affiliation analysis, experiment with macro by @j-lanson in #154
- refactor all remaining repo analyses by @j-lanson in #157
- Improve Hipcheck performance:
- Parallelize grapheme collection across files by @alilleybrinker in #146
- Crate features for benchmarking and new
Drop
based benchmarking struct by @vcfxb in #136
- Clean out dependencies we no longer need:
- Improve Hipcheck container image:
- Use Debian base image instead of Alpine by @cstepanian in #180
- Clean apt cache after installation by @cstepanian in #181
- Keep dependencies up-to-date:
- Bump lazy_static from 1.4.0 to 1.5.0 by @dependabot[bot] in #142
- Bump proc-macro2 from 1.0.85 to 1.0.86 by @dependabot[bot] in #141
- Bump syn from 2.0.66 to 2.0.68 by @dependabot[bot] in #140
- Bump dashmap from 5.5.3 to 6.0.1 by @dependabot[bot] in #164
- Bump log from 0.4.21 to 0.4.22 by @dependabot[bot] in #165
- Bump serde_json from 1.0.117 to 1.0.119 by @dependabot[bot] in #163
- Bump ordered-float from 4.2.0 to 4.2.1 by @dependabot[bot] in #166
- Bump clap from 4.5.7 to 4.5.8 by @dependabot[bot] in #162
- Fix broken Docker build:
- Remove legacy CLI features that aren't actually implemented or supported:
- Removes patch type as option for hc check or hc schema, as that was unsupported by @mchernicoff in #153
- Remove pr analysis and scoring infrastructure by @j-lanson in #158
- @cstepanian made their first contribution in #181
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.3.2...hipcheck-v3.4.0
3.3.2 - 2024-06-21
This patch is intended to workaround a bug in cargo-dist
which caused the
publication of 3.3.1 to fail to include all the expected artifacts. They're
working on fixing the bug, but in the short term this is a reasonable
workaround.
- Stop renaming
target
directory by @alilleybrinker
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.3.1...hipcheck-v3.3.2
3.3.1 - 2024-06-21
This patch release includes two general categories of fixes:
- Getting
Containerfile
builds on new releases working. - Getting
cargo-dist
generation of binaries on new releases working.
- Fix broken Docker Hub action by @alilleybrinker
- Update Containerfile to be accepted on Docker Hub push by @alilleybrinker
- Upgrade
cargo-dist
to 0.16.0 by @alilleybrinker in #137
- Fix broken Containerfile syntax by @alilleybrinker
- Get Containerfile working by @alilleybrinker
- Reduce prebuild targets to ones that work by @alilleybrinker
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.3.0...hipcheck-v3.3.1
3.3.0 - 2024-06-20
Hipcheck version 3.3.0 is mostly focused on refactors and internal
improvements, including a substantial refactor of the hc
Command Line
Interface to be easier to use and easier for us to enhance in the future.
We've also continued to mature our tooling and processes around Hipcheck,
which should hopefully make future advancement easier.
- Add "Hipcheck's Values" RFD by @alilleybrinker in #70
- Added RFD #3, "Plugin Architecture Vision" by @alilleybrinker in #71
- Change
ureq
Agent to use native system certs by @mchernicoff in #85 - Remove OpenSSL as a Hipcheck dependency by @mchernicoff in #80
- Added new types to form the basis of scoring refactor by @j-lanson in #127
- Change hc CLI to use derive instead of builder (WIP) by @mchernicoff
- Small fixes; still panics with no argument by @mchernicoff
- Refactor CLI by @alilleybrinker in #93
- Print help on empty args to
check
andschema
by @j-lanson in #107 - Move error/context to appropriate sub-modules by @j-lanson in #115
- Move
metric
andsession
out ofanalysis
by @mchernicoff in #116 - Move
source
out ofdata
by @mchernicoff in #117 - Creates general
http
module for making requests by @mchernicoff in #118 - Refactor
hc check
CLI by @j-lanson - Initial work on performance by @vcfxb in #131
- Improve performance of grapheme frequency calculation by @vcfxb in #133
- Adds hc ready command by @mchernicoff in #81
- Restore
libc
version to 0.2.153 to match latest version on crates.io by @mchernicoff - Restore
libc
version to 0.2.153 to match latest version on crates.io by @mchernicoff - Make top-level commands for
hc
Option
s to allow for no command by @mchernicoff - Removes unnecessary
use
incli.rs
by @mchernicoff - Remove unnecessary
Default
implementation forhc help
by @mchernicoff - Adds test for CLI commands by @mchernicoff
- Disable built-in
help
command for allhc
commands by @mchernicoff - Rename help flag internally to pass tests by @mchernicoff
- Fix mishandling of
HC_CONFIG
with new CLI by @j-lanson in #114
- Filter GitHub workflow to not run tests if changes to a push or pull-request are outside of code folders by @mchernicoff in #68
- Add "Dependency Tree" task to CI by @alilleybrinker in #79
- Publish tagged HC releases to Dockerhub by @j-lanson in #113
- Add ability to manually exec push-to-dockerhub action by @j-lanson in #119
xtask
is our internal development tooling.
- Add license and description
xtask/src/task/rfd.rs
by @mchernicoff in #90 - Add
xtask
changelog sanity check forgit-cliff
by @j-lanson in #92 - Change
xtask validate
toxtask check
whenxtask ci
is called by @mchernicoff in #89
cargo release
updates Hipcheck version in README by @mchernicoff in #111- Make
cargo-dist
releases includeconfig/
andscripts/
by @alilleybrinker in #135 - Removes missing
/libs
folder from Container file by @mchernicoff in #72
- Bump anyhow from 1.0.83 to 1.0.86 by @dependabot[bot] in #76
- Bump clap from 4.5.6 to 4.5.7 by @dependabot[bot]
- Bump clap from 4.5.4 to 4.5.6 by @dependabot[bot] in #122
- Bump libc from 0.2.154 to 0.2.155 by @dependabot[bot] in #74
- Bump proc-macro2 from 1.0.84 to 1.0.85 by @dependabot[bot] in #109
- Bump regex from 1.10.4 to 1.10.5 by @dependabot[bot] in #121
- Bump schemars from 0.8.19 to 0.8.20 by @dependabot[bot] in #78
- Bump schemars from 0.8.20 to 0.8.21 by @dependabot[bot] in #83
- Bump serde from 1.0.201 to 1.0.202 by @dependabot[bot] in #75
- Bump serde from 1.0.202 to 1.0.203 by @dependabot[bot] in #82
- Bump toml from 0.8.12 to 0.8.13 by @dependabot[bot] in #77
- Bump toml from 0.8.13 to 0.8.14 by @dependabot[bot] in #123
- Bump url from 2.5.0 to 2.5.1 by @dependabot[bot]
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.2.1...hipcheck-v3.3.0
3.2.1 - 2024-05-10
Nothing really new in Hipcheck itself. Publishing this version mostly to work out issues with the machinery for publishing new releases and distributing prebuilt binaries.
- add
xtask changelog
command by @alilleybrinker in #63
- Improved
Cargo.toml
metadata, removed unused deps by @alilleybrinker in #61 - Improved
xtask
experience, removed old commands by @alilleybrinker in #62
- Removed
publish = false
on Hipcheck by @alilleybrinker - Add missing crate description for Hipcheck by @alilleybrinker
- Corrected bad metadata in Hipcheck crate by @alilleybrinker
- Fix broken
cargo-dist
build by @alilleybrinker in #60
Full Changelog: https://github.com/mitre/hipcheck/compare/hipcheck-v3.2.0...hipcheck-v3.3.0
3.2.0 - 2024-05-09
This is the first new version of Hipcheck since our initial open source release, and it represents a lot of housekeeping to get the project up and running! That includes:
- Getting Hipcheck compiling cleanly on the latest stable version of Rust.
- Getting all of Hipcheck's dependencies up to date.
- Shrinking Hipcheck's crate structure down to just a single binary crate.
In addition, we worked on a lot of best-practice related items, including:
- Defining RFD's (Requests for Discussion) as our means of managing the evolution of Hipcheck over time.
- Setting up a DevContainer configuration, for folks who'd like to contribute to Hipcheck without needing to set up their local environment by hand.
- Establishing Continuous Integration testing, to increase confidence in the correctness of future changes we may merge.
- Defining a security policy, a code of conduct, and a guide for potential contributors, so people know how to interact with the project.
- Defining our "Release Engineering" practices, which will help smooth out the flow of future releases of Hipcheck.
Up next we're planning to work on more serious redesigning of Hipcheck's architecture to support third-party plugins for data and analysis. If that's something that appeals to you, please let us know in the Discussions page!
Here's to the first of many more releases!
- Run rustfmt to fix formatting by @alilleybrinker in #20
- Introduce RFD process by @alilleybrinker in #25
- Define devcontainer config by @alilleybrinker in #26
- Enable dependabot version bumps by @alilleybrinker in #32
- Bump
ureq
from 2.9.6 to 2.9.7 by @dependabot[bot] in #35 - Bump
schemars
from 0.8.16 to 0.8.17 by @dependabot[bot] in #34 - Add Conventional Commit check to CI by @alilleybrinker in #36
- Added basic CI testing by @alilleybrinker
- Move common-use crates into
hc_common
by @mchernicoff in #37 - Move data type and retrieval crates into
hc_data
by @mchernicoff in #39 - Merges support crates for
hc_data
intohc_data
by @mchernicoff in #40 - Merge crates into
hc_metric
by @mchernicoff in #43 - Move
hc_pm
intohc_session
by @mchernicoff in #44 - Creates a single analysis crate that handles most of the Hipcheck analysis pipeline by @mchernicoff in #45
- Bump
libc
from 0.2.153 to 0.2.154 by @dependabot[bot] in #46 - Complete unifying Hipcheck in single crate by @alilleybrinker in #47
- Removed dead code by @alilleybrinker in #50
- Added "Release Engineering" RFD by @alilleybrinker in #48
- Add
cargo-dist
for prebuilt binaries by @alilleybrinker in #41 - Removed
pathbuf
module in favor of crate by @alilleybrinker - Organize helper modules under
util/
by @alilleybrinker - Added basic community docs by @alilleybrinker in #54
- Adds security policy by @mchernicoff in #59
- Resolve Cargo warnings by @alilleybrinker
- Move dependabot config back to
.github
folder by @alilleybrinker in #38 - Remove
atty
dep for GHSA-g98v-hv3f-hcfr by @alilleybrinker in #42 - Add missing license notices by @alilleybrinker in #52
- Fix double-version command in
xtask
by @alilleybrinker in #51 - Get
cargo xtask doc --open
working again by @alilleybrinker in #53
- @mchernicoff made their first contribution in #59
- @dependabot[bot] made their first contribution in #46