Skip to content

Releases: redhat-best-practices-for-k8s/certsuite

v4.5.2

26 Oct 20:25
@sebrandon1 sebrandon1
v4.5.2
9ae3271
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.5.2

v4.5.2 is a smaller update with a bunch of dependency updates and CI improvements. Nothing functional as far as test cases are concerned.

Dependency Updates

CNF Suite Operator Work

  • Allow CNF Cert Suite app's go-clients to work in a k8s pod. by @greyerof in #1543
  • Added yaml template to deploy the CNF Cert Suite in a kubernetes/Openshift cluster. by @greyerof in #1557

CI Improvements

QE Coverage Update

  • QE Coverage : Mark dpdk cpu pinning exec probe QE implemented by @bnshr in #1466

Full Changelog: v4.5.1...v4.5.2

Contributors

sebrandon1, dependabot, and 3 other contributors
Assets 2
Loading

v4.5.1

19 Oct 13:38
@sebrandon1 sebrandon1
v4.5.1
3ffc497
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.5.1

New to v4.5.1 is the TNF Config Generator introduced in #1506. This allows users to build a complete TNF config file without having to touch YAML.

There has been a bunch of documentation written about this tool as well so be sure to check that out here.

TNF Config Updates

  • config: add info and a quick demo of the CNF Config Generator by @jmontesi in #1520
  • config: allow interative svg image for the CNF Config Generator demo by @jmontesi in #1526
  • config: fix path for the CNF Config Generator demo image by @jmontesi in #1527
  • cmd/tnf: new tool to generate the CNF config file with user input by @jmontesi in #1506
  • config: delete unused struct definitions by @jmontesi in #1514
  • config: add two more options to the CNF config generator by @jmontesi in #1523
  • config: update documentation by @jmontesi in #1525

Test Improvements

  • check for local storage provisioners for SNO clusters by @edcdavid in #1507
  • Making the lifecycle-pod-scheduling test optional in telco scenarios by @edcdavid in #1521

Claim Compare Tool

  • Official documentation for the claim compare tool. by @greyerof in #1533

Dependency Updates

CI Improvements

Full Changelog: v4.5.0...v4.5.1

Contributors

fredericlepied, sebrandon1, and 5 other contributors
Assets 2
Loading

v4.5.0

03 Oct 15:19
@sebrandon1 sebrandon1
v4.5.0
6be43a6
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.5.0

v4.5.0

There are non-backwards compatible changes as part of this release, hence the reason for moving from v4.4.x to v4.5.x. See the section below about the TNF Config Changes.

In addition to the config changes there has been functionality removed surrounding the container certification status test that was a piece of older functionality that was rarely used. The way it used to check images outside of the labeling scheme, relying on the TNF config to house the image names is not the way we wanted users to interact with the test suite.

TNF Config Changes

New Additions

  • New tool to verify the certification status of container images by @jmontesi in #1450
  • Comparison tool: compare node's labels and annotations. by @greyerof in #1444
  • Claim comparison tool: compare versions. by @greyerof in #1480

Fixes and Improvements

Report Objects

Certified Container Changes

  • Fix: run container-is-certified also in the case of digest only images by @edcdavid in #1439
  • Remove skips for testContainerCertificationStatus by @sebrandon1 in #1462
  • Fix container status by @edcdavid in #1477
  • check for all possible states instead of matching container name with container name in status field by @edcdavid in #1464
  • add check for "manifest_list_digest" when looking for certified containers by digest by @edcdavid in #1489
  • adding digest in the failure reason of of the affiliated-certification-container-is-certified-digest test by @edcdavid in #1473
  • onlinecheck: delete unused function by @jmontesi in #1471

Others

CI Improvements

Dependency Updates

Full Changelog: v4.4.0...v4.4.1

Contributors

rdavid, sebrandon1, and 6 other contributors
Assets 2
Loading

v4.4.0

15 Sep 15:35
@sebrandon1 sebrandon1
v4.4.0
ec84f03
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.4.0

v4.4.0

This minor release of the TNF suite contains a removed test, a couple of bug fixes reported by users, and some TNF config updates as the most notable changes.

In upcoming releases, we will be merging non-backwards compatible changes to the TNF config which might prompt another minor release due to the incompatibility.

Removed Tests

Bug Fixes

TNF Config Updates

  • config: remove unused CertifiedOperatorInfo field by @jmontesi in #1416
  • config: delete helper fields from TestConfiguration by @jmontesi in #1405
  • Update tnf_config.yml file with the correct fields for certifiedcontainerinfo by @ramperher in #1430

Report Objects

  • Add report objects for testContainerCertificationStatus by @sebrandon1 in #1418

Catalog Updates

Claim File Comparison

  • Traverse and compare json nodes in agnostic mode. by @greyerof in #1386

Dependency Updates

CI Improvements

QE Updates

  • Mark security-context as QE test covered by @bnshr in #1382

New Contributors

Full Changelog: v4.3.5...v4.4.0

Contributors

fredericlepied, sebrandon1, and 6 other contributors
Assets 2
Loading

v4.3.5

01 Sep 17:25
@sebrandon1 sebrandon1
v4.3.5
5da38c9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.3.5

Bug Fixes

  • operator: fix the process to check if an operator is ready by @jmontesi in #1337
  • Update message for tainted kernel allowlist being empty by @sebrandon1 in #1356
  • Bug fix: consider the case PDB defined but no label match by @jmontesi in #1374

Improvements

  • Add tnf.Log() function to allow log level selection by @jmontesi in #1353
  • Make use of tnf.Logf() where applicable by @jmontesi in #1357
  • Add non compliant object for internal error when getting PID namespace by @edcdavid in #1355
  • Add a Makefile rule to build the CNF binary for local development by @jmontesi in #1360
  • Update 'compatibility' matrix by @sebrandon1 in #1366
  • Updated diagnostics info format dumped to claim. by @greyerof in #1368

New Tests

access-control

  • access-control: add detection of eBPF filter loading by @jmontesi in #1364

performance

  • Add an optional Far Edge test case to ensure limited use of exec probes by @shimritproj in #1298

platform-alteration

Dependency Updates

Collector Features

CI Updates

QE Updates

New Contributors

Full Changelog: v4.3.4...v4.3.5

Contributors

sebrandon1, theemj, and 8 other contributors
Assets 2
Loading

v4.3.4

11 Aug 17:30
@sebrandon1 sebrandon1
v4.3.4
2037b43
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.3.4

v4.3.4

This release contains the following notable changes:

  • #1324 removes the '-f' focus strings from running the test suites that was previously deprecated.
  • #1305 and #1313 address 'short' image names (from Dockerhub) when being parsed.

Improvements

Bug Fixes

  • Fix regression for short image names by @edcdavid in #1313
  • Fix for parsing the image ID and image by @edcdavid in #1305
  • lifecycle: fix bug preventing dumping test info after a failure by @jmontesi in #1316
  • operator: fix failure reasons in a couple of test cases by @jmontesi in #1325

Dependency Updates

  • Update preflight to v1.6.11 by @sebrandon1 in #1300
  • Bump github.com/operator-framework/api from 0.17.6 to 0.17.7 by @dependabot in #1301
  • Update RHCOS to OCP version map by @github-actions in #1302
  • Update RHCOS to OCP version map by @github-actions in #1308
  • Update RHCOS to OCP version map by @github-actions in #1314
  • Bump github.com/deckarep/golang-set/v2 from 2.3.0 to 2.3.1 by @dependabot in #1311
  • Update Go to v1.20.7 by @sebrandon1 in #1315
  • Update RHCOS to OCP version map by @github-actions in #1317
  • Update RHCOS to OCP version map by @github-actions in #1321
  • Bump ubi8/ubi-minimal from 8.8-860 to 8.8-1037 by @dependabot in #1326
  • Bump github.com/test-network-function/test-network-function-claim from 1.0.24 to 1.0.25 by @dependabot in #1329
  • Bump ubi8/ubi from 8.8-854 to 8.8-1032 by @dependabot in #1327
  • Update RHCOS to OCP version map by @github-actions in #1331
  • Bump github.com/test-network-function/privileged-daemonset from 1.0.7 to 1.0.8 by @dependabot in #1328
  • Update RHCOS to OCP version map by @github-actions in #1334
  • Bump helm.sh/helm/v3 from 3.12.2 to 3.12.3 by @dependabot in #1335

CI Updates

QE Updates

Full Changelog: v4.3.3...v4.3.4

Contributors

rdavid, sebrandon1, and 5 other contributors
Assets 2
Loading

v4.3.3

01 Aug 15:30
@sebrandon1 sebrandon1
v4.3.3
e4a2593
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.3.3

v4.3.3

Notable changes

  • New claim format version is v0.1.0. The new format bundles the catalog and classification information within the claim file, simplifying parsing of results.

  • The HTML parser is now its own project at https://github.com/test-network-function/parser. The claim format version allows checking for compatibility between the claim file and any of the tools able to decode it. Meaning of the claim format version an its relationship with the parser release version is describe at here.

Test Suite Improvements

Claim File Comparison Tool Updates

  • Added claim file version check to tnf claim show failures subcommand. by @greyerof in #1258
  • Fix for tnf claim show failures by @greyerof in #1267
  • Claim comparison tool improvement: results table. by @greyerof in #1283
  • Adding catalog and classification items to the claim file by @edcdavid in #1280
  • dumps claim file as CSV with extra classification information by @edcdavid in #1260

CI Improvements

  • Github CI: clean up unnamed/dangling container images to save disk space. by @greyerof in #1273
  • Github CI: Remove oct container image and the go mod cache to save disk space. by @greyerof in #1282
  • Fix leftovers and CI uploads by @edcdavid in #1296

QE Updates

  • 85% QE coverage : with storage-required-pods + helm-chart-certified by @bnshr in #1291

Dependency Updates

Full Changelog: v4.3.2...v4.3.3

Contributors

sebrandon1, ramperher, and 4 other contributors
Assets 2
Loading

v4.3.2

14 Jul 15:13
@sebrandon1 sebrandon1
v4.3.2
b45d444
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.3.2

v4.3.2 is a smaller update with some notable improvements:

  • The HTML parser is able to now flag when a claim file is out of date with the parser version. See #1241.
  • The access-control-ssh-daemons test was improved in #1219.
  • The HTML parser added ability to pass a claim file via URL in #1243.
  • The CLI tool for comparing claim files was improved in #1249.

Test Case Updates

HTML Parser Updates

  • A new line at the end by @rdavid in #1237
  • Fix for HTML js: avoid crashing when ObjectFieldsKeys or ObjectFields values are null. by @greyerof in #1233
  • Add option to pass claim file via URL and bugfix by @edcdavid in #1243
  • Check claim file format version before attempting rendering results by @edcdavid in #1241

Dependency Updates

  • Update RHCOS to OCP version map by @github-actions in #1239
  • Update Go to v1.20.6 by @sebrandon1 in #1242
  • Update RHCOS to OCP version map by @github-actions in #1244
  • Bump github.com/test-network-function/test-network-function-claim from 1.0.21 to 1.0.22 by @dependabot in #1246
  • Bump github.com/test-network-function/privileged-daemonset from 1.0.5 to 1.0.6 by @dependabot in #1247
  • Update RHCOS to OCP version map by @github-actions in #1248
  • Update RHCOS to OCP version map by @github-actions in #1251

QE Updates

  • Set the undeclared-container-ports-usage test case as QE covered by @jmontesi in #1240
  • Mark ssh-daemons as QE test covered by @bnshr in #1245

Quality of Life Updates

Full Changelog: v4.3.1...v4.3.2

Contributors

rdavid, sebrandon1, and 5 other contributors
Assets 2
Loading

v4.3.1

10 Jul 19:09
@sebrandon1 sebrandon1
v4.3.1
e6f6382
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.3.1

The v4.3.1 release of the CNF Certification Test Suite contains a number of changes all sorted below.

The most notable changes are:

  • Using the all flag when specifying which test suites to run. Users will no longer have to specify certain suites (common,extended,faredge,telco) to run all tests. See #1176.
  • By default now all results files will be placed into a tar.gz bundle after each test run. See #1200 for more details.
  • You can now supply a pre-defined feedback.json to the HTML parser to help with auto-filling response text boxes. See #1213.
  • The CNF Best Practices repo now links directly to-and-from the CATALOG.md. See #1185.

Catalog Improvements

Results Parser Changes

Tests Added

Tests Updated / Improved

  • check that helm version is v3 by @aabughosh in #1103
  • Adding error text to the abort suite message by @edcdavid in #1163
  • CNFCERT-495 : performance refactor code by @bnshr in #1161
  • Adjust the PDB test by @sebrandon1 in #1155
  • Add a single label for running ALL tests instead of having to choose individual labels by @aabughosh in #1176
  • Fix checking semver version against contraint version by @edcdavid in #1195
  • Fix empty service account check by @bnshr in #1197
  • Fix network policy tests with empty podSelector by @javierpena in #1147
  • Fix for hardcoded registry registry.redhat.io by @edcdavid in #1187
  • Add emptystring test case for getIPVersion() by @sebrandon1 in #1211
  • CNFCERT-541 : Make Pod Role Bindings fail with default service account by @bnshr in #1210
  • Fix for the unstable test cases of the performance test suite. by @greyerof in #1226

Failure-Reason Objects

To take better advantage of our new compliant/non-compliant reporting objects in the HTML parser, we have added support for the reporting objects in all of the existing tests.

CI Improvements

Dependency Updates

  • Bump github.com/stretchr/testify from 1.8.3 to 1.8.4 by @dependabot in #1141
  • Update preflight to v1.6.4 by @sebrandon1
  • Bump github.com/onsi/ginkgo/v2 from 2.9.5 to 2.9.7 by @dependabot in #1146
  • Bump github.com/operator-framework/api from 0.17.5 to 0.17.6 by @dependabot in #1148
  • Update operator-sdk to v1.29.0 by @sebrandon1 in #1149
  • Update RHCOS to OCP version map by @github-actions in #1151
  • Update RHCOS to OCP version map by @github-actions in #1153
  • Bump github.com/sirupsen/logrus from 1.9.2 to 1.9.3 by @dependabot in #1154
  • Update GolangCI to 1.53.2 by @sebrandon1 in #1156
  • Update RHCOS to OCP version map by @github-actions in #1158
  • Update preflight to v1.6.5 by @sebrandon1 in #1157
  • Bump github.com/onsi/ginkgo/v2 from 2.9.7 to 2.10.0 by @dependabot in #1159
  • Update RHCOS to OCP version map by @github-actions in #1160
  • Update Go to v1.20.5 by @sebrandon1 in #1165
  • Bump github.com/test-network-function/test-network-function-claim from 1.0.19 to 1.0.20 by @dependabot in #1166
  • Bump github.com/test-network-function/privileged-daemonset from 1.0.3 to 1.0.4 by @dependabot in #1167
  • Update RHCOS to OCP version map by @github-actions in #1168
  • Update preflight to v1.6.6 by @sebrandon1 in #1170
  • Update RHCOS to OCP version map by @github-actions in #1174
  • Bump k8s.io/kubectl from 0.27.2 to 0.27.3 by @dependabot in #1179
  • Bump k8s.io/apiextensions-apiserver from 0.27.2 to 0.27.3 by @dependabot in #1181
  • Bump helm.sh/helm/v3 from 3.12.0 to 3.12.1 by @dependabot in #1183
  • Update RHCOS to OCP version map by @github-actions in #1188
  • Bump github.com/onsi/ginkgo/v2 from 2.10.0 to 2.11.0 by @dependabot in #1190
  • Bump github.com/test-network-function/privileged-daemonset from 1.0.4 to 1.0.5 by @dependabot in #1192
  • Bump github.com/test-network-function/test-network-function-claim from 1.0.20 to 1.0.21 by @dependabot in #1193
  • Update RHCOS to OCP version map by @github-actions in #1196
  • Update preflight to v1.6.7 by @sebrandon1 in #1194
  • Update RHCOS to OCP version map by @github-actions in #1203
  • Update preflight to v1.6.9 by @sebrandon1 in #1204
  • Update RHCOS to OCP version map by @github-actions in #1208
  • Update RHCOS to OCP version map by @github-actions in #1235
  • Update preflight to v1.6.10 by @sebrandon1 in https://github.com/test-network-function/cnf...
Read more

Contributors

rdavid, sebrandon1, and 8 other contributors
Assets 2
Loading

v4.3.0 - New projected volume test, nodeport test change, debug daemonset tagging

30 May 16:44
@sebrandon1 sebrandon1
v4.3.0
988e218
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.3.0 - New projected volume test, nodeport test change, debug daemonset tagging

The v4.3.0 release of the TNF test suite is a minor version bump because we are:

  • Adding a new test, see the projected volumes automount token test from #1046.
  • We have moved the NodePort related test from the networking suite to the access-control suite to better align the purpose of this test along with a name change.
  • This is the first release we have done where we are pinning the version of the debug-daemonset used to match the release version of the TNF suite (4.3.0). Using latest proved to invite failures in older test suite versions.

Bug Fixes

Catalog Updates

Results Parser Changes

Tests Added

  • Accessing kube-api via projected volume bypasses automountToken test by @shimritproj in #1046

Tests Updated / Improved

CI Improvements

Maintenance

Kubernetes Dependency Updates

UBI Base Image Changes

Deprecated

No tests have been deprecated with this release.

Full Changelog: v4.2.4...v4.3.0

Contributors

rdavid, sebrandon1, and 5 other contributors
Assets 2
Loading