v5.0.4

Bug Fixes and Improvements

• autodiscover: modify Istio Service Mesh detection by @jmontesi in #1896
• Only return error when an actual error occurs during a CNF run by @jmontesi in #1901
• Bumping the parser version to v0.4.4 to fix SRI issue (javascript) by @edcdavid in #1905

CI Updates

• Run all the tests in the GitHub CI by @jmontesi in #1889

Dependency Updates

• Update Go to v1.22 by @sebrandon1 in #1902
• build(deps): bump github.com/mittwald/go-helm-client from 0.12.7 to 0.12.8 by @dependabot in #1887
• build(deps): bump github.com/test-network-function/oct from 0.0.5 to 0.0.6 by @dependabot in #1891
• Update GolangCI-lint to v1.56.2 by @sebrandon1 in #1892
• build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 by @dependabot in #1894
• build(deps): bump github.com/test-network-function/test-network-function-claim from 1.0.34 to 1.0.35 by @dependabot in #1899
• build(deps): bump golang.org/x/term from 0.17.0 to 0.18.0 by @dependabot in #1900
• Update preflight to v1.9.1 by @sebrandon1 in #1898
• Update operator-sdk to v1.34.0 by @sebrandon1 in #1897
• build(deps): bump ubi9/ubi from 9.3-1552 to 9.3-1610 by @dependabot in #1903
• build(deps): bump ubi9/ubi-minimal from 9.3-1552 to 9.3-1612 by @dependabot in #1904

Full Changelog: v5.0.3...v5.0.4

v5.0.3

v5.0.3 is a patch level update with dependency updates and CI improvements.

Dependency Updates

• build(deps): bump golang.org/x/term from 0.16.0 to 0.17.0 by @dependabot in #1865
• Update Go to v1.21.7 by @sebrandon1 in #1873
• build(deps): bump k8s.io/kubectl from 0.29.1 to 0.29.2 by @dependabot in #1880
• build(deps): bump helm.sh/helm/v3 from 3.14.0 to 3.14.1 by @dependabot in #1875
• build(deps): bump github.com/k8snetworkplumbingwg/network-attachment-definition-client from 1.5.0 to 1.6.0 by @dependabot in #1877
• build(deps): bump k8s.io/apiextensions-apiserver from 0.29.1 to 0.29.2 by @dependabot in #1881
• build(deps): bump helm.sh/helm/v3 from 3.14.1 to 3.14.2 by @dependabot in #1884
• build(deps): bump github.com/test-network-function/privileged-daemonset from 1.0.20 to 1.0.21 by @dependabot in #1885

CI Improvements

• Github ci: fixed example label for operators autodiscovery. by @greyerof in #1869
• Add new tnf command to check the test results by @jmontesi in #1864
• build(deps): bump golangci/golangci-lint-action from 3 to 4 by @dependabot in #1867
• Add the expected results template for the GitHub CI by @jmontesi in #1870
• workflows: use the tnf check command instead of the gradetool by @jmontesi in #1872
• Delete gradetool leftovers by @jmontesi in #1874

Full Changelog: v5.0.2...v5.0.3

v5.0.2

v5.0.2 includes a number of bug fixes and improvements to the test suite.

The ICMP tests were previously not specifying what interface they would be pinging over so that is corrected.

The tainted kernel test was improved to only test workers that have CNF workloads deployed on them.

The base image test was improved to use podman that was actually installed on the node rather than the compiled podman from the debug container image.

Bug Fixes

• Fix for platform-alteration-base-image test case. by @greyerof in #1861
• Specify network interface for ICMP tests by @sebrandon1 in #1853
• Add filter for tainted node test by @sebrandon1 in #1847

Improvements

• checksdb: turn the labels evaluator into a global variable by @jmontesi in #1834
• Delete empty files by @jmontesi in #1845
• checksdb: delete unused field by @jmontesi in #1828
• flags: allow filtered checks listing by @jmontesi in #1827
• Fix copyright years by @rdavid in #1841
• Propagate errors from preflight in the claim file by @edcdavid in #1684
• certsuite: process flags refactor by @jmontesi in #1848

Dependency Updates

• build(deps): bump github.com/test-network-function/oct from 0.0.4 to 0.0.5 by @dependabot in #1830
• build(deps): bump github.com/test-network-function/test-network-function-claim from 1.0.33 to 1.0.34 by @dependabot in #1831
• Update preflight to v1.8.1 by @sebrandon1 in #1854
• build(deps): bump github.com/operator-framework/api from 0.21.0 to 0.22.0 by @dependabot in #1859

Image Updates

• Use UBI python image by @rdavid in #1832
• build(deps): bump ubi9/ubi-minimal from 9.3-1475 to 9.3-1552 by @dependabot in #1839
• build(deps): bump ubi9/ubi from 9.3-1476 to 9.3-1552 by @dependabot in #1840

TNF Config Updates

• set hard coded collector app endpoint by @shirmoran in #1836

Unit Tests

• Add unit tests for testhelper package by @sebrandon1 in #1835

Scripting Improvements

• Set executable mode initially to the script by @rdavid in #1833
• Simplify operations involving variables by @rdavid in #1849
• Add a new line to results.html by @rdavid in #1852

CI Updates

• Pass Github run ID into QE for collector by @sebrandon1 in #1862

• Github CI: Step to use /mnt for docker storage. by @greyerof in #1857

• Add functionality for running against local CRC instances by @sebrandon1 in #1843

• Fix cluster bootstrapping step. by @greyerof in #1837

• build(deps): bump nick-fields/retry from 2 to 3 by @dependabot in #1850

• build(deps): bump peter-evans/create-pull-request from 5 to 6 by @dependabot in #1851

• adjust collector's ci test to send claim file to prod collector by @shirmoran in #1829

• enable changing cnf suites timeout using env var by @shirmoran in #1846

• Prepare for v5.0.2 by @sebrandon1 in #1863

Full Changelog: v5.0.1...v5.0.2

v5.0.1

v5.0.1 is a patch-level update to bring improved resource allocation for the daemonset the test suite spawns as part of testing.

There are also a number of dependency updates.

Daemonset Resource Usage Improvements

• Adding configurable cpu requests/limit for the debug daemonset by @edcdavid in #1805

Logging Improvements

• log: review logs for autodiscover and provider packages by @jmontesi in #1811

CI Updates

• Add retries for 'make build-image-local' by @sebrandon1 in #1812
• Re-enable gradetool by @sebrandon1 in #1810
• Remove free disk space cleanup by @sebrandon1 in #1821

Dependency Updates

• Update preflight to v1.8.0 by @sebrandon1 in #1809
• build(deps): bump helm.sh/helm/v3 from 3.13.3 to 3.14.0 by @dependabot in #1816
• build(deps): bump k8s.io/kubectl from 0.29.0 to 0.29.1 by @dependabot in #1813
• build(deps): bump ubi8/ubi from 8.9-1107 to 8.9-1107.1705420509 by @dependabot in #1818
• build(deps): bump k8s.io/apiextensions-apiserver from 0.29.0 to 0.29.1 by @dependabot in #1817
• build(deps): bump ubi8/ubi-minimal from 8.9-1108 to 8.9-1108.1705420507 by @dependabot in #1819
• build(deps): bump github.com/test-network-function/privileged-daemonset from 1.0.19 to 1.0.20 by @dependabot in #1822
• build(deps): bump github.com/k8snetworkplumbingwg/network-attachment-definition-client from 1.4.0 to 1.5.0 by @dependabot in #1823
• Update images to ubi9/ubi9-minimal by @sebrandon1 in #1825

Full Changelog: v5.0.0...v5.0.1

v5.0.0

v5.0.0 - No More Ginkgo

Our first release of 2024 is a big one. We are releasing our first v5.x.x release, now without the ginkgo dependency. A lot of work has gone into removing this underlying dependency from the test suite. As far as the end-user knows, everything should act very similar to the previous v4.x.x releases.

The CLI has undergone a good quality-of-life update as well which now shows each test and its corresponding Passed/Failed/Skipped state in realtime.

Bug Fixes

• Fix for the webserver kubeconfig file handling. by @greyerof in #1625
• Graceful exit when no kubeconfig is configured by @edcdavid in #1668
• Revert "Only capture ctrl-c while tests are running" by @edcdavid in #1693
• Only capture ctrl-c while tests are running by @edcdavid in #1694
• Only capture ctrl-c while tests are running by @edcdavid in #1699
• Move flags processing to run function by @edcdavid in #1696
• The find command requires a directory as the first argument by @rdavid in #1709
• Add operator versions by @rdavid in #1708
• Rename poststart test by @edcdavid in #1698
• Fix for fatal error in diagnostics by @edcdavid in #1715
• give precedence to the KUBECONFIG variable instead of the $HOME/.kube/config kubeconfig by @edcdavid in #1755
• autodiscover: delete deprecated default operator label by @jmontesi in #1760
• Bug fix: adjust logic in testPodClusterRoleBindings by @jmontesi in #1792

WebServer Improvements

• webserver: add implementation for uploading tnf config file and add all the fields by @aabughosh in #1644
• webserver: add tests as a table allowing the user to select an option by @aabughosh in #1662

Unit Tests

• Remove gotools; adjust expected/actual by @sebrandon1 in #1667
• Additional unit test coverage by @sebrandon1 in #1780

Documentation Improvements

• docs: add CNFCERT demo by @jmontesi in #1757
• docs: fix CNFCERT demo path by @jmontesi in #1758

Claim File Output Improvements

• Switch to 1-to-1 TestID to Result struct by @sebrandon1 in #1666
• re-add logs in claim file by @edcdavid in #1725
• flags: change "claimloc" for "output-dir" by @jmontesi in #1726
• version: bump claim file format version by @jmontesi in #1738

CLI Improvements

• cli: display information about the CNFCERT run by @jmontesi in #1690
• CLI output improvements: check's elapsed time and last log trace. by @greyerof in #1745
• CLI output: non-tty mode improvements. by @greyerof in #1750
• cli: fix typo in skip reason by @jmontesi in #1756
• cli: cover all possible skip paths to print it in the CLI output by @jmontesi in #1765

Logging Improvements

• log: add new log package that uses slog as global logger by @jmontesi in #1663
• log: remove logrus and migrate all logs to slog by @jmontesi in #1703
• log: add log file to the results tar archive by @jmontesi in #1710
• log: set the configured log level to multiloggers by @jmontesi in #1711
• log: remove loghelper package by @jmontesi in #1721
• observability: review logs by @jmontesi in #1761
• manageability: review logs by @jmontesi in #1770
• operator: review logs by @jmontesi in #1772
• performance: review logs by @jmontesi in #1777
• networking: review logs by @jmontesi in #1782
• certification: review logs by @jmontesi in #1783
• access-control: review logs by @jmontesi in #1787
• platform: review logs by @jmontesi in #1791
• lifecycle: logs review by @jmontesi in #1789
• preflight: review logs by @jmontesi in #1793

Parser Improvements

• Bump parser version to v0.2.0 by @edcdavid in #1676
• extracts results.html from tar.gz and debug mode by @edcdavid in #1706
• update the version of the claim release by @aabughosh in #1784

CI Improvements

• Switch gradetool image to latest by @sebrandon1 in #1681
• Enable QE nightlies for ginkgo_removal by @sebrandon1 in #1723
• Revert "Enable QE nightlies for ginkgo_removal" by @sebrandon1 in #1724
• Enable QE PR tests by @sebrandon1 in #1748
• Remove self-hosted, kind-based QE nightlies by @sebrandon1 in #1776
• Add retries for -partner setup calls by @sebrandon1 in #1775
• Adjust the directories cleaned up in QE nightlies by @sebrandon1 in #1806

Operator Tool Improvements

• Updating Telco list by @edcdavid in #1712
• skip operator if claim file couldn't be parsed by @edcdavid in #1714
• batch operator script - fix long term errors by @edcdavid in #1741

ginkgo_removal work

• Dependency updates for ginkgo removal branch by @sebrandon1 in #1615
• ginkgo_removal branch: enable github checks on new PRs. by @greyerof in #1608
• Fixed deadlock in check.SetResult(). by @greyerof in #1619
• Transition manageability suite to ginkgoless by @sebrandon1 in #1614
• Transition performance suite to ginkgoless. by @greyerof in #1620
• Transition certification suite to ginkgoless. by @greyerof in #1622
• Rebase the ginkgo_removal branch with main by @sebrandon1 in #1642
• Allow more than one skip functions per check. by @greyerof in #1645
• Transition accesscontrol suite to ginkgoless by @sebrandon1 in #1616
• Transition networking suite to ginkgoless by @sebrandon1 in #1621
• Transition lifecycle suite to ginkgoless by @sebrandon1 in #1627
• Lifecycle ts: fix to avoid some checks to be skipped. by @greyerof in #1655
• Load the checks' database using a standard function instead of...

v4.5.7

v4.5.7 is a patch release of the CNF Test Suite.

We improved the test surrounding cluster-role-bindings in #1646 for pods that are owned by a cluster wide operator.

We also bumped the claimVersion to v0.2.0 as there is a new JSON structure update to the claim JSON.

Test Improvements

• Allow cluster role bindings for pods ultimately owned by a cluster wide operator by @edcdavid in #1646
• Fix WaitGroup Done() call by @bnshr in #1688

Claim File Updates

• Cherry pick 1-to-1 result struct change by @sebrandon1 in #1675

Dependency Updates

• Bump github.com/mittwald/go-helm-client from 0.12.3 to 0.12.4 by @dependabot in #1652
• Bump github.com/deckarep/golang-set/v2 from 2.3.1 to 2.4.0 by @dependabot in #1656
• Bump github.com/operator-framework/api from 0.19.0 to 0.20.0 by @dependabot in #1657
• Bump github.com/onsi/ginkgo/v2 from 2.13.1 to 2.13.2 by @dependabot in #1671
• Bump github.com/deckarep/golang-set/v2 from 2.4.0 to 2.5.0 by @dependabot in #1674

CI Improvements

• Temp. disable collector sanity check by @sebrandon1 in #1653
• Add depends-on PRs action to qe-hosted workflow by @sebrandon1 in #1677
• Add-batch-cert-script by @edcdavid in #1604
• Switch gradetool to latest image (main) by @sebrandon1 in #1680
• Increase QE timeout to 90 minutes by @sebrandon1 in #1683
• Fix copyright years by @rdavid in #1686
• Remove GPL commitment by @sebrandon1 in #1689

Full Changelog: v4.5.6...v4.5.7

v4.5.6

Bug Fixes

• Fix catalog links by @edcdavid in #1650
• Fixed function AreCPUResourcesWholeUnits(). by @greyerof in #1631

Dependency Updates

• Update Go to v1.21.4 by @sebrandon1 in #1598
• Bump github.com/gorilla/websocket from 1.4.2 to 1.5.1 by @dependabot in #1596
• Bump github.com/onsi/ginkgo/v2 from 2.13.0 to 2.13.1 by @dependabot in #1610
• Bump github.com/test-network-function/privileged-daemonset from 1.0.14 to 1.0.15 by @dependabot in #1612
• Bump github.com/test-network-function/test-network-function-claim from 1.0.30 to 1.0.31 by @dependabot in #1611
• Bump ubi8/ubi from 8.8-1067.1698056881 to 8.9-1028 by @dependabot in #1633
• Bump github.com/test-network-function/oct from 0.0.3 to 0.0.4 by @jmontesi in #1632
• Bump ubi8/ubi-minimal from 8.8-1072.1697626218 to 8.9-1029 by @dependabot in #1634
• Bump k8s.io/kubectl from 0.28.3 to 0.28.4 by @dependabot in #1636

CI Improvements

• Modify ENV vars for config and report dirs by @sebrandon1 in #1602
• Disable Go cache for self-hosted by @sebrandon1 in #1626
• Fix gchat alert message. by @greyerof in #1630
• Re-enable QE PR checks by @sebrandon1 in #1640
• enable collector's sanity check by @shirmoran in #1606

Full Changelog: v4.5.5...v4.5.6

v4.5.5

Bug Fixes

• Do not consider the exec probes in performance-rt-apps-no-exec-probes by @edcdavid in #1590
• Fix regression in performance-rt-apps-no-exec-probes by @edcdavid in #1593

New Website

• website for our cnf by @aabughosh in #1544

Parser Updates

• Getting bugfix for parser (still needing bootstrap for some elements) by @edcdavid in #1595

Dependency Updates

• Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 by @dependabot in #1588
• Bump github.com/fatih/color from 1.15.0 to 1.16.0 by @dependabot in #1589
• Updating the HTML parser to the latest version (v0.1.3) by @edcdavid in #1587
• Bump helm.sh/helm/v3 from 3.13.1 to 3.13.2 by @dependabot in #1592

CI Updates

• Setup QE OCP 4.14 runner by @sebrandon1 in #1594

Full Changelog: v4.5.4...v4.5.5

v4.5.4

Small update to include a fix for the pod-role-bindings test.

Bug Fixes

• Allow Role bindings across namespaces by @edcdavid in #1583

Dependency Updates

• Bump github.com/operator-framework/api from 0.18.0 to 0.19.0 by @dependabot in #1584

Full Changelog: v4.5.3...v4.5.4

v4.5.3

v4.5.3 is a smaller update with some vulnerability dependency updates and a bug fix to the performance suite.

Bug Fixes

• performance: avoid failing test case if a process cannot be found by @jmontesi in #1564

Improvements

• Add OCP 4.14 to the compatibility matrix by @sebrandon1 in #1574

Dependency Updates

• Bump github.com/go-logr/logr from 1.2.4 to 1.3.0 by @dependabot in #1565
• Bump github.com/docker/docker from 24.0.6+incompatible to 24.0.7+incompatible by @dependabot in #1568
• Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.41.1 to 0.44.0 by @dependabot in #1530
• Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.42.0 to 0.44.0 by @dependabot in #1572
• Bump google.golang.org/grpc from 1.58.2 to 1.58.3 by @dependabot in #1571
• Bump github.com/test-network-function/privileged-daemonset from 1.0.13 to 1.0.14 by @dependabot in #1579
• Bump github.com/test-network-function/test-network-function-claim from 1.0.29 to 1.0.30 by @dependabot in #1580
• Bump github.com/test-network-function/oct from 0.0.2 to 0.0.3 by @dependabot in #1581

CI Improvements

• Add cron to run preflight against unstable image by @sebrandon1 in #1569
• Add YAML for intrusive QE OCP testing by @sebrandon1 in #1573
• enable run of collector's sanity check in CI by @shirmoran in #1567
• Update GolangCI-lint to v1.55.1 by @sebrandon1 in #1575
• Update _typos.toml by @sebrandon1 in #1577

Full Changelog: v4.5.2...v4.5.3