Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GSOC23] - B - Enable the downloading and synchronization of OVAL data #7509

Merged
merged 64 commits into from
Sep 5, 2024
Merged

[GSOC23] - B - Enable the downloading and synchronization of OVAL data #7509

merged 64 commits into from
Sep 5, 2024

Commits on Sep 5, 2024

  1. Read OVAL data sources from a configuration file

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    bae562e View commit details
    Browse the repository at this point in the history

  2. Implement the downloading of OVALs based on the config file

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    01aeecb View commit details
    Browse the repository at this point in the history

  3. Add taskomatic task for syncing OVAL data

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    44ec0e4 View commit details
    Browse the repository at this point in the history

  4. Indicate in UI when vulnerability scan was based on OVAL or Channels … 

    …data
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    d2bd4d9 View commit details
    Browse the repository at this point in the history

  5. Add Leap 15.2 OVAL sources

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    6c3ea0e View commit details
    Browse the repository at this point in the history

  6. Deploy oval.config.json to /usr/share/susemanager/scc/

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    ddbf4d5 View commit details
    Browse the repository at this point in the history

  7. Add support for SLE Micro OVAL-based CVE auditing

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    c298db7 View commit details
    Browse the repository at this point in the history

  8. Add support for openSUSE Micro OVAL-based CVE auditing

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    a180ead View commit details
    Browse the repository at this point in the history

  9. Add SLE and Leap Micro OVAL sources

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    f97857a View commit details
    Browse the repository at this point in the history

  10. Synchronize OVAL data only for products used by registered clients

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    914fb48 View commit details
    Browse the repository at this point in the history

  11. Log the products eligible for OVAL synchronization

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    4160040 View commit details
    Browse the repository at this point in the history

  12. Translate oval sync task related strings

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    03c6bb1 View commit details
    Browse the repository at this point in the history

  13. Test OVAL sources 

    Also, found some dead URLs so I deleted them
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    7c2f376 View commit details
    Browse the repository at this point in the history

  14. Use Leap in oval.config.json

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    51a3a47 View commit details
    Browse the repository at this point in the history

  15. Use proper logging when syncing OVAL data

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    529ddd7 View commit details
    Browse the repository at this point in the history

  16. Rename 'openSUSE_LEAP_MICRO' to 'LEAP_MICRO' 

    - Having enums starting with a capital letter is better for consistency
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    c53980f View commit details
    Browse the repository at this point in the history

  17. Document the OVAL downloader component classes

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    d9102ab View commit details
    Browse the repository at this point in the history

  18. Refactor and fix checkstyle

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    4e7c6da View commit details
    Browse the repository at this point in the history

  19. Use format specifiers instead of concatenation when logging

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    ae819ba View commit details
    Browse the repository at this point in the history

  20. Remove unused methods

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    170d3af View commit details
    Browse the repository at this point in the history

  21. Refactor Server#toOVALProduct

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    7a9ac7f View commit details
    Browse the repository at this point in the history

  22. Add changelogs

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    e783046 View commit details
    Browse the repository at this point in the history

  23. Revert "Use channel data until full OVAL implementation is ready" 

    This reverts commit 6c86362.

It is ready!
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    4298480 View commit details
    Browse the repository at this point in the history

  24. Copyright (c) 2024

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    e96be51 View commit details
    Browse the repository at this point in the history

  25. Prevent synchronizing OVAL data for the same product twice

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    0f13988 View commit details
    Browse the repository at this point in the history

  26. Avoid fetching all servers to detect the set of operating systems the… 

    …y use

- We have deployments with more than 10k systems. This would load all of them when we should have a few of different OS flavors.
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    baed757 View commit details
    Browse the repository at this point in the history

  27. Log the starting and ending of the OVAL syncing task 

    - Since the task runs once a day by default, using log#info instead of log#debug makes more sense.
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    ddcfda1 View commit details
    Browse the repository at this point in the history

  28. Rephrase comment in OVALCleaner

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    89dc45a View commit details
    Browse the repository at this point in the history

  29. Fix typo in oval.config.json

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    00ae7c0 View commit details
    Browse the repository at this point in the history

  30. Refactor OVALDownloader 

    - Also deleted OVALCompressionMethod as it is not needed anymore
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    374bb8e View commit details
    Browse the repository at this point in the history

  31. Refactor to avoid code duplication

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    8e61f4a View commit details
    Browse the repository at this point in the history

  32. Clarify comments

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    518a859 View commit details
    Browse the repository at this point in the history

  33. Increase connection and read timeouts while downloading OVAL data 

    - Increased it to 15 seconds.
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    17d3379 View commit details
    Browse the repository at this point in the history

  34. Store downloaded OVAL files in a cache directory

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    e57ce14 View commit details
    Browse the repository at this point in the history

  35. Move the toOVALProduct method to OsReleasePair

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    4f42473 View commit details
    Browse the repository at this point in the history

  36. Undo OVAL support for ubuntu 

    - Support for ubuntu requires more changes and thus will be added in its own PR.
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    c3c46e4 View commit details
    Browse the repository at this point in the history

  37. Extract the cve from <cve> tag in OVAL files 

    - In openSUSE Leap 15.5 OVAL file, I found <cve> CVE-2018-1000155 at SUSE </cve> which resulted in an exception being thrown when OVAL data is written to the database, because of the length of the CVE (more than 20 characters).
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    954c7dd View commit details
    Browse the repository at this point in the history

  38. Display the list of data sources used to scan each client server 

    - Only two possible data sources: OVAL and Channels
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    a9aa82f View commit details
    Browse the repository at this point in the history

  39. Change regex to not be vulnerable to polynomial runtime due to backtr… 

    …acking
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    1332260 View commit details
    Browse the repository at this point in the history

  40. Introduce UI warning for potential inaccuracies in CVE audit results

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    c4103f0 View commit details
    Browse the repository at this point in the history

  41. Implement the check for existence of erratas in server CVE channels 

    - If result is not empty, then server can be CVE audited normally otherwise a warning should be displayed indicating a lack of channels data and thus inaccuracies are expected.
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    8f7e4cd View commit details
    Browse the repository at this point in the history

  42. Delete cveaudit.less

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    21c56b8 View commit details
    Browse the repository at this point in the history

  43. Remove the "Scan Data" column from the CVE audit page 

    - It doesn't bring much value + UI looks quite overloaded
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    998916e View commit details
    Browse the repository at this point in the history

  44. Translate strings

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    8f66bc9 View commit details
    Browse the repository at this point in the history

  45. Log error on invalid CVE scan data sources

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    f666d1b View commit details
    Browse the repository at this point in the history

  46. Insert missing data related to the sync OVAL taskomatic task

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    3a5dc43 View commit details
    Browse the repository at this point in the history

  47. Rephrase CVE auditing related UI messages 

    Co-authored-by: Abid Mehmood <amehmood@suse.de>
    @HoussemNasri @admd @parlt91
    2 people authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    d9e36a4 View commit details
    Browse the repository at this point in the history

  48. Fix bug where the system name in the CVE auditing table results not d… 

    …isplayed
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    1615168 View commit details
    Browse the repository at this point in the history

  49. Introduce an "unknown" patch status to be returned when OVAL and Chan… 

    …nels metadata not synced
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    44ebb6e View commit details
    Browse the repository at this point in the history

  50. Introduce a shell script that synchronizes the oval.config.json file … 

    …used for testing
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    1cc5f08 View commit details
    Browse the repository at this point in the history

  51. Add missing OVAL data sources

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    f96ddc0 View commit details
    Browse the repository at this point in the history

  52. Move OVAL config file to a more appropriate directory 

    - The 'scc' directory is not the best place to put this json file since it is not coming from SCC.
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    01567d4 View commit details
    Browse the repository at this point in the history

  53. Configure OVAL for SLE/Leap Micro 6.0 and Leap 15.6

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    27d86c7 View commit details
    Browse the repository at this point in the history

  54. Fix linter

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    18c8852 View commit details
    Browse the repository at this point in the history

  55. Frontend lint

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    c838766 View commit details
    Browse the repository at this point in the history

  56. Clear previous OVAL metadata for platform before inserting the newer … 

    …data.
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    0f128b7 View commit details
    Browse the repository at this point in the history

  57. Add an configuration option to enable/disable OVAL metadata usage in … 

    …CVE auditing
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    b0d1d34 View commit details
    Browse the repository at this point in the history

  58. Correct the location of the OVAL config file in RPM spec

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    54372ea View commit details
    Browse the repository at this point in the history

  59. Fix sles and sled OVAL product matching

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    9a64863 View commit details
    Browse the repository at this point in the history

  60. Fix OVAL's release package regex

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    88e06ef View commit details
    Browse the repository at this point in the history

  61. Relocate database migration script

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    3c7637d View commit details
    Browse the repository at this point in the history

  62. Continue syncing other products if OVAL data syncing fails for one

    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    34229c5 View commit details
    Browse the repository at this point in the history

  63. Change string#replace by string#replaceFirst 

    - The former does not accept a regex.
    @HoussemNasri @parlt91
    HoussemNasri authored and parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    81026c5 View commit details
    Browse the repository at this point in the history

  64. Move schema migration to current dir 

    Signed-off-by: Pascal Arlt <parlt@suse.com>
    @parlt91
    parlt91 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    5855da9 View commit details
    Browse the repository at this point in the history