From 7b6537de7b708b37b1921729010281d0e6d03862 Mon Sep 17 00:00:00 2001 From: Konstantinos Tsamis Date: Tue, 10 Sep 2024 15:54:40 +0200 Subject: [PATCH] Add README files for pipeline scripts (#1351) --- .../scripts/json_generator/README.md | 16 +++ .../json_generator/ibs_osc_client_README.md | 91 ++++++++++++ .../maintenance_json_generator_README.md | 134 ++++++++++++++++++ .../json_generator/smash_client_README.md | 95 +++++++++++++ 4 files changed, 336 insertions(+) create mode 100644 jenkins_pipelines/scripts/json_generator/README.md create mode 100644 jenkins_pipelines/scripts/json_generator/ibs_osc_client_README.md create mode 100644 jenkins_pipelines/scripts/json_generator/maintenance_json_generator_README.md create mode 100644 jenkins_pipelines/scripts/json_generator/smash_client_README.md diff --git a/jenkins_pipelines/scripts/json_generator/README.md b/jenkins_pipelines/scripts/json_generator/README.md new file mode 100644 index 000000000..7ff252861 --- /dev/null +++ b/jenkins_pipelines/scripts/json_generator/README.md @@ -0,0 +1,16 @@ +# Pipeline scripts + +This repository contains multiple Python scripts for our pipelines. Below are links to detailed +README files for each script: + +## Scripts + +- [maintenance_json_generator script README](./maintenance_json_generator_README.md) - Automates gathering and +processing of open QAM SLE requests that affect SUMA 4.3 and 5.0 and generates JSON output for BV testsuite +- [ibs_osc_client_README](./ibs_osc_client_README.md) - checks embargo status and processes repository information +- [smash_client_README](./smash_client_README.md) - interacts with SUSE Manager's SMASH API to retrieve and manage embargoed bug IDs and CVEs. + + +## License + +Licensed under the [MIT License](https://opensource.org/licenses/MIT). diff --git a/jenkins_pipelines/scripts/json_generator/ibs_osc_client_README.md b/jenkins_pipelines/scripts/json_generator/ibs_osc_client_README.md new file mode 100644 index 000000000..60d2185b6 --- /dev/null +++ b/jenkins_pipelines/scripts/json_generator/ibs_osc_client_README.md @@ -0,0 +1,91 @@ +# SUSE Maintenance Incident (MI) Embargo Checker + +## Table of Contents + +- [Overview](#overview) +- [Features](#features) +- [Requirements](#requirements) +- [Usage](#usage) + - [Initialize `IbsOscClient`](#initialize-ibsoscclient) + - [Find Maintenance Incidents](#find-maintenance-incidents) + - [Check if MI is Under Embargo](#check-if-mi-is-under-embargo) +- [Output](#output) +- [Dependencies](#dependencies) +- [License](#license) +- [Notes](#notes) + +--- + +## Overview + +This Python script is designed to automate the management of Quality Assurance +Maintenance (QAM) requests for SUSE Linux Enterprise Server (SLES). The script +performs the following tasks: + +- **Queries Open Maintenance Incidents (MIs):** Retrieves a list of open MIs + from the SUSE build service. +- **Checks for Embargo Status:** Determines if an MI is under embargo by + inspecting its attributes. +- **Retrieves and Processes Information:** Collects and processes relevant + details about MIs and their associated repositories. + +The output is a JSON file, which can be used for further testing in the BV +(Business Validation) testsuite pipeline. The script supports both SUSE Manager +4.3 (SUMA 4.3) and SUSE Manager 5.0 (SUMA 5.0). + +## Features + +- Queries open maintenance incidents (MIs) from SUSE build service. +- Checks if an MI is under embargo. +- Retrieves and processes relevant information about MIs and their repositories. + +## Requirements + +- Python 3.11 +- `smash_client` library: Ensure you have the `smash_client` module available in + your environment. + +## Usage + +### Initialize `IbsOscClient` + +```python +from ibs_osc_client import IbsOscClient + +client = IbsOscClient() +``` + +### Find Maintenance Incidents + +Find open maintenance incidents in a specified group: + +```python +mi_ids = client.find_maintenance_incidents(status='open', group='qam-manager') +``` + +### Check if MI is Under Embargo + +Check if a specific MI is under embargo: + +```python +is_under_embargo = client.mi_is_under_embargo(mi_id='123456') +``` + +## Output + +The script outputs a JSON file compatible with the BV testsuite pipeline. + +## Dependencies + +`smash_client` library + +## License + +This script is licensed under the [MIT License](https://opensource.org/licenses/MIT). + +## Notes + +The script uses osc commands for interacting with SUSE build service and may +require these utilities to be installed and properly configured. +Ensure that the environment is set up with the appropriate credentials and +permissions for accessing the SUSE build service. diff --git a/jenkins_pipelines/scripts/json_generator/maintenance_json_generator_README.md b/jenkins_pipelines/scripts/json_generator/maintenance_json_generator_README.md new file mode 100644 index 000000000..9cbe9eef9 --- /dev/null +++ b/jenkins_pipelines/scripts/json_generator/maintenance_json_generator_README.md @@ -0,0 +1,134 @@ +# maintenance_json_generator.py + +## Table of Contents + +- [Overview](#overview) +- [Features](#features) +- [Usage](#usage) +- [Output](#output) +- [Logging](#logging) +- [Functions](#functions) +- [Error Handling](#error-handling) +- [Dependencies](#dependencies) +- [License](#license) + +--- + +## Overview + +This Python script automates the process of gathering and processing open QAM +(Quality Assurance Maintenance) requests for SUSE Linux Enterprise Server (SLES) +that affect SUSE Manager. The output is a JSON file, which can be fed into the +BV (Business Validation) testsuite pipeline for further testing. It supports +both SUSE Manager 4.3 (SUMA 4.3) and SUSE Manager 5.0 (SUMA 5.0). + +The script allows users to input Maintenance Incident (MI) IDs and generates the +appropriate repository information for SUMA 4.3 or 5.0 nodes (servers, proxies, +and clients). It retrieves the necessary information for these nodes and their +associated repositories. + +## Features + +- Support for SUSE Manager 4.3 and 5.0: The script allows users to specify which +version of SUSE Manager they are working with. +- Flexible MI ID Input: MI IDs can be provided via CLI arguments or by reading +from a file. +- Custom Repository Generation: Outputs a JSON file containing repository +information for the SUSE Manager BV testsuite pipeline. +- Embargo Checks: The script has an option to reject Maintenance Incidents (MIs) +that are under embargo. + +## Usage + +Command-Line Arguments + +The script accepts several command-line arguments to control its behavior. + +```bash +python script.py [options] +``` + +Options: + +`-v`, `--version`: Specifies the SUSE Manager version. Options are `43` for SUSE +Manager 4.3 and `50` for SUSE Manager 5.0. Default is 43. +`-i`, `--mi_ids`: A space-separated list of MI IDs. +`-f`, `--file`: Path to a file containing MI IDs, each on a new line. +`-e`, `--no_embargo`: Reject any MIs that are currently under embargo. + +Example: + +```bash +python script.py --version 50 --mi_ids 1234 5678 --file mi_ids.txt --no_embargo +``` + +This command will: + +1. Run the script for SUSE Manager 5.0 (`--version 50`). +2. Use MI IDs 1234 and 5678 along with any additional MI IDs from the file +mi_ids.txt. +3. Reject any MIs that are under embargo (`--no_embargo`). + +## Output + +The script generates a file named custom_repositories.json, which contains the +repository data for the provided MI IDs. + +## Logging + +The script includes basic logging for informational messages. To enable logging, +ensure the setup_logging function is called at the beginning of the script. Log +messages will display timestamped INFO-level messages. + +## Functions + +### Main Functions + +- `parse_cli_args()`: Parses the command-line arguments using argparse. +- `merge_mi_ids()`: Merges MI IDs provided from the CLI or file input. +- `read_mi_ids_from_file()`: Reads MI IDs from a file. +- `clean_mi_ids()`: Cleans and formats MI IDs for consistency. + +### Repository Data + +The script contains two main dictionaries for SUSE Manager client tools +repositories: + +- `v43_client_tools`: Contains repository data for SUMA 4.3 client tools. +- `v50_client_tools_beta`: Contains repository data for SUMA 5.0 beta client tools. +- `merged_client_tools`: Merges the 4.3 and 5.0 beta client tools into a single +dictionary. + +It also defines two dictionaries for SUSE Manager server and proxy +repositories: + +- `v43_nodes`: Includes repository data for SUMA 4.3 server and proxy nodes. +- `v50_nodes`: Includes repository data for SUMA 5.0 server and proxy nodes. + +The final repository information is stored in the nodes_by_version dictionary, +which maps SUMA version numbers (`43`, `50`) to the corresponding repository data. + +## Error Handling + +- If no MI IDs are provided via CLI or file, the script will print an error +message and halt execution. +- Invalid MI IDs or missing files will result in appropriate error messages. + +## Dependencies + +The following Python libraries are required to run this script: + +- `argparse`: For parsing command-line arguments (standard library). +- `functools`: For the cache decorator (standard library). +- `json`: For generating JSON output (standard library). +- `requests`: For sending HTTP requests. +- `logging`: For logging informational messages (standard library). + +### External Dependencies + +`ibs_osc_client`: This module is used to interact with the IBS (Internal Build +Service) Open Service Client. + +## License + +This script is licensed under the [MIT License](https://opensource.org/licenses/MIT). diff --git a/jenkins_pipelines/scripts/json_generator/smash_client_README.md b/jenkins_pipelines/scripts/json_generator/smash_client_README.md new file mode 100644 index 000000000..72f1dc4c3 --- /dev/null +++ b/jenkins_pipelines/scripts/json_generator/smash_client_README.md @@ -0,0 +1,95 @@ +# SmashClient + +## Table of Contents + +- [Overview](#overview) +- [Features](#features) +- [Requirements](#requirements) +- [Installation](#installation) +- [Usage](#usage) + - [Initialize `SmashClient`](#initialize-smashclient) + - [Get Embargoed Bug IDs](#get-embargoed-bug-ids) +- [Dependencies](#dependencies) +- [License](#license) +- [Notes](#notes) + +--- + +## Overview + +The `SmashClient` Python script provides a straightforward interface to interact +with the SUSE Manager's SMASH API to retrieve a list of embargoed bug IDs. This +is useful for managing and tracking software issues that are currently under +embargo, ensuring that you are informed about which bugs and CVEs (Common +Vulnerabilities and Exposures) are restricted and cannot be discussed or +published publicly. + +## Features + +- Fetches and caches embargoed bug IDs from the SMASH API. +- Returns a set of embargoed bug IDs including associated CVEs. +- Caches results to optimize performance and reduce redundant API calls. + +## Requirements + +- Python 3.6 or higher +- `requests` library + +## Installation + +To install the required dependencies, ensure you have `requests` installed: + +```bash +pip install requests +``` + +## Usage + +### Initialize SmashClient + +First, you need to create an instance of the SmashClient class. This class +handles interaction with the SMASH API and manages caching of embargoed bug IDs. + +```python +from your_module_name import SmashClient + +client = SmashClient() +``` + +## Get Embargoed Bug IDs + +Once you have an instance of SmashClient, you can fetch the embargoed bug IDs by +calling the get_embargoed_bugs_ids method. This method retrieves the IDs from +the SMASH API and caches them for future use. + +```python +embargoed_ids = client.get_embargoed_bugs_ids() +print(embargoed_ids) +``` + +This method performs the following steps: + +1. Checks Cache: If the embargoed IDs are already cached, it returns them +immediately, avoiding unnecessary API calls. +2. Fetches Data: If the cache is empty, it makes a request to the SMASH API to +retrieve the latest embargoed bug IDs. +3. Processes Response: Parses the JSON response to extract bug IDs and CVEs. +4. Updates Cache: Stores the fetched data in the cache for subsequent calls. + +## Dependencies + +`requests`: A popular Python library for making HTTP requests. It is used here to +handle communication with the SMASH API. + +## License + +This script is licensed under the [MIT License](https://opensource.org/licenses/MIT). + +## Notes + +Ensure that the requests library is installed in your environment. +This script relies on the SMASH API being available and responsive. Handle +possible exceptions appropriately in production environments. +Caching helps in reducing the load on the API and speeds up access to the +embargoed IDs, but make sure to handle cache invalidation if the data can change +frequently.