adding metadata propagation

README.md

@@ -10,6 +10,7 @@ Join the community in the [ContainerCraft Community Discord](https://discord.gg/
 
 ## Features
 
+- **AWS LandingZone**: Automated Kubernetes cluster setup using Talos.
 - **Kubernetes Deployment**: Automated Kubernetes cluster setup using Talos.
 - **Pulumi IaC Integration**: Infrastructure as Code management with Pulumi.
 - **Runme Integration**: Execute documented tasks directly from the README.md.

ROADMAP.md

@@ -32,7 +32,14 @@
 
 ## Introduction
 
-This roadmap outlines the development of a next-generation, cloud-agnostic platform engineering environment. The goal is to establish a robust, scalable, and secure multi-cloud infrastructure that automates provisioning, enforces compliance, and centralizes operational data. This environment will empower application teams to operate safely within their own isolated accounts, supported by a streamlined, code-driven landing zone setup.
+This roadmap outlines the development of a next-generation, cloud-agnostic platform engineering environment.
+The goal is to establish a robust, scalable, and secure multi-cloud infrastructure that automates provisioning, enforces compliance, and centralizes operational data.
+This environment will empower application teams to operate safely within their own isolated accounts, supported by a streamlined, code-driven landing zone setup.
+
+The architecture consists of a hierarchical organization with multiple organizational units (OUs) and accounts across AWS, Azure, and GCP.
+Infrastructure provisioning and configuration are fully automated using Infrastructure as Code (IaC) practices.
+Compliance controls are embedded within the configuration code, ensuring consistent policy enforcement.
+Centralized governance is achieved through policy propagation and centralized services for logging, monitoring, and cost management.
 
 ---
 
@@ -45,13 +52,6 @@ This roadmap outlines the development of a next-generation, cloud-agnostic platf
 - **Centralized Governance**: Maintain centralized policies, secrets, and configurations for consistent management across all environments.
 - **Scalability and Modularity**: Design for horizontal scalability and modularity to accommodate growth and technological changes.
 
----
-
-## Architecture Overview
-
-The architecture consists of a hierarchical organization with multiple organizational units (OUs) and accounts across AWS, Azure, and GCP. Infrastructure provisioning and configuration are fully automated using Infrastructure as Code (IaC) practices. Compliance controls are embedded within the configuration code, ensuring consistent policy enforcement. Centralized governance is achieved through policy propagation and centralized services for logging, monitoring, and cost management.
-
----
 
 ## Key Components
 
@@ -224,33 +224,33 @@ The architecture consists of a hierarchical organization with multiple organizat
 - **Policy as Code**: Embed compliance and governance policies directly into your codebase.
 - **Automation API**: Integrate Pulumi into CI/CD pipelines and other automation workflows.
 
-### C. 1.1 Infrastructure Provisioning
+### 1.1 Infrastructure Provisioning
 
 - **Resource Management**: Define and manage cloud resources using code.
 - **Complex Logic Handling**: Utilize programming constructs for loops, conditionals, and abstractions.
 - **Reusable Components**: Create modules and packages for shared infrastructure code.
 
-### C. 1.2 Multi-Cloud Capabilities
+### 1.2 Multi-Cloud Capabilities
 
 - **Unified Interface**: Manage different cloud providers using the same codebase.
 - **Cross-Cloud Abstractions**: Develop higher-level components that abstract away provider specifics.
 
-### C. 1.3 State Management
+### 1.3 State Management
 
 - **State Persistence**: Track infrastructure state for accurate deployments.
 - **Backend Options**: Use local files, cloud storage, or Pulumi Cloud for state management.
 
-### C. 1.4 Policy and Compliance
+### 1.4 Policy and Compliance
 
 - **Policy as Code**: Define and enforce policies within your infrastructure code.
 - **Compliance Integration**: Include compliance controls (e.g., FISMA, NIST) in configurations.
 
-### C. 1.5 CI/CD Integration
+### 1.5 CI/CD Integration
 
 - **Automation Support**: Seamlessly integrate with CI/CD pipelines for automated deployments.
 - **GitOps Workflow**: Adopt GitOps practices with Pulumi for infrastructure changes.
 
-### C. 1.6 Collaboration and Secrets Management
+### 1.6 Collaboration and Secrets Management
 
 - **Team Collaboration**: Use Pulumi Cloud for role-based access control and collaboration.
 - **Secure Secrets Management**: Handle secrets securely with Pulumi's Federated OIDC, and Secrets Federation suppport.

pulumi/__main__.py

@@ -31,10 +31,6 @@ def main():
         modules_to_deploy = [
             "aws",
             #"cert_manager",
-            #"kubevirt",
-            #"multus",
-            #"hostpath_provisioner",
-            #"containerized_data_importer",
             #"prometheus"
         ]
 

pulumi/core/types.py

@@ -64,14 +64,15 @@ def merge(user_config: Dict[str, Any]) -> 'ComplianceConfig':
             ComplianceConfig: The merged compliance configuration object.
         """
         default_config = ComplianceConfig()
+        valid_keys = {'fisma', 'nist', 'scip'}
         for key, value in user_config.items():
-            if hasattr(default_config, key):
+            if key in valid_keys:
                 nested_config = getattr(default_config, key)
                 for nested_key, nested_value in value.items():
                     if hasattr(nested_config, nested_key):
                         setattr(nested_config, nested_key, nested_value)
                     else:
                         pulumi.log.warn(f"Unknown key '{nested_key}' in compliance.{key}")
             else:
-                pulumi.log.warn(f"Unknown compliance configuration key: {key}")
+                pulumi.log.debug(f"Ignored non-compliance key: {key}")
         return default_config

pulumi/modules/aws/config.py

@@ -1,52 +1,115 @@
-# pulumi/modules/aws/types.py
+# konductor/pulumi/modules/aws/config.py
 
 """
-AWS Module Configuration Loader.
+AWS Module Configuration
 
-This module defines functions to load and parse the AWS module configurations
-using the Pydantic-based AWSConfig model.
+This module handles the initialization and configuration of AWS resources and providers
+within the Pulumi stack. It includes functions to load AWS-specific settings, initialize
+the AWS provider, and propagate compliance and Git metadata as AWS resource tags.
 
 The module defines the following functions:
-    - load_config: Defines the function to load the configuration for the AWS module.
+    - load_aws_config: Loads and parses the AWS configuration.
+    - initialize_aws_provider: Initializes the AWS provider with credentials and region.
+    - generate_tags: Creates global tags including compliance and Git metadata.
+    - load_tenant_account_configs: Loads tenant account configurations.
 """
 
-from typing import Dict
+import os
+from typing import Dict, Any
 import pulumi
-from pulumi import Config, log
-from .types import AWSConfig, TenantAccountConfig, ControlTowerConfig, IAMUserConfig
+from pulumi import log, Config
+from pulumi_aws import Provider
+from core.metadata import get_global_labels, generate_compliance_labels, generate_git_labels
+from core.types import ComplianceConfig
+from .types import AWSConfig, TenantAccountConfig
 
+# Constants
+MODULE_NAME = "aws"
+
+def initialize_aws_provider(config: AWSConfig) -> Provider:
+    """
+    Initializes the AWS provider with the supplied configuration.
+
+    Args:
+        config (AWSConfig): The AWS configuration object with provider details.
+
+    Returns:
+        Provider: An initialized AWS Provider for resource management.
+    """
+    aws_config = pulumi.Config("aws")
+    aws_access_key = os.getenv('AWS_ACCESS_KEY_ID', aws_config.get("access_key_id"))
+    aws_secret_key = os.getenv('AWS_SECRET_ACCESS_KEY', aws_config.get("secret_access_key"))
+    profile = os.getenv('AWS_PROFILE', config.profile)
+
+    return Provider(
+        "awsProvider",
+        access_key=aws_access_key,
+        secret_key=aws_secret_key,
+        profile=profile,
+        region=config.region,
+    )
+
+def generate_tags(config: AWSConfig, compliance_config: ComplianceConfig, git_info: Dict[str, str]) -> Dict[str, str]:
+    """
+    Generates tags for AWS resources, including compliance and Git metadata.
+
+    Args:
+        config (AWSConfig): The AWS configuration object.
+        compliance_config (ComplianceConfig): The compliance configuration object.
+        git_info (Dict[str, str]): Information about the Git repository and commit.
+
+    Returns:
+        Dict[str, str]: A dictionary containing key-value pairs of tags to apply to AWS resources.
+    """
+    global_labels = get_global_labels()
+    compliance_labels = generate_compliance_labels(compliance_config)
+    git_labels = generate_git_labels(git_info)
+
+    aws_module_tags = {
+        **global_labels,
+        **compliance_labels,
+        **git_labels,
+        "iac_module_name": MODULE_NAME
+    }
+    pulumi.export("aws_module_tags", aws_module_tags)
+
+    return aws_module_tags
 
 def load_aws_config() -> AWSConfig:
     """
-    Loads the configuration for the AWS module.
+    Loads the AWS module configuration using Pulumi Config.
 
     Returns:
-        AWSConfig: The AWS configuration.
+        AWSConfig: The parsed AWS configuration object.
+
+    Raises:
+        ValueError: If there is an issue with the AWS configuration format.
     """
     config = Config()
     aws_config_dict = config.get_object('aws') or {}
 
     try:
-        aws_config = AWSConfig(**aws_config_dict)
+        # Isolate AWS configurations, ignoring compliance inline
+        aws_config = AWSConfig.merge(aws_config_dict)
     except Exception as e:
         log.error(f"Invalid AWS configuration: {e}")
         raise
 
     return aws_config
 
-
 def load_tenant_account_configs() -> Dict[str, TenantAccountConfig]:
     """
-    Loads tenant account configurations from Pulumi config.
+    Loads configurations for tenant accounts.
 
     Returns:
-        Dict[str, TenantAccountConfig]: A dictionary of tenant account configurations.
+        Dict[str, TenantAccountConfig]: A dictionary containing configurations for each tenant account.
     """
     config = Config()
     aws_config_dict = config.get_object('aws') or {}
     tenant_accounts_list = aws_config_dict.get('landingzones', [])
     tenant_accounts = {}
 
+    # TODO: use pulumi .apply method to ensure type Output objects are resolved
     for tenant in tenant_accounts_list:
         try:
             tenant_config = TenantAccountConfig(**tenant)