diff --git a/docs/modules/core/README.md b/docs/modules/core/README.md index f32b8e0..f3328a9 100644 --- a/docs/modules/core/README.md +++ b/docs/modules/core/README.md @@ -22,105 +22,105 @@ This README provides a high-level overview of the core module’s purpose, archi ## Introduction -Modern, multi-cloud IaC environments demand consistent, secure, and compliant patterns. The Konductor Core Module addresses these needs by serving as the backbone of the entire IaC codebase. Through a well-defined set of Pydantic models, typed configurations, protocols, and DRY design, the core module ensures that all modules—no matter what provider or technology they manage—adhere to a unified standard of quality, compliance, and maintainability. +Modern, multi-cloud IaC environments demand consistent, secure, and compliant patterns. The Konductor Core Module addresses these needs by serving as the backbone of the entire IaC codebase. Through a well-defined set of Pydantic models, strict typing, runtime validation, and DRY design principles, the core module ensures that all modules—regardless of the provider or technology—adhere to a unified standard of quality, compliance, and maintainability. ## Purpose and Scope **What the Core Module Does:** -- Defines fundamental data models and protocols for resources, configurations, compliance, and deployment results. +- Defines fundamental Pydantic-based data models and protocols for resources, configurations, compliance, and deployment results. - Provides a uniform abstraction for all providers and modules, ensuring consistent tagging, labeling, annotations, compliance checks, and lifecycle management. -- Offers compliance frameworks (FISMA, NIST) and ATO enforcement integrated at the foundational level. -- Introduces shared base classes and metadata fields (e.g., `CommonMetadataFields`) to promote a DRY codebase. +- Offers integrated compliance frameworks (FISMA, NIST) and ATO enforcement at the foundational level, ensuring production stacks meet stringent requirements. +- Introduces shared base classes and metadata fields (e.g., `CommonMetadataFields`) to keep the code DRY, easier to maintain, and more coherent. **What It Does Not Do:** -- The core module does not implement provider-specific logic. Instead, it sets universal standards. -- It does not impose project-specific logic. Instead, it provides building blocks for more specialized modules to leverage. +- The core module does not implement provider-specific logic. Instead, it establishes universal standards for all modules to follow. +- It does not impose project-specific application logic. Instead, it provides flexible building blocks that specialized modules can extend. ## Core Principles -1. **Provider-Agnostic**: The core module never assumes a particular provider. It ensures AWS, GCP, Azure, OpenStack, Kubernetes, and beyond can integrate seamlessly. -2. **Strict Typing and Validation**: Pydantic models and typed dictionaries guarantee correctness and clarity. Early validation prevents runtime misconfigurations. -3. **Security and Compliance Embedded**: Compliance-oriented fields (FISMA, NIST, ATO) are baked in, ensuring production stacks meet stringent requirements. -4. **DRY and Maintainable**: Common fields and patterns (e.g., metadata, enabled/disabled states) are factored into shared base classes, reducing duplication. -5. **Pulumi-Native Integration**: Patterns and interfaces align with Pulumi’s stack-based workflows, ensuring smooth integration and state management. +1. **Provider-Agnostic**: No assumptions about specific cloud providers. AWS, GCP, Azure, OpenStack, Kubernetes, and beyond integrate seamlessly. +2. **Strict Typing and Runtime Validation**: Fully Pydantic-based models ensure early detection of misconfigurations and adherence to schemas at runtime. +3. **Security and Compliance Embedded**: Compliance models (FISMA, NIST, ATO) are integrated at the core, ensuring production stacks cannot bypass essential authorization and lifecycle controls. +4. **DRY and Maintainable**: Common fields (like `metadata`) and patterns (like `enabled` states) are centralized, reducing duplication and making the codebase more maintainable. +5. **Pulumi-Native Integration**: Designed to align with Pulumi stack patterns, ensuring smooth configuration loading, state management, and resource orchestration. ## Provider-Agnostic and Unified Interface -By defining universal interfaces (`ModuleInterface`, `DeploymentContext`) and standardized configuration patterns, the core module creates a single “language” that all modules speak. This ensures: +By defining universal interfaces (`ModuleInterface`, `DeploymentContext`) and standardized configuration patterns, the core module creates a single "language" all modules speak. This ensures: -- Every module follows the same lifecycle hooks (validate, deploy, manage dependencies). -- Every resource is enriched with consistent metadata and compliance checks. -- Teams can integrate new providers without rethinking foundational IaC patterns. +- Every module follows the same lifecycle steps (validate, deploy, handle dependencies). +- Every resource benefits from consistent metadata and compliance checks. +- Teams can integrate new providers easily without re-engineering fundamental IaC patterns. -The result is a uniform development experience, simpler onboarding, and more predictable deployments. +A unified interface leads to a consistent development experience, simpler onboarding, and more predictable deployments. ## Foundational Data Models and Patterns -Key patterns include: +Key elements: -- **`CommonMetadataFields`**: A central model for tags, labels, annotations, ensuring consistent metadata application. -- **`BaseConfigModel`**: A shared base class for configuration, providing `enabled`, `dependencies`, `configuration`, and `metadata`. -- **`InitializationConfig`**: Augments `BaseConfigModel` with stack/project details, compliance, and git metadata. -- **`ResourceModel`**: A unified resource representation including `metadata` and timestamps. -- **`ComplianceConfig`, `FismaConfig`, `NistConfig`**: Compliance models to ensure production environments meet authorization and controls requirements. +- **`CommonMetadataFields`**: A central model for tags, labels, and annotations, ensuring uniform metadata application. +- **`BaseConfigModel`**: A shared base class providing common fields (`enabled`, `dependencies`, `configuration`, `metadata`) for all configuration models. +- **`InitializationConfig`**: Augments `BaseConfigModel` with stack/project details, compliance configs, and Git metadata. +- **`ResourceModel`**: A standardized resource representation, including timestamps and metadata. +- **`ComplianceConfig`, `FismaConfig`, `NistConfig`**: Models that ensure production environments meet required authorization and controls. -By consolidating these common elements, the core module reduces complexity and ensures a cohesive experience across the entire IaC codebase. +These common elements reduce complexity and enforce coherence across the entire IaC stack. ## Compliance and Security -Compliance is not optional. The core module integrates compliance configurations at the base level, ensuring: +Compliance is integral to the core module: -- **ATO Enforcement**: Production stacks must provide authorized and eol dates. -- **FISMA/NIST Integration**: Controls, modes, and exceptions are captured in a single compliance model, preventing ad-hoc compliance logic. -- **Consistent Compliance Checks**: Modules and deployments inherit these patterns automatically, reducing risk and audit overhead. +- **ATO Enforcement**: Production stacks must specify authorized and eol dates. +- **FISMA/NIST Controls**: Centralized compliance models prevent ad-hoc logic and ensure consistent enforcement. +- **Consistent Validation**: Pydantic validation catches invalid compliance configurations early, reducing risk and audit overhead. ## Global Metadata Management -Global metadata is managed via a thread-safe `MetadataSingleton`: +Global metadata is managed via `MetadataSingleton`: -- **Global Tags, Labels, Annotations**: Applied once and inherited universally. -- **Git Metadata**: Every resource can trace its lineage back to a specific commit or tag. -- **Consistent Application**: `setup_global_metadata()` ensures a single, authoritative source for all metadata. +- **Global Tags, Labels, Annotations**: Set once, apply everywhere. +- **Git Metadata**: Resources traceable to a specific commit or tag. +- **Consistent Application**: `setup_global_metadata()` ensures a single source of truth. -This approach enables governance, compliance, and observability at scale. +This empowers governance, compliance, and platform-wide observability at scale. ## Module Integration and Interfaces -**`ModuleInterface`** and **`DeploymentContext`** define how modules interact: +`ModuleInterface` and `DeploymentContext` define how modules interact: -- **ModuleInterface**: Requires `validate()`, `deploy()`, and `dependencies()` methods, ensuring all modules follow a known lifecycle pattern. -- **DeploymentContext**: Provides a consistent deployment environment, retrieving configurations and executing deployments in a predictable manner. +- **ModuleInterface**: Modules must implement `validate()`, `deploy()`, `dependencies()` uniformly. +- **DeploymentContext**: Provides a consistent environment for retrieving configurations and performing deployments. -This uniform interface simplifies maintenance, testing, and extensibility across providers and technologies. +This uniform interface simplifies maintenance, testing, and integrating new modules, regardless of the underlying provider. ## Configuration Management -**`ConfigManager`** centralizes configuration loading and caching: +`ConfigManager` centralizes loading and caching Pulumi configurations: -- **Unified Config Access**: Modules request their configurations from a single source. -- **Layered and Overridden Configs**: Stacks can override defaults, and modules can adapt dynamically. -- **Early Validation**: Pydantic ensures invalid configurations never reach runtime deployments. +- **Unified Config Access**: Modules load configurations from a single, consistent source. +- **Validation-First**: Pydantic ensures that misconfigurations never proceed undetected. +- **Dynamic, Layered Configs**: Override defaults and adapt to different environments without code duplication. -This reduces duplication and errors, increasing reliability of the IaC workflows. +This leads to more reliable and predictable IaC workflows. ## Extensibility and Modularity -By focusing on provider-agnostic abstractions and DRY models: +By focusing on provider-agnostic abstractions and DRY patterns: -- **Easy Provider Onboarding**: Adding a new provider module only requires conforming to the defined interfaces and base models. -- **Scalable and Future-Ready**: As new technologies or compliance frameworks emerge, integrating them is straightforward since the core sets stable standards. +- **Easy Provider Onboarding**: Adding a new provider requires conforming to defined interfaces and models, not reinventing patterns. +- **Scalable and Future-Ready**: As new technologies or compliance frameworks appear, integrating them is straightforward with stable, universal standards. -This approach ensures that Konductor can evolve gracefully as infrastructure needs change. +The result is a codebase that evolves gracefully as infrastructure needs change. ## Usage Guidelines -- **Validate Early**: Leverage `ComplianceConfig.from_pulumi_config()` and Pydantic validations to catch configuration issues before deployments begin. -- **Use Common Models**: Apply `CommonMetadataFields` to ensure consistent tagging and labeling practices. -- **Implement Protocols**: Adhere to `ModuleInterface` and `DeploymentContext` for a uniform, testable, and maintainable module structure. -- **Manage Compliance Centrally**: Rely on `ComplianceConfig` to enforce production requirements and maintain a single source of compliance truth. -- **Global Metadata Once**: Use `setup_global_metadata()` and `MetadataSingleton` to ensure consistent metadata across the entire platform. +- **Validate Early**: Use `ComplianceConfig.from_pulumi_config()` and Pydantic validations to prevent invalid deployments. +- **Use Common Models**: Apply `CommonMetadataFields` to ensure consistent tagging and labeling. +- **Implement Protocols**: Follow `ModuleInterface` and `DeploymentContext` for uniform, testable module structures. +- **Centralize Compliance**: Rely on `ComplianceConfig` for production authorization and controls. +- **Global Metadata Once**: Use `setup_global_metadata()` and `MetadataSingleton` to achieve consistent metadata across the platform. ## Related Documentation @@ -132,6 +132,6 @@ This approach ensures that Konductor can evolve gracefully as infrastructure nee ## Conclusion -The Konductor Core Module establishes a robust, provider-agnostic foundation for building, scaling, and governing IaC deployments. By consolidating metadata management, compliance enforcement, resource modeling, and configuration standardization into a single framework, the core module transforms a complex multi-cloud environment into a uniform, secure, and maintainable platform. +The Konductor Core Module establishes a robust, provider-agnostic foundation for building, scaling, and governing IaC deployments. By moving entirely to Pydantic models, we gain runtime validation, richer schemas, and DRY patterns, ensuring a more maintainable, secure, and consistent environment as Konductor evolves. -As Konductor evolves, the core module’s DRY patterns and consistent interfaces ensure that integrating new providers, applying compliance frameworks, and managing infrastructure remains straightforward and predictable, supporting the platform’s adaptability and future readiness. +With these stable standards in place, integrating new providers, applying compliance frameworks, and managing infrastructure remains straightforward, predictable, and aligned with the highest quality standards. diff --git a/docs/modules/core/TYPES.md b/docs/modules/core/TYPES.md index 445fe1c..f9ebef1 100644 --- a/docs/modules/core/TYPES.md +++ b/docs/modules/core/TYPES.md @@ -1,8 +1,8 @@ -# Core Module Types (revision08) +# Core Module Types (revision10) -Welcome to the **Core Module Types** reference. This document provides a detailed explanation of the `core/types` module after the DRY-focused refactoring (revision08). The refactoring introduces common models to reduce duplication and clarifies responsibilities of each class and interface. These types serve as the foundational building blocks for Konductor's Infrastructure as Code (IaC) environment, ensuring consistency, security, compliance, and maintainability. +Welcome to the **Core Module Types** reference. This document provides a detailed, technical explanation of the `core/types` module after the comprehensive Pydantic-only refactoring (revision10). In this iteration, all data structures previously defined with `TypedDict` have been replaced by Pydantic models, ensuring runtime validation, consistency, and richer modeling capabilities. This approach maintains the project's DRY principles, strict typing, and compliance/security integration, while enhancing the overall resilience and clarity of the type system. -This guide is suitable for engineers at all levels. By understanding these core types, you can design, implement, and extend Konductor IaC confidently, leveraging Pulumi's capabilities and industry best practices. +This guide is suitable for engineers at all levels. By understanding these pure Pydantic models and interfaces, you can design, implement, and extend Konductor IaC confidently, leveraging Pulumi and industry best practices for stable, secure, and compliant infrastructure deployments. ## Table of Contents @@ -14,7 +14,7 @@ This guide is suitable for engineers at all levels. By understanding these core - [GitInfo](#gitinfo) 5. [Compliance and Configuration Models](#compliance-and-configuration-models) - [OwnershipInfo, AtoConfig, ProjectOwnership](#ownershipinfo-atoconfig-projectownership) - - [StackConfig (Compliance Project)](#stackconfig-compliance-project) + - [ScipConfig (Project Compliance)](#scipconfig-project-compliance) - [FismaConfig and NistConfig](#fismaconfig-and-nistconfig) - [ComplianceConfig](#complianceconfig) - [BaseConfigModel](#baseconfigmodel) @@ -34,7 +34,7 @@ This guide is suitable for engineers at all levels. By understanding these core 10. [Stack-Level Types](#stack-level-types) - [GlobalMetadata](#globalmetadata) - [SourceRepository](#sourcerepository) - - [StackConfig for Outputs](#stackconfig-for-outputs) + - [StackConfig](#stackconfig) - [StackOutputs](#stackoutputs) - [ModuleDefaults](#moduledefaults) 11. [Best Practices and Usage Guidelines](#best-practices-and-usage-guidelines) @@ -43,199 +43,180 @@ This guide is suitable for engineers at all levels. By understanding these core ## Introduction -The `core/types` module defines a set of Pydantic models, TypedDicts, and Protocols that establish the core schema for configuration, compliance, resource metadata, and module lifecycle in Konductor's Pulumi-based IaC environment. This revision08 focuses on a DRY approach, reducing redundancy and consolidating common attributes into reusable models. +In revision10, the `core/types` module adopts a fully Pydantic-based approach, eliminating all `TypedDict` usages. This choice strengthens runtime validation, ensures uniform schema definitions, and enables advanced Pydantic features (defaults, validators, type coercion) across all data models. The result is a more robust, consistent, and maintainable type system that fully aligns with Pulumi’s stack-based IaC workflows and Konductor’s compliance and security mandates. ## Core Principles and Conventions -- **Strict Typing**: All models use strict typing, leveraging Pydantic and TypedDict for validation and clarity. -- **Pulumi Alignment**: Models and interfaces integrate smoothly with Pulumi stack configurations and resource management patterns. -- **Compliance and Security**: Compliance configurations (FISMA, NIST, ATO) are central. Production deployments enforce stricter requirements. -- **DRY and Maintainable**: Common fields (e.g., tags, labels, annotations) and patterns (e.g., enabling/disabling modules) are factored into shared base models. -- **No Hardcoded Secrets**: All sensitive data must be handled securely outside the code or through Pulumi secrets. +- **Strict Typing and Runtime Validation**: All models are Pydantic-based, providing runtime checks that catch invalid data early. +- **Pulumi-Native Alignment**: Patterns and interfaces mesh seamlessly with Pulumi stacks, ensuring consistent configuration, resource modeling, and lifecycle management. +- **Compliance and Security-First**: Models like `ComplianceConfig`, `FismaConfig`, and `NistConfig` ensure production stacks meet FISMA/NIST controls and ATO requirements out of the box. +- **DRY and Maintainable**: Common patterns—such as resource metadata and enabling/disabling modules—are encapsulated in shared base classes (e.g., `BaseConfigModel`). +- **No Hardcoded Secrets**: All sensitive data remains outside the code or handled via Pulumi secrets, following security best practices. ## Exceptions -- **CoreError**: Base exception for the core module. -- **ModuleLoadError**: Raised if a module cannot be loaded, aiding in clear error reporting and debugging. +- **CoreError**: The base exception for the core module, providing a foundation for more specific errors. +- **ModuleLoadError**: Raised when a module cannot be loaded, aiding in diagnosing configuration or dependency issues clearly. ## Metadata Models ### CommonMetadataFields **Class:** `CommonMetadataFields` -Provides `tags`, `labels`, and `annotations` fields reused across various models, ensuring uniform application of metadata. +Centralizes `tags`, `labels`, and `annotations`, ensuring every resource or configuration that requires metadata can use a common schema. This simplifies the propagation of governance, compliance, and organizational metadata across the entire IaC estate. ### GitInfo **Class:** `GitInfo` -Captures essential Git repository metadata (commit, branch, remote, optional release tag) at deployment time, allowing traceability of deployed code versions. +Captures repository commit hash, branch name, remote URL, and optional release tag. Tracing infrastructure changes back to specific code commits or release tags becomes effortless, improving auditability and helping with rollbacks or debugging. ## Compliance and Configuration Models ### OwnershipInfo, AtoConfig, ProjectOwnership -- **OwnershipInfo** and **ProjectOwnership**: TypedDict structures defining contact and ownership details. -- **AtoConfig**: Includes `authorized`, `eol`, and `last_touch` timestamps for production environments, ensuring no unapproved deployments. +All previously `TypedDict`-based structures are now Pydantic models: -### StackConfig (Compliance Project) +- **OwnershipInfo**: Defines contact details for project owners or teams. +- **AtoConfig**: Specifies Authority to Operate dates (authorized, end-of-life), critical for production compliance. +- **ProjectOwnership**: Bundles `OwnershipInfo` for both owner and operations, ensuring that accountability and escalation paths are clearly documented. -**Class:** `StackConfig` (within ComplianceConfig) -Represents the project's environment configuration, production flags, ownership details, and ATO configuration. This ensures environment-specific compliance and governance. +### ScipConfig (Project Compliance) + +**Class:** `ScipConfig` +Represents project-specific compliance settings: +- `environment` and `production` fields mark which environment and whether it’s production. +- `ownership` and `ato` fields enforce that production stacks must have explicit authorization. ### FismaConfig and NistConfig **Classes:** `FismaConfig`, `NistConfig` -Describe compliance levels and modes for FISMA and NIST frameworks. Enable or disable controls and handle exceptions. Together, they guide compliance-enforcing logic in the IaC pipelines. +These classes define compliance levels and exceptions for FISMA and NIST controls. Pydantic ensures that invalid compliance configurations never reach runtime deployments. ### ComplianceConfig **Class:** `ComplianceConfig` -Holds the combined `StackConfig`, `FismaConfig`, and `NistConfig`. Provides a `from_pulumi_config()` method to build configurations directly from Pulumi stack data. If production, authorized/eol must be present; otherwise defaults apply. +Consolidates `ScipConfig` (project details), `FismaConfig`, and `NistConfig`. +`from_pulumi_config()` enables runtime construction from Pulumi stack configs. Production stacks are validated to ensure authorized and eol dates are present, preventing non-compliant deployments. ### BaseConfigModel **Class:** `BaseConfigModel` -A base configuration model providing: -- `enabled`: Whether a module or configuration set is active -- `parent`: Parent module or resource reference -- `dependencies`: A list of module dependencies -- `configuration`: Arbitrary config dictionaries -- `metadata`: `CommonMetadataFields` for consistent tagging and labeling - -This base class minimizes duplication across various config models. +A fundamental building block providing common fields: `enabled`, `parent`, `dependencies`, `configuration`, and `metadata`. Using `CommonMetadataFields` and `BaseConfigModel` ensures all modules and configurations follow a consistent pattern. ### InitializationConfig **Class:** `InitializationConfig` -Extends `BaseConfigModel` with: -- `pulumi_config`: Pulumi configuration -- `stack_name`, `project_name`: Identifiers for stack and project -- `global_depends_on`: Global resource dependencies -- `git_info`: Git metadata -- `compliance_config`: Compliance settings -- `deployment_date_time`, `deployment_manager`: Deployment context details - -This ensures the core module initialization has all required info before any module deployments. +Extends `BaseConfigModel` to handle stack/project initialization details, compliance config, Git info, and other essential fields (`pulumi_config`, `stack_name`, `project_name`). This ensures that the core module has all necessary context before modules are deployed. ### ModuleRegistry and ModuleBase **Classes:** `ModuleRegistry`, `ModuleBase` -`ModuleRegistry` adds a required `name` field. -`ModuleBase` defines a standard pattern for modules, ensuring `name`, `enabled`, `dependencies`, and `metadata` are uniformly managed. +`ModuleRegistry` introduces a `name` field for identifying modules. +`ModuleBase` inherits from `BaseConfigModel` and enforces a `name` field, representing a deployable module. Together, they define a standard interface and lifecycle pattern for all modules. ## Resource and Deployment Models ### ResourceModel **Class:** `ResourceModel` -Represents a resource with: -- `name` -- `metadata` (via `CommonMetadataFields`) -- `created_at`, `updated_at` timestamps - -This simplifies resource representation, ensuring that all resources follow a consistent metadata pattern. +Represents a resource with `name`, `metadata`, and timestamps (`created_at`, `updated_at`). This standardized model ensures that all resources share a uniform metadata structure and are traceable through time. ### ModuleDeploymentResult **Class:** `ModuleDeploymentResult` -Captures outcomes of a module deployment: -- `config`: `ResourceModel` instances representing deployed configurations -- `compliance`: Compliance/tracing info -- `errors`: Any errors encountered -- `metadata`: Additional deployment-level metadata +Captures the outcome of a module deployment: +- `config`: List of `ResourceModel` instances for the deployed resources or configurations. +- `compliance`: Compliance/tracing information. +- `errors`: Issues encountered during deployment. +- `metadata`: Additional deployment-level metadata. -This helps clients and consumers understand what changed during deployment, what compliance context applies, and any encountered issues. +This provides a clear, consistent return value for module deployments, aiding in diagnostics and auditing. ## Interfaces and Protocols ### DeploymentContext **Protocol:** `DeploymentContext` -Defines the interface for executing deployments and retrieving the configuration model. This ensures modules can be deployed in a consistent manner regardless of underlying logic. +Defines how the deployment environment provides configurations and executes deployments. +`get_config()` returns the configuration model, and `deploy()` initiates the deployment, returning a `ModuleDeploymentResult`. ### ModuleInterface **Protocol:** `ModuleInterface` -Defines minimal lifecycle methods for a module: -- `validate()`: Validate its configuration -- `deploy(ctx)`: Deploy using the given context -- `dependencies()`: Declare module dependencies +Specifies that modules must implement: +- `validate(config)`: Check configuration correctness. +- `deploy(ctx)`: Perform the deployment using `DeploymentContext`. +- `dependencies()`: Declare dependencies on other modules. -This common interface ensures a uniform pattern across all modules, simplifying integration and testing. +This ensures a consistent lifecycle for all modules, irrespective of the provider or service they manage. ## Global Metadata Management ### MetadataSingleton **Class:** `MetadataSingleton` -A thread-safe singleton providing a global store for tags, labels, annotations, and git metadata. `setup_global_metadata(init_config)` uses this to populate global metadata once, ensuring a single source of truth. +A thread-safe singleton that holds global tags, labels, annotations, and Git metadata. Once set, these metadata fields are uniformly available to all modules and resources, ensuring governance and compliance data is globally consistent. ### InitConfig Protocol and setup_global_metadata() **Protocol:** `InitConfig` -Defines `project_name`, `stack_name`, `git_info`, and `metadata` needed to set global metadata. +Defines required fields (`project_name`, `stack_name`, `git_info`, `metadata`) needed for global metadata setup. + **Function:** `setup_global_metadata(init_config)` -Merges provided metadata with git and project info, populating the `MetadataSingleton`. +Merges global metadata and Git info into the `MetadataSingleton`, ensuring all resources and modules share a consistent global metadata context from the start. ## Configuration Management Utilities ### ConfigManager **Class:** `ConfigManager` -Handles loading and caching of Pulumi configuration: -- `get_config()`: Loads full stack config. -- `get_module_config(module_name)`: Retrieves a module's config. -- `get_enabled_modules()`: Lists modules that are enabled. +Centralizes configuration loading: +- `get_config()`: Retrieves the entire stack config once, caching results. +- `get_module_config()`: Extracts a specific module’s configuration. +- `get_enabled_modules()`: Identifies which modules have `enabled: true`. -By centralizing config retrieval, `ConfigManager` reduces repeated code and improves maintainability. +Unified config management reduces code duplication and improves reliability. ## Stack-Level Types ### GlobalMetadata **Class:** `GlobalMetadata` -Represents global tags, labels, and annotations at the stack level. Supports platform-wide governance by applying consistent metadata. +Stores global tags, labels, and annotations at the stack level. This ensures that platform-wide governance and compliance tags are consistently applied to all resources and configurations. ### SourceRepository **Class:** `SourceRepository` -Holds branch, commit, and remote URL (and optional tag) for auditing and rollback capabilities. Integrates with compliance and metadata to ensure full traceability of deployed code. - -### StackConfig for Outputs +Holds branch, commit, remote, and optional tag for the source repository. This makes it easy to correlate deployed infrastructure with a specific code commit or release, improving traceability. -**Note:** There are now two `StackConfig` concepts. One inside `ComplianceConfig.project` (representing compliance project environment), and one at the stack outputs level. For clarity: -- `ComplianceConfig.project` uses `StackConfig` to define environment and ATO details. -- `StackConfig` at the outputs level holds `compliance`, `metadata`, `source_repository`. +### StackConfig -This naming can be adjusted if desired. As is, it differentiates project compliance config (internal) from the top-level stack outputs `StackConfig`. +**Class:** `StackConfig` +Combines `compliance` (via `ComplianceConfig`), `metadata` (`GlobalMetadata`), and `source_repository` details into a coherent top-level model representing the entire stack’s configuration. ### StackOutputs **Class:** `StackOutputs` -Combines: -- `stack`: A `StackConfig` with compliance, metadata, and source repository details -- `secrets`: Optional sensitive data - -This final aggregation can be shared with other systems or teams. +Aggregates `StackConfig` and optional `secrets` into a final output model. This can be shared with other teams or pipelines, providing a complete view of the stack’s final state and compliance posture. ### ModuleDefaults **Class:** `ModuleDefaults` -Defines a baseline configuration (e.g., `enabled`, `config`) that modules can inherit. Ensures new modules have a consistent starting point. +Provides a baseline configuration for modules (`enabled`, `config`). New modules can start from these defaults, ensuring a consistent initial configuration across the platform. ## Best Practices and Usage Guidelines -- **Validate Early**: Use `from_pulumi_config()` methods and Pydantic validation to prevent malformed deployments. -- **Embrace Common Models**: Leverage `CommonMetadataFields` to avoid duplicating metadata fields across models. -- **Stick to Protocols**: Implement `ModuleInterface` and use `DeploymentContext` to ensure modules are deployable in a uniform manner. -- **Centralize Compliance**: Maintain compliance via `ComplianceConfig`, ensuring production stacks meet authorization and expiration requirements. -- **Utilize `MetadataSingleton`**: Set global metadata once, ensuring a single, authoritative source for tagging and labeling. +- **Validate Early**: Use `ComplianceConfig.from_pulumi_config()` and Pydantic validation to catch issues before deployments start. +- **Embrace Common Models**: `CommonMetadataFields` and `BaseConfigModel` reduce code duplication and ensure consistent patterns. +- **Implement Protocols**: Following `ModuleInterface` and `DeploymentContext` yields a uniform, testable, and maintainable module ecosystem. +- **Enforce Compliance Centrally**: `ComplianceConfig` guarantees that production stacks meet authorization and control requirements. +- **Set Metadata Once**: Use `setup_global_metadata()` and `MetadataSingleton` for a single authoritative source of metadata. ## Conclusion -The revision08 `core/types` module presents a cleaner, DRY, and more maintainable type system for Konductor IaC. By consolidating common fields, standardizing compliance and resource models, and leveraging protocols for modules and deployment contexts, this approach ensures a robust foundation that scales with complexity and supports multi-cloud environments gracefully. +With revision10, the `core/types` module fully adopts Pydantic-based models, enhancing runtime validation, consistency, and extensibility. The removal of TypedDicts clarifies schemas, while the established patterns continue to ensure a robust, secure, and maintainable IaC foundation. -Through these models, teams can confidently manage infrastructure configurations, apply compliance and metadata consistently, and integrate smoothly with Pulumi and other modules. +Armed with these models and interfaces, teams can confidently manage complex, multi-cloud environments, enforce compliance uniformly, and integrate new providers or compliance frameworks with minimal friction—ensuring Konductor’s future-readiness in a rapidly evolving infrastructure landscape. ## Related Documentation diff --git a/modules/aws/types/__init__.py b/modules/aws/types/__init__.py index e49e510..baa02fc 100644 --- a/modules/aws/types/__init__.py +++ b/modules/aws/types/__init__.py @@ -1,6 +1,8 @@ # ./modules/aws/types/__init__.py """ AWS Module Configuration Types + +This is still a large omnibus module for testing before splitting into submodules. """ from .components.eks import ( diff --git a/modules/core/types/__init__.py b/modules/core/types/__init__.py index c93a506..1e45fc4 100644 --- a/modules/core/types/__init__.py +++ b/modules/core/types/__init__.py @@ -1,34 +1,32 @@ # ./modules/core/types/__init__.py """ -Core module types implementation. +Core module types implementation (revision10). -This module defines all shared data classes, interfaces, and types used across the core and other modules. -It provides type-safe configuration structures using Pydantic models and TypedDicts, plus validation protocols. -We unify and streamline a large previously scattered codebase into a single coherent file for initial testing -before splitting into submodules. +This revision removes all TypedDict usage and uses Pydantic models exclusively. +This enhances runtime validation, consistency, and the ability to leverage Pydantic's +advanced features (validation, defaults, and type conversion) throughout the codebase. + +Changes from previous revision: +- Converted OwnershipInfo, AtoConfig, ProjectOwnership from TypedDict to Pydantic models. +- Ensured compliance and project config classes now consistently use Pydantic models. +- Maintained DRY principles, strict typing, and compliance/security considerations. Conventions and Requirements: -- Strict typing enforced by Pyright in strict mode. -- No `Any` type usage: replaced with `object` or more specific types. -- Pydantic models for configuration and validation. -- TypedDict for strictly typed dictionaries. -- Protocols for interfaces with clear docstrings and return annotations. -- Comprehensive docstrings for all public classes and methods. -- No broad `except Exception` blocks, except where explicitly needed to log and re-raise. -- Security and compliance considerations baked in. +- Strict typing with Pyright in strict mode. +- No use of Any or typed dicts; replaced with Pydantic models. +- Pydantic models for validation and runtime checks. +- Clear docstrings and alignment with Pulumi IaC best practices. +- No broad `except Exception` unless re-raised after logging. +- Security, compliance, and maintainability integrated. - No hardcoded credentials or secrets. -- Code follows PEP 8, PEP 257, and all documented style standards. -- After verification and testing, we will refactor into a proper module structure. +- Follow PEP 8, PEP 257, and all style/documentation standards. -This is a single large omnibus module to allow testing before final refactoring into multiple files. +This remains a large omnibus module for testing before splitting into submodules. """ from datetime import datetime, timezone, timedelta from threading import Lock -from typing import ( - Dict, List, Optional, Union, Protocol, ClassVar, cast -) -from typing_extensions import TypedDict +from typing import Dict, List, Optional, Union, Protocol, ClassVar, cast from pydantic import BaseModel, Field, validator, ConfigDict import pulumi from pulumi import Resource, log @@ -38,15 +36,35 @@ # Exceptions # ----------------------- class CoreError(Exception): - """Base exception for the core module.""" + """Base exception for core module errors.""" + pass class ModuleLoadError(CoreError): """Exception raised when a module cannot be loaded.""" + pass +# ----------------------- +# Common Metadata Fields +# ----------------------- +class CommonMetadataFields(BaseModel): + """ + Common metadata fields used across multiple classes for tagging, labeling, and annotating resources. + + Attributes: + tags: Key-value tags for resources. + labels: Key-value labels for resources. + annotations: Key-value annotations for resources. + """ + + tags: Dict[str, str] = Field(default_factory=dict, description="Key-value tags for resources.") + labels: Dict[str, str] = Field(default_factory=dict, description="Key-value labels for resources.") + annotations: Dict[str, str] = Field(default_factory=dict, description="Key-value annotations for resources.") + + # ----------------------- # Git Metadata # ----------------------- @@ -54,99 +72,138 @@ class GitInfo(BaseModel): """ Git Repository Metadata. - Provides structured information about the Git repository associated with the current project: - commit hash, branch name, and remote URL. + Attributes: + commit_hash: Current git commit hash or "unk" if unknown. + branch_name: Current git branch name or "unk" if unknown. + remote_url: Current git remote URL or "unk" if unknown. + release_tag: Optional release tag if the current commit is associated with a semver tag. """ - commit_hash: str = Field(default="unknown", description="git commit hash") - branch_name: str = Field(default="unknown", description="git branch name") - remote_url: str = Field(default="unknown", description="git remote URL") - def model_dump(self) -> Dict[str, str]: - """ - Convert the GitInfo instance to a dictionary format. + commit_hash: str = Field(default="unk", description="Current git commit hash") + branch_name: str = Field(default="unk", description="Current git branch name") + remote_url: str = Field(default="unk", description="Current git remote URL") + release_tag: Optional[str] = Field(default=None, description="Git release tag if applicable.") - Returns: - Dict[str, str]: Dictionary with keys 'commit', 'branch', and 'remote'. - """ - return { + def model_dump(self) -> Dict[str, str]: + """Return a dict with commit, branch, remote, and optional tag fields.""" + data = { "commit": self.commit_hash, "branch": self.branch_name, "remote": self.remote_url, } + if self.release_tag: + data["tag"] = self.release_tag + return data # ----------------------- -# Compliance Configuration +# Compliance and Project Configuration # ----------------------- -class OwnershipInfo(TypedDict): - """Owner contact information.""" +class OwnershipInfo(BaseModel): + """ + Owner contact information. + + Attributes: + name: Name of the owner or owning team. + contacts: List of contact methods (emails, slack channels, etc.). + """ + name: str - contacts: List[str] + contacts: List[str] = Field(default_factory=list) + +class AtoConfig(BaseModel): + """ + Authority to Operate (ATO) configuration for production deployments. + + Attributes: + id: ATO identifier. + authorized: ISO datetime the project was authorized to operate. + eol: ISO datetime marking end-of-life. + last_touch: ISO datetime of the last compliance touch. + """ -class AtoConfig(TypedDict): - """ATO (Authority to Operate) configuration.""" id: str - authorized: str # ISO datetime - eol: str # ISO datetime - last_touch: str # ISO datetime + authorized: str + eol: str + last_touch: str + +class ProjectOwnership(BaseModel): + """ + Project ownership structure, including owner and operations contacts. + + Attributes: + owner: OwnershipInfo for the project's owner. + operations: OwnershipInfo for operations team contacts. + """ -class ProjectOwnership(TypedDict): - """Project ownership structure.""" owner: OwnershipInfo operations: OwnershipInfo -class ProjectProviders(TypedDict): - """Cloud provider configuration.""" - name: List[str] - regions: List[str] +class ScipConfig(BaseModel): + """ + SCIP (project) configuration for compliance. + Attributes: + environment: Environment name, e.g. 'prod-us-west' + production: Boolean indicating if this is a production environment. + ownership: ProjectOwnership model with owner and operations info. + ato: AtoConfig with authorization details. + """ -class ProjectConfig(BaseModel): - """Project configuration.""" - environment: str = Field(..., description="e.g., 'prod-us-west'") - production: bool = Field(default=False) + environment: str = Field(..., description="Environment name, e.g. 'prod-us-west'") + production: bool = Field(default=False, description="Whether environment is production.") ownership: ProjectOwnership ato: AtoConfig - providers: ProjectProviders class FismaConfig(BaseModel): - """FISMA compliance configuration.""" - level: str = Field(default="moderate") - mode: str = Field(default="warn") + """ + FISMA compliance configuration. + + Attributes: + level: FISMA compliance level ('low', 'moderate', 'high'). + mode: Enforcement mode ('disabled', 'warn', or 'enforcing'). + """ + + level: str = Field(default="moderate", description="FISMA compliance level.") + mode: str = Field(default="warn", description="FISMA compliance mode.") class NistConfig(BaseModel): - """NIST compliance configuration.""" - auxiliary: List[str] = Field(default_factory=list) - exceptions: List[str] = Field(default_factory=list) + """ + NIST compliance configuration. + + Attributes: + auxiliary: Enabled auxiliary NIST controls. + exceptions: Disabled NIST exception controls. + """ + + auxiliary: List[str] = Field(default_factory=list, description="Enabled auxiliary NIST controls.") + exceptions: List[str] = Field(default_factory=list, description="Disabled NIST exception controls.") class ComplianceConfig(BaseModel): """ - Consolidated compliance configuration. - This maps to the 'compliance' section of the stack configuration. + Consolidated compliance configuration mapping to 'compliance' in stack config. + + Includes: + - project: ScipConfig with environment and ATO details. + - fisma: FismaConfig for FISMA compliance. + - nist: NistConfig for NIST compliance. """ - project: ProjectConfig + + project: ScipConfig fisma: FismaConfig nist: NistConfig @classmethod def from_pulumi_config(cls, config: pulumi.Config, timestamp: datetime) -> "ComplianceConfig": """ - Create ComplianceConfig from Pulumi config. - - If there's an error in loading or parsing config, returns a default config. - - Args: - config: Pulumi configuration object - timestamp: Current timestamp for configuration creation - - Returns: - ComplianceConfig: Configuration instance + Create ComplianceConfig from Pulumi stack config. + If environment is production, 'authorized' and 'eol' must be present or defaults are used. """ try: raw = config.get_object("compliance") or {} @@ -158,39 +215,27 @@ def from_pulumi_config(cls, config: pulumi.Config, timestamp: datetime) -> "Comp ato_data = cast(Dict[str, object], project_data.get("ato", {})) current_time = timestamp.isoformat() - if is_production: - # Production must have 'authorized' date - if "authorized" not in ato_data: - raise ValueError("Production environments require 'authorized' date in ATO configuration") - authorized_date = str(ato_data["authorized"]) - else: - authorized_date = current_time + if is_production and "authorized" not in ato_data: + raise ValueError("Production environments require 'authorized' date in ATO configuration") - if "eol" in ato_data: - eol_date = str(ato_data["eol"]) - else: - if is_production: - raise ValueError("Production environments require 'eol' date in ATO configuration") - else: - # Dev/non-prod gets 24h EOL from now - eol_date = (timestamp + timedelta(hours=24)).isoformat() + authorized_date = str(ato_data.get("authorized", current_time)) + eol_date = str(ato_data.get("eol", (timestamp + timedelta(hours=24)).isoformat())) - # Ensure project data structure + # If project not defined, provide defaults if "project" not in compliance_data: compliance_data["project"] = { "environment": "dev", "production": False, "ownership": { "owner": {"name": "default", "contacts": []}, - "operations": {"name": "default", "contacts": []} + "operations": {"name": "default", "contacts": []}, }, "ato": { "id": "default", "authorized": authorized_date, "eol": eol_date, - "last_touch": current_time + "last_touch": current_time, }, - "providers": {"name": [], "regions": []} } else: proj = cast(Dict[str, object], compliance_data["project"]) @@ -198,12 +243,11 @@ def from_pulumi_config(cls, config: pulumi.Config, timestamp: datetime) -> "Comp "id": str(ato_data.get("id", "dev")), "authorized": authorized_date, "eol": eol_date, - "last_touch": current_time + "last_touch": current_time, } if "fisma" not in compliance_data: compliance_data["fisma"] = {"level": "moderate", "mode": "warn"} - if "nist" not in compliance_data: compliance_data["nist"] = {"auxiliary": [], "exceptions": []} @@ -216,110 +260,76 @@ def from_pulumi_config(cls, config: pulumi.Config, timestamp: datetime) -> "Comp @classmethod def create_default(cls) -> "ComplianceConfig": """ - Create a default compliance configuration. - - Returns: - ComplianceConfig: Default configuration instance + Create a default compliance configuration for a non-production dev environment. """ timestamp = datetime.now(timezone.utc) current_time = timestamp.isoformat() return cls( - project=ProjectConfig( + project=ScipConfig( environment="dev", production=False, - ownership={ - "owner": {"name": "default", "contacts": []}, - "operations": {"name": "default", "contacts": []} - }, - ato={ - "id": "default", - "authorized": current_time, - "eol": (timestamp + timedelta(hours=24)).isoformat(), - "last_touch": current_time - }, - providers={"name": [], "regions": []} - ), - fisma=FismaConfig( - level="low", - mode="warn" + ownership=ProjectOwnership( + owner=OwnershipInfo(name="default", contacts=[]), + operations=OwnershipInfo(name="default", contacts=[]), + ), + ato=AtoConfig( + id="default", + authorized=current_time, + eol=(timestamp + timedelta(hours=24)).isoformat(), + last_touch=current_time, + ), ), - nist=NistConfig( - auxiliary=[], - exceptions=[] - ) + fisma=FismaConfig(level="low", mode="warn"), + nist=NistConfig(auxiliary=[], exceptions=[]), ) - def model_dump(self) -> Dict[str, object]: - """Convert the model to a dictionary.""" - return { - "project": self.project.dict(), - "fisma": self.fisma.dict(), - "nist": self.nist.dict() - } - # ----------------------- -# Base Resource and Config Types +# Configuration Base Models # ----------------------- -class ResourceBase(BaseModel): +class BaseConfigModel(BaseModel): """ - Base for all infrastructure resources. - Provides a minimal set of fields common to all resources. + Base configuration model providing common fields: + - enabled: Whether the module or config is active. + - parent: Optional parent identifier. + - dependencies: List of dependencies. + - configuration: Arbitrary configuration dictionary. + - metadata: CommonMetadataFields for tags, labels, annotations. """ - name: str = Field(..., description="Resource name") - urn: Optional[str] = Field(None, description="Pulumi resource identifier") - id: Optional[str] = Field(None, description="Provider-assigned resource ID") - metadata: Dict[str, object] = Field( - default_factory=lambda: { - "tags": {}, - "labels": {}, - "annotations": {}, - }, - description="Runtime resource metadata, including tags, labels, annotations." - ) - created_at: str = Field( - default_factory=lambda: datetime.now(timezone.utc).isoformat(), - description="Resource creation timestamp" - ) - -class ConfigBase(BaseModel): - """ - Base for all configurations. - Includes enabling/disabling a module, dependencies, and user config. - """ enabled: bool = Field(default=False) parent: Optional[str] = None dependencies: List[str] = Field(default_factory=list) configuration: Dict[str, object] = Field(default_factory=dict) - metadata: Dict[str, object] = Field( - default_factory=lambda: {"labels": {}, "annotations": {}} - ) + metadata: CommonMetadataFields = Field(default_factory=CommonMetadataFields) -class InitializationConfig(ConfigBase): +class InitializationConfig(BaseConfigModel): """ Configuration for core module initialization. - Provides details necessary for initializing the core module, - including Pulumi settings, stack/project names, providers, and compliance info. + + Adds: + - pulumi_config: Pulumi configuration object or dict + - stack_name, project_name: Identifiers + - global_depends_on: Global resource dependencies + - git_info: Git repository metadata + - compliance_config: Compliance configuration + - metadata: CommonMetadataFields + - deployment_date_time: Timestamp of deployment + - deployment_manager: Optional deployment manager object """ + pulumi_config: Union[pulumi.Config, Dict[str, object]] stack_name: str project_name: str global_depends_on: List[Resource] = Field(default_factory=list) - platform_providers: Dict[str, object] = Field(default_factory=dict) git_info: GitInfo = Field(default_factory=GitInfo) compliance_config: ComplianceConfig = Field( - default_factory=ComplianceConfig.create_default, - description="Compliance configuration for the deployment" - ) - metadata: Dict[str, Dict[str, str]] = Field( - default_factory=lambda: {"labels": {}, "annotations": {}} - ) - deployment_date_time: str = Field( - default_factory=lambda: datetime.now(timezone.utc).isoformat() + default_factory=ComplianceConfig.create_default, description="Compliance configuration for the deployment" ) + metadata: CommonMetadataFields = Field(default_factory=CommonMetadataFields) + deployment_date_time: str = Field(default_factory=lambda: datetime.now(timezone.utc).isoformat()) deployment_manager: Optional[object] = None model_config = ConfigDict(arbitrary_types_allowed=True) @@ -331,29 +341,42 @@ def validate_configuration(cls, v: Dict[str, object]) -> Dict[str, object]: return v -class ModuleRegistry(ConfigBase): +class ModuleRegistry(BaseConfigModel): """ Module registration information. + Requires a 'name' field. """ + name: str - parent: Optional[str] = None - dependencies: List[str] = Field(default_factory=list) -class ModuleBase(BaseModel): +class ModuleBase(BaseConfigModel): """ Base class for all modules. - Represents a deployable unit of infrastructure. + Adds a required 'name' field, representing a deployable unit of infrastructure. """ + name: str = Field(..., description="Module name") - enabled: bool = Field(default=False, description="Whether module is enabled") - parent: Optional[str] = Field(None, description="Parent resource name") - dependencies: List[str] = Field(default_factory=list, description="Module dependencies") - config: Dict[str, object] = Field(default_factory=dict, description="Module configuration") - metadata: Dict[str, object] = Field( - default_factory=lambda: {"labels": {}, "annotations": {}}, - description="Module metadata" - ) + + +# ----------------------- +# Resource Model +# ----------------------- +class ResourceModel(BaseModel): + """ + Resource model representing infrastructure resources. + + Attributes: + name: Resource name + metadata: Common metadata fields + created_at: Timestamp of resource creation + updated_at: Timestamp of last update + """ + + name: str = Field(..., description="Resource name") + metadata: CommonMetadataFields = Field(default_factory=CommonMetadataFields) + created_at: datetime = Field(default_factory=datetime.utcnow) + updated_at: datetime = Field(default_factory=datetime.utcnow) # ----------------------- @@ -364,30 +387,29 @@ class ModuleDeploymentResult(BaseModel): Results from a module deployment operation. Attributes: - success: Whether the deployment was successful - config: List of ResourceBase configurations for sharing + config: List of ResourceModel configurations compliance: List of compliance/tracing info - resources: Dictionary of resources created or updated - errors: List of error messages if any occurred + errors: List of errors encountered metadata: Additional deployment metadata """ - success: bool - config: List[ResourceBase] = Field(default_factory=list) + + config: List[ResourceModel] = Field(default_factory=list) compliance: List[str] = Field(default_factory=list) - resources: Dict[str, object] = Field(default_factory=dict) errors: List[str] = Field(default_factory=list) metadata: Dict[str, object] = Field(default_factory=dict) class DeploymentContext(Protocol): - """Protocol defining the deployment context interface.""" + """ + Protocol defining the deployment context interface. + """ - def get_config(self) -> ConfigBase: + def get_config(self) -> BaseConfigModel: """ Retrieve the configuration for the deployment. Returns: - ConfigBase: The configuration model for this deployment context. + BaseConfigModel: The configuration model. """ ... @@ -396,7 +418,7 @@ def deploy(self) -> ModuleDeploymentResult: Execute the deployment and return results. Returns: - ModuleDeploymentResult: The result of the deployment process. + ModuleDeploymentResult: Deployment result data. """ ... @@ -404,72 +426,23 @@ def deploy(self) -> ModuleDeploymentResult: class ModuleInterface(Protocol): """ Module interface defining lifecycle and validation methods. - """ - - def validate_config(self, config: ConfigBase) -> List[str]: - """ - Validate module configuration and return a list of errors if any. - - Args: - config (ConfigBase): The configuration to validate. - - Returns: - List[str]: A list of validation error messages, empty if valid. - """ - ... - - def validate_resources(self, resources: List[ResourceBase]) -> List[str]: - """ - Validate module resources and return a list of errors if any. - - Args: - resources (List[ResourceBase]): The resources to validate. - - Returns: - List[str]: A list of validation error messages, empty if valid. - """ - ... - def pre_deploy_check(self) -> List[str]: - """ - Run checks before deploying resources. + Methods: + validate(config): Validate module configuration + deploy(ctx): Deploy the module using a given context + dependencies(): Return a list of dependent modules + """ - Returns: - List[str]: A list of pre-deployment error messages, empty if valid. - """ + def validate(self, config: BaseConfigModel) -> List[str]: + """Validate module configuration and return errors if any.""" ... def deploy(self, ctx: DeploymentContext) -> ModuleDeploymentResult: - """ - Deploy the module using the provided deployment context. - - Args: - ctx (DeploymentContext): The deployment context. - - Returns: - ModuleDeploymentResult: The result of the module deployment. - """ + """Deploy the module using the provided context.""" ... - def post_deploy_validation(self, result: ModuleDeploymentResult) -> List[str]: - """ - Validate the deployment result after resources are created. - - Args: - result (ModuleDeploymentResult): The result to validate. - - Returns: - List[str]: A list of post-deployment validation errors, empty if valid. - """ - ... - - def get_dependencies(self) -> List[str]: - """ - Return a list of module dependencies. - - Returns: - List[str]: The names of dependent modules. - """ + def dependencies(self) -> List[str]: + """Return a list of module dependencies.""" ... @@ -478,8 +451,9 @@ def get_dependencies(self) -> List[str]: # ----------------------- class MetadataSingleton: """ - Global metadata thread-safe singleton class. - Provides a centralized store for all cross-module metadata. + Global metadata thread-safe singleton. + + Stores global tags, labels, annotations, git metadata, and module-specific metadata. """ _instance: Optional["MetadataSingleton"] = None @@ -561,11 +535,18 @@ def get_module_metadata(self, module_name: str) -> Dict[str, object]: class InitConfig(Protocol): """ Protocol for initial configuration used when setting up global metadata. + + Attributes: + project_name: Name of the project + stack_name: Name of the stack + git_info: GitInfo model with repository metadata + metadata: CommonMetadataFields for tags, labels, annotations """ + project_name: str stack_name: str git_info: GitInfo - metadata: Dict[str, Dict[str, str]] + metadata: CommonMetadataFields # ----------------------- @@ -574,17 +555,15 @@ class InitConfig(Protocol): def setup_global_metadata(init_config: InitConfig) -> None: """ Initialize global metadata for resources using the provided InitConfig. - - This sets global tags, labels, annotations, and git metadata in the MetadataSingleton. """ try: metadata = MetadataSingleton() git_info = init_config.git_info.model_dump() git_labels = { - "git.commit": git_info["commit"], - "git.branch": git_info["branch"], - "git.repository": git_info["remote"], + "git.commit": git_info.get("commit", "unk"), + "git.branch": git_info.get("branch", "unk"), + "git.repository": git_info.get("remote", "unk"), } base_metadata = { @@ -594,29 +573,12 @@ def setup_global_metadata(init_config: InitConfig) -> None: "git": git_info, } - base_tags = { - **base_metadata, - **init_config.metadata.get("tags", {}), - } - - base_labels = { - **base_metadata, - **init_config.metadata.get("labels", {}), - } - - base_annotations = { - **base_metadata, - **init_config.metadata.get("annotations", {}), - } - - all_labels = { - **base_labels, - **git_labels, - **(init_config.metadata.get("labels", {})), - } + base_tags = {**base_metadata, **init_config.metadata.tags} + base_labels = {**base_metadata, **init_config.metadata.labels, **git_labels} + base_annotations = {**base_metadata, **init_config.metadata.annotations} metadata.set_tags({k: str(v) for k, v in base_tags.items()}) - metadata.set_labels({k: str(v) for k, v in all_labels.items()}) + metadata.set_labels({k: str(v) for k, v in base_labels.items()}) metadata.set_annotations({k: str(v) for k, v in base_annotations.items()}) metadata.set_git_metadata({k: str(v) for k, v in git_info.items()}) @@ -631,8 +593,7 @@ def setup_global_metadata(init_config: InitConfig) -> None: # ----------------------- class ConfigManager: """ - Configuration Management Class - Retrieves and caches Pulumi configuration for modules. + Configuration Management class for retrieving and caching Pulumi configuration. """ def __init__(self) -> None: @@ -641,7 +602,7 @@ def __init__(self) -> None: self._module_configs: Dict[str, Dict[str, object]] = {} def get_config(self) -> Dict[str, object]: - """Get the full configuration from Pulumi stack config.""" + """Retrieve full configuration from Pulumi stack config, caching results.""" if self._config_cache is None: raw_config: Dict[str, object] = {} for module_name in ["aws", "kubernetes"]: @@ -650,35 +611,29 @@ def get_config(self) -> Dict[str, object]: raw_config[module_name] = module_cfg except KeyError as e: log.debug(f"No config found for module {module_name}: {e}") - self._config_cache = raw_config log.debug(f"Loaded config: {raw_config}") - return cast(Dict[str, object], self._config_cache) def get_module_config(self, module_name: str) -> Dict[str, object]: - """Get configuration for a specific module.""" + """Get configuration for a specific module from cached config.""" if module_name in self._module_configs: return self._module_configs[module_name] - config = self.get_config() module_config = cast(Dict[str, object], config.get(module_name, {})) - self._module_configs[module_name] = module_config log.debug(f"Module {module_name} config: {module_config}") return module_config def get_enabled_modules(self) -> List[str]: - """Get list of enabled modules.""" + """Return a list of enabled modules from the cached configuration.""" enabled_modules: List[str] = [] config = self.get_config() - for module_name in ["aws", "kubernetes"]: module_config = cast(Dict[str, object], config.get(module_name, {})) if bool(module_config.get("enabled", False)): enabled_modules.append(module_name) log.info(f"Module {module_name} is enabled") - log.info(f"Enabled modules: {enabled_modules}") return enabled_modules @@ -688,16 +643,19 @@ def get_enabled_modules(self) -> List[str]: # ----------------------- class GlobalMetadata(BaseModel): """ - Global metadata structure. - Used for top-level stack outputs metadata. + Global metadata structure for top-level stack outputs. """ + tags: Dict[str, str] = Field(default_factory=dict, description="Global public cloud resource tags") labels: Dict[str, str] = Field(default_factory=dict, description="Global Kubernetes labels") annotations: Dict[str, str] = Field(default_factory=dict, description="Global Kubernetes annotations") class SourceRepository(BaseModel): - """Source repository information.""" + """ + Source repository information. + """ + branch: str commit: str remote: str @@ -707,7 +665,9 @@ class SourceRepository(BaseModel): class StackConfig(BaseModel): """ Root stack configuration mapping. + Holds compliance, metadata, and source repository info. """ + compliance: ComplianceConfig metadata: GlobalMetadata source_repository: SourceRepository @@ -717,53 +677,37 @@ class StackOutputs(BaseModel): """ Complete stack outputs structure. """ + stack: StackConfig - secrets: Optional[Dict[str, object]] = Field( - default=None, - description="Sensitive information like credentials or tokens" - ) + secrets: Optional[Dict[str, object]] = Field(default=None, description="Sensitive credentials or tokens") class ModuleDefaults(BaseModel): """ Default configuration for modules. """ + enabled: bool = Field(default=False, description="Whether the module is enabled") config: Dict[str, object] = Field(default_factory=dict, description="Module-specific configuration") -class ResourceMetadata(ResourceBase): - """ - Resource metadata structure extending ResourceBase. - Adds tags, labels, annotations, and timestamps. - """ - tags: Dict[str, str] = Field(default_factory=dict) - labels: Dict[str, str] = Field(default_factory=dict) - annotations: Dict[str, str] = Field(default_factory=dict) - created_at: datetime = Field(default_factory=datetime.utcnow) - updated_at: datetime = Field(default_factory=datetime.utcnow) - - -# ----------------------- -# Public API (exports) -# ----------------------- __all__ = [ "CoreError", "ModuleLoadError", + "CommonMetadataFields", "GitInfo", - "ProjectConfig", - "FismaConfig", - "NistConfig", - "ComplianceConfig", "OwnershipInfo", "AtoConfig", "ProjectOwnership", - "ProjectProviders", - "ResourceBase", - "ConfigBase", + "ScipConfig", + "FismaConfig", + "NistConfig", + "ComplianceConfig", + "BaseConfigModel", "InitializationConfig", "ModuleRegistry", "ModuleBase", + "ResourceModel", "ModuleDeploymentResult", "DeploymentContext", "ModuleInterface", @@ -776,5 +720,4 @@ class ResourceMetadata(ResourceBase): "StackConfig", "StackOutputs", "ModuleDefaults", - "ResourceMetadata", ] diff --git a/modules/core/types/base.py b/modules/core/types/base.py index c6cf1cc..18cccbb 100644 --- a/modules/core/types/base.py +++ b/modules/core/types/base.py @@ -1,701 +1,6 @@ -# ./modules/core/types/__init__.py +# ./modules/core/types/base.py """ -Core module types implementation (revision07). +Base types for the core module. -This revision reduces duplication (DRY) by: - -- Introducing a shared `CommonMetadataFields` model for tags, labels, annotations. -- Unifying repeated fields in configuration-related classes into a common base model (`BaseConfigModel`). -- Removing redundant `ResourceBase` in favor of a single `ResourceModel` that includes all needed resource fields. -- Ensuring that all attributes from the previous revision remain present but organized more coherently. - -Conventions and Requirements: -- Strict typing with Pyright in strict mode. -- No `Any` type usage: replaced with `object` or more specific types. -- Pydantic models for validation and TypedDict where appropriate. -- Clear docstrings and alignment with Pulumi IaC best practices. -- No broad `except Exception` unless re-raised after logging. -- Security, compliance, and maintainability baked in. -- No hardcoded credentials or secrets. -- Follow PEP 8, PEP 257, and all style/documentation standards. - -This is still a large omnibus module for testing before splitting into submodules. +This module defines the base types for the core module. """ - -from datetime import datetime, timezone, timedelta -from threading import Lock -from typing import Dict, List, Optional, Union, Protocol, ClassVar, cast -from typing_extensions import TypedDict -from pydantic import BaseModel, Field, validator, ConfigDict -import pulumi -from pulumi import Resource, log - - -# ----------------------- -# Exceptions -# ----------------------- -class CoreError(Exception): - """Base exception for the core module errors.""" - - pass - - -class ModuleLoadError(CoreError): - """Exception raised when a module cannot be loaded.""" - - pass - - -# ----------------------- -# Common Metadata Fields -# ----------------------- -class CommonMetadataFields(BaseModel): - """ - Common metadata fields used repeatedly across multiple classes. - Provides a standard structure for tags, labels, and annotations. - - Attributes: - tags: Dict of key-value pairs for resource tagging. - labels: Dict of key-value pairs for labeling (e.g., Kubernetes labels). - annotations: Dict of key-value pairs for annotations. - """ - - tags: Dict[str, str] = Field(default_factory=dict, description="Key-value tags for resources.") - labels: Dict[str, str] = Field(default_factory=dict, description="Key-value labels for resources.") - annotations: Dict[str, str] = Field(default_factory=dict, description="Key-value annotations for resources.") - - -# ----------------------- -# Git Metadata -# ----------------------- -class GitInfo(BaseModel): - """ - Git Repository Metadata. - Provides commit hash, branch name, and remote URL of the current project. - - Default values are "unk" to indicate unknown state if not provided. - - Attributes: - commit_hash: Current git commit hash - branch_name: Current git branch name - remote_url: Current git remote URL - release_tag: Current git release tag if current commit is a valid semver tag - """ - - commit_hash: str = Field(default="unk", description="Git commit hash") - branch_name: str = Field(default="unk", description="Git branch name") - remote_url: str = Field(default="unk", description="Git remote URL") - release_tag: Optional[str] = Field(default=None, description="Git release tag") - - def model_dump(self) -> Dict[str, str]: - """Return a dict with commit, branch, and remote fields.""" - return { - "commit": self.commit_hash, - "branch": self.branch_name, - "remote": self.remote_url, - "tag": self.release_tag, - } - - -# ----------------------- -# Compliance Configuration -# ----------------------- -class OwnershipInfo(TypedDict): - """Owner contact information.""" - - name: str - contacts: List[str] - - -class AtoConfig(TypedDict): - """ATO (Authority to Operate) configuration with required timestamps (ISO).""" - - id: str - authorized: str # ISO datetime - eol: str # ISO datetime - last_touch: str # ISO datetime - - -class ProjectOwnership(TypedDict): - """Project ownership structure with owner and operations info.""" - - owner: OwnershipInfo - operations: OwnershipInfo - - -class StackConfig(BaseModel): - """ - Pulumi stack configuration. - Defines environment, production flag, ownership, and ATO details. - """ - - environment: str = Field(..., description="Arbitrary environment name, e.g. 'prod-us-west'") - production: bool = Field(default=False, description="Is this a production environment?") - ownership: ProjectOwnership - ato: AtoConfig - - -class FismaConfig(BaseModel): - """ - FISMA compliance configuration. - """ - - level: str = Field(default="moderate", description="FISMA compliance level: 'low', 'moderate', or 'high'") - mode: str = Field(default="warn", description="FISMA compliance mode: 'disabled', 'warn', or 'enforcing'") - - -class NistConfig(BaseModel): - """ - NIST compliance configuration. - """ - - auxiliary: List[str] = Field(default_factory=list, description="Enabled auxiliary NIST controls.") - exceptions: List[str] = Field(default_factory=list, description="Disabled NIST control exceptions.") - - -class ComplianceConfig(BaseModel): - """ - Consolidated compliance configuration mapping to 'compliance' in stack config. - Includes Project, FISMA, and NIST configurations. - """ - - project: StackConfig - fisma: FismaConfig - nist: NistConfig - - @classmethod - def from_pulumi_config(cls, config: pulumi.Config, timestamp: datetime) -> "ComplianceConfig": - """ - Instantiate ComplianceConfig from Pulumi config. - If production, authorized and eol dates must be present; otherwise defaults are used. - """ - try: - raw = config.get_object("compliance") or {} - compliance_data = cast(Dict[str, object], raw) - - project_data = cast(Dict[str, object], compliance_data.get("project", {})) - is_production = bool(project_data.get("production", False)) - - ato_data = cast(Dict[str, object], project_data.get("ato", {})) - current_time = timestamp.isoformat() - - if is_production and "authorized" not in ato_data: - raise ValueError("Production environments require 'authorized' date in ATO configuration") - - authorized_date = str(ato_data.get("authorized", current_time)) - eol_date = str(ato_data.get("eol", (timestamp + timedelta(hours=24)).isoformat())) - - # Ensure project structure - if "project" not in compliance_data: - compliance_data["project"] = { - "environment": "dev", - "production": False, - "ownership": { - "owner": {"name": "default", "contacts": []}, - "operations": {"name": "default", "contacts": []}, - }, - "ato": { - "id": "default", - "authorized": authorized_date, - "eol": eol_date, - "last_touch": current_time, - }, - } - else: - proj = cast(Dict[str, object], compliance_data["project"]) - proj["ato"] = { - "id": str(ato_data.get("id", "dev")), - "authorized": authorized_date, - "eol": eol_date, - "last_touch": current_time, - } - - if "fisma" not in compliance_data: - compliance_data["fisma"] = {"level": "moderate", "mode": "warn"} - if "nist" not in compliance_data: - compliance_data["nist"] = {"auxiliary": [], "exceptions": []} - - return cls(**compliance_data) - - except (KeyError, ValueError) as e: - log.error(f"Failed to load compliance config: {str(e)}") - return cls.create_default() - - @classmethod - def create_default(cls) -> "ComplianceConfig": - """ - Create a default compliance configuration for non-production dev environment. - """ - timestamp = datetime.now(timezone.utc) - current_time = timestamp.isoformat() - - return cls( - project=StackConfig( - environment="dev", - production=False, - ownership={ - "owner": {"name": "default", "contacts": []}, - "operations": {"name": "default", "contacts": []}, - }, - ato={ - "id": "default", - "authorized": current_time, - "eol": (timestamp + timedelta(hours=24)).isoformat(), - "last_touch": current_time, - }, - ), - fisma=FismaConfig(level="low", mode="warn"), - nist=NistConfig(auxiliary=[], exceptions=[]), - ) - - -# ----------------------- -# Configuration Base Models -# ----------------------- -class BaseConfigModel(BaseModel): - """ - Base configuration model providing common fields: - - enabled: Whether the module or config is active - - parent: Optional parent identifier - - dependencies: List of dependencies - - configuration: Arbitrary configuration dictionary - - metadata: CommonMetadataFields for tags, labels, annotations - """ - - enabled: bool = Field(default=False) - parent: Optional[str] = None - dependencies: List[str] = Field(default_factory=list) - configuration: Dict[str, object] = Field(default_factory=dict) - metadata: CommonMetadataFields = Field(default_factory=CommonMetadataFields) - - -class InitializationConfig(BaseConfigModel): - """ - Configuration for core module initialization. - Adds fields necessary for stack/project initialization, compliance, and git info. - """ - - pulumi_config: Union[pulumi.Config, Dict[str, object]] - stack_name: str - project_name: str - global_depends_on: List[Resource] = Field(default_factory=list) - git_info: GitInfo = Field(default_factory=GitInfo) - compliance_config: ComplianceConfig = Field( - default_factory=ComplianceConfig.create_default, description="Compliance configuration for the deployment" - ) - # Overriding metadata to store tags/labels/annotations as needed - # But can remain CommonMetadataFields for uniformity. - metadata: CommonMetadataFields = Field(default_factory=CommonMetadataFields) - deployment_date_time: str = Field(default_factory=lambda: datetime.now(timezone.utc).isoformat()) - deployment_manager: Optional[object] = None - model_config = ConfigDict(arbitrary_types_allowed=True) - - @validator("configuration") - def validate_configuration(cls, v: Dict[str, object]) -> Dict[str, object]: - """Validate that all configuration values are dictionaries.""" - if not all(isinstance(config, dict) for config in v.values()): - raise ValueError("All configurations must be dictionaries") - return v - - -class ModuleRegistry(BaseConfigModel): - """ - Module registration information. - Adds a required 'name' field. - """ - - name: str - - -class ModuleBase(BaseConfigModel): - """ - Base class for all modules. - Represents a deployable unit of infrastructure. - Inherits from BaseConfigModel. - Adds a required 'name' field. - """ - - name: str = Field(..., description="Module name") - - -# ----------------------- -# Resource Model -# ----------------------- -class ResourceModel(BaseModel): - """ - Resource model representing infrastructure resources. - - Attributes: - name: The resource's name. - metadata: Common metadata fields (tags, labels, annotations). - created_at: Resource creation timestamp. - updated_at: Resource last update timestamp. - """ - - name: str = Field(..., description="Resource name") - metadata: CommonMetadataFields = Field(default_factory=CommonMetadataFields) - created_at: datetime = Field(default_factory=datetime.utcnow) - updated_at: datetime = Field(default_factory=datetime.utcnow) - - -# ----------------------- -# Deployment Result and Interfaces -# ----------------------- -class ModuleDeploymentResult(BaseModel): - """ - Results from a module deployment operation. - - Attributes: - config: List of ResourceModel configurations for sharing - compliance: List of compliance/tracing info - errors: List of error messages if any occurred - metadata: Additional deployment metadata (tags, labels, annotations, etc.) - """ - - config: List[ResourceModel] = Field(default_factory=list) - compliance: List[str] = Field(default_factory=list) - errors: List[str] = Field(default_factory=list) - metadata: Dict[str, object] = Field(default_factory=dict) - - -class DeploymentContext(Protocol): - """ - Protocol defining the deployment context interface. - """ - - def get_config(self) -> BaseConfigModel: - """ - Retrieve the configuration for the deployment. - - Returns: - BaseConfigModel: The configuration model for this deployment context. - """ - ... - - def deploy(self) -> ModuleDeploymentResult: - """ - Execute the deployment and return results. - - Returns: - ModuleDeploymentResult: The result of the deployment process. - """ - ... - - -class ModuleInterface(Protocol): - """ - Module interface defining lifecycle and validation methods. - Minimalistic: just validate config and deploy, plus dependencies listing. - """ - - def validate(self, config: BaseConfigModel) -> List[str]: - """ - Validate module configuration and return a list of errors if any. - - Args: - config (BaseConfigModel): The configuration to validate. - - Returns: - List[str]: Validation error messages, empty if valid. - """ - ... - - def deploy(self, ctx: DeploymentContext) -> ModuleDeploymentResult: - """ - Deploy the module using the provided deployment context. - - Args: - ctx (DeploymentContext): The deployment context. - - Returns: - ModuleDeploymentResult: Deployment results. - """ - ... - - def dependencies(self) -> List[str]: - """ - Return a list of module dependencies. - - Returns: - List[str]: The names of dependent modules. - """ - ... - - -# ----------------------- -# Metadata Singleton -# ----------------------- -class MetadataSingleton: - """ - Global metadata thread-safe singleton class. - Provides a centralized store for cross-module metadata. - - Attributes: - global_tags: Global tags - global_labels: Global labels - global_annotations: Global annotations - git_metadata: Git metadata - modules_metadata: Module-specific metadata - """ - - _instance: Optional["MetadataSingleton"] = None - _lock: ClassVar[Lock] = Lock() - - def __new__(cls) -> "MetadataSingleton": - if cls._instance is None: - with cls._lock: - if cls._instance is None: - instance = super().__new__(cls) - instance.__init__() # type: ignore - cls._instance = instance - return cls._instance - - def __init__(self) -> None: - if not hasattr(self, "_initialized"): - self._global_tags: Dict[str, str] = {} - self._global_labels: Dict[str, str] = {} - self._global_annotations: Dict[str, str] = {} - self._git_metadata: Dict[str, object] = {} - self._modules_metadata: Dict[str, Dict[str, object]] = {} - self._initialized = True - - @property - def global_tags(self) -> Dict[str, str]: - with self._lock: - return self._global_tags.copy() - - @property - def global_labels(self) -> Dict[str, str]: - with self._lock: - return self._global_labels.copy() - - @property - def global_annotations(self) -> Dict[str, str]: - with self._lock: - return self._global_annotations.copy() - - @property - def git_metadata(self) -> Dict[str, object]: - with self._lock: - return self._git_metadata.copy() - - @property - def modules_metadata(self) -> Dict[str, Dict[str, object]]: - with self._lock: - return self._modules_metadata.copy() - - def set_tags(self, tags: Dict[str, str]) -> None: - with self._lock: - self._global_tags.update(tags) - - def set_labels(self, labels: Dict[str, str]) -> None: - with self._lock: - self._global_labels.update(labels) - - def set_annotations(self, annotations: Dict[str, str]) -> None: - with self._lock: - self._global_annotations.update(annotations) - - def set_git_metadata(self, metadata: Dict[str, str]) -> None: - with self._lock: - self._git_metadata.update(metadata) - - def set_module_metadata(self, module_name: str, metadata: Dict[str, object]) -> None: - with self._lock: - if module_name not in self._modules_metadata: - self._modules_metadata[module_name] = {} - self._modules_metadata[module_name].update(metadata) - - def get_module_metadata(self, module_name: str) -> Dict[str, object]: - with self._lock: - return self._modules_metadata.get(module_name, {}).copy() - - -# ----------------------- -# InitConfig Protocol (for setup_global_metadata) -# ----------------------- -class InitConfig(Protocol): - """ - Protocol for initial configuration used when setting up global metadata. - """ - - project_name: str - stack_name: str - git_info: GitInfo - metadata: CommonMetadataFields - - -# ----------------------- -# Setup Global Metadata -# ----------------------- -def setup_global_metadata(init_config: InitConfig) -> None: - """ - Initialize global metadata for resources using the provided InitConfig. - """ - try: - metadata = MetadataSingleton() - - git_info = init_config.git_info.model_dump() - git_labels = { - "git.commit": git_info["commit"], - "git.branch": git_info["branch"], - "git.repository": git_info["remote"], - } - - base_metadata = { - "managed-by": f"pulumi-{init_config.project_name}-{init_config.stack_name}", - "project": init_config.project_name, - "stack": init_config.stack_name, - "git": git_info, - } - - # Merge global tags - base_tags = {**base_metadata, **init_config.metadata.tags} - # Merge global labels and add git_labels - base_labels = {**base_metadata, **init_config.metadata.labels, **git_labels} - # Merge global annotations - base_annotations = {**base_metadata, **init_config.metadata.annotations} - - metadata.set_tags({k: str(v) for k, v in base_tags.items()}) - metadata.set_labels({k: str(v) for k, v in base_labels.items()}) - metadata.set_annotations({k: str(v) for k, v in base_annotations.items()}) - metadata.set_git_metadata({k: str(v) for k, v in git_info.items()}) - - log.info("Global metadata initialized successfully") - except Exception as e: - log.error(f"Failed to setup global metadata: {e}") - raise - - -# ----------------------- -# Configuration Management -# ----------------------- -class ConfigManager: - """ - Configuration Management class for retrieving and caching Pulumi configuration. - """ - - def __init__(self) -> None: - self.pulumi_config = pulumi.Config() - self._config_cache: Optional[Dict[str, object]] = None - self._module_configs: Dict[str, Dict[str, object]] = {} - - def get_config(self) -> Dict[str, object]: - """Retrieve full configuration from Pulumi stack config, caching results.""" - if self._config_cache is None: - raw_config: Dict[str, object] = {} - for module_name in ["aws", "kubernetes"]: - try: - module_cfg = pulumi.Config(module_name).get_object("") or {} - raw_config[module_name] = module_cfg - except KeyError as e: - log.debug(f"No config found for module {module_name}: {e}") - self._config_cache = raw_config - log.debug(f"Loaded config: {raw_config}") - return cast(Dict[str, object], self._config_cache) - - def get_module_config(self, module_name: str) -> Dict[str, object]: - """Get configuration for a specific module from cached config.""" - if module_name in self._module_configs: - return self._module_configs[module_name] - config = self.get_config() - module_config = cast(Dict[str, object], config.get(module_name, {})) - self._module_configs[module_name] = module_config - log.debug(f"Module {module_name} config: {module_config}") - return module_config - - def get_enabled_modules(self) -> List[str]: - """Return a list of enabled modules from the cached configuration.""" - enabled_modules: List[str] = [] - config = self.get_config() - for module_name in ["aws", "kubernetes"]: - module_config = cast(Dict[str, object], config.get(module_name, {})) - if bool(module_config.get("enabled", False)): - enabled_modules.append(module_name) - log.info(f"Module {module_name} is enabled") - log.info(f"Enabled modules: {enabled_modules}") - return enabled_modules - - -# ----------------------- -# Global Metadata and Stack Config -# ----------------------- -class GlobalMetadata(BaseModel): - """ - Global metadata structure for top-level stack outputs. - """ - - tags: Dict[str, str] = Field(default_factory=dict, description="Global public cloud resource tags") - labels: Dict[str, str] = Field(default_factory=dict, description="Global Kubernetes labels") - annotations: Dict[str, str] = Field(default_factory=dict, description="Global Kubernetes annotations") - - -class SourceRepository(BaseModel): - """ - Source repository information. - """ - - branch: str - commit: str - remote: str - tag: Optional[str] = None - - -class StackConfig(BaseModel): - """ - Root stack configuration mapping. - """ - - compliance: ComplianceConfig - metadata: GlobalMetadata - source_repository: SourceRepository - - -class StackOutputs(BaseModel): - """ - Complete stack outputs structure. - """ - - stack: StackConfig - secrets: Optional[Dict[str, object]] = Field(default=None, description="Sensitive info like credentials/tokens") - - -class ModuleDefaults(BaseModel): - """ - Default configuration for modules. - """ - - enabled: bool = Field(default=False, description="Whether the module is enabled") - config: Dict[str, object] = Field(default_factory=dict, description="Module-specific configuration") - - -__all__ = [ - "CoreError", - "ModuleLoadError", - "CommonMetadataFields", - "GitInfo", - "OwnershipInfo", - "AtoConfig", - "ProjectOwnership", - "StackConfig", - "FismaConfig", - "NistConfig", - "ComplianceConfig", - "BaseConfigModel", - "InitializationConfig", - "ModuleRegistry", - "ModuleBase", - "ResourceModel", - "ModuleDeploymentResult", - "DeploymentContext", - "ModuleInterface", - "MetadataSingleton", - "InitConfig", - "setup_global_metadata", - "ConfigManager", - "GlobalMetadata", - "SourceRepository", - "StackConfig", - "StackOutputs", - "ModuleDefaults", -]