Mega update and bug fixes (#445)

Docs/ci-cd-ado-pipelines.md

@@ -0,0 +1,123 @@
+# Azure DevOps Pipelines
+
+This page covers the specifics for Azure DevOps (ADO) pipelines. It si based on a simplified GitHub Flow as documented in [CI/CD Overview](./ci-cd-overview.md)
+
+Previously [setup App Registrations](./ci-cd-app-registrations.md) are a pre-requisite.
+
+This repository contains starter pipelines
+
+* Azure DevOps (Single Tenant)
+* Azure DevOps (Multi Tenant)
+* Azure DevOps (Simplified)
+
+## Service connections for Azure DevOps CI/CD
+
+Create ADO service connections for each of the previously created [App Registrations](./ci-cd-app-registrations.md). You will need to retrieve the client id and create a client secret or authenticate with a X509 certificate configured for the SPN.
+
+When creating a Service Connection in Azure DevOps you can set up the service connections on a Subscription or a Management Group scope level. If you are using subscriptions to simulate a hierarchy during EPAC development, configure the service connection(s) scope level as **Subscription**. When creating a Service Connections for management groups (any EPAC environments) Deployment and EPAC Role Assignment the service connection scope level is **Management Group**.
+
+Subscription scope level | Management Group scope level
+:-----------:|:----------------:
+![image](./Images/azdoServiceConnectionSubConf.png) | ![image](./Images/azdoServiceConnectionMGConf.png)
+
+## Single Tenant Pipeline
+
+### Single Tenant Stages
+
+| Stage | Purpose | Trigger | Scripts |
+|-------|---------|---------|---------|
+| devStage | Feature branch DEV environment build, deploy and test | CI, Manual | Build-DeploymentPlans.ps1 <br> Deploy-PolicyPlan.ps1 <br/> Deploy-RolesPlan.ps1 |
+| tenantPlanFeatureStage | Feature branch based plan for prod deployment | CI, Manual | Build-DeploymentPlans.ps1 |
+| tenantPlanMainStage | Main branch based plan for prod deployment | PR Merged, Manual | Build-DeploymentPlans.ps1 |
+| tenantDeployStage | Deploy Policies defined by Main branch based plan | Prod stage approved | Deploy-PolicyPlan.ps1 |
+| tenantRolesStage | Assign roles defined by Main branch based plan | Role stage approved | Deploy-RolesPlan.ps1 |
+
+### Single Tenant Service Connections and Roles
+
+Create Service Principals and associated service connections in Azure DevOps or the equivalent in your CI/CD tool. The SPNs require the following roles to adhere to the least privilege principle. If you have a single tenant, remove the last column and rows with connections ending in "-2".
+
+| Connection | Stages  | MG: epac-dev-mg | MG: Tenant Root |
+| :--- | :--- | :--- | :--- |
+| sc-pac-dev | devStage  | Owner <br/> MS Graph Permissions ||||
+| sc-pac-plan | tenantPlanFeatureStage <br/> tenantPlanMainStage || EPAC Policy Reader <br/> MS Graph Permissions |
+| sc-pac-prod | tenantDeployStage || Policy Contributor |
+| sc-pac-roles | tenantRolesStage-1 || User Access Administrator <br/> MS Graph Permissions |
+
+## Multi Tenant Pipeline
+
+### Multi Tenant Stages
+
+| Stage | Purpose | Trigger | Scripts |
+|-------|---------|---------|---------|
+| devStage | Feature branch EPAC DEV environment build, deploy and test | CI, Manual | Build-DeploymentPlans.ps1 <br> Deploy-PolicyPlan.ps1 <br/> Deploy-RolesPlan.ps1 |
+| tenantPlanFeatureStage-1 | Feature branch based plan for prod deployment (tenant 1) | CI, Manual | Build-DeploymentPlans.ps1 |
+| tenantPlanFeatureStage-2 | Feature branch based plan for prod deployment (tenant 2) | CI, Manual | Build-DeploymentPlans.ps1 |
+| completedFeature | Empty stage to complete feature branch | None | None |
+| tenantPlanMainStage-1 | Main branch based plan for prod deployment (tenant 1) | PR Merged, Manual | Build-DeploymentPlans.ps1 |
+| tenantDeployStage-1 | Deploy Policies defined by Main branch based plan (tenant 1) | Prod stage approved | Deploy-PolicyPlan.ps1 |
+| tenantRolesStage-1 | Assign roles defined by Main branch based plan (tenant 1) | Role stage approved | Deploy-RolesPlan.ps1 |
+| tenantPlanMainStage-2 | Main branch based plan for prod deployment (tenant 2) | PR Merged, Manual | Build-DeploymentPlans.ps1 |
+| tenantDeployStage-2 | Deploy Policies defined by Main branch based plan (tenant 2) | Prod stage approved | Deploy-PolicyPlan.ps1 |
+| tenantRolesStage-2 | Assign roles defined by Main branch based plan (tenant 2) | Role stage approved | Deploy-RolesPlan.ps1 |
+
+### Multi Tenant Service Connections and Roles
+
+Create Service Principals and associated service connections in Azure DevOps or the equivalent in your CI/CD tool. The SPNs require the following roles to adhere to the least privilege principle. If you have a single tenant, remove the last column and rows with connections ending in "-2".
+
+| Connection | Stages  | MG: epac-dev-mg | MG: Tenant 1 Root | MG: Tenant 2 Root |
+| :--- | :--- | :--- | :--- | :--- |
+| sc-pac-dev | devStage  | Owner <br/> MS Graph Permissions ||||
+| sc-pac-plan-1 | tenantPlanFeatureStage-1 <br/> tenantPlanMainStage-1 || EPAC Policy Reader <br/> MS Graph Permissions ||
+| sc-pac-plan-2 | tenantPlanFeatureStage-2 <br/> tenantPlanMainStage-2 ||| EPAC Policy Reader <br/> MS Graph Permissions |
+| sc-pac-prod-1 | tenantDeployStage-1 || Policy Contributor ||
+| sc-pac-prod-2 | tenantDeployStage-2 ||| Policy Contributor |
+| sc-pac-roles-1 | tenantRolesStage-1 || User Access Administrator ||
+| sc-pac-roles-2 | tenantRolesStage-2 ||| User Access Administrator |
+| none | completedPlanFeatureStage ||||
+
+
+## Azure DevOps (Simplified Pipeline)
+
+If you have less complex requirements for a pipeline deployment using Azure DevOps you can utilize the ```simplified-pipeline.yaml``` file and the ```templates``` folder in the ```StarterKit``` folder to quickly get started in Azure Pipelines.
+
+This template requires the creation of two environments in Azure Pipelines and can easily have approvals added for deployment control.
+
+## Deployment Environments
+
+Create distinct ADO environment to configure approval gates. Refer to the following documentation:  <https://docs.microsoft.com/en-us/azure/devops/pipelines/process/environments?view=azure-devops>
+
+
+## Pipeline Execution
+
+In Azure Devops pipelines the following happens. Your CI/CD tools will display progress differently.
+
+### `Commit` to a feature branch or a manual pipeline run
+
+* Stage devStage to deploy Policies, Policy Sets and Policy Assignments to the PAC DEV environment.
+* Calculates the plan for PROD environment deployment based on the Feature branch.
+  * This plan is never executed. Instead the logs and if desired the artifact generated are used by the developer to verify the definition files and to determine if the code is ready for a Pull Request.
+  * The PR approver(s) will use the same input plus the source code changes to decide the PR approval or rejection.
+
+![image.png](Images/feature-run.png)
+
+Detail view:
+
+![image.png](Images/feature-run-details.png)
+
+### `Pull Request` is approved and branch merged into main
+
+* Calculates the plan for PROD environment deployment based on the merged Main branch.
+* The pipeline stops for PROD gate(s) approval at this time.
+  * The logs and if desired the artifacts generated are used by the PROD gate(s) approver(s) to decide on the PROD stage approval(s) or rejection(s).
+* ![image.png](Images/prod-approval.png)
+* ![image.png](Images/prod-approval-dialog.png)
+* After the approval deployments to PROD will begin.
+* Optional a second approval before role assignments is required.
+* ![image.png](Images/prod-complete.png)
+* After the ntire run the overview page looks like this:
+* ![image.png](Images/pipeline-runs.png)
+
+### No changes
+
+* Deployment steps and stages are skipped. Skipped stages do not need approvals.
+* ![image.png](Images/prod-no-changes.png)

Docs/ci-cd-app-registrations.md

@@ -0,0 +1,136 @@
+# App Registrations Setup
+
+CI/CD pipelines/workflows require the creation of App Registrations in your Entra ID (Azure AD) tenants. The App Registrations are used by the EPAC pipeline to deploy the EPAC Management Group and the EPAC Management Group Policy Definitions.
+
+The following screenshot shows the Management Group hierarchy that used for the App Registrations.
+
+![Management Group hierarchy](./images/ci-cd-mg.png)
+
+## Custom `EPAC Resource Policy Reader Role`
+
+EPAC uses a set of Entra ID App Registrations (Service Principals). To build the deployment plan and adhere to the least-privilege-principle, a Resource Policy Reader role is required. This role is not built-in. EPAC contains script `New-AzPolicyReaderRole.ps1` to create this role or you can use the below JSON in Azure Portal.
+
+```json
+{
+    "properties": {
+        "roleName": "EPAC Resource Policy Reader",
+        "description": "Provides read access to all Policy resources for the purpose of planning the EPAC deployments.",
+        "assignableScopes": [
+            "/"
+        ],
+        "permissions": [
+            {
+                "actions": [
+                    "Microsoft.Authorization/policyassignments/read",
+                    "Microsoft.Authorization/policydefinitions/read",
+                    "Microsoft.Authorization/policyexemptions/read",
+                    "Microsoft.Authorization/policysetdefinitions/read",
+                    "Microsoft.PolicyInsights/*",
+                    "Microsoft.Management/register/action",
+                    "Microsoft.Management/managementGroups/read"
+                ],
+                "notActions": [],
+                "dataActions": [],
+                "notDataActions": []
+            }
+        ]
+    }
+}
+```
+
+## Create single App Registration and Role assignments for `epac-dev`
+
+Create the App Registrations for:
+
+- epac-dev environment with Owner rights to the epac-dev Management Group
+- Optional: epac-test environment with Owner rights to the epac-test Management Group (repeat the steps below for epac-test)
+
+### Create the App Registration for `epac-dev` environment
+
+![App Registration](./images/ci-cd-app-reg-perm-1.png)
+
+### Grant the App Registration the necessary Microsoft Graph permissions
+
+![App Registration](./images/ci-cd-app-reg-perm-2.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-3.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-4.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-5.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-6.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-7.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-8.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-9.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-a.png)
+
+### Grant the App Registration the necessary Azure `Owner` permissions for the epac Management Group
+
+![App Registration](./images/ci-cd-app-reg-perm-b.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-c.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-d1.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-d2.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-d3.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-d4.png)
+
+![App Registration](./images/ci-cd-app-reg-perm-d5.png)
+
+## Create App Registrations and Role assignments for prod environments (per tenant)
+
+### App Registration  with permissions to read Policy resources and Azure roles
+
+#### Create the App Registration the same as above with the same Microsoft Graph permissions
+
+![App Registration](./images//ci-cd-app-reg-root-reader.png)
+
+#### Create custom Azure role with permissions to read Policy resources
+
+![Reader Role](Images/ci-cd-role-policy-reader-1.png)
+
+![Reader Role](Images/ci-cd-role-policy-reader-2.png)
+
+![Reader Role](Images/ci-cd-role-policy-reader-3.png)
+
+![Reader Role](Images/ci-cd-role-policy-reader-4.png)
+
+![Reader Role](Images/ci-cd-role-policy-reader-5.png)
+
+#### Grant the App Registration the custom Azure role at the root Management Group
+
+![App Registration](./images//ci-cd-app-reg-root-reader-perm-1.png)
+
+![App Registration](./images//ci-cd-app-reg-root-reader-perm-2.png)
+
+### App Registration with permissions to deploy Policy resources
+
+### Create the App Registration ***without*** Microsoft Graph permissions
+
+![App Registration](./images//ci-cd-app-reg-root-contributor.png)
+
+#### Grant the App Registration the `ResourcePolicy Contributor` role at the root Management Group
+
+![App Registration](./images//ci-cd-app-reg-root-contributor-perm-1.png)
+
+![App Registration](./images//ci-cd-app-reg-root-contributor-perm-2.png)
+
+### App Registration with permissions to assign Roles at root Management Group
+
+#### Create the App Registration the same as above with the same Microsoft Graph permissions
+
+![App Registration](./images//ci-cd-app-reg-root-roles.png)
+
+#### Grant the App Registration the `User Access Administrator` role at the root Management Group
+
+![App Registration](./images//ci-cd-app-reg-root-role-assignments-perm-1.png)
+
+![App Registration](./images//ci-cd-app-reg-root-role-assignments-perm-2.png)

Docs/github-actions.md → Docs/ci-cd-github-actions.md

@@ -1,8 +1,54 @@
-# CI/CD with Github Actions
+# Github Actions
 
 The starter kit contains a sample pipeline to use GitHub Actions to deploy Enterprise Policy as Code. It features a review process and is driven by pull requests and approvals.
 
-## Action Flow
+We have revised our approach to GitHub Actions simplifying the process and make it easier to understand. The new approach is documented below and is included in the starter kit with v8.5 and later.
+
+The previous version is still available in the starter kit in folder `Legacy` and the [documentation is retained](#previous-cicd-workflow----legacy) after the documentation for the new approach.
+
+## Setup in GitHub
+
+### Create GitHub Deployment Environments
+
+You will need one [GitHub deployment environment](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment) for the `epac-dev` workflow and three environments each for your epac-prod or tenant workflows. This documentation assumes the include starter kit pipelines.
+
+| Environment | Purpose | [App Registration](./ci-cd-app-registrations.md) (SPN) |
+|---|---|---|
+| EPAC-DEV | Plan and deploy to `epac-dev` | ci-cd-epac-dev-owner |
+| TENANT-PLAN | Build deployment plan for `tenant` | ci-cd-root-policy-reader |
+| TENANT-DEPLOY-POLICY | Deploy Policy resources for `tenant` | ci-cd-root-policy-contributor |
+| TENANT-DEPLOY-ROLES | Deploy Roles for `tenant` | ci-cd-root-user-assignments |
+
+For each environment, [add to the environment secrets](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#environment-secrets) for the tenant id, client id and client secret for the SPN. The secrets must be named `TENANT_ID`, `CLIENT_ID` and `CLIENT_SECRET` respectively.
+
+### Hardening each Environment
+
+* Global setting for the repo: protect the `main` branch with [branch protection rules](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches).
+* [Configure required reviewers](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#required-reviewers) for TENANT-DEPLOY-POLICY and TENANT-DEPLOY-ROLES environments.
+* [Restrict branch](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#deployment-branches-and-tags) to `main` branch for TENANT-DEPLOY-POLICY and TENANT-DEPLOY-ROLES environments.
+* Do not [allow administrators to bypass configured protection rules](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#allow-administrators-to-bypass-configured-protection-rules) for TENANT-DEPLOY-POLICY and TENANT-DEPLOY-ROLES environments.
+
+## New GitHub Workflows
+
+Each of the workflows has three (3) jobs:
+* `Build` - Builds the deployment plan
+* `Deploy Policy` - Deploys the policy plan
+* `Deploy Roles` - Deploys the role plan
+
+The starter kit contains the following workflows:
+
+* `epac-dev.yaml` is the workflow for the `epac-dev` environment. It is triggered on push to any `feature/**` branch.
+* `tenant.yaml` is the workflow for the `epac-prod` or `tenant` environments. It is triggered on PR closed to the `main` branch.
+
+## Testing your Workflows
+
+To test the actions make a change to a file in the definitions folder.
+
+## Previous CI/CD Workflow -- Legacy
+
+This section is retained from the previous documentation to enable you to continue using the previous approach. It is recommended that you migrate to the new approach as soon as possible.
+
+### Action Flow -- Legacy
 
 1. Changes are made to files in the Definitions folder (e.g. new policy definition/assignment, removing files) and pushed to GitHub
 2. The `Build Deployment Plan` action is triggered. This runs the `Build-DeploymentPlans` function against the environment specified in the `pacEnvironment` variable in `global-settings.jsonc`
@@ -19,7 +65,7 @@ The starter kit contains a sample pipeline to use GitHub Actions to deploy Enter
 !!! note
     This is a sample method of deploying policy and role changes - feel free to modify it to suit your environment and contribute to this project if you want to share an update.
 
-## Setup in GitHub
+### Setup in GitHub -- Legacy
 
 There are some steps to be performed in GitHub to ensure the action runs correctly.
 
@@ -44,14 +90,10 @@ There are some steps to be performed in GitHub to ensure the action runs correct
 
 4. In the `.github\workflows\build.yaml` and `.github\workflows\deploy.yaml` file updated the trigger's path setting to ensure it is triggered when a file change is made.
 
-## Testing
-
-To test the actions make a change to a file in the definitions folder.
-
-## Skipping a Workflow Run
+### Skipping a Workflow Run -- Legacy
 
 To skip the workflow run add a file called `NO_ACTIONS` in the definitions folder. An action will be started on push however the build will not occur.
 
-## Generating Build Results Only
+### Generating Build Results Only -- Legacy
 
 If you want to run just the `Build-DeploymentPlans` but not save the output - add a file called `NO_DEPLOY`. This will run the build step and then cancel the action. You can review the summary output by checking the logs in the cancelled action.