Sync from ADO (#840)

Co-authored-by: Anthony Watherston <Anthony.Watherston@microsoft.com>

Docs/operational-scripts-reference.md

@@ -39,7 +39,7 @@ Include Policies with effect Manual. Default: do not include Policies with effec
 The New-AzRemediationTasks PowerShell creates remediation tasks for all non-compliant resources in the current AAD tenant. If one or multiple remediation tasks fail, their respective objects are added to a PowerShell variable that is outputted for later use in the Azure DevOps Pipeline.
 
 ```ps1
-New-AzRemediationTasks [[-PacEnvironmentSelector] <String>] [-DefinitionsRootFolder <String>] [-Interactive <Boolean>] [-OnlyCheckManagedAssignments] [-PolicyDefinitionFilter <String[]>] [-PolicySetDefinitionFilter <String[]>] [-PolicyAssignmentFilter <String[]>] [-PolicyEffectFilter <String[]>] [-NoWait] [-WhatIf] [-Confirm] [<CommonParameters>]
+New-AzRemediationTasks [[-PacEnvironmentSelector] <String>] [-DefinitionsRootFolder <String>] [-Interactive <Boolean>] [-OnlyCheckManagedAssignments] [-PolicyDefinitionFilter <String[]>] [-PolicySetDefinitionFilter <String[]>] [-PolicyAssignmentFilter <String[]>] [-PolicyEffectFilter <String[]>] [-NoWait] [-TestRun] [-Confirm] [<CommonParameters>]
 ```
 
 ### Parameters
@@ -79,7 +79,7 @@ Filter by Policy effect (array).
 #### `-NoWait [<SwitchParameter>]`
 Indicates that the script should not wait for the remediation tasks to complete.
 
-#### `-WhatIf [<SwitchParameter>]`
+#### `-TestRun [<SwitchParameter>]`
 Simulates the actions of the command without actually performing them. Useful for testing.
 
 #### `-Confirm [<SwitchParameter>]`

Docs/policy-assignments.md

@@ -201,6 +201,22 @@ Both `scope` and `notScopes` are specific to an [EPAC Environment using the pacS
 }
 ```
 
+> [!NOTE]
+> Prerelease Feature - Subscription Pattern Matching for Excluded Scopes
+> As of version v10.7.6-beta you can define a pattern to match on subscriptions for an excluded scope. This allows an assignment to add matched subscription names to the excluded scope. It is not dynamic i.e. if you add subscriptions later and want to include them you would have to run the plan again.
+> The syntax is:
+
+```jsonc
+"notScopes": {
+    "tenant": [
+        "/subscriptions/subscriptionsPattern/wildcard-pattern-to-match" 
+    ]
+}
+E.g. /subscriptions/subscriptionsPattern/Con* would match a subscription with the name Connectivity
+
+The Powershell -like expression is used for matching. 
+```
+
 ## Managed Identities and role assignments
 
 Policies with a `DeployIfNotExists` or `Modify` effect need a Managed Identity (MI) and role assignments to execute remediation tasks. EPAC calculates the necessary role assignments based on the `roleDefinitionIds` in the policy definition. By default EPAC uses a system-assigned Managed Identity. The team maintaining EPAC recommend using system-assigned identities; however, your organization may have role assignment reasons to use user-assigned Managed Identities.

Docs/start-changes.md

@@ -29,7 +29,7 @@ Per `pacEnvironment`:
 - `strategy`: was optional and defaulted to `full`. We recommend setting it to `full`, except during a short transition period to EPAC. This was changed to require an explicit decision.
 - `keepDfcSecurityAssignments`: replaces `deleteDfcSecurityAssignments`  which defaulted to `true`. We highly recommend setting it to `false` and assigning any desired Initiative at management groups.
 
-`desiredState` fields `deleteExpiredExemptions` and `deleteOrphanedExemptions` are deprecated and removed. Exemptions with an ``unknownOwner` are only deleted when `strategy` is `full`. 
+`desiredState` fields `deleteExpiredExemptions` and `deleteOrphanedExemptions` are deprecated and removed. Exemptions with an ``unknownOwner` are only deleted when `strategy` is `full`.
 
 The recommended `desiredState` settings are now as follows:
 
@@ -51,7 +51,7 @@ During a brief transition from a pre-EPAC to an EPAC usage, you can set `desired
 
 ### Desired State Handling for Policy Assignments
 
-Field `desiredState.includeResourceGroups` is deprecated/removed. This change removes all Policy Assignments in resource groups not defined in the Policy Assignment definition files. To keep the previous behavior, add a pattern `"/subscriptions/*/resourceGroups/*" to the `"excludedScopes"` array.
+Field `desiredState.includeResourceGroups` is deprecated/removed. This change removes all Policy Assignments in resource groups not defined in the Policy Assignment definition files. To keep the previous behavior, add a pattern `"/subscriptions/*/resourceGroups/*" to the`"excludedScopes"` array.
 
 Desired state handling for Policy Assignments related to Defender for Cloud (DfC) automatic Policy Assignments has been reworked. DfC creates two different types of Policy Assignments at the subscription level.
 
@@ -78,28 +78,21 @@ EPAC had multiple operational scripts which are not Policy as Code related. Thes
 
 We recommend that you use [Azure Governance Visualizer (AzGovViz)](https://github.com/JulianHayward/Azure-MG-Sub-Governance-Reporting) for these tasks.
 
-## Enhancements planned for v10.1.0
-
-- Script to update CSV effect/parameter files preserving extra columns: https://github.com/Azure/enterprise-azure-policy-as-code/issues/498.
-- Automatically disable deprecated Policies: https://github.com/Azure/enterprise-azure-policy-as-code/issues/516.
-- Cleanup/Improve `Export-PolicyResources` and `Build-PolicyDocumentation` scripts: https://github.com/Azure/enterprise-azure-policy-as-code/issues/517 and https://github.com/Azure/enterprise-azure-policy-as-code/issues/498.
-- Simplify exemption creation by allowing lists of scopes and Policy definitions: https://github.com/Azure/enterprise-azure-policy-as-code/issues/518.
-- Clarify SPNs, Least Privilege, and environments for CI/CD: https://github.com/Azure/enterprise-azure-policy-as-code/issues/519.
-
 ## Enhancements in v10.0.0
 
 ### Support for Cloud environments with limited Support for Resource Graph Queries
 
 - US Government Cloud handling of Role Assignments
 - China cloud (21v) handling for Role Assignments and Exemptions.
 
-### Cross-tenant (Lighthouse) support for Role Assignments.
+### Cross-tenant (Lighthouse) support for Role Assignments
 
 Cross-tenant Role Assignments are now supported. This is used if log collection is directed to a resource (Log Analytics, Event Hub. Storage) in a management tenant (e.g, Azure Lighthouse, and similar constructs) which requires you to use `additionalRoleAssignments` in the Policy Assignment file.
 
 ### Simplified Exemption definitions
 
 Exemptions can be specified with a `policyDefinitionName` or `policyDefinitionId` instead of a `policyAssignmentId` and `policyDefinitionReferenceId`. EPAC creates as many Exemptions as needed to cover all Policy Assignments occurrences of the specified Policy
+
 - Support for Microsoft release flow in addition to GitHub flow (documentation and starter kit)
 - Schema updated to latest draft specification
 
@@ -130,3 +123,7 @@ Updating JSON schema to the latest [specification 2020-12](https://json-schema.o
 ### Documentation Updates
 
 Reorganized the documentation to make it easier to find information. Added a new section on how to use the starter kit and how to use the Microsoft release flow.
+
+## Prerelease Features
+
+- v10.7.6-alpha - Subscription pattern matching for excluded scopes in assignments

Docs/start-forking-github-repo.md

@@ -2,6 +2,16 @@
 
 Instead of installing `EnterprisePolicyAsCode` from the PowerShell Gallery, you can clone the GitHub repository and use the scripts described below to install the script source code. This is useful, if your organization has overly restrictive policies on installing PowerShell modules from the PowerShell Gallery. It can also be useful if you want to contribute EPAC source code to the project.
 
+## Changes to the Forking Process
+
+With the ability to provide prerelease versions via release and PowerShell - if you are working in a forked repo you should also clone the tags from the original source to allow your forked repo to pin to specific releases.
+
+1. Add an upstream remote containing the original release `git remote add upstream https://github.com/Azure/enterprise-azure-policy-as-code.git`
+1. Fetch the tags from the upstream - `git fetch --tags upstream`
+1. Push tags to the fork - `git push --tags`
+
+Tags are not automatically synced to a forked repo so you must perform this task each time you sync your fork with the main project.
+
 ## Setting up your Repo
 
 1. Initial setup
@@ -124,4 +134,4 @@ This is a guide on how to release a new version of the project - including autom
 7. Click **Publish Release**
 8. Click on **Actions**
 9. Verify that a workflow run has started with the same name as the release.
-12. Verify that the module has been published to the [PowerShell Gallery](https://www.powershellgallery.com/packages/EnterprisePolicyAsCode).
+10. Verify that the module has been published to the [PowerShell Gallery](https://www.powershellgallery.com/packages/EnterprisePolicyAsCode).

Docs/start-implementing.md

@@ -139,6 +139,9 @@ EPAC can be installed in two ways:
     Install-Module EnterprisePolicyAsCode -Scope CurrentUser
 ```
 
+> [!IMPORTANT]
+> Experimental or features containing large breaking changes may be available as a prerelease version in GitHub and in PowerShell. To install one of these versions use the ```-AllowPrerelease``` parameter in Install-Module. Be aware that these version are not supported for production use and may introduce breaking changes. Where a feature is implemented in a prerelease it will be documented accordingly.
+
 Many scripts use parameters for input and output folders. They default to the current directory. We recommend that you do one of the following approaches instead of accepting the default to prevent your files being created in the wrong location:
     - [Preferred] Set the environment variables `PAC_DEFINITIONS_FOLDER`, `PAC_OUTPUT_FOLDER`, and `PAC_INPUT_FOLDER`.
     - [Alternative] Use the script parameters `-DefinitionsRootFolder`, `-OutputFolder`, and `-InputFolder`.

Module/EnterprisePolicyAsCode.prerelease.psd1

@@ -0,0 +1,68 @@
+@{
+	# Script module or binary module file associated with this manifest
+	RootModule        = 'EnterprisePolicyAsCode.psm1'
+
+	# Version number of this module.
+	ModuleVersion     = ''
+
+	# ID used to uniquely identify this module
+	GUID              = '197a34e5-115d-4c15-a593-b004228be78b'
+
+	# Author of this module
+	Author            = 'Microsoft Corporation'
+
+	# Company or vendor of this module
+	CompanyName       = 'Microsoft'
+
+	# Copyright statement for this module
+	Copyright         = 'Copyright (c) 2022,2023,2024 Microsoft Corporation'
+
+	# Description of the functionality provided by this module
+	Description       = 'Enterprise Policy as Code PowerShell Module'
+
+	# Minimum version of the Windows PowerShell engine required by this module
+	PowerShellVersion = '7.0'
+
+	# Functions to export from this module
+	FunctionsToExport = ''
+
+	# Cmdlets to export from this module
+	CmdletsToExport   = ''
+
+	# Variables to export from this module
+	VariablesToExport = ''
+
+	# Aliases to export from this module
+	AliasesToExport   = ''
+
+	# List of all files packaged with this module
+	FileList          = @()
+
+	# Private data to pass to the module specified in ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell.
+	PrivateData       = @{
+
+		#Support for PowerShellGet galleries.
+		PSData = @{
+
+			# Tags applied to this module. These help with module discovery in online galleries.
+			# Tags = @()
+
+			# Prerelease tag to append
+			Prerelease   = ''
+
+			# A URL to the license for this module.
+			LicenseUri   = 'https://github.com/Azure/enterprise-azure-policy-as-code/blob/main/LICENSE'
+
+			# A URL to the main website for this project.
+			ProjectUri   = 'https://github.com/Azure/enterprise-azure-policy-as-code'
+
+			# A URL to an icon representing this module.
+			IconUri      = 'https://raw.githubusercontent.com/Azure/enterprise-azure-policy-as-code/refs/heads/main/Docs/Images/10316-icon-service-Policy.svg'
+
+			# ReleaseNotes of this module
+			ReleaseNotes = 'https://github.com/Azure/enterprise-azure-policy-as-code/releases'
+
+		} # End of PSData hashtable
+
+	} # End of PrivateData hashtable
+}

Module/build.ps1

@@ -1,6 +1,16 @@
 New-Item .\Module\EnterprisePolicyAsCode\internal\functions -ItemType Directory -Force
 New-Item .\Module\EnterprisePolicyAsCode\functions -ItemType Directory -Force
 
+$tag_name = $env:TAG_NAME -replace "v", ""
+
+if ($tag_name -match "-") {
+    Copy-Item -Path .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.prerelease.psd1 -Destination .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1 -Force
+    $isPreRelease = $true
+}
+else {
+    Copy-Item -Path .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.release.psd1 -Destination .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1 -Force
+}
+
 Get-ChildItem -Path .\Scripts\Helpers\*.ps1 -Recurse -File -Exclude Add-HelperScripts.ps1 | Copy-Item -Destination .\Module\EnterprisePolicyAsCode\internal\functions
 
 # Deploy Functions
@@ -37,20 +47,18 @@ $functionNames | Foreach-Object {
 
 Get-ChildItem -Path .\Scripts\HydrationKit\*.ps1 -Recurse -File | Copy-Item -Destination .\Module\EnterprisePolicyAsCode\functions
 
-# $functionNames = (Get-ChildItem .\Scripts\HydrationKit\* -File -Include *.ps1).BaseName
-
-# $functionNames | Foreach-Object {
-#     "function $_ {" | Set-Content ".\Module\EnterprisePolicyAsCode\functions\$_.ps1" -Force
-#     Get-Content .\Scripts\HydrationKit\$_.ps1 | Where-Object { $_ -notmatch "^#Requires" } | Add-Content ".\Module\EnterprisePolicyAsCode\functions\$_.ps1" -Force
-#     "}" | Add-Content ".\Module\EnterprisePolicyAsCode\functions\$_.ps1" -Force
-# }
-
-
-
 Copy-Item -Path .\Scripts\CloudAdoptionFramework\policyAssignments -Destination .\Module\EnterprisePolicyAsCode -Force -Recurse
 
 (Get-Content -Path .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1) -replace "FunctionsToExport = ''", "FunctionsToExport = @($((Get-ChildItem -Path .\Module\EnterprisePolicyAsCode\functions | Select-Object -ExpandProperty BaseName) | Join-String -Separator "," -DoubleQuote))" | Set-Content .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1
 
-$tag_name = $env:TAG_NAME -replace "v", ""
+if ($isPreRelease) {
+    $version = ($tag_name -split "-")[0]
+    $prereleaseString = ($tag_name -split "-")[1]
+    (Get-Content -Path .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1) -replace "ModuleVersion     = ''", "ModuleVersion     = '$version'" | Set-Content .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1
+    (Get-Content -Path .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1) -replace "Prerelease = ''", "Prerelease = '$prereleaseString'" | Set-Content .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1
+}
+else {
+    (Get-Content -Path .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1) -replace "ModuleVersion     = ''", "ModuleVersion     = '$tag_name'" | Set-Content .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1
+}
+
 
-(Get-Content -Path .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1) -replace "ModuleVersion     = ''", "ModuleVersion     = '$tag_name'" | Set-Content .\Module\EnterprisePolicyAsCode\EnterprisePolicyAsCode.psd1

Scripts/Helpers/Build-AssignmentDefinitionNode.ps1

@@ -400,13 +400,22 @@ function Build-AssignmentDefinitionNode {
                     $thisScopeGlobalNotScopeTable = $thisScopeDetails.notScopesTable
                     foreach ($notScope in $definition.notScopesList) {
                         $individualResource = $false
+                        if ($notScope -match "subscriptionsPattern") {
+                            $thisScopeChildren.Keys | Foreach-Object {
+                                if ($thisScopeChildren.$_.type -eq "/subscriptions") {
+                                    if ($thisScopeChildren.$_.displayName -like $notScope.split("/")[-1]) {
+                                        $null = $thisNotScopeList.Add($thisScopeChildren.$_.id)
+                                    }
+                                }
+                            }
+                        }
                         $notScopeTrimmed = $notScope
                         $splits = $notScope -split "/"
                         if ($splits.Count -gt 5) {
                             $individualResource = $true
                             $notScopeTrimmed = $splits[0..4] -join "/"
                         }
-                        if (-not $thisScopeGlobalNotScopeTable.ContainsKey($notScopeTrimmed)) {
+                        if (-not $thisScopeGlobalNotScopeTable.ContainsKey($notScopeTrimmed) -or ($notScope -match "subscriptionsPattern")) {
                             if ($thisScopeChildren.ContainsKey($notScopeTrimmed)) {
                                 $null = $thisNotScopeList.Add($notScope)
                             }

Scripts/Helpers/Build-ScopeTableForSubscription.ps1

@@ -50,6 +50,16 @@ function Build-ScopeTableForSubscription {
                     $IsExcluded = $true
                     break
                 }
+                if ($SubscriptionName -like $globalExcludedScope) {
+                    $IsExcluded = $true
+                    break
+                }
+                if ($globalExcludedScope -match "/subscriptions/subscriptionsPattern/") {
+                    if ($SubscriptionName -match $globalExcludedScope.Split("/")[-1]) {
+                        $IsExcluded = $true
+                        break
+                    }
+                }
             }
         }
     }

Scripts/Helpers/Get-GlobalSettings.ps1

@@ -261,19 +261,21 @@ function Get-GlobalSettings {
                             }
                             else {
                                 $null = $excludedScopesList.Add($excludedScope)
-                                if ($excludedScope.StartsWith("/subscriptions/") -and $desired.excludeSubscriptions -eq $false) {
-                                    if ($excludedScope.Contains("/resourceGroups/", [System.StringComparison]::OrdinalIgnoreCase)) {
-                                        $null = $globalExcludedScopesResourceGroupsList.Add($excludedScope)
-                                    }
-                                    else {
-                                        $null = $globalExcludedScopesSubscriptionsList.Add($excludedScope)
+                                if ($excludedScope.StartsWith("/subscriptions/")) {
+                                    if ($desired.excludeSubscriptions -eq $false -or $null -eq $desired.excludeSubscriptions) {
+                                        if ($excludedScope.Contains("/resourceGroups/", [System.StringComparison]::OrdinalIgnoreCase)) {
+                                            $null = $globalExcludedScopesResourceGroupsList.Add($excludedScope)
+                                        }
+                                        else {
+                                            $null = $globalExcludedScopesSubscriptionsList.Add($excludedScope)
+                                        }
                                     }
                                 }
                                 elseif ($excludedScope.StartsWith("/providers/Microsoft.Management/managementGroups/")) {
                                     $null = $globalExcludedScopesManagementGroupsList.Add($excludedScope)
                                 }
                                 else {
-                                    Add-ErrorMessage -ErrorInfo $errorInfo -ErrorString "Global settings error: pacEnvironment $pacSelector field desiredState.excludedScopes ($excludedScope) must be a valid scope."
+                                    Write-Host "Global settings error: pacEnvironment $pacSelector field desiredState.excludedScopes ($excludedScope) must be a valid scope."
                                 }
                             }
                         }

Scripts/Operations/Export-NonComplianceReports.ps1

@@ -244,7 +244,7 @@ else {
         $subscriptionScope = "/subscriptions/$($subscriptionId)"
         $subscriptionName = $subscriptionId
         if ($scopeTable.ContainsKey($subscriptionScope)) {
-            $subscriptionName = $scopeTable.$subscriptionScope.name
+            $subscriptionName = $scopeTable.$subscriptionScope.displayName
         }
         $splits = $resourceId -split "/"
         $segments = $splits.Length