Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated What-If Step #86

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
01b8b3c
Updated What-If Step
marvinbuss Nov 24, 2021
b10ff6b
updated what if stage
marvinbuss Nov 24, 2021
44ba66d
added deployment name
marvinbuss Nov 24, 2021
300cb18
added dev and prod deployment workflows for gh
marvinbuss Dec 7, 2021
504bb5d
updated triggers
marvinbuss Dec 7, 2021
5d416e5
updated dependency
marvinbuss Dec 7, 2021
4814e5f
updated environments
marvinbuss Dec 7, 2021
1538749
updated workflows again
marvinbuss Dec 8, 2021
d475947
updated workflows based on guidance
marvinbuss Dec 9, 2021
6f0d2e0
updated ref
marvinbuss Dec 9, 2021
93a6397
test
marvinbuss Dec 9, 2021
30ae365
test
marvinbuss Dec 9, 2021
af239a1
test
marvinbuss Dec 9, 2021
f7ffd9e
updated trigger
marvinbuss Dec 9, 2021
0d12032
test
marvinbuss Dec 9, 2021
0c9b692
updated workflows
marvinbuss Dec 9, 2021
5d8fc61
test
marvinbuss Dec 9, 2021
0895524
test
marvinbuss Dec 9, 2021
66fd3fa
update reference
marvinbuss Dec 9, 2021
777fc57
updated reference back
marvinbuss Dec 9, 2021
1dbaa3d
Updated links in PR and Issue Templates
marvinbuss Dec 20, 2021
61dab4a
Merge branch 'main' into marvinbuss/what_if
marvinbuss Dec 20, 2021
126673f
updated gh workflows and templates
marvinbuss Dec 20, 2021
847568b
Added ado template
marvinbuss Dec 20, 2021
3a99b8b
Updated ado pipelines
marvinbuss Dec 20, 2021
2278a15
updated trigger
marvinbuss Dec 20, 2021
9f58d02
updated workflows
marvinbuss Mar 4, 2022
6254401
Merge branch 'main' of https://github.com/Azure/data-product-streamin…
marvinbuss Mar 4, 2022
d7f6ce3
updated workflows
marvinbuss Mar 4, 2022
e467a9f
test
marvinbuss Mar 4, 2022
61ce92f
updated wf
marvinbuss Mar 4, 2022
1639886
update linting
marvinbuss Mar 4, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .ado/workflows/dataProductDeploymentDev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Data Product Deployment - Dev

trigger:
branches:
include:
- main
paths:
include:
- code/*
- infra/*
- .ado/workflows/*
pr:
branches:
include:
- main
paths:
include:
- code/*
- infra/*
- .ado/workflows/*

variables:
DEV_ENVIRONMENT_NAME: "dev" # Update to '{devEnvironmentName}'
DEV_VARIABLE_GROUP_NAME: "dev-product-streaming" # Update to '{devVariableGroupName}'

stages:
- stage: Validation_Dev
displayName: "Validation of IaC templates - Dev"
jobs:
- template: templates/validation.yml
parameters:
environment_name: ${{ variables.DEV_ENVIRONMENT_NAME }}
variable_group_name: ${{ variables.DEV_VARIABLE_GROUP_NAME }}

- stage: Deployment_Dev
displayName: "Deployment of IaC templates - Dev"
dependsOn: [ Validation_Dev ]
# condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI'))

jobs:
- template: templates/deployment.yml
parameters:
environment_name: ${{ variables.DEV_ENVIRONMENT_NAME }}
variable_group_name: ${{ variables.DEV_VARIABLE_GROUP_NAME }}
61 changes: 61 additions & 0 deletions .ado/workflows/dataProductDeploymentTestProd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Data Product Deployment - Test/Prod

trigger:
branches:
include:
- main
- releases/*
tags:
include:
- v*
paths:
include:
- code/*
- infra/*
- .ado/workflows/*

variables:
TEST_ENVIRONMENT_NAME: "test" # Update to '{testEnvironmentName}'
TEST_VARIABLE_GROUP_NAME: "test-product-streaming" # Update to '{testVariableGroupName}'
PROD_ENVIRONMENT_NAME: "prod" # Update to '{prodEnvironmentName}'
PROD_VARIABLE_GROUP_NAME: "prod-product-streaming" # Update to '{prodVariableGroupName}'

stages:
- stage: Validation_Test
displayName: "Validation of IaC templates - Test"
continue-on-error: false
jobs:
- template: templates/validation.yml
parameters:
environment_name: ${{ variables.TEST_ENVIRONMENT_NAME }}
variable_group_name: ${{ variables.TEST_VARIABLE_GROUP_NAME }}

- stage: Validation_Prod
displayName: "Validation of IaC templates - Prod"
jobs:
- template: templates/validation.yml
parameters:
environment_name: ${{ variables.PROD_ENVIRONMENT_NAME }}
variable_group_name: ${{ variables.PROD_VARIABLE_GROUP_NAME }}

- stage: Deployment_Test
displayName: "Deployment of IaC templates - Test"
dependsOn: [ Validation_Test, Validation_Prod ]
# condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI'))

jobs:
- template: templates/deployment.yml
parameters:
environment_name: ${{ variables.TEST_ENVIRONMENT_NAME }}
variable_group_name: ${{ variables.TEST_VARIABLE_GROUP_NAME }}

- stage: Deployment_Prod
displayName: "Deployment of IaC templates - Prod"
dependsOn: [ Deployment_Test ]
# condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI'))

jobs:
- template: templates/deployment.yml
parameters:
environment_name: ${{ variables.PROD_ENVIRONMENT_NAME }}
variable_group_name: ${{ variables.PROD_VARIABLE_GROUP_NAME }}
61 changes: 61 additions & 0 deletions .ado/workflows/templates/deployment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
parameters:
- name: environment_name
displayName: Environment Name
type: string
- name: variable_group_name
displayName: Variable Group Name
type: string

jobs:
- job: Deployment
displayName: "Deployment to ${{ parameters.environment_name }}"
continueOnError: false
pool:
vmImage: "ubuntu-latest"
variables:
- group: ${{ parameters.variable_group_name }}

steps:
# Checkout code
- checkout: self
name: checkout_repository
displayName: "Checkout repository"
submodules: true
lfs: false
clean: true
continueOnError: false
enabled: true

# Generate Password 001
- task: PowerShell@2
name: generate_password_001
displayName: Generate Password 001
enabled: true
continueOnError: false
inputs:
targetType: "filePath"
filePath: "$(System.DefaultWorkingDirectory)/code/GeneratePassword.ps1"
errorActionPreference: "stop"
failOnStderr: false
ignoreLASTEXITCODE: false
pwsh: true

# Deploy Data Product
- task: AzureResourceManagerTemplateDeployment@3
name: data_product_deployment
displayName: Deploy Data Product
enabled: true
continueOnError: false
inputs:
deploymentScope: "Resource Group"
azureResourceManagerConnection: $(AZURE_RESOURCE_MANAGER_CONNECTION_NAME)
subscriptionId: $(AZURE_SUBSCRIPTION_ID)
action: "Create Or Update Resource Group"
resourceGroupName: $(AZURE_RESOURCE_GROUP_NAME)
location: $(AZURE_LOCATION)
templateLocation: "Linked artifact"
csmFile: "$(System.DefaultWorkingDirectory)/infra/main.json"
csmParametersFile: "$(System.DefaultWorkingDirectory)/infra/params.${{ parameters.environment_name }}.json"
deploymentMode: "Incremental"
overrideParameters: >
-administratorPassword "$(password)"
89 changes: 89 additions & 0 deletions .ado/workflows/templates/validation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
parameters:
- name: environment_name
displayName: Environment Name
type: string
- name: variable_group_name
displayName: Variable Group Name
type: string

jobs:
- job: Validation
displayName: "Validation of ${{ parameters.environment_name }}"
continueOnError: false
pool:
vmImage: "ubuntu-latest"
variables:
- group: ${{ parameters.variable_group_name }}

steps:
# Checkout code
- checkout: self
name: checkout_repository
displayName: "Checkout repository"
submodules: true
lfs: false
clean: true
continueOnError: false
enabled: true

# Generate Password 001
- task: PowerShell@2
name: generate_password_001
displayName: Generate Password 001
enabled: true
continueOnError: false
inputs:
targetType: "filePath"
filePath: "$(System.DefaultWorkingDirectory)/code/GeneratePassword.ps1"
errorActionPreference: "stop"
failOnStderr: false
ignoreLASTEXITCODE: false
pwsh: true

# Deploy Data Product - validation
- task: AzureResourceManagerTemplateDeployment@3
name: data_product_validation
displayName: Deploy Data Product - validation
enabled: true
continueOnError: false
inputs:
deploymentScope: "Resource Group"
azureResourceManagerConnection: $(AZURE_RESOURCE_MANAGER_CONNECTION_NAME)
subscriptionId: $(AZURE_SUBSCRIPTION_ID)
action: "Create Or Update Resource Group"
resourceGroupName: $(AZURE_RESOURCE_GROUP_NAME)
location: $(AZURE_LOCATION)
templateLocation: "Linked artifact"
csmFile: "$(System.DefaultWorkingDirectory)/infra/main.json"
csmParametersFile: "$(System.DefaultWorkingDirectory)/infra/params.${{ parameters.environment_name }}.json"
deploymentMode: "Validation"
overrideParameters: >
-administratorPassword "$(password)"

# Deploy Data Product - what-if
- task: AzureCLI@2
name: data_product_whatif
displayName: Deploy Data Product - what-if
enabled: true
continueOnError: false
inputs:
azureSubscription: $(AZURE_RESOURCE_MANAGER_CONNECTION_NAME)
scriptType: pscore
scriptLocation: inlineScript
inlineScript: |
az account set `
--subscription $(AZURE_SUBSCRIPTION_ID)

az deployment group what-if `
--resource-group $(AZURE_RESOURCE_GROUP_NAME) `
--exclude-change-types Ignore NoChange Unsupported `
--mode "Incremental" `
--template-file "$(System.DefaultWorkingDirectory)/infra/main.json" `
--parameters "$(System.DefaultWorkingDirectory)/infra/params.${{ parameters.environment_name }}.json" administratorPassword="$(password)" `
--result-format "FullResourcePayloads"

powerShellErrorActionPreference: "stop"
addSpnToEnvironment: false
useGlobalConfig: false
failOnStandardError: false
powerShellIgnoreLASTEXITCODE: false
6 changes: 3 additions & 3 deletions .github/ISSUE_TEMPLATE/BUG_REPORT.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this report! Before submitting, please make sure to [search through existing issues](https://github.com/Azure/data-product-streaming/issues) and [check the Known Issues section](/docs/DataManagementAnalytics-KnownIssues.md).
Thanks for taking the time to fill out this report! Before submitting, please make sure to [search through existing issues](https://github.com/Azure/data-product-streaming/issues) and [check the Known Issues section](https://github.com/Azure/data-product-streaming/blob/main/docs/EnterpriseScaleAnalytics-KnownIssues.md).

- type: dropdown
id: deployment
Expand Down Expand Up @@ -39,7 +39,7 @@ body:
label: Error Message
description: If possible, please share the error message that you received.
placeholder: Error Messages
render: json
render: JSON
validations:
required: false

Expand All @@ -58,7 +58,7 @@ body:
label: Code of Conduct
description: The Code of Conduct helps create a safe space for everyone. We require that everyone must abide by it.
options:
- label: I agree to follow this project's [Code of Conduct](/CODE_OF_CONDUCT.md)
- label: I agree to follow this project's [Code of Conduct](https://github.com/Azure/data-product-streaming/blob/main/CODE_OF_CONDUCT.md)
required: true

- type: markdown
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/DOCUMENTATION_ISSUE.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this report! Before submitting, please make sure to [search through existing issues](https://github.com/Azure/data-product-streaming/issues) and [check the Known Issues section](/docs/DataManagementAnalytics-KnownIssues.md).
Thanks for taking the time to fill out this report! Before submitting, please make sure to [search through existing issues](https://github.com/Azure/data-product-streaming/issues) and [check the Known Issues section](https://github.com/Azure/data-product-streaming/blob/main/docs/EnterpriseScaleAnalytics-KnownIssues.md).

- type: textarea
id: description
Expand All @@ -25,7 +25,7 @@ body:
label: Code of Conduct
description: The Code of Conduct helps create a safe space for everyone. We require that everyone must abide by it.
options:
- label: I agree to follow this project's [Code of Conduct](/CODE_OF_CONDUCT.md)
- label: I agree to follow this project's [Code of Conduct](https://github.com/Azure/data-product-streaming/blob/main/CODE_OF_CONDUCT.md)
required: true

- type: markdown
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/FEATURE_REQUEST.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this report! Before submitting, please make sure to [search through existing issues](https://github.com/Azure/data-product-streaming/issues) and [check the Known Issues section](/docs/DataManagementAnalytics-KnownIssues.md).
Thanks for taking the time to fill out this report! Before submitting, please make sure to [search through existing issues](https://github.com/Azure/data-product-streaming/issues) and [check the Known Issues section](https://github.com/Azure/data-product-streaming/blob/main/docs/EnterpriseScaleAnalytics-KnownIssues.md).

- type: textarea
id: feature_description
Expand All @@ -34,7 +34,7 @@ body:
label: Code of Conduct
description: The Code of Conduct helps create a safe space for everyone. We require that everyone must abide by it.
options:
- label: I agree to follow this project's [Code of Conduct](/CODE_OF_CONDUCT.md)
- label: I agree to follow this project's [Code of Conduct](https://github.com/Azure/data-product-streaming/blob/main/CODE_OF_CONDUCT.md)
required: true

- type: markdown
Expand Down
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [ ] Closes Issue #xxx
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Code of Conduct signed. If not, go to [Code of Conduct](https://github.com/Azure/data-management-zone/blob/main/CODE_OF_CONDUCT.md).

<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Loading