Merge pull request #2 from turbot/release/v1.0.0

Release/v1.0.0

CHANGELOG.md

@@ -0,0 +1,5 @@
+## v1.0.0 [2024-12-16]
+
+_What's new?_
+
+- Added 109 new 'detect and correct' pipelines to identify Azure resources that are non-compliant with common security and compliance checks. These pipelines can also remediate non-compliant automatically or with approval steps. For usage information and a full list of pipelines, please see [Azure Compliance Mod](https://hub.flowpipe.io/mods/turbot/azure_compliance).

README.md

@@ -1,2 +1,193 @@
-# flowpipe-mod-azure-compliance
-Flowpipe mod for Azure compliance
+# Azure Compliance Mod for Flowpipe
+
+Pipelines to detect and remediate misconfigurations in Azure resources.
+
+## Documentation
+
+- **[Pipelines →](https://hub.flowpipe.io/mods/turbot/azure_compliance/pipelines)**
+
+## Getting Started
+
+### Requirements
+
+Docker daemon must be installed and running. Please see [Install Docker Engine](https://docs.docker.com/engine/install/) for more information.
+
+### Installation
+
+Download and install Flowpipe (https://flowpipe.io/downloads) and Steampipe (https://steampipe.io/downloads). Or use Brew:
+
+```sh
+brew install turbot/tap/flowpipe
+brew install turbot/tap/steampipe
+```
+
+Install the Azure plugin with [Steampipe](https://steampipe.io):
+
+```sh
+steampipe plugin install azure
+```
+
+Steampipe will automatically use your default Azure credentials. Optionally, you can [setup multiple subscriptions](https://hub.steampipe.io/plugins/turbot/azure#multi-subscription-connections) or [customize Azure credentials](https://hub.steampipe.io/plugins/turbot/azure#configuring-azure-credentials).
+
+Create a `connection_import` resource to import your Steampipe Azure connections:
+
+```sh
+vi ~/.flowpipe/config/azure.fpc
+```
+
+```hcl
+connection_import "azure" {
+  source      = "~/.steampipe/config/azure.spc"
+  connections = ["*"]
+}
+```
+
+For more information on importing connections, please see [Connection Import](https://flowpipe.io/docs/reference/config-files/connection_import).
+
+For more information on connections in Flowpipe, please see [Managing Connections](https://flowpipe.io/docs/run/connections).
+
+Install the mod:
+
+```sh
+mkdir azure-compliance
+cd azure-compliance
+flowpipe mod install github.com/turbot/flowpipe-mod-azure-compliance
+```
+
+### Running Detect and Correct Pipelines
+
+To run your first detection, you'll need to ensure your Steampipe server is up and running:
+
+```sh
+steampipe service start
+```
+
+To find your desired detection, you can filter the `pipeline list` output:
+
+```sh
+flowpipe pipeline list | grep "detect_and_correct"
+```
+
+Then run your chosen pipeline:
+
+```sh
+flowpipe pipeline run detect_and_correct_sql_databases_with_public_access_enabled
+```
+
+This will then run the pipeline and depending on your configured running mode; perform the relevant action(s), there are 3 running modes:
+- Wizard
+- Notify
+- Automatic
+
+#### Wizard
+
+This is the `default` running mode, allowing for a hands-on approach to approving changes to resources by prompting for [input](https://flowpipe.io/docs/build/input) for each detected resource.
+
+Whilst the out of the box default is to run the workflow directly in the terminal. You can use Flowpipe [server](https://flowpipe.io/docs/run/server) and [external integrations](https://flowpipe.io/docs/build/input#create-an-integration) to prompt in `http`, `slack`, `teams`, etc.
+
+#### Notify
+
+This mode as the name implies is used purely to report detections via notifications either directly to your terminal when running in client mode or via another configured [notifier](https://flowpipe.io/docs/reference/config-files/notifier) when running in server mode for each detected resource.
+
+To run in `notify` mode, you will need to set the `approvers` variable to an empty list `[]` and ensure the resource-specific `default_action` variable is set to `notify`, either in your `flowpipe.fpvars` file:
+
+```hcl
+approvers = []
+sql_databases_with_public_access_enabled_default_action = "notify"
+```
+
+or pass the `approvers` and `default_action` arguments on the command-line.
+
+```sh
+flowpipe pipeline run detect_and_correct_sql_databases_with_public_access_enabled --arg='default_action=notify' --arg='approvers=[]'
+```
+
+#### Automatic
+
+This behavior allows for a hands-off approach to remediating resources.
+
+To run in `automatic` mode, you will need to set the `approvers` variable to an empty list `[]` and the the resource-specific `default_action` variable to one of the available options in your `flowpipe.fpvars` file:
+
+```hcl
+approvers = []
+sql_databases_with_public_access_enabled_default_action = "revoke_firewall_rule"
+```
+
+or pass the `approvers` and `default_action` argument on the command-line.
+
+```sh
+flowpipe pipeline run detect_and_correct_sql_databases_with_public_access_enabled --arg='approvers=[] --arg='default_action=revoke_firewall_rule'
+```
+
+To further enhance this approach, you can enable the pipelines corresponding [query trigger](#running-query-triggers) to run completely hands-off.
+
+### Running Query Triggers
+
+> Note: Query triggers require Flowpipe running in [server](https://flowpipe.io/docs/run/server) mode.
+
+Each `detect_and_correct` pipeline comes with a corresponding [Query Trigger](https://flowpipe.io/docs/flowpipe-hcl/trigger/query), these are _disabled_ by default allowing for you to _enable_ and _schedule_ them as desired.
+
+Let's begin by looking at how to set-up a Query Trigger to automatically resolve our SQL databases that do not block public access.
+
+Firsty, we need to update our `flowpipe.fpvars` file to add or update the following variables - if we want to run our remediation `hourly` and automatically `apply` the corrections:
+
+```hcl
+sql_databases_with_public_access_enabled_trigger_enabled  = true
+sql_databases_with_public_access_enabled_trigger_schedule = "1h"
+sql_databases_with_public_access_enabled_default_action   = "revoke_firewall_rule"
+```
+
+Now we'll need to start up our Flowpipe server:
+
+```sh
+flowpipe server
+```
+
+This will run every hour and detect SQL databases that do not block public access and apply the corrections without further interaction!
+
+### Configure Variables
+
+Several pipelines have [input variables](https://flowpipe.io/docs/build/mod-variables#input-variables) that can be configured to better match your environment and requirements.
+
+Each variable has a default defined in its source file, e.g, `sql/sql_databases_with_public_access_enabled.fp` (or `variables.fp` for more generic variables), but these can be overwritten in several ways:
+
+The easiest approach is to setup your `flowpipe.fpvars` file, starting with the sample:
+
+```sh
+cp flowpipe.fpvars.example flowpipe.fpvars
+vi flowpipe.fpvars
+
+flowpipe pipeline run detect_and_correct_sql_databases_with_public_access_enabled
+```
+
+Alternatively, you can pass variables on the command line:
+
+```sh
+flowpipe pipeline run detect_and_correct_sql_databases_with_public_access_enabled --var notifier=notifier.default
+```
+
+Or through environment variables:
+
+```sh
+export FP_VAR_notifier="notifier.default"
+flowpipe pipeline run detect_and_correct_sql_databases_with_public_access_enabled
+```
+
+For more information, please see [Passing Input Variables](https://flowpipe.io/docs/build/mod-variables#passing-input-variables)
+
+Finally, each detection pipeline has a corresponding [Query Trigger](https://flowpipe.io/docs/flowpipe-hcl/trigger/query), these are disabled by default allowing for you to configure only those which are required, see the [docs](https://hub.flowpipe.io/mods/turbot/azure_compliance/triggers) for more information.
+
+## Open Source & Contributing
+
+This repository is published under the [Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0). Please see our [code of conduct](https://github.com/turbot/.github/blob/main/CODE_OF_CONDUCT.md). We look forward to collaborating with you!
+
+[Flowpipe](https://flowpipe.io) and [Steampipe](https://steampipe.io) are products produced from this open source software, exclusively by [Turbot HQ, Inc](https://turbot.com). They are distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our [Open Source FAQ](https://turbot.com/open-source).
+
+## Get Involved
+
+**[Join #flowpipe on Slack →](https://turbot.com/community/join)**
+
+Want to help but don't know where to start? Pick up one of the `help wanted` issues:
+
+- [Flowpipe](https://github.com/turbot/flowpipe/labels/help%20wanted)
+- [Azure Compliance Mod](https://github.com/turbot/flowpipe-mod-azure-compliance/labels/help%20wanted)

flowpipe.fpvars.example

@@ -0,0 +1,8 @@
+# Core options
+approvers          = [notifier.default]
+notifier           = notifier.default
+notification_level = "info"
+
+# Advanced options
+database        = connection.steampipe.default
+max_concurrency = 1

locals.fp

@@ -0,0 +1,42 @@
+// Tags
+locals {
+  azure_compliance_common_tags = {
+    category = "Compliance"
+    plugin   = "azure"
+    service  = "Azure"
+  }
+}
+
+// Consts
+locals {
+  level_verbose = "verbose"
+  level_info    = "info"
+  level_error   = "error"
+  style_ok      = "ok"
+  style_info    = "info"
+  style_alert   = "alert"
+}
+
+locals {
+  notification_level_enum = ["verbose", "info", "error"]
+}
+
+// Common Texts
+locals {
+  description_resource         = "The name of the resource"
+  description_database         = "Database connection string."
+  description_approvers        = "List of notifiers to be used for obtaining action/approval decisions."
+  description_connection       = "Name of the connection to be used for any authenticated actions."
+  description_resource_group   = "Azure Resource Group. Examples: my-rg, my-rg-123."
+  description_subscription_id  = "Azure Subscription Id. Examples: d46d7416-f95f-4771-bbb5-529d4c766."
+  description_title            = "Title of the resource, to be used as a display name."
+  description_max_concurrency  = "The maximum concurrency to use for responding to detection items."
+  description_notifier         = "The name of the notifier to use for sending notification messages."
+  description_notifier_level   = "The verbosity level of notification messages to send. Valid options are 'verbose', 'info', 'error'."
+  description_default_action   = "The default action to use when there are no approvers."
+  description_enabled_actions  = "The list of enabled actions to provide to approvers for selection."
+  description_trigger_enabled  = "If true, the trigger is enabled."
+  description_trigger_schedule = "If the trigger is enabled, run it on this schedule."
+  description_items            = "A collection of detected resources to run corrective actions against."
+}
+

mod.fp

@@ -1,10 +1,10 @@
 mod "azure_compliance" {
   title         = "Azure Compliance"
   description   = "Run pipelines to detect and correct Azure resources that are non-compliant."
-  color         = "#ea4335"
+  color         = "#0089D6"
   documentation = file("./README.md")
   icon          = "/images/mods/turbot/azure-compliance.svg"
-  categories    = ["azure", "cost", "compliance", "public cloud"]
+  categories    = ["azure", "compliance", "public cloud", "standard"]
 
   opengraph {
     title       = "Azure Compliance Mod for Flowpipe"
@@ -13,11 +13,14 @@ mod "azure_compliance" {
   }
 
   require {
-    mod "github.com/turbot/flowpipe-mod-azure" {
-      version = "*"
+    flowpipe {
+      min_version = "1.0.0"
     }
     mod "github.com/turbot/flowpipe-mod-detect-correct" {
-      version = "*"
+      version = "^1"
+    }
+    mod "github.com/turbot/flowpipe-mod-azure" {
+      version = "^1"
     }
   }
 }

pipelines/appservice/appservice.fp

@@ -0,0 +1,5 @@
+locals {
+  appservice_common_tags = merge(local.azure_compliance_common_tags, {
+    service = "Azure/AppService"
+  })
+}