Skip to content

Commit

Permalink
docs: update documentation for release
Browse files Browse the repository at this point in the history
Signed-off-by: Kevin Lefevre <lefevre.kevin@gmail.com>
  • Loading branch information
ArchiFleKs committed Feb 20, 2020
1 parent 08b5039 commit 2444ad6
Show file tree
Hide file tree
Showing 13 changed files with 395 additions and 152 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ The main additionals features are the curated addons list, see [here](https://gi
* Default PSP is removed and sensible defaults are enforced
* All addons have specific PSP enabled
* No IAM credentials on instances, everything is enforced with [IRSA](https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/) or [KIAM](https://github.com/uswitch/kiam)
* Each addons is deployed in it's own namespace with sensible default network policies

### Out of the box monitoring

Expand Down
17 changes: 17 additions & 0 deletions docs/user-guides/ecr.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# ECR module

`ecr` is a custom module maintained [here](https://github.com/clusterfrak-dynamics/terraform-aws-ecr) and allows creation of AWS ECR repository to host docker images.

It provides:

* ECR repository
* ECR repository policy
* Custom IAM user with Access Key and Secret Key to be able to push images to ECR (eg. for CI purposes)

## Customization

Just like the other modules, custom configuration is done in `terragrunt.hcl`.

```json
{!terraform/live/demo/eu-west-3/ecr/terragrunt.hcl!}
```
143 changes: 143 additions & 0 deletions docs/user-guides/eks-addons.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# EKS addons module

`eks-addons` is a custom module maintained [here](https://github.com/clusterfrak-dynamics/terraform-kubernetes-addons) and provides either:

* helm v3 charts
* manifests
* operators

For commonly used addons one Kubernetes and most specifically with EKS. The deployments are curated to be tightly integrated with AWS and EKS.

The following addons are available and work out of the box.

## Helm charts

All charts have been tested with Helm v3 and the `terraform-provider-helm` v1.0 which supports Helm v3. They can be easily customize with custom values.

* [cluster-autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler): scale worker nodes based on workload.
* [external-dns](https://github.com/kubernetes-incubator/external-dns): sync ingress and service records in route53.
* [cert-manager](https://github.com/jetstack/cert-manager): automatically generate TLS certificates, supports ACME v2.
* [kiam](https://github.com/uswitch/kiam): prevents pods to access EC2 metadata and enables pods to assume specific AWS IAM roles.
* [nginx-ingress](https://github.com/kubernetes/ingress-nginx): processes *Ingress* object and acts as a HTTP/HTTPS proxy (compatible with cert-manager).
* [metrics-server](https://github.com/kubernetes-incubator/metrics-server): enable metrics API and horizontal pod scaling (HPA).
* [prometheus-operator](https://github.com/coreos/prometheus-operator): Monitoring / Alerting / Dashboards.
* [fluentd-cloudwatch](https://github.com/helm/charts/tree/master/incubator/fluentd-cloudwatch): forwards logs to AWS Cloudwatch.
* [node-problem-detector](https://github.com/kubernetes/node-problem-detector): Forwards node problems to Kubernetes events
* [flux](https://github.com/weaveworks/flux): Continuous Delivery with Gitops workflow.
* [sealed-secrets](https://github.com/bitnami-labs/sealed-secrets): Technology agnostic, store secrets on git.
* [kong](https://konghq.com/kong): API Gateway ingress controller.
* [keycloak](https://www.keycloak.org/) : Identity and access management

## Kubernetes Manifests

Kubernetes manifests are deployed with [`terraform-provider-kubectl`](https://github.com/gavinbunney/terraform-provider-kubectl)

* [cni-metrics-helper](https://docs.aws.amazon.com/eks/latest/userguide/cni-metrics-helper.html): Provides cloudwatch metrics for VPC CNI plugins.

## Operator

Some project are transitioning to [Operators](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/). Istio is going to drop Helm support and is not compatible with Helm v3 so it has been removed and replaced with the [Istio operator](https://istio.io/blog/2019/introducing-istio-operator/)

* [istio-operator](https://istio.io): Service mesh for Kubernetes.

## IAM permissions

Some addons require specific IAM permission. This can be done by either:

* IRSA: [IAM role for service account](https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/) which is the default and recommended way
* [Kiam](https://github.com/uswitch/kiam)

Addons that need IAM access have two variables:

* `create_resources_irsa`: default to true and uses IAM role for service account
* `create_resources_kiam`: default to false and uses KIAM to manage IAM permissions

There is no specific config, everything is taken care of by the module.

## Customization

All the configuration is done in `eks-addons/terragrunt.hcl`.

```json
{!terraform/live/demo/eu-west-3/eks-addons/terragrunt.hcl!}
```

### Default charts values

Some values are defined by default directly into the module. These can off course be overridden and or merged/replaced. You can find the defaults values in the [upstream module](https://github.com/clusterfrak-dynamics/terraform-kubernetes-addons). Eg. default values for `cluster-autoscaler` are in [`cluster-autoscaler.tf`](https://github.com/clusterfrak-dynamics/terraform-kubernetes-addons/blob/master/cluster-autoscaler.tf)

### Overriding Helm provider values

Helm provider have defaults values defined [here](https://github.com/clusterfrak-dynamics/terraform-kubernetes-addons/blob/master/locals.tf):

```json
helm_defaults_defaults = {
atomic = false
cleanup_on_fail = false
dependency_update = false
disable_crd_hooks = false
disable_webhooks = false
force_update = false
recreate_pods = false
render_subchart_notes = true
replace = false
reset_values = false
reuse_values = false
skip_crds = false
timeout = 3600
verify = false
wait = true
extra_values = ""
}
```

These can be overridden globally with the `helm_defaults` input variable or can be overridden per chart in `terragrunt.hcl`:

```json
helm_defaults = {
replace = true
verify = true
timeout = 300
}


cluster_autoscaler = {
create_iam_resources_kiam = false
create_iam_resources_irsa = true
iam_policy_override = ""
version = "v1.14.7"
chart_version = "6.4.0"
enabled = true
default_network_policy = true
cluster_name = dependency.eks.outputs.cluster_id
timeout = 3600 <= here you can add any helm provider override
}
```

### Overriding charts values.yaml

It is possible to add or override values per charts. Helm provider use the same merge logic as Helm so you can basically rewrite the whole `values.yaml` if needed.

Each chart has a `extra_values` variable where you can specify custom values.

```json
flux = {
create_iam_resources_kiam = false
create_iam_resources_irsa = true
version = "1.18.0"
chart_version = "1.2.0"
enabled = false
default_network_policy = true

extra_values = <<EXTRA_VALUES
git:
url: "ssh://git@gitlab.com/myrepo/gitops-${local.env}.git"
pollInterval: "2m"
rbac:
create: false
registry:
automationInterval: "2m"
EXTRA_VALUES
```

There are some examples in the `terragrunt.hcl` file. Not all the variables available are present. If you want a full list of variable, you can find them in the [upstream module](https://github.com/clusterfrak-dynamics/terraform-kubernetes-addons). For example for `cluster-autoscaler` you can see the default [here](https://github.com/clusterfrak-dynamics/terraform-kubernetes-addons/blob/master/cluster-autoscaler.tf#L2).
16 changes: 16 additions & 0 deletions docs/user-guides/eks-namespaces.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# EKS namespaces module

`eks-namespace` is a custom module maintained [here](https://github.com/clusterfrak-dynamics/terraform-kubernetes-namespaces) and allow to create and manage Kubernetes namespaces with Terragrunt/Terraform.

It provides:

* Namespaces [quotas](https://kubernetes.io/docs/concepts/policy/resource-quotas/)
* Namesapces [limitranges](https://kubernetes.io/docs/concepts/policy/limit-range/)

## Customization

Just like the other modules, custom configuration is done in `terragrunt.hcl`. It takes a list of namespaces with their configuration as an input variable.

```json
{!terraform/live/demo/eu-west-3/eks-namespaces/terragrunt.hcl!}
```
86 changes: 86 additions & 0 deletions docs/user-guides/eks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# EKS module

## Upstream configuration

EKS module is also [upstream](https://github.com/terraform-aws-modules/terraform-aws-eks) and allow to deploy an EKS cluster which supports:

* managed node pools
* launch configuration node pools
* launch template node pools

tEKS uses launch template by default and use one node pool per availability zone.

You can use any inputs from the upstream module to configure the cluster in `eks/terragrunt.hcl`.

```json
{!terraform/live/demo/eu-west-3/eks/terragrunt.hcl!}
```

## Customizations

Besides the upstream module, there are some customizations. If you look at the directory structure:

```bash
.
├── manifests
│   ├── calico.yaml
│   ├── psp-default-clusterrole.yaml
│   ├── psp-default-clusterrolebinding.yaml
│   ├── psp-default.yaml
│   ├── psp-privileged-clusterrole.yaml
│   ├── psp-privileged-clusterrolebinding.yaml
│   ├── psp-privileged-node-rolebinding.yaml
│   └── psp-privileged.yaml
├── manifests.tf
├── providers.tf
└── terragrunt.hcl
```

### Terragrunt hooks

In addition to the upstream module there is some `hooks` included, these `hooks` can be remove if necessary. In order:

* Download the `terraform-provider-kubectl` to manage manifests (necessary if using `manifests.tf)
* Copy the kubeconfig locally
* Label the `kube-system` namespace with its name
* Remove the defaults EKS podSecurityPolicies (see [#401](https://github.com/aws/containers-roadmap/issues/401))

### Custom manifests

`terraform-provider-kubectl` allows to deploy Kubernetes manifests in a cleaner way than using a `local-exec`. `manifests.tf` is parsing the manifest folder and applying them to the cluster.

#### Calico

Calico is used to enable network policies enforcement on the cluster. To disable calico you can remove the `calico.yaml` file.

#### Pod Security Policies

The default EKS [Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) is privileged. By default it is removed (by the previous hooks) and more sensible non privileged default pod security policies are deployed:

* all the service account in `kube-system` can use `privileged`

```yaml
{!terraform/live/demo/eu-west-3/eks/manifests/psp-privileged.yaml!}
```

* any other authenticated user can use `default`

```yaml
{!terraform/live/demo/eu-west-3/eks/manifests/psp-default.yaml!}
```


The input variable `psp_privileged_ns` allow to give privileged to services account inside a namespace. Eg. in `terragrunt.hcl`:

```json
psp_privileged_ns = [
"cluster-autoscaler", #waiting for https://github.com/helm/charts/pull/20891
"istio-system" #istio does not support psp by default
]
```

This gives to all the service accounts inside `cluster-autoscaler` and `istio-system` access to the `privileged` pod security policy.



##
105 changes: 105 additions & 0 deletions docs/user-guides/getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Getting started

## Tooling requirements

The necessary tools are in `requirements.yaml` you can install them any way you want, make sure they are available in your $PATH.

The following dependencies are required on the deployer host:

* [Terraform](https://www.terraform.io/intro/getting-started/install.html)
* [Terragrunt](https://github.com/gruntwork-io/terragrunt#install-terragrunt)
* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
* [helm](https://helm.sh/)
* [aws-iam-authenticator](https://github.com/kubernetes-sigs/aws-iam-authenticator)

## AWS requirements

* At least one AWS account
* `awscli` configured ([see installation instructions](https://aws.amazon.com/cli/)) to access your AWS account.
* A route53 hosted zone if you plan to use `external-dns` or `cert-manager` but it is not a hard requirement.

## Getting the template repository

You can either clone the repo locally or generate/fork a template from github.

```bash
git clone https://github.com/clusterfrak-dynamics/teks.git --branch v6.0.0
```

The terraform directory structure is the following:

```bash
.
└── live
└── demo
├── common_tags.yaml
├── common_values.yaml
├── eu-west-3
│   ├── ecr
│   │   ├── provider.tf
│   │   └── terragrunt.hcl
│   ├── eks
│   │   ├── kubeconfig
│   │   ├── manifests
│   │   │   ├── calico.yaml
│   │   │   ├── psp-default-clusterrole.yaml
│   │   │   ├── psp-default-clusterrolebinding.yaml
│   │   │   ├── psp-default.yaml
│   │   │   ├── psp-privileged-clusterrole.yaml
│   │   │   ├── psp-privileged-clusterrolebinding.yaml
│   │   │   ├── psp-privileged-node-rolebinding.yaml
│   │   │   └── psp-privileged.yaml
│   │   ├── manifests.tf
│   │   ├── providers.tf
│   │   └── terragrunt.hcl
│   ├── eks-addons
│   │   ├── examples
│   │   │   ├── keycloak-values.yaml
│   │   │   └── kong-values.yaml
│   │   ├── providers.tf
│   │   └── terragrunt.hcl
│   ├── eks-namespaces
│   │   ├── providers.tf
│   │   └── terragrunt.hcl
│   └── vpc
│   ├── provider.tf
│   └── terragrunt.hcl
└── terragrunt.hcl
```

Each cluster in inside the `terraform/live` folder and then modules are group by AWS region.

## Start a new cluster

Create a new cluster beside `demo`:

```bash
cp -ar demo mycluster
```

## Configuring Terragrunt remote state

`live/demo/terragrunt.hcl` is the parent terragrunt file use to configure remote state.

The configuration is done automatically from the `common_values.yaml` file.

```yaml
{!terraform/live/demo/common_values.yaml!}
```

The values here will generate automatically the parent terragrunt file.

```json
{!terraform/live/demo/terragrunt.hcl!}
```

You can either customize the values or edit directly the `terragrunt.hcl` file.

## Running Terragrunt command

Terragrunt command are run inside their respective folder, for example, to run the `vpc` module:

```bash
cd vpc
terragrunt apply
```
Loading

0 comments on commit 2444ad6

Please sign in to comment.