Pre-work Checklist: DevOps-Security-Member: drakeredwind01

[drakeredwind01]

Prerequisite

We are looking forward to having you on our team. Please ensure you have prior experience with the HfLA website team before contributing to our repository.

Overview

As a new member on the HfLA devops-security team, fill in the following fields as you complete each onboarding item.

Special Notes

1. Keep this issue open until you've completed all steps, including learning to provide updates for longer tasks.
2. Normally, handle one issue at a time, but this one is an exception as it teaches how to manage multiple tasks.
3. Work on action items sequentially, but proceed if possible. For example, set up your dev environment without waiting for the weekly meeting.

Action Items

• Before starting to work on the below instructions, make sure to join the #ops Slack Channel. And are a member of devops-security repository.

• Self-assign this issue (gear in right side panel).

• Add this issue to the Project Board under the Projects section (gear in right side panel).

• Attend weekly team meeting, Wednesdays 6-8pm PST.

  • Note: There are no meetings on the 1st-7th of every month.

• Complete the steps in Creating a personal AWS account and Login as root user & setup MFA.

• Read and follow the instructions in Setting up IAM and AWS CLI for:

  • Creating an IAM User
  • Creating an IAM Group
  • Attaching IAM user to IAM Group
  • Providing AdministratorAccess policy to IAM group
  • Log in as the newly created user instead of continuing to log in as the root user (it is not recommended to login with root access).
  • Generating user access keys

• Complete the instructions in AWS Documentation and choose your operating system to install AWS CLI.

• Complete the instruction in AWS Documentation to setup the AWS CLI.

• Read follow the instructions in Creating a backend state.

• Install Terraform locally by following the instructions of the installation guide mentioned in Installing Terraform

• Install Terraform Docs locally by following the instructions of the installation guide mentioned in Installing Terraform docs

• Complete the instructions in Clone the repository

• Submit a new request to create new AWS user account and then self-assign this issue.

• Create a new branch from main by executing the command

  git checkout -b issue-number-add-new-iam-user
  

• Navigate to the aws-user.tf file and add your user information and follow the below template.

  module "iam_user_testiamuser" {
  source = "./modules/aws-users"
  
  user_name = "testiamuser"
  user_tags = {
    "Project"      = "devops-security"
    "Access Level" = "1"
  }
  user_groups = ["read-only-group"]
  }
  

• In your code editor navigate to terraform directory. cd terraform

• Execute the command terraform init to initialize terraform in the directory. Address any failures that arise (if any).

• Execute the command terraform plan this will output a plan replicating the same IAM resources as the devops security account. Address any failures that arise (if any).

• Then execute the command terraform apply this will create all of the resources that are currently managed by Devops Security. All of the resources created here incur zero cost except for the Dynamo DB installation, which should remain in the free tier.
  - [ ] ** If you have cost concerns, Run a Terraform Destroy to take down all of the resources you created (don't worry, you can recreate them just as quickly). **

• Once you have tested your changes, stage them in git with

  • git status command.
  • then git add path/to/file (you can copy from above output for the file path).

• Commit the changes by executing git commit -m "briefly describing the changes".

• Push the changes with git push --set-upstream origin name-of-branch

[drakeredwind01]

look for HFLA note: for notes

prerequisites needs a new title and the template needs real prerequisites

intro

We are looking forward to having you on our team. Please ensure you have prior experience with the HfLA website team before contributing to our repository.

features

• special instructions for windows included

prerequisites

• join
  • added hyperlinks for ease ops Slack Channel
  • added hyperlinks for ease devops-security repository

Action Items

• Add this issue to the Project Board under the Projects section (gear in right side panel). # add CoP: DevOps: Project Board

• Add this issue to the Project Board under the Projects section (gear in right side panel). as title CoP: DevOps: Project Board

• Attend weekly team meeting, Wednesdays 6-8pm PST.
  • Note: There are no meetings on the 1st-7th of every month. # should be attention getting instead of a step
    Note: There are no meetings on the 1st-7th of every month.

AWS

• for the following watch video guide Getting Started With AWS Cloud | Step-by-Step Guide
  • create MFA for ROOT
  • Complete Creating a personal AWS account
    • HFLA note: link was broken Login as root user & setup MFA
    • HFLA note: link led to creating iam user group not user Creating an IAM User

AWS IAM new user

^ part of Getting Started With AWS Cloud | Step-by-Step Guide

• create admin user
  • click users category on left
  • click add user
  • enter username
  • click "Provide user access to the AWS Management Console - optional"
  • select "create IAM user"
  • click "custom password"
  • enter password
  • unclick "Users must create a new password at next sign-in - Recommended"
  • click next
  • click "Attach policies directly"
  • only check "AdministratorAccess"
  • click "next"
  • click "create user"
  • save "Console sign-in URL" to bookmark for easy access
  • sign in to new admin user

AWS CLI

HFLA note: maybe this process can be skipped using the following code in cloudshell or local
aws iam create-user --user-name drakeredwind01
aws iam create-login-profile --user-name drakeredwind01 --password Black2BlackHFLA
aws iam create-access-key --user-name drakeredwind01 > drakeredwind01_access_key.json
nano drakeredwind01_access_key.json

^ part of Getting Started With AWS Cloud | Step-by-Step Guide

• go to AWS CLI
• click "getting started" (under the big "1")
• SideNav click Get started > Install/Update
  HFLA note: I had to do this part again. i did it before but the aws commands weren't working on my computer
• for windows
  • in CMD run the following:
    • msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi
    • confirm the installation by running:
    • aws --version
    • for instance mine said "aws-cli/2.17.39 Python/3.11.9 Windows/10 exe/AMD64"
• link AWS CLI with your account
  • go to IAM > Security credentials
  • click "create Access key"
  • click "use case" > "Command Line Interface (CLI)"
  • click "I understand the above recommendation and want to proceed to create an access key."
  • click "next"
  • click "create access key"
  • save your "Access key" and "Secret access key"
  • check to see it worked by using CMD to enter:
  • aws
  • configure it:
    • enter into cmd: "aws configure"
    • enter your "Access key" you saved
    • enter your "Secret access key" you saved
    • enter "Default region name" "us-west-2"
    • enter "Default output format" "json"
    • check to see it worked
    • enter: "aws s3 ls"
    • optional: for more commands you can go to Use the AWS CLI
    • optional: save the above link in bookmarks
  • in CMD run following:
    • aws iam create-group --group-name AdminGroup
    • aws iam add-user-to-group --group-name AdminGroup --user-name HFLA
    • aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AdministratorAccess --group-name AdminGroup

HFLA note: maybe replace?

• Read and follow the instructions in Setting up IAM and AWS CLI for:
  • Creating an IAM Group
  • Attaching IAM user to IAM Group
  • Providing AdministratorAccess policy to IAM group
  • Log in as the newly created user instead of continuing to log in as the root user (it is not recommended to login with root access).
  • Generating user access keys

AWS CLI quick quide

aws iam create-group --group-name AdminGroup
aws iam create-user --user-name drakeredwind01
aws iam create-login-profile --user-name drakeredwind01 --password Black2BlackHFLA
aws iam add-user-to-group --group-name AdminGroup --user-name drakeredwind01
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AdministratorAccess --group-name AdminGroup
aws iam create-access-key --user-name drakeredwind01 > drakeredwind01_access_key.json
nano drakeredwind01_access_key.json

• HFLA note: maybe remove? [x] Complete the instructions in AWS Documentation and choose your operating system to install AWS CLI.
• HFLA note: maybe remove? [x] Complete the instruction in AWS Documentation to setup the AWS CLI.

oldbackend state

• Read follow the instructions in Creating a backend state.
• Create S3 bucket
  • Region: us-west-2 (Oregon)
  • Name: hfla-ops-terraform-state
  • Enable versioning
  • Enable server-side encryption
• Set up DynamoDB to store backend
  • Create table hfla_ops_terraform_table
  • Set partition key to LockID with a type of String
  • Choose on-demand capacity

aws s3api put-bucket-encryption --bucket drakeredwind01-hfla-ops-terraform-state --server-side-encryption-configuration '{
"Rules": [
{
"ApplyServerSideEncryptionByDefault": {
"SSEAlgorithm": "AES256"
}
}
]
}'

Resources/Instructions

https://developer.hashicorp.com/terraform/language/settings/backends/s3

newbackend state

• Read follow the instructions in Creating a backend state.
• Create S3 bucket
  • added for ease search for s3
  • AWS Region: US West (Oregon) us-west-2
  • ? Bucket type General purpose or Directory - New
  • Bucket name hfla-ops-terraform-state
  • Enable versioning
  • ? Enable server-side encryption
    • ? Server-side encryption with Amazon S3 managed keys (SSE-S3)
    • ? Server-side encryption with AWS Key Management Service keys (SSE-KMS)
  • ? Bucket Key default Enable
• Set up DynamoDB to store backend
  • added for ease search for DynamoDB
  • added for ease Table details
    • Create table hfla_ops_terraform_table
    • Set partition key to LockID with a type of String
    • Choose on-demand capacity
  • added for ease Table settings
    • added Customize settings
  • added for ease Read/write capacity settings
    • On-demand
  • ? Deletion protection

Terraform

• Install Terraform locally by following the instructions of the installation guide mentioned in Installing Terraform

  • (windows) make sure terraform is in your path

• Install Terraform Docs locally by following the instructions of the installation guide mentioned in Installing Terraform docs

  WINDOWS

  • if you have windows you first need scoop
  • run scoop command

    • scoop bucket add terraform-docs https://github.com/terraform-docs/scoop-bucket
      scoop install terraform-docs

    • if you don't have scoop run install it by running the following in powershell

      • iex (new-object net.webclient).DownloadString('https://get.scoop.sh')

      • if you get following error in red letters run the bellow command PowerShell requires an execution policy in [Unrestricted, RemoteSigned, ByPass] to run Scoop. For example, to set the execution policy to 'RemoteSigned' please run 'Set-ExecutionPolicy RemoteSigned -Scope CurrentUser'.

        • Set-ExecutionPolicy RemoteSigned -Scope CurrentUser

• Complete the instructions in Clone the repository

• Submit a new request to create new AWS user account and then self-assign this issue.

  • ? {github handle} or {service account}. which service account? the one for aws? Account Name:
  • Project(s) Name: https://github.com/hackforla/devops-security.git
  • ? no instructions given. should i put in: in order to complete "Pre-work Checklist: DevOps-Security-Member: drakeredwind01 #36" Reason for access:

• Create a new branch from main by executing the command

  git checkout -b issue-number-add-new-iam-user

  • added for ease example

    git checkout -b 36-add-new-iam-user-drakeredwind01

HFLA note: potentialy deprecated (unsure if you should add a user)

• Navigate to the aws-user.tf file and add your user information and follow the below template.

  module "iam_user_testiamuser" {
  source = "./modules/aws-users"
  
  user_name = "testiamuser"
  user_tags = {
    "Project"      = "devops-security"
    "Access Level" = "1"
  }
  user_groups = ["read-only-group"]
  }

  example

      module "iam_user_drakeredwind01" {
        source = "./modules/aws-users"
      
        user_name = "drakeredwind01"
        user_tags = {
          "Project"      = "devops-security"
          "Access Level" = "1"
        }
        user_groups = ["read-only-group"]
      }

• In your code editor navigate to terraform directory. cd terraform

left off

• Execute the command terraform init to initialize terraform in the directory. Address any failures that arise (if any).
  • if using pycharm go to settings then plugins
    • install AWS Core
    • install AWS Toolkit
    • restart pycharm
      • on the left side bar you will see project,commit,pull request,structure,AWS Toolkit
    • click AWS Toolkit
    • click Add Another Connection
    • click IAM Credentials
    • click Continue
    • enter profile name,access key,secret key found in access_key.json
      • go to cloudshell and type nano access_key.json
    • click Continue
  • if after input terraform init get message

      ╷
      │ Error: validating provider credentials: retrieving caller identity from STS: operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: 38c7ba80-a5e4-4a1b-bf31-589bb74b8f4a, api error InvalidClientTokenId: The security token included in the request is invalid.
      │ 
      │ 
      ╵
    

    Configure the AWS CLI with IAM Identity Center authentication

    from https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html#cli-configure-sso-prereqs
      - (Recommended) SSO token provider configuration.
        if you do not have established access through IAM Identity Center
        - #### Get started with common tasks in IAM Identity Center
    

Configure the AWS CLI with IAM Identity Center authentication

from https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html#cli-configure-sso-prereqs
- (Recommended) SSO token provider configuration.
if you do not have established access through IAM Identity Center

Get started with common tasks in IAM Identity Center

extra THE BLOCKER DESTROYED

put in cloud shell
aws dynamodb create-table \ --table-name hfla_ops_terraform_table \ --attribute-definitions AttributeName=LockID,AttributeType=S \ --key-schema AttributeName=LockID,KeyType=HASH \ --billing-mode PAY_PER_REQUEST
if .terraform file does not have providers
delete .terraform
run terraform init --backend-config=backend.tfvars

• Execute the command terraform plan this will output a plan replicating the same IAM resources as the devops security account. Address any failures that arise (if any).
• Then execute the command terraform apply -auto-approve this will create all of the resources that are currently managed by Devops Security. All of the resources created here incur zero cost except for the Dynamo DB installation, which should remain in the free tier.
  - [x] ** If you have cost concerns, Run a Terraform Destroy to take down all of the resources you created (don't worry, you can recreate them just as quickly). **
  HFLA note: how to test your changes as mentioned below
• Once you have tested your changes, stage them in git with
  • add your profile to aws-users.tf
  • git status command.
  • then git add path/to/file/aws-users.tf (you can copy from above output for the file path).
  • make sure you added it by using get status again
• Commit the changes by executing git commit -m "briefly describing the changes".
• example git commit -m "drakeredwind01 added to aws-users.tf".
• Push the changes with git push --set-upstream origin name-of-branch
• example git push --set-upstream origin git push --set-upstream origin 36-add-new-iam-user-drakeredwind01

[ExperimentsInHonesty]

@drakeredwind01 please make a new issue that outlines the problem like you have done in this issue... but that is specific to the changes that you are suggesting. that way, we can give your issue (this one) back to you and still have an issue specifically for updating the prework template.

[drakeredwind01]

@ExperimentsInHonesty
Thank you for your advice. I believe it may be easier (after looking at it) to finish all the changes first and then create the separate issues from the derivatives, as I’m constantly restructuring the list.

Please let me know if you would prefer to have play-by-play issues instead of final, concrete suggested changes and an overall example, which is what this current method would ultimately provide. I can see the merits of both methods.

[ExperimentsInHonesty]

@drakeredwind01 please document all the changes you are suggesting in a comment on this issue.

[drakeredwind01]

zz Pre-work Checklist DevOps-Security-Member drakeredwind01 #36

20240626T182821Z

Work through this template
#36

Made from Issue: Pre-work Template - DevOps-Security
Add comments as working through template
Ask questions here:
Questions / Answers from ops members #93
hackforla/ops#93

Slack ops channel
https://app.slack.com/client/T04502KQX/CV7QGL66B

issue and possible solutions

My problem
I get error 
╷
│ Error: Required plugins are not installed
│
│ The installed provider plugins are not consistent with the packages selected in the dependency lock file:
│   - registry.terraform.io/hashicorp/aws: there is no package for registry.terraform.io/hashicorp/aws 5.34.0 cached in .terraform\providers
│
│ Terraform uses external plugins to integrate with a variety of different infrastructure services. To download the
│ plugins required for this configuration, run:
│   terraform init
╵

I think the important part is:

│ - registry.terraform.io/hashicorp/aws: there is no package for registry.terraform.io/hashicorp/aws 5.34.0 cached in .terraform\providers

Possible solution:

https://github.com/gruntwork-io/terragrunt/issues/1960
It seems they fixed this by deleting “.terraform.lock.hcl”

Can confirm removing the lock files solves it, but you need to do it on the current module and any it depends on.
terragrunt version v0.36.1
Terraform v1.1.5
on darwin_amd64

Itsayellow says:
He also had this issue and deleting the terraform lock files but that wasn't helping.
But “Finally I found if I deleted ./.terragrunt-cache from where I was executing terragrunt, it finally worked without error for me.”

Extra advice:

luogedai commented on Jan 13, 2022
If you have dependencies, try to clear and init or upgrade the dependencies first. This step solved my issue.

AWS support

connectria
possible aws support through 3rd party (still waiting for them to get back to me)

connectria.com
A dedicated team of AWS experts, available 24/7.
Markus will leave number for sales team to see if we can have a small consultation fee for help

AWS Support plans

https://us-east-1.console.aws.amazon.com/support/plans/home#/
https://aws.amazon.com/premiumsupport/pricing/ 

[drakeredwind01]

QUEST COMPLETED!

[sudhara]

@drakeredwind01

Please provide update

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.

[drakeredwind01]

Just complexly finished the prework checklist and altered the "aws-users.tf" file and pushed, and created a merge request for approval successfully.

I am also searching for more CloudShell commands to integrate a more programmatic process for newer members to enhance the cohesion process.

[drakeredwind01]

2. blockers
   the blockers have been systematicly anihilated and accounted for in the new tut I created for a smooth induction process.

3. my time is highly variable in this uncertain time in my life. I have already devote much time into this singular process and will continue to work on the process to enhance the new user experience.

4. finished

5. link
   link for the pull request: 36 add new iam user drakeredwind01 #83