Thursday, July 30, 2020

Terraform - first configuration


1. How to install terraform on Mac( i am using macos > 10 , however installation on linux is also a simple process)


Ashoks-MacBook-Pro:~ ashokkafle$ brew install terraform

Updating Homebrew...

==> Auto-updated Homebrew!

Updated 1 tap (homebrew/core).

==> New Formulae

act                   chart-testing         cubejs-cli            git-hooks-go          k9s                   lunchy-go             ory-hydra             scw@1                 thanos

arb                   chrony                dnsprobe              gofish                kona                  mandown               osi                   sdns                  vgrep

argo                  clair                 dosbox-staging        golangci-lint         ksync                 marked                pandoc-include-code   simdjson              wgcf

argocd                cloudformation-cli    duckscript            gradle-profiler       kubie                 naabu                 pipgrip               smlpkg                yj

arrayfire             coconut               eksctl                gulp-cli              ldpl                  never                 promtail              so                    z.lua

awsweeper             colfer                fennel                hy                    litecli               ngs                   python@3.7            standardese

buildozer             copilot               functionalplus        jimtcl                logcli                notmuch-mutt          reg                   subfinder

cadence               cortex                gateway-go            jinx                  loki                  oci-cli               rqlite                termcolor

chalk-cli             croaring              gcc@9                 jsonnet-bundler       lunchy                omake                 saltwater             terraform-ls

==> Updated Formulae

Updated 3947 formulae.

==> Renamed Formulae

elasticsearch@6.8 -> elasticsearch@6                                                                   kibana@6.8 -> kibana@6

==> Deleted Formulae

cargo-completion      elasticsearch@2.4     elasticsearch@5.6     kibana@5.6            lumo                  python                sflowtool             tomee-jax-rs          unravel


==> Downloading https://homebrew.bintray.com/bottles/terraform-0.12.29.catalina.bottle.tar.gz

==> Downloading from https://d29vzk4ow07wi7.cloudfront.net/f7c787a4c42bb1291200f19b112aae5f725fc0fad068ad2422003e73ab74e4f7?response-content-disposition=attachment%3Bfilename%3D%22terraform-0.12.29.catali

######################################################################## 100.0%



2. Create a working directory for practice:

Ashoks-MacBook-Pro:~ ashokkafle$ mkdir terraform-practice


The first thing we need to do is create a working folder on your home directory or so and inside the folder we will create a file called main.tf

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ touch main.tf


The content of the file looks like below which will simply create a private s3 bucket on the aws.

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ cat main.tf 



resource "aws_s3_bucket" "b" {

  bucket_prefix = "my-tf-test-bucket-"

  acl    = "private"


  tags = {

    Name        = "My bucket"

    Environment = "Dev"

  }



3.  After having the main.tf file , run the terraform init command

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ terraform init


Initializing the backend...


Initializing provider plugins...

- Checking for available provider plugins...

- Downloading plugin for provider "aws" (hashicorp/aws) 2.70.0...


The following providers do not have any version constraints in configuration,

so the latest version was installed.


To prevent automatic upgrades to new major versions that may contain breaking

changes, it is recommended to add version = "..." constraints to the

corresponding provider blocks in configuration, with the constraint strings

suggested below.


* provider.aws: version = "~> 2.70"


Terraform has been successfully initialized!


You may now begin working with Terraform. Try running "terraform plan" to see

any changes that are required for your infrastructure. All Terraform commands

should now work.


If you ever set or change modules or backend configuration for Terraform,

rerun this command to reinitialize your working directory. If you forget, other

commands will detect it and remind you to do so if necessary




4. Now we need to have aws-cli configured on our local system to connect with aws:
. Below are the series of commands and steps to install aws-cli on macOs using aws bundlezip . There are various ways to do it. For further reference refer:

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

100 16.0M  100 16.0M    0     0  4065k      0  0:00:04  0:00:04 --:--:-- 4065k

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ ls -ltr

total 34544

-rw-r--r--  1 ashokkafle  staff       163 Jul 30 19:16 main.tf

-rw-r--r--  1 ashokkafle  staff  16796306 Jul 30 19:24 awscli-bundle.zip

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ unzip awscli-bundle.zip

Archive:  awscli-bundle.zip

  inflating: awscli-bundle/install   

  inflating: awscli-bundle/packages/six-1.15.0.tar.gz  

  inflating: awscli-bundle/packages/docutils-0.15.2.tar.gz  

  inflating: awscli-bundle/packages/urllib3-1.25.10.tar.gz  

  inflating: awscli-bundle/packages/botocore-1.17.32.tar.gz  

  inflating: awscli-bundle/packages/virtualenv-16.7.8.tar.gz  

  inflating: awscli-bundle/packages/PyYAML-5.3.1.tar.gz  

  inflating: awscli-bundle/packages/futures-3.3.0.tar.gz  

  inflating: awscli-bundle/packages/pyasn1-0.4.8.tar.gz  

  inflating: awscli-bundle/packages/rsa-3.4.2.tar.gz  

  inflating: awscli-bundle/packages/awscli-1.18.109.tar.gz  

  inflating: awscli-bundle/packages/python-dateutil-2.8.0.tar.gz  

  inflating: awscli-bundle/packages/jmespath-0.10.0.tar.gz  

  inflating: awscli-bundle/packages/colorama-0.4.1.tar.gz  

  inflating: awscli-bundle/packages/colorama-0.4.3.tar.gz  

  inflating: awscli-bundle/packages/PyYAML-5.2.tar.gz  

  inflating: awscli-bundle/packages/s3transfer-0.3.3.tar.gz  

  inflating: awscli-bundle/packages/urllib3-1.25.7.tar.gz  

  inflating: awscli-bundle/packages/setup/setuptools_scm-3.3.3.tar.gz  

  inflating: awscli-bundle/packages/setup/wheel-0.33.6.tar.gz  

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ ls -ltr

total 34544

-rw-r--r--  1 ashokkafle  staff       163 Jul 30 19:16 main.tf

-rw-r--r--  1 ashokkafle  staff  16796306 Jul 30 19:24 awscli-bundle.zip

drwxr-xr-x  4 ashokkafle  staff       128 Jul 30 19:25 awscli-bundle

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ ./awscli-bundle/install -b ~/bin/aws

Running cmd: /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python virtualenv.py --no-download --python /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /Users/ashokkafle/.local/lib/aws

Running cmd: /Users/ashokkafle/.local/lib/aws/bin/pip install --no-binary :all: --no-cache-dir --no-index --find-links file://. setuptools_scm-3.3.3.tar.gz

Running cmd: /Users/ashokkafle/.local/lib/aws/bin/pip install --no-binary :all: --no-cache-dir --no-index --find-links file://. wheel-0.33.6.tar.gz

Running cmd: /Users/ashokkafle/.local/lib/aws/bin/pip install --no-binary :all: --no-build-isolation --no-cache-dir --no-index  --find-links file:///Users/ashokkafle/terraform-practice/simple-aws-configuration/awscli-bundle/packages awscli-1.18.109.tar.gz

You can now run: /Users/ashokkafle/bin/aws --version

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ echo $PATH | grep ~/bin     // See if $PATH contains ~/bin (output will be empty if it doesn't)

-bash: syntax error near unexpected token `('

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ export PATH=~/bin:$PATH     // Add ~/bin to $PATH if necessary

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ echo $PATH | grep ~/bin 

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ export PATH=~/bin:$PATH 

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ echo $PATH | grep ~/bin 

/Users/ashokkafle/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands


Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ vi ~/.bash_profile

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ source ~/.bash

.bash_history         .bash_profile.pysave  .bashrc               

.bash_profile         .bash_sessions/       

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ source ~/.bash

.bash_history         .bash_profile.pysave  .bashrc               

.bash_profile         .bash_sessions/       

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ source ~/.bash_profile

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ aws --version

aws-cli/1.18.109 Python/2.7.16 Darwin/19.5.0 botocore/1.17.32




5. After installing aws-cli, we also need to have a user created on aws with access-key and secret to configure aws locally as shown below. For detailed explanation of this process, refer: https://ashokkafle.blogspot.com/2020/05/start-and-stop-ec2-instances-using.html

Once user is created and correct permission is assigned, we will get the access-key and secret which can be used for connecting with aws. Those can also be downloaded as csv file and keep safe somewhere.



6. Configure aws locally


Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ aws configure

AWS Access Key ID [None]: <access-key>

AWS Secret Access Key [None]: <secret-key>

Default region name [None]: us-east

Default output format [None]: json

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ cat ~/.aws/config 

[default]

output = json

region = us-east

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ cat ~/.aws/credentials 

[default]

aws_access_key_id = < access-key>

aws_secret_access_key = <secret-key>




7. Now execute terraform plan  to see if the config file will be doing what you intended ( It won't execute but just give a display of what it is going to once executed)


Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ terraform plan

provider.aws.region

  The region where AWS operations will take place. Examples

  are us-east-1, us-west-2, etc.


  Enter a value: us-east-1


Refreshing Terraform state in-memory prior to plan...

The refreshed state will be used to calculate this plan, but will not be

persisted to local or remote state storage.



------------------------------------------------------------------------


An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

  + create


Terraform will perform the following actions:


  # aws_s3_bucket.b will be created

  + resource "aws_s3_bucket" "b" {

      + acceleration_status         = (known after apply)

      + acl                         = "private"

      + arn                         = (known after apply)

      + bucket                      = (known after apply)

      + bucket_domain_name          = (known after apply)

      + bucket_prefix               = "my-tf-test-bucket-"

      + bucket_regional_domain_name = (known after apply)

      + force_destroy               = false

      + hosted_zone_id              = (known after apply)

      + id                          = (known after apply)

      + region                      = (known after apply)

      + request_payer               = (known after apply)

      + tags                        = {

          + "Environment" = "Dev"

          + "Name"        = "My bucket"

        }

      + website_domain              = (known after apply)

      + website_endpoint            = (known after apply)


      + versioning {

          + enabled    = (known after apply)

          + mfa_delete = (known after apply)

        }

    }


Plan: 1 to add, 0 to change, 0 to destroy.


------------------------------------------------------------------------


Note: You didn't specify an "-out" parameter to save this plan, so Terraform

can't guarantee that exactly these actions will be performed if

"terraform apply" is subsequently run.



8. Next execute the actual terraform apply  to do the actual task 


Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ terraform apply

provider.aws.region

  The region where AWS operations will take place. Examples

  are us-east-1, us-west-2, etc.


  Enter a value: us-east-1



An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

  + create


Terraform will perform the following actions:


  # aws_s3_bucket.b will be created

  + resource "aws_s3_bucket" "b" {

      + acceleration_status         = (known after apply)

      + acl                         = "private"

      + arn                         = (known after apply)

      + bucket                      = (known after apply)

      + bucket_domain_name          = (known after apply)

      + bucket_prefix               = "my-tf-test-bucket-"

      + bucket_regional_domain_name = (known after apply)

      + force_destroy               = false

      + hosted_zone_id              = (known after apply)

      + id                          = (known after apply)

      + region                      = (known after apply)

      + request_payer               = (known after apply)

      + tags                        = {

          + "Environment" = "Dev"

          + "Name"        = "My bucket"

        }

      + website_domain              = (known after apply)

      + website_endpoint            = (known after apply)


      + versioning {

          + enabled    = (known after apply)

          + mfa_delete = (known after apply)

        }

    }


Plan: 1 to add, 0 to change, 0 to destroy.


Do you want to perform these actions?

  Terraform will perform the actions described above.

  Only 'yes' will be accepted to approve.


  Enter a value: yes


aws_s3_bucket.b: Creating...

aws_s3_bucket.b: Creation complete after 3s [id=my-tf-test-bucket-20200731003742999700000001]


Apply complete! Resources: 1 added, 0 changed, 0 destroyed.





9. Run some verfications like check if terraform state file is there or s3 bucket is created and so on



Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ terraform state list

aws_s3_bucket.b



Also verified on the aws console that s3 bucket with my-tf-test-bucket-20200731003742999700000001  is created



10. Destroy the resource using terraform destroy


Now we can destroy the resource that is created using terraform or the resources that resides on the terraform state file.


Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ terraform destroy

provider.aws.region

  The region where AWS operations will take place. Examples

  are us-east-1, us-west-2, etc.


  Enter a value: us-east-1


aws_s3_bucket.b: Refreshing state... [id=my-tf-test-bucket-20200731003742999700000001]


An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

  - destroy


Terraform will perform the following actions:


  # aws_s3_bucket.b will be destroyed

  - resource "aws_s3_bucket" "b" {

      - acl                         = "private" -> null

      - arn                         = "arn:aws:s3:::my-tf-test-bucket-20200731003742999700000001" -> null

      - bucket                      = "my-tf-test-bucket-20200731003742999700000001" -> null

      - bucket_domain_name          = "my-tf-test-bucket-20200731003742999700000001.s3.amazonaws.com" -> null

      - bucket_prefix               = "my-tf-test-bucket-" -> null

      - bucket_regional_domain_name = "my-tf-test-bucket-20200731003742999700000001.s3.amazonaws.com" -> null

      - force_destroy               = false -> null

      - hosted_zone_id              = "Z3AQBSTGFYJSTF" -> null

      - id                          = "my-tf-test-bucket-20200731003742999700000001" -> null

      - region                      = "us-east-1" -> null

      - request_payer               = "BucketOwner" -> null

      - tags                        = {

          - "Environment" = "Dev"

          - "Name"        = "My bucket"

        } -> null


      - versioning {

          - enabled    = false -> null

          - mfa_delete = false -> null

        }

    }


Plan: 0 to add, 0 to change, 1 to destroy.


Do you really want to destroy all resources?

  Terraform will destroy all your managed infrastructure, as shown above.

  There is no undo. Only 'yes' will be accepted to confirm.


  Enter a value: yes


aws_s3_bucket.b: Destroying... [id=my-tf-test-bucket-20200731003742999700000001]

aws_s3_bucket.b: Destruction complete after 1s


Destroy complete! Resources: 1 destroyed.



11. Confirm it has been destroyed (  terrform state list)  gives no output and also s3 bucket is no longer present on aws


Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$ terraform state list

Ashoks-MacBook-Pro:simple-aws-configuration ashokkafle$