How To Automate HashiCorp Vault Installation For AWS EC2

Introduction

HashiCorp Vault is an identity-based secrets and encryption management system. It is used to secure, store and protect secrets and other sensitive data using a UI, CLI, or HTTP API.

Makes the creation of the instance and installation of Vault easy using automation tools.

Tools:

• AWS Account

• Ansible

• Terraform

• Jenkins

Let's play around and learn:

• First, create an IAM user with an administrator role in the AWS account and get the access key and secret key to access it.

• Create the security group for the vault with the inbound rule and allow ports 8200, 8201 and 22.

• Create a key pair to connect instances from AWS.

• Start Jenkins and install plugins: Terraform, Ansible and AWS credentials

• If Terraform and Ansible are installed in your server where Jenkins is running, update the path in the plugin configuration; Otherwise, use the install automatically feature.

• Add new credential with the help of AWS credentials plugin select kind as AWS Credentials and stores the access key and secret key. Use this credential for creating instance through Terraform.

  

• Add a new credential with an SSH Username with the private key and give the details of the generated key pair file from AWS. Use this created credential ID in an ansible playbook stage.

• Create a new pipeline in Jenkins, copy the content from the below file/repository and update the credentials details and parameter names in the pipeline.

• Click on Build with Parameters and select the apply to start our vault instance.

Jenkins Pipeline:

pipeline{
    agent any
    tools{
        terraform 'terraform'
        ansible 'Ansible2'
    }
    stages{
        stage('git checkout'){
            steps{
                git branch: 'main', url: 'https://github.com/Dhruvp-11/vault.git' //get the files from github repo
            }
        }
        stage('terraform initialization'){
            steps{
                    sh 'terraform init'  // terraform initialization
            }
        }
        stage("terraform plan"){
            steps{
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'TD',
                accessKeyVariable: 'AWS_ACCESS_KEY_ID',
                secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) {  
                    sh 'terraform plan -out=plan.txt'  //running plan with aws credentials 
                }
            }
        }
        stage("terraform apply"){
            steps{
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: '6',
                accessKeyVariable: 'AWS_ACCESS_KEY_ID',
                secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) {
                    sh 'terraform ${Action} --auto-approve'    //running apply with choice parameter apply/destory with aws credentials 
                }
            }
        }
        stage("IP addition in host file"){
            when {
                expression {params.Action == 'apply'}  // runs only when choice parameter is Apply
            }
            steps{
                sh """terraform output | grep -Eo '[[:digit:]]{1,3}\\.[[:digit:]]{1,3}\\.[[:digit:]]{1,3}\\.[[:digit:]]{1,3}' >> hosts.ini"""
            } // get IP and adding into hosts file for ansible 
        }
        stage('ansible playbook'){
            when {
                expression {params.Action == 'apply'}
            }
            steps{
                ansiblePlaybook become: true, credentialsId: 'ID', disableHostKeyChecking: true, installation: 'Ansible2', inventory: 'hosts.ini', playbook: 'playbook.yaml'
            } // running Ansible playbook with security key for AWS instance to install vault
        }
    }
}

GitHub Repository details :

• https://github.com/Dhruvp-11/vault.git

• I am using the above git repository in our Jenkins pipeline, as in this we are creating only 1 instance of vault with storage option as raft with minimal setup if you want some other storage or config please update the config in "vault.hcl" file where the configuration are stored.

  Note : The Terraform and Ansible configuration works with installation of vault in AWS Amazon Linux image.

• Overview of files :

  1. Terraform : main.tf, provider.tf and backend.tf file contains the details of vault instance creation.

  2. Ansible : playbook.yaml and hosts.ini file contains tasks and host details.

• vault.hcl , script.sh and vault.service files are required to complete our installation of vault on server.

1. Jenkinsfile : contains pipeline for Jenkins.

Conclusion:

This article covers the installation of Vault on AWS EC2 Amazon Linux image using the Jenkins pipeline.

I can be found on @Linkedin.

Stay safe and secure! , If you've enjoyed it or have any queries, please feel free to reach out.