Terraform Automation with CI/CD Pipelines

Learn how to automate your infrastructure deployments by integrating Terraform with popular CI/CD platforms.

Why Automate Terraform?

1. Consistent deployments
2. Reduced human error
3. Automated testing
4. Version control integration
5. Audit trail

CI/CD Pipeline Components

1. Version Control
2. Testing
3. Plan Generation
4. Approval Gates
5. Deployment
6. Validation

GitHub Actions Pipeline

# .github/workflows/terraform.yml
name: Terraform CI/CD

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  terraform:
    runs-on: ubuntu-latest
    
    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      TF_VAR_environment: ${{ github.ref == 'refs/heads/main' && 'prod' || 'dev' }}
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Setup Terraform
      uses: hashicorp/setup-terraform@v2
      with:
        terraform_version: 1.0.0
        
    - name: Terraform Format
      run: terraform fmt -check
        
    - name: Terraform Init
      run: terraform init
        
    - name: Terraform Validate
      run: terraform validate
        
    - name: Terraform Plan
      run: terraform plan -out=tfplan
      
    - name: Upload Plan
      uses: actions/upload-artifact@v3
      with:
        name: tfplan
        path: tfplan
        
    - name: Terraform Apply
      if: github.ref == 'refs/heads/main' && github.event_name == 'push'
      run: terraform apply -auto-approve tfplan

GitLab CI Pipeline

# .gitlab-ci.yml
image: hashicorp/terraform:1.0.0

variables:
  TF_VAR_environment: ${CI_COMMIT_REF_NAME}
  
cache:
  paths:
    - .terraform

stages:
  - validate
  - plan
  - apply

before_script:
  - terraform init

validate:
  stage: validate
  script:
    - terraform fmt -check
    - terraform validate
  
plan:
  stage: plan
  script:
    - terraform plan -out=tfplan
  artifacts:
    paths:
      - tfplan
      
apply:
  stage: apply
  script:
    - terraform apply -auto-approve tfplan
  only:
    - main
  when: manual

Azure DevOps Pipeline

# azure-pipelines.yml
trigger:
  - main

pool:
  vmImage: 'ubuntu-latest'

variables:
  - group: terraform-secrets

stages:
- stage: Validate
  jobs:
  - job: ValidateAndPlan
    steps:
    - task: TerraformInstaller@0
      inputs:
        terraformVersion: '1.0.0'
        
    - task: TerraformTaskV3@3
      inputs:
        provider: 'aws'
        command: 'init'
        
    - task: TerraformTaskV3@3
      inputs:
        provider: 'aws'
        command: 'plan'
        environmentServiceNameAWS: 'AWS-Connection'
        
- stage: Deploy
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
  jobs:
  - deployment: ApplyTerraform
    environment: 'production'
    strategy:
      runOnce:
        deploy:
          steps:
          - task: TerraformTaskV3@3
            inputs:
              provider: 'aws'
              command: 'apply'
              environmentServiceNameAWS: 'AWS-Connection'

Jenkins Pipeline

// Jenkinsfile
pipeline {
    agent any
    
    environment {
        TF_VAR_environment = "${BRANCH_NAME}"
        AWS_CREDENTIALS = credentials('aws-credentials')
    }
    
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        
        stage('Terraform Init') {
            steps {
                sh 'terraform init'
            }
        }
        
        stage('Terraform Plan') {
            steps {
                sh 'terraform plan -out=tfplan'
            }
        }
        
        stage('Approval') {
            when {
                branch 'main'
            }
            steps {
                input message: 'Apply Terraform changes?'
            }
        }
        
        stage('Terraform Apply') {
            when {
                branch 'main'
            }
            steps {
                sh 'terraform apply -auto-approve tfplan'
            }
        }
    }
    
    post {
        always {
            cleanWs()
        }
    }
}

CircleCI Pipeline

# .circleci/config.yml
version: 2.1

orbs:
  terraform: circleci/terraform@3.0.0

workflows:
  version: 2
  terraform:
    jobs:
      - terraform/fmt:
          checkout: true
          context: terraform
          
      - terraform/validate:
          checkout: true
          context: terraform
          requires:
            - terraform/fmt
            
      - terraform/plan:
          checkout: true
          context: terraform
          requires:
            - terraform/validate
            
      - terraform/apply:
          checkout: true
          context: terraform
          filters:
            branches:
              only: main
          requires:
            - terraform/plan

Best Practices

1. State Management

# backend.tf
terraform {
  backend "s3" {
    bucket         = "terraform-state"
    key            = "env/${var.environment}/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    dynamodb_table = "terraform-locks"
  }
}

2. Variable Management

# variables.tf
variable "environment" {
  type        = string
  description = "Environment name (dev/staging/prod)"
}

# terraform.tfvars.example
environment = "dev"
region      = "us-west-2"

3. Workspace Separation

# Use workspaces for environment isolation
terraform workspace new dev
terraform workspace new prod
terraform workspace select dev

Security Considerations

1. Secrets Management

# GitHub Actions secrets
env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
  TF_VAR_db_password: ${{ secrets.DB_PASSWORD }}

2. Access Control

# IAM policy for CI/CD
resource "aws_iam_policy" "terraform_ci" {
  name = "terraform-ci-policy"
  
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:PutObject"
        ]
        Resource = "${aws_s3_bucket.terraform_state.arn}/*"
      }
    ]
  })
}

Testing in CI/CD

1. Unit Tests

# test/main_test.go
package test

import (
    "testing"
    "github.com/gruntwork-io/terratest/modules/terraform"
)

func TestTerraformBasicExample(t *testing.T) {
    terraformOptions := &terraform.Options{
        TerraformDir: "../examples/basic",
        Vars: map[string]interface{}{
            "environment": "test",
        },
    }
    
    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)
}

2. Policy Compliance

# policy/naming.rego
package terraform

deny[msg] {
    resource := input.planned_values.root_module.resources[_]
    not startswith(resource.values.tags.Name, "company-")
    msg = sprintf("Resource %v must have company prefix in Name tag", [resource.address])
}

Monitoring and Notifications

1. Slack Notifications

# GitHub Actions
- name: Notify Slack
  uses: 8398a7/action-slack@v3
  with:
    status: ${{ job.status }}
    fields: repo,message,commit,author,action,eventName,ref,workflow
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

2. Email Notifications

// Jenkins
post {
    success {
        emailext subject: "Pipeline Successful",
                 body: "Terraform changes applied successfully",
                 to: "team@company.com"
    }
    failure {
        emailext subject: "Pipeline Failed",
                 body: "Terraform pipeline failed. Please check logs.",
                 to: "team@company.com"
    }
}

Drift Detection

# Scheduled drift detection
name: Terraform Drift Detection

on:
  schedule:
    - cron: '0 0 * * *'

jobs:
  detect-drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        
      - name: Terraform Plan
        run: |
          terraform init
          terraform plan -detailed-exitcode
        continue-on-error: true
        id: plan
        
      - name: Notify on Drift
        if: steps.plan.outcome == 'failure'
        run: |
          echo "Infrastructure drift detected!"
          # Add notification logic

Conclusion

Automating Terraform with CI/CD:

• Ensures consistent deployments
• Reduces human error
• Provides audit trail
• Enables automated testing
• Improves collaboration

Remember to:

1. Use version control
2. Implement proper testing
3. Secure sensitive data
4. Monitor deployments
5. Handle state properly