Module 3: Terraform secrets state and remote backends tutorial

This comprehensive tutorial covers best practices for managing secrets, state, and remote backends in Terraform, focusing specifically on using Azure Storage for remote state management.

Secrets, state, and remote backends: core concepts

Terraform operates on the concept of Infrastructure as Code (IaC), relying heavily on managing three critical components: Secrets, State, and Backends.

Understanding terraform state

Terraform utilizes a state file (by default named terraform.tfstate) to maintain a mapping between the configuration defined in your HCL files and the actual resources deployed in the real world (cloud infrastructure). This state records resource configurations, metadata, and identifies remote objects, allowing Terraform to calculate the necessary changes (Create, Read, Update, Delete) for future deployments.

Key facts about state:

• The state file is crucial for Terraform’s functionality and acts as an internal database for the tool.
• The state file format is a private API and should generally never be edited manually due to the high risk of corruption; official CLI commands like terraform import or terraform state should be used instead.
• State files contain resource attributes, often including sensitive data such as passwords and API keys, usually stored in plain text.

Managing secrets

Sensitive data, including passwords, tokens, and API keys, must be handled carefully. Exposing secrets through version control or accidental logging poses significant security risks. While Terraform allows input variables to be marked as sensitive, this is only a partial security measure (see section 2). Ideally, secrets should be managed outside of Terraform using dedicated Secret Management Services like Google Cloud Secret Manager, Azure Key Vault, or HashiCorp Vault, and retrieved dynamically when needed.

Utilizing remote backends

For individual development or experimentation, the state can be stored locally (local backend), residing as a terraform.tfstate file on your filesystem. However, this approach is insufficient for collaborative, production, or long-term projects because it lacks conflict resolution, risks data loss, and exposes sensitive data.

A remote backend solves these issues by storing the state file in a centralized, shared location (e.g., cloud storage services like S3, GCS, or Azure Storage). Remote backends enable team collaboration, provide state locking mechanisms, and offer inherent security features like encryption and access controls.

Managing secrets with environment variables and terraform.tfvars

To mitigate the risk of exposing sensitive data in source control, developers must avoid hard-coding values directly into Terraform configuration files or passing them through unsecured methods.

Using environment variables for secrets

Environment variables are the recommended method for inputting sensitive information into your root module configuration, as they keep the data separate from the codebase.

1. Define the Variable: Declare the input variable, optionally including a description and a type constraint, and apply the sensitive = true flag.

   variable "database_password" {
     description = "The password for the database"
     type        = string
     sensitive   = true  # Masks the value in CLI output
   }
   

2. Set the Environment Variable: Set the environment variable using the naming convention TF_VAR_<variable_name>.

   • On Linux/macOS: export TF_VAR_database_password="supersecret"
   • On Windows: set TF_VAR_database_password="supersecret"

Terraform will automatically read this value when terraform plan or terraform apply is executed.

The sensitive flag and state files

Setting sensitive = true instructs Terraform to mask the variable’s value in the CLI output and logs during execution. However, this flag does not encrypt the value; the actual secret remains stored in clear text within the Terraform state file (terraform.tfstate). This vulnerability underscores the necessity of using secure remote backends for state storage.

Using terraform.tfvars files

.tfvars files are used to specify values for variables defined in your configuration. While convenient for non-sensitive data like resource names or regional settings, storing secrets in .tfvars files is discouraged, as they are frequently saved in version control, leading to exposure.

Using azure storage account for remote state

To ensure data residency, resilience, and security in an Azure environment, the AzureRM backend (azurerm) is used to store state files in an Azure Storage Account.

Prerequisites for azure remote state

1. Azure Storage Account: You need a dedicated Azure Storage Account and a container (blob) where the state file will be stored. The Storage Account name must be globally unique.
2. Access Key/Credentials: Authentication details are required to allow Terraform access to the Storage Account. This is often done using the Storage Account access key, typically set via the ARM_ACCESS_KEY environment variable.

Configuring the azurerm backend

The remote backend configuration is declared within the terraform settings block in the root module (often in a dedicated file like backend.tf or providers.tf).

# backend.tf 

terraform {
  required_version = ">= 1.0"
  # Backend configuration must be defined in the terraform block
  backend "azurerm" {
    # Azure Resource Group containing the Storage Account
    resource_group_name  = "RG-TFBACKEND" 
    # Name of the Storage Account (Must be globally unique)
    storage_account_name = "storagetfbackend" 
    # Name of the container (blob) inside the Storage Account
    container_name       = "tfstate" 
    # Path/Name of the state file within the container
    key                  = "myapp.tfstate" 
    
    # Optionally, pass the access key here, though environment variables are safer:
    # access_key = "xxxxx-xxxx-xxxxx-xxxxx" 
  }
}

Initializing the backend

After defining the configuration, you must initialize the backend using terraform init.

• If you are migrating state from a local file, Terraform will prompt you to confirm the migration to the new remote backend.
• If you use a partial configuration (omitting sensitive details like the access key or bucket name in the .tf file), you pass those details during initialization using the -backend-config flag:

# Example if using partial configuration file (backend.tfvars):
terraform init -backend-config="backend.tfvars" 

Locking and versioning with azurerm_backend

Implementing state locking and versioning is mandatory for production use to guarantee resiliency, avoid state file corruption, and provide an audit trail.

State locking

State locking prevents multiple concurrent operations (e.g., multiple developers running terraform apply simultaneously) from attempting to modify the state file, which would lead to conflicts and corruption. Remote backends, including azurerm, support state locking natively, ensuring that when one user is working with the state file, others must wait until the lock is released.

Versioning and auditing

The Azure Storage platform supports file versioning. Enabling Versioning on the underlying Storage Account container ensures that every revision of your state file is stored. If the current state file becomes corrupted or is accidentally deleted, you can roll back to a previous, known-good version saved by the backend.

When configuring the Storage Account resource using Terraform (or manually via Azure CLI), ensure versioning is enabled on the associated container.

State introspection and troubleshooting

Terraform provides command-line interface (CLI) tools for inspecting and manipulating state safely without resorting to manual file editing.

Template production

The following templates demonstrate the configuration required to set up Azure Storage for remote state management.

azurerm_storage_account_module/main.tf

This pseudo-module defines the necessary Azure Storage resources needed to host the remote state file, configured for resilience and security.

# This file defines the resources required for state management.

# 1. Resource Group for Backend State Infrastructure
resource "azurerm_resource_group" "backend" {
  name     = "RG-TFBACKEND-${var.environment_suffix}"
  location = var.location
}

# 2. Azure Storage Account for State File Storage (Blobs)
# Note: Storage Account names must be globally unique and contain only lowercase letters/numbers.
resource "azurerm_storage_account" "tfbackend" {
  name                     = "sttfbackend${var.environment_suffix}" 
  resource_group_name      = azurerm_resource_group.backend.name
  location                 = azurerm_resource_group.backend.location
  account_tier             = "Standard"
  account_replication_type = "GRS" # Geo-Redundant Storage for high resilience
  
  # Optional: Enforce HTTPS access for security
  enable_https_traffic_only = true 
}

# 3. Blob Container to Hold the State Files
resource "azurerm_storage_container" "tfstate" {
  name                  = "tfstate"
  storage_account_name  = azurerm_storage_account.tfbackend.name
  container_access_type = "private" # Restrict public access
}

output "storage_account_name" {
    value       = azurerm_storage_account.tfbackend.name
    description = "Name of the storage account hosting the remote state."
}

output "storage_container_name" {
    value       = azurerm_storage_container.tfstate.name
    description = "Name of the container hosting the remote state."
}

backend.tf

This file configures the azurerm backend using the resources created by the module above (assuming outputs from the infrastructure module are passed, or values are hardcoded after resource creation).

# backend.tf

# This block must be defined in the root module before terraform init is run.
terraform {
  # Defines the backend as Azure Resource Manager
  backend "azurerm" {
    # Specify the resource identifiers for the backend infrastructure
    resource_group_name  = "RG-TFBACKEND-dev" # Example hardcoded value
    storage_account_name = "sttfbackenddev" # Example hardcoded value
    container_name       = "tfstate" 
    
    # The unique key path for the state file (isolates state per configuration)
    key                  = "my_application/infrastructure.tfstate" 
    
    # Note: Authentication details are typically provided via environment variables (ARM_ACCESS_KEY, etc.)
  }
}

state.tf

This file, placed in a consuming root module, defines outputs to inspect generated state attributes, demonstrating basic introspection capability.

# state.tf

# Output demonstrating introspection of the resource name recorded in state
output "backend_storage_account_id" {
  # Accessing the computed ID attribute of the backend storage account resource.
  # Assuming the azurerm_storage_account resource address is accessible.
  # This example requires the consuming module to have visibility of the resource ID.
  value = azurerm_storage_account.tfbackend.id 
  description = "The full Resource ID of the Azure Storage Account hosting the state file."
}

# Output demonstrating a sensitive attribute should be flagged when extracted
resource "random_password" "db_pass" {
  length  = 16
  special = true
}

output "initial_password_value" {
  value = random_password.db_pass.result
  sensitive = true # Ensures this value is masked in CLI output
  description = "Initial generated sensitive password (stored in state)."
}