Module 4: Modularisation and Reusability in Terraform

The ability to organize configuration into reusable blocks is a pillar of Infrastructure as Code (IaC). Modules are the key ingredient for writing maintainable, reusable, and testable Terraform code.

Below is a detailed tutorial demonstrating modularization, dynamic infrastructure creation, and publication practices, structured using example from previous tutorials in the series.

Modularization and reusability in terraform

Modularization addresses the common problem of code reuse among developers. The practice aims to minimize side effects and adhere to the Don’t Repeat Yourself (DRY) principle by avoiding code duplication.

A Terraform module is essentially a container for multiple resources that work together to achieve a specific goal. Modules allow experts to encapsulate complex configurations, such as an entire network stack or a Kubernetes cluster, into reusable building blocks.

1. Creating and structuring reusable terraform modules

Any directory containing Terraform configuration files (.tf files) is considered a module. The directory from which the terraform init and apply commands are run is called the root module, and any modules it calls are generally referred to as child modules or submodules.

Standard module file structure

For clarity, documentation, and tooling integration, modules should adhere to a standard directory and file layout.

Implementing the scaffold structure

We will use your requested scaffold to demonstrate three composable modules:

.
├── main.tf (Root Configuration)
└── modules
    ├── network
    │   ├── main.tf
    │   ├── variables.tf
    │   └── outputs.tf
    ├── storage
    │   ├── main.tf
    │   ├── variables.tf
    │   └── outputs.tf
    └── vm
        ├── main.tf
        ├── variables.tf
        └── outputs.tf

A. Network module (modules/network)

This module is responsible for defining the networking layer (VPC/VNet and subnets). It takes configurations (like CIDR blocks) as input and exposes connectivity information as outputs.

• Role: Defines network topology (e.g., aws_vpc, azurerm_virtual_network).
• Composition: This module must be provisioned first, as its outputs are crucial inputs for the VM module.

modules/network/variables.tf (Inputs) Input variables are the module’s API, allowing users to customize its behavior without modifying the source code.

variable "resource_group_name" {
  description = "Resource group name where network will reside"
  type        = string
}

variable "vnet_cidr" {
  description = "The main CIDR block for the Virtual Network"
  type        = string
  default     = "10.0.0.0/16"
}

modules/network/outputs.tf (Outputs) Outputs are used to retrieve values from provisioned resources after execution, making them available for other configurations or external programs.

output "vnet_name" {
  description = "The name of the provisioned Virtual Network"
  value       = azurerm_virtual_network.vnet.name 
}

output "private_subnet_ids" {
  description = "A list of IDs for private subnets"
  value       = azurerm_subnet.private_subnets[*].id 
  # This uses the splat operator [*] which is common when modules create multiple resources.
}

B. VM module (modules/vm)

This module provisions compute instances (e.g., aws_instance or azurerm_linux_virtual_machine).

• Composition: This module depends on consuming the network outputs (Subnet IDs) as its inputs.

modules/vm/variables.tf (Inputs)

variable "subnet_id" {
  description = "The ID of the Subnet to launch the VM into"
  type        = string
}

variable "instance_type" {
  description = "The machine type for the compute instance"
  type        = string
  default     = "Standard_B1s"
}

C. Root configuration (composition)

The root module stitches these components together using the module block and passing output values as inputs:

# This file is typically named main.tf in the root of your project
resource "azurerm_resource_group" "rg" {
  name     = "App-Resource-Group"
  location = "westeurope"
}

# 1. Provision the Network Module
module "app_network" {
  source              = "./modules/network"
  resource_group_name = azurerm_resource_group.rg.name
  # This module is implicitly applied first due to the dependency chain
}

# 2. Provision the VM Module using Network Outputs
module "app_vm" {
  source        = "./modules/vm"
  # Pull the subnet ID from the network module's output:
  subnet_id     = module.app_network.private_subnet_ids 
  instance_type = "Standard_D2s_v3"
}

2. Dynamic infrastructure with locals, count, and for_each

To create scalable and non-redundant configurations, Terraform provides mechanisms for repetition and conditional logic.

Using locals for internal logic

Local values (defined in a locals {} block) allow you to assign a name to an expression, improving readability and avoiding repetitive logic throughout a module. They function as variables scoped exclusively within the module and cannot be redefined or accessed externally.

Example: Generating complex, standardized resource names (modules/vm/main.tf enhancement).

locals {
  # Define a reusable naming convention for the VM
  standard_name = upper(format("%s-VM-%s", var.prefix, var.environment))
}

resource "azurerm_virtual_machine" "app_vm" {
  name                = local.standard_name # Reference the calculated value
  location            = var.location
  # ... other configuration
}

Using count for resource repetition and conditionals

The count meta-argument is a basic iteration construct that defines how many identical copies of a resource (or a module call) should be created.

Repetition example: provisioning multiple storage accounts

If you needed two identical storage accounts, you could use count within your modules/storage/main.tf:

resource "azurerm_storage_account" "data_store" {
  count                    = var.storage_account_count # e.g., defined as 2 in variables.tf
  name                     = "datastore${count.index}" # Use count.index (starts at 0) for uniqueness
  resource_group_name      = var.resource_group_name
  # ... other configuration
}

Conditional logic (the binary toggle)

By setting count to the result of a conditional expression (the ternary operator: <CONDITION> ? <TRUE_VAL> : <FALSE_VAL>), you can toggle resources on (1) or off (0) based on an input variable.

Example: Conditionally enabling a separate logging component based on a boolean variable.

variable "enable_logging" {
  description = "Set to true to create a logging container."
  type        = bool
  default     = false
}

resource "azurerm_log_analytics_workspace" "logging" {
  count = var.enable_logging ? 1 : 0 # Resource is created if true (1), destroyed if false (0)
  name = "logger-${var.resource_group_name}"
  # ...
}

Using for_each for varied collections and dynamic blocks

The for_each meta-argument iterates over collections (maps or sets) and is generally preferred over count for collections that are not identical or whose size changes often. Using keys (from a map or set) allows Terraform to safely remove items from the middle of the collection without destroying and recreating resources that precede them.

Looping over a map of objects

This is useful when creating multiple resources of the same type (like multiple VMs), but each instance requires a unique configuration (instance size, specific tags, disk size, etc.).

Example: Creating multiple storage accounts with unique tiers.

# In variables.tf (or terraform.tfvars)
variable "storage_definitions" {
  type = map(object({
    tier = string
    replication = string
  }))
  default = {
    "primary" = { tier = "Standard", replication = "LRS" }
    "backup"  = { tier = "Premium", replication = "ZRS" }
  }
}

# In main.tf
resource "azurerm_storage_account" "custom_stores" {
  for_each                 = var.storage_definitions 
  name                     = "store-${each.key}" # 'primary' or 'backup'
  account_tier             = each.value.tier       # 'Standard' or 'Premium'
  account_replication_type = each.value.replication
  # ...
}

Generating multiple blocks with dynamic

The dynamic block uses for_each to generate multiple inline configuration blocks within a single resource definition. This is commonly used for defining lists of network rules, tags, or settings that vary dynamically.

Example: Applying a list of custom firewall rules to a security group.

resource "azurerm_network_security_group" "nsg" {
  # ... configuration ...

  dynamic "security_rule" { # The 'dynamic' keyword generates blocks of type 'security_rule'
    for_each = var.nsg_rules # Iterate over an input list of rule objects

    content {
      name                       = security_rule.value.name
      priority                   = security_rule.value.priority
      direction                  = security_rule.value.direction
      # ... other rule properties using security_rule.value
    }
  }
}

3. Publishing modules to github or terraform registry

Once a module is developed and tested, it must be shared to realize the promise of reusability. Modules can be sourced locally, via Git, or via a module registry.

Versioning modules with git

Regardless of the hosting location, modules should be versioned using Git tags in the Semantic Versioning format (vX.Y.Z). Versioning allows downstream consumers to pin their usage to a stable release, ensuring consistency and enabling safe testing of changes in staging environments before moving to production.

Sharing via git repository (github/private repos)

The simplest remote method is to use a Git repository directly as the source.

1. Repository Setup: Commit the module code (modules/vm, modules/network, etc.) to a GitHub repository.
2. Tagging: Create and push a Git tag (e.g., v1.0.0) corresponding to a stable release.

Module call example (using git source with version pinning):

When calling this module in the root configuration, the source attribute is set to the repository URL, and the ref parameter pins the version:

module "app_network" {
  # Note the double slash // to specify the subdirectory containing the module
  source = "git@github.com:ORG/repo.git//modules/network?ref=v1.0.0" 
  # ... inputs ...
}

When terraform init is executed, Terraform clones the specified module version locally. This technique is essential for private Git repositories (like Azure Repos or internal GitLab) where modules cannot be exposed publicly.

Publishing to a module registry

Registries provide discovery, version tracking, and documentation parsing.

A. Public terraform/OpenTofu registry

For open-source modules, the public registry offers a centralized index.

1. Naming Convention: The module repository must be named following the convention: terraform-<PROVIDER>-<NAME> (e.g., terraform-azurerm-network).
2. Publication: After logging in (via GitHub credentials for the Terraform registry), you publish the repository via the web UI. The registry automatically detects the versions based on Git tags pushed to the repository.

Module Call Example (Using Registry Source):

The registry uses a simplified source alias that is easier to read than a full Git URL:

module "network" {
  # Source format: <OWNER>/<NAME>/<PROVIDER> 
  source  = "Azure/network/azurerm" 
  version = "3.0.1" # Version is specified separately
  # ... inputs ...
}

B. Private module registry (e.g., terraform cloud)

For enterprises, private registries offer centralized sharing of proprietary modules, documentation, and access control. The process involves linking the organization’s VCS (e.g., GitHub) to the platform and publishing the module repository.

Once published in a private registry, the module can be consumed using a distinct source path tied to the organization’s domain: source = "app.terraform.io/<TFC organisation>/<module_name>/<provider>".