Preventing Terraform state conflicts when deploying multiple VMs

Published on 04 May 2026 by Adam Lloyd-Jones

If you use the same state file for every pipeline run, yes, Terraform will attempt to destroy or modify the first VM to make it match the second one.

This happens because Terraform is a declarative tool; the state file acts as its “memory” of what it is responsible for. If your code defines one VM and your state file records one VM, running the pipeline with a new name tells Terraform: “The existing VM should now have this new name,” which often triggers a replacement (destroy then create).

To manage multiple VMs independently using the same code, you must ensure each VM has its own isolated state. Your proposed approach of using a dynamic state file per VM is correct and is a standard industry practice often achieved through Partial Configuration or Workspaces.

1. The Right Approach: Partial Backend Configuration

Your idea of using vm-$(vm_name).tfstate is highly effective for CI/CD pipelines. This is known as partial configuration.

In this setup, you leave the backend configuration “empty” in your .tf files and provide the specific details at runtime during the terraform init phase.

In your code:

terraform {
  backend "azurerm" {} # Leave the settings empty here
}

In your Azure DevOps Pipeline: When you run terraform init, pass the dynamic key via a command-line argument: terraform init -backend-config="key=vm-$(vm_name).tfstate" ....

Why this works: This forces Terraform to look at a unique state file for every VM name. When the pipeline runs for “VM-B,” Terraform will see an empty state file (since vm-B.tfstate doesn’t exist yet) and create it from scratch without ever knowing “VM-A” exists in a different state file.

2. Alternative: Terraform Workspaces

Terraform has a built-in feature called Workspaces designed specifically for this “one codebase, many deployments” scenario.

How it works: You use the command terraform workspace new $(vm_name) to create a separate state instance for each VM.
Storage: In Azure Blob Storage, Terraform automatically creates a folder structure to keep these states separate (e.g., env:/VM-A/path/to/state and env:/VM-B/path/to/state).
Drawback: Workspaces are not always visible in the code itself, which can occasionally lead to confusion in complex team environments.

3. Managing “Sync” Fear (Idempotency)

You mentioned concerns about Terraform trying to “sync” the state. In Terraform, this is actually a feature called idempotency.

If you use separate state files: Running terraform apply for VM-B will never affect VM-A because their states are completely isolated “bulkheads”.
If you run the same VM again: If you run the pipeline for VM-A a second time using its vm-A.tfstate, Terraform will compare the code to the existing VM. If nothing has changed, it will do nothing (Idempotency). If only the RAM changed, it will only update that specific attribute.

Summary and Best Practices

Isolation is Safety: Treating each VM as a separate state “component” protects you from a mistake in one deployment destroying another.
Namespace your Resources: Ensure your Hyper-V VM resource name in HCL uses the variable: resource "hyperv_machine_instance" "vm" { name = var.vm_name ... }.
Ansible Integration: Since you are using Ansible afterward, you can use Terraform outputs (like the new VM’s IP address) to dynamically generate your Ansible inventory.
State Security: Because state files can contain sensitive data (like Windows admin passwords), ensure your Azure Storage container is encrypted and access-restricted.

Partial backend configuration is a specialized technique in Terraform that allows you to omit some or all configuration parameters from a backend block within your .tf files, providing them instead at runtime during the initialization phase. This method is essential for creating flexible, secure, and reusable infrastructure modules, as Terraform’s backend block does not natively support the use of variables or references.

Core Purpose and Benefits

The primary reasons for utilizing partial backend configuration include:

Handling Variable Limitations: Because the backend block is processed very early by Terraform, it cannot use standard input variables or resource attributes. Partial configuration is the standard workaround to inject dynamic values like bucket names or state file paths.
Environment Reusability: It allows a single codebase to be deployed across multiple environments (e.g., development, staging, and production). By keeping the backend block “empty” or partial, you can point each environment to a unique state location without modifying the core code.
Enhanced Security: It prevents sensitive information, such as access keys or specific storage paths, from being hardcoded in version control.
Testing Isolation: In automated testing scenarios (such as using Terratest), partial configuration enables tests to provision real infrastructure while using temporary, isolated backend locations to avoid overwriting “live” state files.

Implementation Methods

To implement this, you first define a terraform block with a backend block that is either entirely empty or only contains non-sensitive, static parameters.

# main.tf
terraform {
  backend "s3" {
    # Leave dynamic or sensitive settings out of this block
  }
}

You then provide the missing configuration values during the terraform init phase using the -backend-config flag. There are two primary ways to pass these values:

1. Using External Configuration Files

You can create a separate file (e.g., backend.hcl, prod.tfbackend, or backend.tfvars) containing the specific key-value pairs required by your backend.

Example configuration file (backend.hcl):

bucket         = "my-terraform-state-bucket"
region         = "us-east-2"
dynamodb_table = "terraform-locks"
encrypt        = true

Initialization command: terraform init -backend-config=path/to/backend.hcl.

2. Using Command-Line Flags

Parameters can also be passed directly as individual key-value pairs on the command line.

Initialization command: terraform init -backend-config="key=path/to/my/state/prod.tfstate" -backend-config="region=us-east-1".

Important Considerations

Precedence: If you use both a configuration file and command-line flags, individual flags passed in the command line typically take precedence over values in the file.
Persistence in .terraform: When you run init with partial configuration, Terraform saves these parameters in the .terraform directory. This allows subsequent terraform plan and terraform apply commands to function without re-specifying the backend settings.
Security Risks: Because parameters provided via these methods may be saved on disk in the .terraform directory, you should be cautious when passing sensitive credentials directly through command-line flags or files in shared environments.
Migration Prompts: If you change your backend configuration (e.g., moving from a local backend to a partial remote backend), Terraform will detect this change during init and prompt you to migrate the existing state to the new location.
Interactive Prompts: If you execute terraform init and have omitted required backend parameters without providing them via -backend-config, Terraform will fall back to interactively prompting the user for the missing data.

In the context of Infrastructure as Code (IaC), workspaces serve as isolated environments that allow teams to manage multiple instances of the same infrastructure codebase. While the term is most prominently associated with Terraform, other tools like Pulumi also utilize the workspace concept to facilitate collaboration and environment separation.

Terraform CLI Workspaces

Terraform CLI workspaces allow a developer to use the same configuration files to build multiple, isolated environments, each with its own state file. By default, every Terraform project starts with a single workspace named default.

Mechanics and State Isolation

State Separation: Each workspace maintains a distinct state file. If you create a resource in a workspace named “dev,” it will not be visible when you switch to the “prod” workspace, even though you are using the exact same .tf files.
Remote Backend Storage: When using a remote backend like Amazon S3, Terraform automatically organizes these states into a specific folder structure, typically using an env: prefix followed by the workspace name (e.g., env:/dev/path/to/state).
Interpolation: You can reference the current workspace name in your code using the global variable ${terraform.workspace}. This is commonly used to dynamically name resources or set environment-specific parameters, such as instance types.

Common CLI Commands

Workspaces are managed through several subcommands:

terraform workspace list: Displays all available workspaces in the project.
terraform workspace new <name>: Creates a new, empty workspace and automatically switches to it.
terraform workspace select <name>: Switches to a different existing workspace.
terraform workspace show: Displays the name of the workspace currently in use.
terraform workspace delete <name>: Removes a workspace (note: its state must be empty or deleted first).

Pros and Cons for Production

Use Cases: Workspaces are ideal for quick, isolated tests or experimenting with code refactors without affecting stable infrastructure.
Limitations: They are often considered unsuitable for production-grade isolation because all workspaces in a project typically share the same backend, meaning they use the same authentication and access controls.
Visibility Risks: Workspaces are not explicitly visible in the code structure. It is easy for a developer to forget which workspace is active and accidentally run terraform destroy in the wrong environment. Consequently, many practitioners prefer Isolation via File Layout (using separate folders for /stage and /prod) for critical production environments.

2. HCP Terraform (Cloud) Workspaces

Workspaces in HCP Terraform (TFC) are fundamentally different from CLI workspaces. While CLI workspaces share code but have separate states, TFC workspaces are centralized logical containers that manage code, variables, state, and permissions for a specific component.

Integrated Workflow: TFC workspaces can be linked directly to a Version Control System (VCS) like GitHub. Merging code can automatically trigger a plan or apply within that specific workspace.
Variable Management: Unlike CLI workspaces, TFC workspaces provide a UI for managing environment and Terraform variables, including the ability to mark them as sensitive to hide their values in logs.
Execution Modes: TFC workspaces can run in remote mode (where Terraform executes on HashiCorp’s infrastructure) or local mode (where only the state is stored in the cloud).

3. Pulumi Workspaces

Pulumi utilizes the workspace concept to foster team collaboration by creating isolated units within a project.

Concurrency: Isolation prevents conflicts by segregating changes made by different developers. For instance, one team member can optimize database settings in their workspace without disrupting another developer working on network configurations.
Scoped Configurations: Pulumi workspaces accommodate settings tailored to specific environments, such as unique cloud provider credentials or regions. This allows a single project to target multiple cloud providers (AWS, Azure, GCP) across different workspaces seamlessly.
Git Integration: Workspaces are designed to be version-control friendly, allowing changes to be tracked, reviewed, and audited through standard pull request workflows.

4. Other Tools: SaltStack Environments

While not called “workspaces,” SaltStack achieves similar goals through Environments defined in the file_roots parameter.

Segregation: Configurations are segregated into directories like base, dev, and prod.
Top Files: A central Top File acts as a traffic controller, directing specific “minions” to their designated environment-specific configurations. This ensures, for example, that production passwords are not accidentally deployed to development servers.

Adam Lloyd-Jones

Adam is a privacy-first SaaS builder, technical educator, and automation strategist. He leads modular infrastructure projects across AWS, Azure, and GCP, blending deep cloud expertise with ethical marketing and content strategy.