How do I upgrade to Terraform 12 incrementally?

[yorinasub17]

I am on a legacy project that uses Gruntwork with an old version of Terraform and Terragrunt (0.11.x and 0.18.x). I would like to upgrade to the latest version, but I have too much infrastructure to do it all at once. Is there a way I can incrementally update my projects?

[yorinasub17]

There is a process we have identified for a partial migration that has worked well with a few of our customers. It involves upgrading to terragrunt 19+ first, and then taking on the terraform 12 upgrade. The reason for doing this is that the newer terragrunt features allow you to better manage cross module dependencies that are not bound by the terraform version (terraform_remote_state has limitations where you can't read state that is written by a different terraform version).

Here is the rough process:

Incrementally upgrade to using terragrunt 0.19+

Terragrunt 0.19 has a bunch of features that support mixed terraform versions, so your first step is to upgrade to the latest terragrunt to take advantage of those features. Here is a process for how to do this incrementally in your repo:

• Prerequisite: Install tgswitch and tfswitch. These tools have a bunch of nice features that make it easy to manage mixed version repos. See our blog post How to manage multiple versions of terragrunt and terraform as a team for more info.

• Prerequisite: Install terragrunt 0.18 and 0.23 using tgswitch

  • tgenv 0.18.7
  • tgenv 0.23.31

• Prerequisite: Install terraform 0.11 and 0.12 using tfswitch

  • tfenv 0.11.15
  • tfenv 0.12.31

• Prerequisite: Add .tgswitchrc​ and .tfswitchrc files at root of repo, pointing to 0.18.7 and 0.11.15 respectively. This ensures that the default version used for your modules will be tf11/tg18.

Once the prerequisites are done, you are ready to incrementally switch to tg19+:

• Make a copy of your root terraform.tfvars file, and convert it to terragrunt.hcl.

• Update any tfvars in your directory structure (e.g. region.tfvars) to use yaml files. Replace the relevant extra_arguments subblocks in the root terragrunt.hcl with the yaml pattern as described in our upgrade guide.

• For each module:

  1. Replace terraform.tfvars with terragrunt.hcl file.
  2. Add terraform_version_constraint = ">= 0.11"​ to terragrunt.hcl.
  3. Add .tgswitchrc​ in the upgraded module to point to 0.23.31.
  4. Verify terragrunt plan works and no changes are reported
  5. Open PR and merge to trunk.

• Once all modules are upgraded to terragrunt 0.23, replace the root .tgswitchrc file to point to 0.23.31.

Switch to using dependency blocks

Terragrunt 0.19 introduced a new feature for passing outputs across modules that doesn't depend on the terraform version and remote state: dependency blocks. Using this feature, it is possible to hook up the outputs of a module using terraform 11 with a module using terraform 12. This is important because, as mentioned above, terraform_remote_state data sources are terraform version sensitive.

However, this is a big change for the module so you will want to do this incrementally, in isolation of terraform 12 changes.

• For each module that has a dependency, update the module to replace terraform_remote_state usage to instead take in variables of the ARN/ID of the resources. Use data sources to your advantage to limit the amount of dependency inputs.

• Introduce dependency blocks in terragrunt.hcl to pull in the dependency information from other modules. Note that if a module doesn't export the output you need to feed to the data source, you will need to add this into the upstream modules.

• Run terragrunt plan​ and terragrunt apply ​to ensure the changes are saved in state.

• Open PR and merge to trunk.

• Repeat for every module that has dependencies.

Upgrade modules to terraform 12

Once terragrunt is upgraded and the dependency blocks are introduced, you can start to have mixed terraform version modules. At this point, you can start to incrementally upgrade your modules to terraform 0.12, in a similar fashion to terragrunt:

• Follow https://github.com/gruntwork-io/docs/blob/legacy/content/guides/upgrading-to-tf12-tg19/index.md to upgrade the module to terraform 12

• Add .tfswitchrc ​in the upgraded module to point to 0.12.29.

• Remove the terraform_version_constraint attribute in terragrunt.hcl.

• Run terragrunt plan and terragrunt apply to ensure the changes are saved in state.

• Open PR and merge to trunk.

• Repeat for every module.