How to configure Terraform logging

How to configure logging for Terraform

Sometimes things just go wrong, and the default error messages coming out is less than helpful.

This was the case for me this morning, I still don’t know why this happened, but when running terraform init, I would get the following error:

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "2.9.0"...
- Installing hashicorp/azurerm v2.9.0...

Error: Failed to install provider

Error while installing hashicorp/azurerm v2.9.0: open
.terraform\plugins\registry.terraform.io\hashicorp\azurerm\2.9.0\windows_amd64\terraform-provider-azurerm_v2.9.0_x5.exe:
The system cannot find the path specified.

While this is a good error, it wasn’t enough to work out what the problem is. Fortunately, Terraform offers verbose logging, which gives way more information making it easier to debug through the issue.

You can find out more about Terraforms logging info in the Debugging Terraform section of the documentation. I highly suggest enabling verbose logging locally. The information is written out is much more helpful when running init, plan or apply.

It wasn’t immediately apparent how I would enable more verbose logging, so below are instructions for enabling this on both Windows and Linux. I set mine to TRACE, but you can also set the following log levels DEBUG, INFO, WARN, or ERROR. I went straight for the most verbose setting, as I wanted to dig through exactly what had gone on in hopes of finding out what was going wrong.

Configuring Terraform logging

To enable the different levels of logging, Terraform requires you to configure two environment variables, these are TF_LOG and TF_LOG_PATH. You need to configure both of these. Otherwise, you won’t get any additional logs. I chose to call my log file terraform.log, but you can name it whatever you like. You can also output the file where ever you want.

Enabling in your current session

If you only want to enable this level of logging temporarily to work through a problem, then you can configure these settings just for the session you are working in. Here’s how to enable these both in PowerShell and Bash; the next time you run a terraform command, your log file will be created and will contain the verbose logging.

PowerShell

> $env:TF_LOG="TRACE"
> $env:TF_LOG_PATH="./logs/terraform.log"

Bash

$ export TF_LOG="TRACE"
$ export TF_LOG_PATH="./logs/terraform.log"

This works great when you just need these detailed logs for a single session. There are times when I’ve run Terraform within VS Code. The output to the console is so large that it overwrites the terminal buffer, preventing my ability to scroll back far enough to capture all the info I want. It’s for this reason that I’ve set this up as a permanent option, though I do suggest you add the log file to your .gitignore file.

Setting up verbose logging permanently for your profile

I’ve become a big fan of this setting, as it means my logs are persisted. There’s no need to rerun a command that failed with a deeper logging level to determine what went wrong as the error is already in my logs.

PowerShell Profile

To set this up as a permanent option in your PowerShell profile, you’ll need to open your Powershell profile. This is simple enough to do with the $profile command in a PowerShell console. Just simply type $profile, and it will output the location of your profile file.

Open that file up and add the following two lines (you can change the name and location to suit you):

# Terraform log settings
$env:TF_LOG="TRACE"
$env:TF_LOG_PATH="./logs/terraform.log"

Close and reopen PowerShell and type the following to verify that the change has worked:

> echo $env:TF_LOG
TRACE
> echo $env:TF_LOG_PATH
/logs/terraform.log

Bash Profile

This is almost exactly the same as the PowerShell method, except the file name and location is different. Open your .bashrc file, which you can find located in your $home directory, then add the following lines:

# Terraform log settings
export TF_LOG=TRACE
export TF_LOG_PATH="./logs/terraform.logs"s

Close your bash console and reopen, and type the following to confirm the change has worked correctly.

$ echo $TF_LOG
TRACE
$ echo $TF_LOG_PATH
/logs/terraform.log

Wrap Up

That’s it. That’s all you need to do to get some really useful logs out of Terraform. Thanks to the verbose logging, I was able to find out what had gone wrong, though why I still don’t know.

The azurerm provider exe wouldn’t populate in the plugins folder. The folder’s paths would, just not the exe. The logs showed that the file was accessible over TCP but still nothing.

In the end, I just downloaded the file manually and placed it in the correct location. While that’s not the most ideal solution, it was enough to allow me to crack on with the rest of my day.

Thanks for reading. I hope this has been some use.