Terraform examines each resource and uses a graph-based approach to model and apply the desired state.Each resource is placed in the graph, its relationships with other resources are calculated, and then each resource is automatically built in the correct order to produce your infrastructure.
When Terraform runs inside a directory it will load any Terraform configuration files ( e.g. *.tf). Any non-configuration files are ignored and Terraform will not recurse into any sub-directories. Each file is loaded in alphabetical order, and the contents of each configuration file are appended into one configuration.Terraform also has an “override” file construct. Override files are merged rather than appended.
Terraform then constructs a DAG, or Directed Acyclic Graph, of that configuration. The vertices of that graph— its nodes—are resources—for example, a host, subnet, or unit of storage; the edges are the relationships, the order, between resources. For example, a network may be required before a host can be created in order to assign it an IP address. The graph determines this relationship and then ensures Terraform builds your configuration in the right order to satisfy this. This is a similar technique to what Puppet uses to construct its configuration catalog.
Terraform configuration files are normal text files. They are suffixed with either .tf or .tf.json. Files suffixed with .tf are in Terraform’s native file format, and .tf.json files are JSON-formatted.
After creating our resource, Terraform has saved the current state of our infrastructure into a file called terraform.tfstate in our base directory. This is called a state file. The state file contains a map of resources and their data to resource IDs.
The state is the canonical record of what Terraform is managing for you. This file is important because it is canonical. If you delete the file Terraform will not know what resources you are managing, and it will attempt to apply all configuration from scratch. This is bad. You should ensure you preserve this file.
As this file is the source of truth for the infrastructure being managed, it’s critical to only use Terraform to manage that infrastructure. If you make a change to your infrastructure manually, or if you use another tool, it can be easy for this state to get out of sync with reality. You can then lose track of the state of your infrastructure and its configuration, or have Terraform reset your infrastructure back to a potentially non-functioning configuration when it runs. We strongly recommend that if you want to manage specific infrastructure with Terraform that it becomes the sole way of managing that infrastructure.
If our execution plan had failed, then Terraform would not roll back the resources. It’ll instead mark the failed resource as tainted. The tainted state is Terraform’s way of saying, “This resource may not be right.” Why tainting instead of rolling back? Terraform always holds to the execution plan: it was asked to create a resource, not delete one. If you run the execution plan again, Terraform will attempt to destroy and recreate any tainted resources.
Terraform is a declarative system; you specify the proposed state of your resources rather than the steps needed to create those resources. When you specify resources, Terraform builds a dependency graph of your configuration. The dependency graph represents the relationships between the resources in your configuration.
When you plan or apply that configuration, Terraform walks that graph, works out which resources are related, and hence knows the order in which to apply them.
Usage: terraform [-version] [-help] <command> [args]
The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.
Common commands:
apply Builds or changes infrastructure
console Interactive console for Terraform interpolations
destroy Destroy Terraform-managed infrastructure
env Workspace management
fmt Rewrites config files to canonical format
get Download and install modules for the configuration
graph Create a visual graph of Terraform resources
import Import existing infrastructure into Terraform
init Initialize a Terraform working directory
output Read an output from a state file
plan Generate and show an execution plan
providers Prints a tree of the providers used in the configuration
push Upload this Terraform module to Atlas to run
refresh Update local state file against real resources
show Inspect Terraform state or plan
taint Manually mark a resource for recreation
untaint Manually unmark a resource as tainted
validate Validates the Terraform files
version Prints the Terraform version
workspace Workspace management
All other commands:
debug Debug output management (experimental)
force-unlock Manually unlock the terraform state
state Advanced state management
Initialize a new or existing Terraform working directory by creating initial files, loading any remote state, downloading modules, etc.
This is the first command that should be run for any new or existing Terraform configuration per machine. This sets up all the local data necessary to run Terraform that is typically not committed to version control.
This command is always safe to run multiple times. Though subsequent runs may give errors, this command will never delete your configuration or state. Even so, if you have important information, please back it up prior to running this command, just in case.
If no arguments are given, the configuration in this working directory is initialized.
One of the useful flags is -backend-config= which allows you to use custom configuration file for initialization.
When performing changes it is recommended to use flag -out flag with terraform plan -out change-$(date +'%%s').plan which can be used for apply later on.This approach can help us to make changes to the infrastructure incrementally and carefully.Also no confirmation is needed for executing terraform apply change-$(date +'%%s').plan
Another useful flag for terraform plan and terraform apply is flag -target to restrict resource which will be managed.
By default, terraform plan will check for changes between current state and current configuration in *.tf files.In case of removing resources (aka destroy) terraform plan has handy flag -destroy which can be also used with -out flag for incremental changes.
Warning: The generated plan file will contain all your variable values potentially including any credentials or secrets. It is not encrypted or otherwise protected. Handle this file with appropriate caution!
The current implementation of Terraform import can only import resources into the state. It does not generate configuration. A future version of Terraform will also generate configuration.
Because of this, prior to running terraform import it is necessary to write manually a resource configuration block for the resource, to which the imported object will be attached.Only required arguments are needed.
While this may seem tedious, it still gives Terraform users an avenue for importing existing resources. A future version of Terraform will fully generate configuration, significantly simplifying this process.
terraform import ${PROVIDER}_${TYPE}.${NAME} ${RESOURCE_ID}
# here is an example for aws
terraform import aws_instance.web i-12345678
or import block
import {
to = "aws_instance.web"
id = "i-12345678"
}
Terraform has 2 useful commands when working with *.tf files.With terraform validate it will check if all variables are defined and do a basic syntax check.On the other hand, with terraform fmt -diff=true it will format related tf file to the canonical format and style.Thus one doesn’t need to worry about indention or style.
A <computed> value is one that Terraform does not know the value
of yet. The value of the configuration item will only be known when the resource
is actually created.
Terraform apply changes as:
-: removing resource+: adding resource-/+: replacing resource~: modifying resourceWhere possible, Terraform will aim to perform the smallest incremental change rather than rebuilding every resource. In some cases, however, changing a resource requires recreating it. Since this is a destructive action, you should always carefully read the proposed actions in a terraform apply before saying yes or run terraform plan first to understand the impact of executing the change. The last thing you want to do is inadvertently destroy a running application.
Terraform also has the concept of tainting and untainting resources. Tainting resources marks a single resource to be destroyed and recreated on the next apply. It doesn’t change the resource but rather the current state of the resource. Untainting reverses the marking.
tainting an resource from the module
terraform taint -module=remote_state aws_s3_bucket.remote_state
The console is a Terraform REPL that allows you to work with interpolations and other logic. It’s a good way to explore working with Terraform syntax.This command loads the current state.This command will never modify your state.
$ terraform console
> lookup(map("id", "1","message", "hello world" ), "id")
1
> lookup(map("id", "1","message", "hello world" ), "message")
hello world
> max("1", "2","4")
4
> min("1", "2","4")
1
> concat(list("1", "2"),list("3","5"))
[
"1",
"2",
"3",
"5",
]
> element(concat(list("1", "2"),list("3","5")),0)
1
> element(concat(list("1", "2"),list("3","5")),2)
3
The terraform output command is used to extract the value of an output variable from the state file.One of the useful flags is -module=module_name to extract output from specific module.
The terraform show command is used to provide human-readable output from a state or plan file. This can be used to inspect a plan to ensure that the planned operations are expected, or to inspect the current state as Terraform sees it.
The terrafirn moved and removed is used to rename and remove resource from terraform state.
moved {
from = module.foo.aws_iam_user[0]
to = module.foo.aws_iam_user_0
}
removed {
from = module.foo.aws_iam_user_0
lifecycle {
destroy = false
}
}
They represent the infrastructure components you want to manage: hosts, networks, firewalls, DNS entries, etc. The resource object is constructed of a type, name, and a block containing the configuration of the resource. The combination of type and name must be unique in your configuration. Each resource is a vertex or node in that graph. Each node is uniquely identified, its edges examined to determine its relationship with other nodes in the graph, and an order determined for the creation of those resources.
resource "PROVIDER_TYPE" "NAME" {
[ CONFIG ... ]
}
variable "NAME" {
description = "..."
type = "string|list|map"
# string
default = "bla"
# list
default = [1,2,3]
# map
default = {
key1 = "value1"
key2 = "value2"
}
}
Variables with and without defaults behave differently.A defined, but empty variable is a required value for an execution plan.
$ terraform plan -var 'access_key=abc123' -var 'secret_key=abc123'
$ terraform plan -var 'ami={ us-east-1 = "ami-0d729a60", us-west-1 = "ami-
7c4b331c" }'
$ terraform plan -var 'security_group_ids=["sg-4f713c35", "sg-4f713c35", "sg-
4f713c35"]'
When Terraform runs it will automatically load the terraform.tfvars file and assign any variable values in it.If you name it differently you will need to add flag -var-file to the terraform plan.You can use more than one -var-file flag to specify more than one file. If you specify more than one file, the files are evaluated from first to last, in the order specified on the command line. If a variable value is
specified multiple times, the last value defined is used.
cat > terraform.tfvars
access_key = "abc123"
secret_key = "abc123"
ami = {
us-east-1 = "ami-0d729a60"
us-west-1 = "ami-7c4b331c"
}
security_group_ids = [
"sg-4f713c35",
"sg-4f713c35",
"sg-4f713c35"
]
Terraform will also parse any ENV variables that are prefixed with TF_VAR. For example following ENVs
TF_VAR_access_code=abc123
TF_VAR_ami='{us-east-1 = "ami-0d729a60", us-west-1 = "ami-7c4b331c"}'
TF_VAR_roles='["sg-4f713c35", "sg-4f713c35", "sg-4f713c35"]'
can be used as var.access_code, var.ami and var.roles in terraform configuration.
using string
"${var.VARIABLE_NAME}"
using list
"${var.LIST}"
or
"${var.LIST[idx]}" - where idx is element of the list
using map
"${var.MAP["KEY"]}"
Lists are zero-indexed.
variable "security_group_ids" {
type = "list"
description = "List of security group IDs."
default = ["sg-1", "sg-2", "sg-3"]
}
variable "ami" {
type = "map"
default = {
us-east-1 = "ami-0d729a60"
us-west-1 = "ami-7c4b331c"
}
description = "The AMIs to use."
}
variable "region" {
description = "The AWS region."
default = "us-east-1"
}
count (int) - create identical resources multiple times,use as index for function elementdepends_on ( list of strings ) - explicitly specify dependenciesprovider (string) - use alias providerlifecycle - customize resource lifecycle
create_before_destroy (bool) - create first replacement resourceprevent_destroy (bool) - do not destroyignore_changes (list of strings)output "NAME" {
value = "${PROVIDER_TYPE.NAME.item}"
description = "some description"
}
Modules are defined with the module block. Modules are a way of constructing reusable bundles of resources. They allow you to organize collections of Terraform code that you can share across configurations.
Hashicorp makes available a collection of verified and community modules in the Terraform Module Registry. These include modules for a large number of purposes and are a good point to start if you need a module. You can learn more about the Terraform Module Registry in the documentation.
NOTE Modules with a blue tick on the Terraform Registry are verified andfrom a Hashicorp partner. These modules should be more resilient and tested than others. You can also publish your own modules on the Registry.
example of module
.
├── child/
│ └── main.tf
├── main.tf
└── terraform.tfvars
Content of each files.Note that variable "memory" {} needs to be provided in parent folder in order to be used via terraform.tfvars.
==> main.tf <==
variable "memory" {}
module "child" {
source = "./child"
memory = "${var.memory}"
}
output "child_memory" {
value = "${module.child.received}"
}
==> terraform.tfvars <==
memory = "1G"
==> child/main.tf <==
variable "memory" {}
output "received" {
value = "${var.memory}"
}
So what’s module root? Well, remember that modules are just folders containing files. Terraform considers every folder of configuration files a module. Modules in Terraform are folders with Terraform files. In fact, when you run terraform apply, the current working directory holding the Terraform files you’re applying comprise what is called the root module. This itself is a valid module.
More info - Standard structure The standard module structure is a file and folder layout we recommend for reusable modules.
The standard module expects the structure documented below:
main.tf,variables.tf,outputs.tfvariables.tf and outputs.tf should have descriptions/modules folder/examples folder/test used by golang library terratestBefore using a module you need to load it or get it.You need to execute terraform get from your root module directory.This command gets the module code and stores it in the .terraform/modules directory inside the root module directory.If you change your module, or the module you’re using has been updated, you’ll need to run the terraform get command again, with the -update flag set
Data sources provide read-only data that can be used in your configuration. Data sources are linked to providers. Not every provider has data sources—generally they exist if there are sources of information that are useful in the configuration managed by the provider. For example, the aws provider has a data source called aws_ami that allows you to search for and return the IDs of specific AMIs.
data source for template provider
data "template_file" "init" {
template = "${file("${path.module}/init.tpl")}"
vars = {
consul_address = "${aws_instance.consul.private_ip}"
}
}
where init.tpl is
#!/bin/bash
echo "CONSUL_ADDRESS = ${consul_address}" > /tmp/iplist
Terraform templates are very simple. They can only contain string primitives and don’t support lists or maps. They do not support control flow logic like optional content, if/else clauses, or loops.
Via data provider terraform_remote_state one can retrieve variables and states from different remote state configs ( used when you have folders ).Before one can use it in source state file the variable must be present in output ( defined in ouputs.tf ).
data "terraform_remote_state" "db" {
backend = "s3"
config {
bucket = "${YOUR_BUCKET_NAME}"
key = "${YOUR_KEY}"
region = "${REGION}"
}
We need to trigger terraform apply in current root module folder (which is using above data state) as data source is a resource.Afterwards, output variables can be viewed via terraform show.
terraform {
backend "s3" {
bucket = "xxxx"
key = "terraform.tfstate"
region = "us-east-1"
# optinal locking
dynamodb_table = "yyyy"
}
}
Your backend configuration cannot contain interpolated variables. This is because this configuration is initialized prior to Terraform parsing these variables.Before using s3 remote state backend s3 bucket must be created or local terraform state must be used in the first place.Afterwards, after defining s3 as backend and triggering reinitialization s3 can be used.
resource "aws_dynamodb_table" "state-lock" {
name = "yyyy"
hash_key = "LockID"
read_capacity = 20
write_capacity = 20
attribute {
name = "LockID"
type = "S"
}
tags {
Name = "${var.prefix}-state-lock-${var.environment}"
ENV = "${var.environment}"
}
}
"${some_function(...)}
${count.index} - the current index in a multi-count resource${length(list)} - returns the length of the list or map${format(format, args)} - format a string according to the given format${lookup(map, key, [default]} - dynamic lookup on map.If key does not exist in map, the interpolation will fail unless you specify a third argument, default, which should be a string value to return if no key is found in map${file(path)} - read contents from a file to a stringprovider_type.name.*.id[#] - where # is the index number of the resourceprovider_type.name.*.id - all resources${element(list, index)} - returns a single element from a list as the given indexThe syntax is path.TYPE. TYPE can be cwd, module, or root.
${path.cwd} - will interpolate the current working directory${path.module} will interpolate the path to the current module${path.root} will interpolate the path of the root moduleIn general, you probably want the path.module variable.
condition ? true : false
resource "aws_sns_topic_subscription" "cloudwatch_alarms" {
count = "${var.pagerduty_alerts_endpoint != "" ? 1 : 0}"
endpoint = "${var.pagerduty_alerts_endpoint}"
endpoint_auto_confirms = true
protocol = "https"
topic_arn = "${aws_sns_topic.alarms.arn}"
}
supported operators
Equality: == and !=
Numerical comparison: >, <, >=, <=
Boolean logic: &&, ||, unary !
Terraform also has the concept of local value configuration. Local values assign a name to an expression, essentially allowing you to create repeatable function-like values.
locals {
instance_ip_count = "${length(var.instance_ips)}"
default_name_prefix = "${var.project_name}-web"
name_prefix = "${var.name_prefix != "" ? var.name_prefix : local.default_name_prefix}"
common_tags = {
Component = "awesome-app"
Environment = "production"
}
}
A local is only available in the context of the module it is defined in. It will not work cross-module.
# Local values can be interpolated elsewhere using the "local." prefix.
resource "aws_s3_bucket" "files" {
bucket = "${local.name_prefix}-files"
# ...
}
When you create and delete a resource, you can run provisioners in those resources to configure them. This can be used to bootstrap the resource—for example, to add it to a cluster or to update a network device. Provisioners can also trigger other tools, like configuration management tools or patch and update systems. The most important piece of information here is that provisioners only run when a resource is created or destroyed. They do not run when a resource is changed or updated. This means that if you want to trigger provisioning again, you will need to destroy and recreate the resource, which is often not convenient or practical. Again, Terraform’s provisioning is not a replacement for configuration management. Provisioners are the exception to the rule for Terraform resources: they are executed in the sequence they are specified. This means you can daisy chain provisioning actions and trust them to work in the order specified.
type of provisioners:
file - is used to upload file,tempalte,dir or content to the remote host
provisioner "file" {
content = "Instance ID: ${self.id}"
destination = "/etc/instance_id"
}
What’s self? It refers to the value of one of the resource’s own attributes. An attribute name prefixed with self refers to the attribute in that resource. For example, self.id would refer to id attribute of an aws_instance resource. You can only use self variables in provisioners. They do not work anywhere else.
remote-exec - runs one or more scripts on remote host.Possible options are script and ordered scripts without args and inline for ordered commands with args.
provisioner "remote-exec" {
script = "files/bootstrap_puppet.sh"
}
local-exec - runs commands locally
provisioner "local-exec" {
command = "echo ${aws_instance.web.private_ip} >> private_ips.txt"
}