I need to deploy a list of GCP compute instances. How do I loop for_each through the "vms" in a list of objects like this:
"gcp_zone": "us-central1-a", "image_name": "centos-cloud/centos-7", "vms": [ { "hostname": "test1-srfe", "cpu": 1, "ram": 4, "hdd": 15, "log_drive": 300, "template": "Template-New", "service_types": [ "sql", "db01", "db02" ] }, { "hostname": "test1-second", "cpu": 1, "ram": 4, "hdd": 15, "template": "APPs-Template", "service_types": [ "configs" ] } ] } 9 Answers
I work a lot with iterators in Terraform, they always gave me bad headaches. Therefore I identified five of the most common iterator patterns (code examples are given below), which helped me construct a lot of nice modules (source).
- Using for_each on a list of strings
- Using for_each on a list of objects
- Using for_each to combine two lists
- Using for_each in a nested block
- Using for_each as a conditional
Using for_each and a list of strings is the easiest to understand, you can always use the toset() function. When working with a list of objects you need to convert it to a map where the key is a unique value. The alternative is to put a map inside your Terraform configuration. Personally, I think it looks cleaner to have a list of objects instead of a map in your configuration. The key usually doesn't have a purpose other than to identify unique items in a map, which can thus be constructed dynamically. I also use iterators to conditionally deploy a resource or resource block, especially when constructing more complex modules.
1. Using for_each on a list of strings
locals { ip_addresses = ["10.0.0.1", "10.0.0.2"] } resource "example" "example" { for_each = toset(local.ip_addresses) ip_address = each.key } 2. Using for_each on a list of objects
locals { virtual_machines = [ { ip_address = "10.0.0.1" name = "vm-1" }, { ip_address = "10.0.0.1" name = "vm-2" } ] } resource "example" "example" { for_each = { for index, vm in local.virtual_machines: vm.name => vm # Perfect, since VM names also need to be unique # OR: index => vm (unique but not perfect, since index will change frequently) # OR: uuid() => vm (do NOT do this! gets recreated everytime) } name = each.value.name ip_address = each.value.ip_address } 3. Using for_each to combine two lists
locals { domains = [ "", "" ] paths = [ "/one", "/two", "/three" ] } resource "example" "example" { # Loop over both lists and flatten the result urls = flatten([ for domain in local.domains : [ for path in local.paths : { domain = domain path = path } ] ])) } 4. Using for_each on a nested block
# Using the optional() keyword makes fields null if not present variable "routes" { type = list( name = string path = string config = optional(object({ cache_enabled = bool https_only = bool })) default = [] } resource "example" "example" { name = ... dynamic "route" { for_each = { for route in var.routes : route.name => route } content { # Note: <top_level_block>.value.<object_key> name = route.value.name } dynamic "configuration" { # Note: <top_level_block>.value.<optional_object_key> for_each = route.value.config != null ? [1] : [] content { cache_enabled = route.value.config.cache_enabled https_only = route.value.config.https_only } } } 5. Using for_each as a conditional
variable "deploy_example" { type = bool description = "Indicates whether to deploy something." default = true } # Using count and a conditional, for_each is also possible here. # See the next solution using a for_each with a conditional. resource "example" "example" { count = var.deploy_example ? 0 : 1 name = ... ip_address = ... } variable "enable_logs" { type = bool description = "Indicates whether to enable something." default = false } resource "example" "example" { name = ... ip_address = ... # Note: dynamic blocks cannot use count! # Using for_each with an empty list and list(1) as a readable alternative. dynamic "logs" { for_each = var.enable_logs ? [] : [1] content { name = "logging" } } } Seem's like I found what to do. If you pass not the maps of maps but the list of maps you can use such code
resource "google_compute_instance" "node" { for_each = {for vm in var.vms: vm.hostname => vm} name = "${each.value.hostname}" machine_type = "custom-${each.value.cpu}-${each.value.ram*1024}" zone = "${var.gcp_zone}" boot_disk { initialize_params { image = "${var.image_name}" size = "${each.value.hdd}" } } network_interface { network = "${var.network}" } metadata = { env_id = "${var.env_id}" service_types = "${join(",",each.value.service_types)}" } } It will create actual number of instance and when you remove for example middle one of three(if you create three:)), terraform will remove what we asked.
3From Terraform 1.3, you can use the for_each and objects with modules like the following:
modules/google_compute_instance/variables.tf
variable "hosts" { type = map(object({ cpu = optional(number, 1) ram = optional(number, 4) hdd = optional(number, 15) log_drive = optional(number, 300) template = optional(string, "Template-New") service_types = list(string) })) } modules/google_compute_instance/main.tf
resource "google_compute_instance" "gcp_instance" { for_each = { for key, value in var.hosts : key => value } hostname = each.key cpu = each.value.cpu ram = each.value.ram hdd = each.value.hdd log_drive = each.value.log_drive template = each.value.template service_types = each.value.service_types } servers.tf
module "gcp_instances" { source = "./modules/google_compute_instance" hosts = { "test1-srfe" = { hdd = 20, log_drive = 500, service_types = ["sql", "db01", "db02"] }, "test1-second" = { cpu = 2, ram = 8, template = "APPs-Template", service_types = ["configs"] }, } } Of course, you can add as many variables as needed and use them in the module.
5You can do the following:
for_each = toset(keys({for i, r in var.vms: i => r})) cpu = var.vms[each.value]["cpu"] Assuming you had the following:
variable "vms" { type = list(object({ hostname = string cpu = number ram = number hdd = number log_drive = number template = string service_types = list(string) })) default = [ { cpu: 1 ... } ] } Using the for_each block is pretty new and there's not too much documentation. Some of the best info comes from their announcement blog post:
Also make sure to check out the Dynamic Blocks section of their documentation:
From what your example looks like you need to have a set of values for each instance that is created so you'll have a map of maps:
Below is an example I created using Terraform 0.12.12:
variable "hostnames" { default = { "one" = { "name" = "one", "machine" = "n1-standard-1", "os" = "projects/coreos-cloud/global/images/coreos-stable-2247-5-0-v20191016", "zone" = "us-central1-a" }, "two" = { "name" = "two", "machine" = "n1-standard-2", "os" = "projects/centos-cloud/global/images/centos-8-v20191018", "zone" = "us-central1-b" } } } resource "google_compute_instance" "default" { for_each = var.hostnames name = each.value.name machine_type = each.value.machine zone = each.value.zone boot_disk { initialize_params { image = each.value.os } } scratch_disk { } network_interface { network = "default" } } Terraform plan output:
Terraform will perform the following actions: # google_compute_instance.default["one"] will be created + resource "google_compute_instance" "default" { + can_ip_forward = false + cpu_platform = (known after apply) + deletion_protection = false + guest_accelerator = (known after apply) + id = (known after apply) + instance_id = (known after apply) + label_fingerprint = (known after apply) + machine_type = "n1-standard-1" + metadata_fingerprint = (known after apply) + name = "one" + project = (known after apply) + self_link = (known after apply) + tags_fingerprint = (known after apply) + zone = "us-central1-a" + boot_disk { + auto_delete = true + device_name = (known after apply) + disk_encryption_key_sha256 = (known after apply) + kms_key_self_link = (known after apply) + mode = "READ_WRITE" + source = (known after apply) + initialize_params { + image = "projects/coreos-cloud/global/images/coreos-stable-2247-5-0-v20191016" + labels = (known after apply) + size = (known after apply) + type = (known after apply) } } + network_interface { + address = (known after apply) + name = (known after apply) + network = "default" + network_ip = (known after apply) + subnetwork = (known after apply) + subnetwork_project = (known after apply) } + scheduling { + automatic_restart = (known after apply) + on_host_maintenance = (known after apply) + preemptible = (known after apply) + node_affinities { + key = (known after apply) + operator = (known after apply) + values = (known after apply) } } + scratch_disk { + interface = "SCSI" } } # google_compute_instance.default["two"] will be created + resource "google_compute_instance" "default" { + can_ip_forward = false + cpu_platform = (known after apply) + deletion_protection = false + guest_accelerator = (known after apply) + id = (known after apply) + instance_id = (known after apply) + label_fingerprint = (known after apply) + machine_type = "n1-standard-2" + metadata_fingerprint = (known after apply) + name = "two" + project = (known after apply) + self_link = (known after apply) + tags_fingerprint = (known after apply) + zone = "us-central1-b" + boot_disk { + auto_delete = true + device_name = (known after apply) + disk_encryption_key_sha256 = (known after apply) + kms_key_self_link = (known after apply) + mode = "READ_WRITE" + source = (known after apply) + initialize_params { + image = "projects/centos-cloud/global/images/centos-8-v20191018" + labels = (known after apply) + size = (known after apply) + type = (known after apply) } } + network_interface { + address = (known after apply) + name = (known after apply) + network = "default" + network_ip = (known after apply) + subnetwork = (known after apply) + subnetwork_project = (known after apply) } + scheduling { + automatic_restart = (known after apply) + on_host_maintenance = (known after apply) + preemptible = (known after apply) + node_affinities { + key = (known after apply) + operator = (known after apply) + values = (known after apply) } } + scratch_disk { + interface = "SCSI" } } Plan: 2 to add, 0 to change, 0 to destroy. Yes this is possible, you need to use the for expression in Terraform to achieve this though, the for loop converts the list of objects into a value in which Terraform can loop over using for_each, without the for expression, Terraform cannot loop over the list of objects because there is no key value for Terraform to reference.
Below is a is a simple example:
# variables.tf variable "nsg_rules" { description = "list of maps consisting of nsg rules" type = list(object({ access = string destination_address_prefixes = list(string) destination_port_ranges = list(string) direction = string name = string priority = number protocol = string source_address_prefixes = list(string) source_port_range = string })) default = [ { access = "Deny" destination_address_prefixes = ["10.10.1.0/24", "10.10.2.0/24"] destination_port_ranges = ["80"] direction = "Inbound" name = "DenyHTTPInbound" priority = 100 protocol = "*" source_address_prefixes = ["10.0.0.0/24"] source_port_range = "*" }, { access = "Deny" destination_address_prefixes = ["10.10.10.0/24", "10.10.11.0/24"] destination_port_ranges = ["22"] direction = "Inbound" name = "DenySSHInbound" priority = 200 protocol = "*" source_address_prefixes = ["10.0.0.0/24"] source_port_range = "*" } ] } Use the for expression wrapped in curl brackets to convert the variable value, each maps key will be given the value of each maps name input, for example the first map would be given a key of "DenyHTTPInbound"
resource "azurerm_network_security_rule" "nsg_rules" { for_each = { for rule in var.nsg_rules : rule.name => rule } access = each.value.access destination_address_prefixes = each.value.destination_address_prefixes destination_port_ranges = each.value.destination_port_ranges direction = each.value.direction name = each.value.name network_security_group_name = azurerm_network_security_group.nsg.name priority = each.value.priority protocol = each.value.protocol resource_group_name = azurerm_resource_group.rg.name source_address_prefixes = each.value.source_address_prefixes source_port_range = each.value.source_port_range } I took reference from the for_each example above and used below. This did not work for me, link below has details. Terraform for_each on custom registry
module "az"{ source="./modules/az" vpc_id = module.vpc.vpc_id for_each = toset(keys({for i,v in var.az_sub: i => v})) availability_zone = var.az_sub[each.value]["az"] public_cidr_block = var.az_sub[each.value]["public_cidr_block"] private_cidr_block =var.az_sub[each.value]["private_cidr_block"] } Error:module.az is object with 2 attributes If I replace for_each with actual values, the module is working perfectly.
This is a pretty confusing structure in terraform, but given:
variable services { type = list(map(string)) description = "services" default = [ { name = "abc" target_port = 9097 health_port = 3780 health_code = 200 protocol = "HTTP" }, { name = "def" target_port = 8580 health_port = 3580 health_code = 200 protocol = "HTTP" }, { name = "ghi" target_port = 80 health_port = 3680 health_code = 200 protocol = "HTTP" } ] } You iterate through resource as so:
resource "aws_lb_listener" "listeners" { for_each = { for service in var.services: service.name => service } load_balancer_arn = aws_lb.internal.arn port = each.value.target_port protocol = each.value.protocol tags = var.tags You reference ANOTHER resource which uses a list of objects as so:
default_action { type = "forward" target_group_arn = aws_lb_target_group.target_groups[each.value.name].id } resource "aws_lb_target_group" "target_groups" { for_each = { for service in var.services: service.name => service } Notice since aws_lb_target_group is also using an array of maps, you must specify the map property when referencing from another resource, as shown above! That could trip people up.
And if you want to output the list of objects, you do as so:
output "alb_listener_arns" { value = values(aws_lb_listener.listeners)[*].arn } output "target_group_ids" { value = values(aws_lb_target_group.target_groups)[*].id } I've kept together some snippets for remembering how to do do various list comprehension and looping in TF. Some examples, e.g. if given a users = { "foo": { a = true, b = []}} in a tfvar file.
Make from map into a list:
output "reminderHowToListComprehensionEach" { value = [for e in var.users : e] } Keep as a map:
output "reminderHowToListComprehensionKeyValue" { value = {for k,v in var.users : k => v} } Another way
output "reminderHowToListComprehensionKey" { value = [for k,v in var.users : k] } And more useful probably, filtering the map:
output "reminderHowToListComprehensionValueWithConditional" { value = [for k,v in var.users : v if !v["xyz_bool_attribute"]] } See also:
