terraform: filter list of maps based on key

I'm implementing a security group modules such that it will create security group rules by taking & filtering cidr & source_security_group_id to create a security group rule.

The current module configuration.

securty_group_module.tf

resource "aws_security_group" "this" { name = var.name description = var.description vpc_id = var.vpc_id revoke_rules_on_delete = var.revoke_rules_on_delete } ## CIDR Rule resource "aws_security_group_rule" "cidr_rule" { count = length(var.security_group_rules) type = var.security_group_rules[count.index].type from_port = var.security_group_rules[count.index].from_port to_port = var.security_group_rules[count.index].to_port protocol = var.security_group_rules[count.index].protocol cidr_blocks = var.security_group_rules[count.index].cidr_block description = var.security_group_rules[count.index].description security_group_id = aws_security_group.this.id } ## Source_security_group_id Rule resource "aws_security_group_rule" "source_sg_id_rule" { count = length(var.security_group_rules) type = var.security_group_rules[count.index].type from_port = var.security_group_rules[count.index].from_port to_port = var.security_group_rules[count.index].to_port protocol = var.security_group_rules[count.index].protocol source_security_group_id = var.security_group_rules[count.index].source_security_group_id description = var.security_group_rules[count.index].description security_group_id = aws_security_group.this.id } 

main.tf

module "sample_sg" { source = "./modules/aws_security_group" name = "test-sg" vpc_id = "vpc-xxxxxx" security_group_rules = [ { type = "ingress", from_port = 22, to_port = 22, protocol = "tcp", cidr_block = [var.vpc_cidr], description = "ssh" }, { type = "ingress", from_port = 80, to_port = 80, protocol = "tcp", cidr_block = [var.vpc_cidr], description = "http" }, { type = "ingress", from_port = 0, to_port = 0, protocol = "-1", source_sg_id = "sg-xxxx", description = "allow all" } { type = "egress", from_port = 0, to_port = 0, protocol = "-1", source_sg_id = "sg-xxxx", description = "allow all" } ] } 

So, the problem statement here is when I call the security group rules in the module with the above list of maps, it should check if it is source_sg_id or cidr.

Then filter those maps & pass it to respective resources in the module.

Ex:

module ""{ ... security_group_rules = [ { type = "ingress", from_port = 22, to_port = 22, protocol = "tcp", cidr_block = [var.vpc_cidr], description = "ssh" }, { type = "ingress", from_port = 0, to_port = 65535, protocol = "-1", source_sg_id = "sg-xxxx", description = "allow all" } ] } 

These rules should be looked up & pass the first one to CIDR rule & second one to Source_security_group_id rule.

I'm thinking of making it as below

locals { sid_rules = some_function{var.security_group_rules, "source_security_group_id"} cidr_rules = some_function{var.security_group_rules, "cidr"} } resource "aws_security_group_rule" "cidr_rule" { count = count(local.cidr_rules) .... cidr_blocks = local.cidr_rules[count.index].cidr_block .... } resource "aws_security_group_rule" "sid_rule" { count = count(local.sid_rules) .... source_security_group_id = local.sid_rules[count.index].source_sg_id .... } 

So, I'm looking for a way to filter the maps from list based on a key

I have tried lookup but was no help in case of list of string.

5 Answers

I figured out a clever way to do this.

Let's say I am trying to filter only the pets that are cats kind = "cat" from a list of pets.

variable "pets" { type = list(object({ name = string kind = string })) default = [ { name = "Fido" kind = "dog" }, { name = "Max" kind = "dog" }, { name = "Milo" kind = "cat" }, { name = "Simba" kind = "cat" } ] } 

1. First convert the list of pets to a map pets_map of pets using the index tostring(i) as the key. This will be used in step 3 to lookup the filtered pets.

locals { pets_map = { for i, pet in var.pets : tostring(i) => pet } } 

2. Next create a filtered list of the keys that respectively matches the condition pet.kind == "cat" by looping over the keys in the pets_map and setting the respective keys that do not match to an empty string. Then compact the list which removes the empty strings from the list.

locals { cats_keys = compact([for i, pet in local.pets_map : pet.kind == "cat" ? i : ""]) } 

3. Loop over the filtered keys cats_keys and lookup the respective pet from the pets_map. Now you have the filtered list of pets that are cats kind = "cat".

locals { cats = [for key in local.cats_keys : lookup(local.pets_map, key)] } 

You can now access the cats with local.cats, which will give you the following map.

{ name = "Milo" kind = "cat" }, { name = "Simba" kind = "cat" } 

Below is the full example.

variable "pets" { type = list(object({ name = string kind = string })) default = [ { name = "Fido" kind = "dog" }, { name = "Max" kind = "dog" }, { name = "Milo" kind = "cat" }, { name = "Simba" kind = "cat" } ] } locals { pets_map = { for i, pet in var.pets : tostring(i) => pet } cats_keys = compact([for i, pet in local.pets_map : pet.kind == "cat" ? i : ""]) cats = [for key in local.cats_keys : lookup(local.pets_map, key)] } 

Consider creating another module to handle the rules, and setting the security group resources inside that module.

module "security_groups" { count = length(var.security_group_rules) source_sg_id_rule = var.security_group_rules[count.index].source_sg_id_rule } 

Then, in the new module, use a count statement as a test to create optional items:

resource "aws_security_group_rule" "source_sg_id_rule" { count = length(var.source_sg_id_rule) == 0 ? 0 : 1 type = var.type from_port = var.from_port to_port = var.to_port protocol = var.protocol source_security_group_id = var.source_security_group_id description = var.description security_group_id = var.security_group_id } 

This will create the resources as an array of one or zero items, and drop any lists of zero.

Given:

[{foo: "...", baz: "..."}, ...] 

And you want the baz of some element in this list, use:

element([ for o in [{"foo"="kick","baz"="5"},{"foo"="bar","baz"="100"}] : o if o.foo == "bar" ],0).baz 

Thanks for the response @dan-monego.

I sorted it out with single module itslef.

Following is the module file.

aws_sg_module.tf

 # Security group ########################## resource "aws_security_group" "this" { name = var.name description = var.description vpc_id = var.vpc_id revoke_rules_on_delete = var.revoke_rules_on_delete tags = merge( { "Name" = format("%s", var.name) }, local.default_tags, var.additional_tags ) } resource "aws_security_group_rule" "cidr" { count = var.create ? length(var.cidr_sg_rules) : 0 type = var.cidr_sg_rules[count.index].type from_port = var.cidr_sg_rules[count.index].from to_port = var.cidr_sg_rules[count.index].to protocol = var.cidr_sg_rules[count.index].protocol cidr_blocks = var.cidr_sg_rules[count.index].cidr description = var.cidr_sg_rules[count.index].description security_group_id = local.this_sg_id } resource "aws_security_group_rule" "source_sg" { count = var.create ? length(var.source_sg_rules) : 0 type = var.source_sg_rules[count.index].type from_port = var.source_sg_rules[count.index].from to_port = var.source_sg_rules[count.index].to protocol = var.source_sg_rules[count.index].protocol source_security_group_id = var.source_sg_rules[count.index].source_sg_id description = var.source_sg_rules[count.index].description security_group_id = local.this_sg_id } resource "aws_security_group_rule" "self" { count = var.create ? length(var.self_sg_rules) : 0 self = true type = var.source_sg_rules[count.index].type from_port = var.source_sg_rules[count.index].from to_port = var.source_sg_rules[count.index].to protocol = var.source_sg_rules[count.index].protocol description = var.source_sg_rules[count.index].description security_group_id = local.this_sg_id } 

Call it using following module block.

security_groups.tf

module "stack_sg" { source = "./modules/aws_security_group" name = "stack-sg" vpc_id = module.network.vpc_id cidr_sg_rules = [ { type = "ingress", from = 80, to = 80, protocol = "tcp", cidr = [module.network.vpc_cidr], description = "http" }, { type = "egress", from = 0, to = 65535, protocol = "-1", cidr = ["0.0.0.0/0"], description = "allow all " } ] source_sg_rules = [ { type = "ingress", from = 0, to = 65535, protocol = "tcp", source_sg_id = module.alb_sg.sg_id, description = "alb" } ] } 

In order to filter a list of maps using a specific key value. You can use the following simple statement:

Assuming:

• key is map's key you're filtering on
• val is the value of the key
• list is the original list of maps

element([ for element in list) : env if element.key == "val" ], 0) 

the result of the above statement will be a map.