Introducing Edge Delta Terraform Provider: Manage Hundreds of Agents in a Single Terraform Config
.avif)
See Edge Delta in Action
Edge Delta features, like agents and monitors, enrich your observability pipeline. It does so by analyzing your data as it’s created at the source and surfacing insights, such as changes in your systems and anomalous behavior.
As customers go from monitoring multiple data sources to managing hundreds, we’ve received feedback to automate the configurations of agents, monitors, and other features. As a result, we’ve come up with a solution that’s easy to use and developer-friendly: a Terraform provider for Edge Delta resources!
We present you terraform-provider-edgedelta. After considering and trying different solutions, we found that Terraform's HCL provides an excellent declarative language to define various Edge Delta resources clearly and precisely in a single configuration file. It also supports iterative constructs and templates, which helps you avoid lengthy, repetitive configuration files declaring hundreds of resources quite similar to each other.
At the time being, Edge Delta Terraform provider supports the following resources:
- Agent configuration (
edgedelta_config) - - Monitor (
edgedelta_monitor)
This post will go over the current features of terraform-provider-edgedelta with step-by-step examples for edgedelta_config and edgedelta_monitor resources.
Setting Up the Environment
Before using the provider, you need to have Terraform CLI installed. If you don't, you can install it following the steps here.
The setup is pretty straightforward, we just need to initialize a .tf config file using provider "edgedelta/edgedelta":
terraform {
required_providers {
edgedelta = {
source = "edgedelta/edgedelta"
version = "0.0.5"
}
}
}
variable "ED_API_TOKEN" {
type = string
}
provider "edgedelta" {
org_id = ""
api_secret = var.ED_API_TOKEN
}
We can save this config file to edgedelta.tf and run terraform init to initialize the provider:
terraform init
Before we start, we need to provide some credentials to the provider. We can do this by setting the ED_API_TOKEN variable in the environment:
export TF_VAR_ED_API_TOKEN=
If you don't have an API token, you can create one from the global settings. Here, you can find the API token under the "Token" section. Click on the "Create Token" button and give your token a name. Be sure to add "Agent Configurations" and "Monitors" permissions to your token. You can then use the token in the environment variable TF_VAR_ED_API_TOKEN.
Creating an Agent Configuration
Let's start by defining a new agent configuration called new_config:
resource "edgedelta_config" "new_config" {
config_content = file("new_config.yml")
}
Here, edgedelta_config is the type of the resource we're defining. We have also used config_content to specify the content of the agent configuration, and the file function to read the configuration content from a YAML file. This is a Terraform built-in function that allows us to read files from the disk.
Let's also put the configuration content in a file called new_config.yml:
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
region: us-west-2
log_group_name: /ecs/microservice
log_stream_name: test-stream
Here, we defined a dummy agent configuration using Edge Delta's Amazon CloudWatch log stream integration. This configuration will be used to create a new agent.
We are using the Amazon CloudWatch integration just for the sake of example. You can use any other integration.
To see the configuration ID that will be generated for the new agent configuration, we can define an output variable:
output "new_config_id" {
value = edgedelta_config.new_config.id
description = "The config ID created for the edgedelta_config.new_config instance"
}
To put the configuration all together, we now have:
terraform {
required_providers {
edgedelta = {
source = "edgedelta/edgedelta"
version = "0.0.5"
}
}
}
variable "ED_API_TOKEN" {
type = string
}
provider "edgedelta" {
org_id = ""
api_secret = var.ED_API_TOKEN
}
output "new_config_id" {
value = edgedelta_config.new_config.id
description = "The config ID created for the edgedelta_config.new_config instance"
}
resource "edgedelta_config" "new_config" {
config_content = file("new_config.yml")
}
To see the actions that Terraform will apply using the configuration we have just created, we can use terraform plan:
terraform plan
This will create a plan that Terraform will execute to create the new agent configuration. We can see that the new agent configuration will be created and the output variable new_config_id will be set to the ID of the new agent configuration:
Terraform will perform the following actions:
# edgedelta_config.new_config will be created
+ resource "edgedelta_config" "new_config" {
+ config_content = <<-EOT
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
region: us-west-2
log_group_name: /ecs/microservice
log_stream_name: test-stream
EOT
+ id = (known after apply)
+ tag = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ new_config_id = (known after apply)
Now we can run terraform apply to create the new agent configuration:
terraform apply
After typing yes and the command finishes, we can see the new agent configuration:
edgedelta_config.new_config: Creating...
edgedelta_config.new_config: Creation complete after 2s [id=11111111-2222-3333-4444-555555555555]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
new_config_id = "11111111-2222-3333-4444-555555555555"
We can also navigate to app.edgedelta.com/agent-settings to see the new agent configuration:
Updating the Agent Configuration
Now that we have created a new agent configuration, we can update it by changing the config_content attribute. To do this, we just need to update the configuration content in the new_config.yml file. Let's change the region attribute to us-west-1 and apply terraform plan:
terraform plan
The plan shows that the agent configuration will be updated:
Terraform will perform the following actions:
# edgedelta_config.new_config will be updated in-place
~ resource "edgedelta_config" "new_config" {
~ config_content = <<-EOT
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
- region: us-west-2
+ region: us-west-1
log_group_name: /ecs/microservice
log_stream_name: test-stream
EOT
id = "11111111-2222-3333-4444-555555555555"
# (1 unchanged attribute hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
As we can see, Terraform will update the agent configuration by only changing the region attribute to us-west-1. Now we can run terraform apply to update the agent configuration:
terraform apply
edgedelta_config.new_config: Modifying... [id=11111111-2222-3333-4444-555555555555]
edgedelta_config.new_config: Modifications complete after 1s [id=11111111-2222-3333-4444-555555555555]
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Outputs:
new_config_id = "11111111-2222-3333-4444-555555555555"
The agent configuration is now updated on the Edge Delta side and the new region is us-west-1.
Deleting the Agent Configuration
If we want to delete the agent configuration, we can use terraform destroy. This will delete the agent configuration:
terraform destroy
After typing yes, we can see that the agent configuration is deleted:
Terraform will perform the following actions:
# edgedelta_config.new_config will be destroyed
- resource "edgedelta_config" "new_config" {
- config_content = <<-EOT
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
region: us-west-1
log_group_name: /ecs/microservice
log_stream_name: test-stream
EOT -> null
- id = "11111111-2222-3333-4444-555555555555" -> null
- tag = "Edge" -> null
}
Plan: 0 to add, 0 to change, 1 to destroy.
Changes to Outputs:
- new_config_id = "11111111-2222-3333-4444-555555555555" -> null
Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
edgedelta_config.new_config: Destroying... [id=11111111-2222-3333-4444-555555555555]
edgedelta_config.new_config: Destruction complete after 0s
Destroy complete! Resources: 1 destroyed.
Creating a Monitor Resource
Just like the agent configuration, we can create monitors using the edgedelta_monitor resource in Terraform. In Edge Delta we have 4 types of monitors: correlated-signal, mettric-alert, pattern-check and pattern-skyline. In this example we will create a pattern-check monitor.
You can find the full list of monitor types in the Edge Delta documentation.
Let's create a monitor resource named pattern_check_monitor in our Terraform config. The resource definition is as follows:
output "monitor_id" {
value = edgedelta_monitor.pattern_check_monitor.id
}
resource "edgedelta_monitor" "pattern_check_monitor" {
name = "pattern-check-example-monitor"
type = "pattern-check"
enabled = true
payload = file("payload.json")
creator = "creator@email.domain"
}
And also provide the payload file payload.json in the same directory as the Terraform config file. The payload file is as follows:
{
"merge_level": "none",
"mail_recipients": "test1@example.org",
"triggers": ["slack-trigger"],
"suppression_window": "12h",
"timezone": "Europe/London",
"trigger_template": ""
}
To create the monitor, run terraform apply:
terraform apply
Which outputs the following:
Terraform will perform the following actions:
# edgedelta_monitor.pattern_check_monitor will be created
+ resource "edgedelta_monitor" "pattern_check_monitor" {
+ creator = "creator@email.domain"
+ enabled = true
+ id = (known after apply)
+ name = "pattern-check-example-monitor"
+ payload = jsonencode(
{
+ mail_recipients = "test1@example.org"
+ merge_level = "none"
+ suppression_window = "12h"
+ timezone = "Europe/London"
+ trigger_template = ""
+ triggers = [
+ "slack-trigger",
]
}
)
+ type = "pattern-check"
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ monitor_id = (known after apply)
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
edgedelta_monitor.pattern_check_monitor: Creating...
edgedelta_monitor.pattern_check_monitor: Creation complete after 2s [id=222222-3333-4444-5555-666666666666]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
monitor_id = "222222-3333-4444-5555-666666666666"
Updating a Monitor Resource
After making changes to the monitor definition in the Terraform configuration, we can update the monitor by running terraform apply.
Let's update the payload of the pattern-check monitor we have created above and change the timezone to America/Los_Angeles:
{
"merge_level": "none",
"mail_recipients": "test1@example.org",
"triggers": ["slack-trigger"],
"suppression_window": "12h",
"timezone": "America/Los_Angeles",
"trigger_template": ""
}
Now we can run terraform apply to update the monitor:
terraform apply
We can see that the monitor is updated successfully in the output:
Terraform will perform the following actions:
# edgedelta_monitor.pattern_check_monitor will be updated in-place
~ resource "edgedelta_monitor" "pattern_check_monitor" {
id = "222222-3333-4444-5555-666666666666"
name = "pattern-check-example-monitor"
~ payload = jsonencode(
~ {
~ timezone = "Europe/London" -> "America/Los_Angeles"
# (5 unchanged elements hidden)
}
)
# (3 unchanged attributes hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
edgedelta_monitor.pattern_check_monitor: Modifying... [id=222222-3333-4444-5555-666666666666]
edgedelta_monitor.pattern_check_monitor: Modifications complete after 2s [id=222222-3333-4444-5555-666666666666]
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Outputs:
monitor_id = "222222-3333-4444-5555-666666666666"
Deleting a Monitor Resource
To delete a monitor, we can run terraform destroy:
terraform destroy
This will delete our existing monitor resource with ID 222222-3333-4444-5555-666666666666:
Terraform will perform the following actions:
# edgedelta_monitor.pattern_check_monitor will be destroyed
- resource "edgedelta_monitor" "pattern_check_monitor" {
- creator = "creator@email.domain" -> null
- enabled = true -> null
- id = "222222-3333-4444-5555-666666666666" -> null
- name = "pattern-check-example-monitor" -> null
- payload = jsonencode(
{
- mail_recipients = "test1@example.org"
- merge_level = "none"
- suppression_window = "12h"
- timezone = "America/Los_Angeles"
- trigger_template = ""
- triggers = [
- "slack-trigger",
]
}
) -> null
- type = "pattern-check" -> null
}
Plan: 0 to add, 0 to change, 1 to destroy.
Changes to Outputs:
- monitor_id = "222222-3333-4444-5555-666666666666" -> null
Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
edgedelta_monitor.pattern_check_monitor: Destroying... [id=222222-3333-4444-5555-666666666666]
edgedelta_monitor.pattern_check_monitor: Destruction complete after 2s
Destroy complete! Resources: 1 destroyed.
As we can see, the monitor is deleted successfully in the output.
Using Templates to Create Multiple Resources
One of the most useful features of the provider is the ability to create multiple resources at once. To do this, we can use the templatefile function and for_each construct in Terraform. templatefile function allows us to use a template file to create multiple resources. The template file for this example should be in YAML format. Let's use config.yml.tpl as the template file:
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
region: ${region}
log_group_name: /ecs/microservice
log_stream_name: ${log_stream_name}
Here we are using the ${region} and ${log_stream_name} variables to replace the values in the template file. We can use the for_each construct to iterate over a list of values:
output "multi_conf_ids" {
value = {
for k, v in edgedelta_config.multi_confs : k => v.id
}
description = "The config ID created for the edgedelta_config.new_config instance"
}
variable "config_instances" {
type = map(map(string))
default = {
"instance_1" = {
"region" = "us-west-1",
"log_stream_name" = "test-stream-1"
},
"instance_2" = {
"region" = "us-west-1",
"log_stream_name" = "test-stream-2"
},
"instance_3" = {
"region" = "us-west-2",
"log_stream_name" = "test-stream-3"
},
"instance_4" = {
"region" = "us-west-2",
"log_stream_name" = "test-stream-4"
},
}
}
resource "edgedelta_config" "multi_confs" {
for_each = var.config_instances
config_content = templatefile("${path.module}/config.yml.tpl", {
region = each.value["region"]
log_stream_name = each.value["log_stream_name"]
})
}
The for_each construct will iterate over the values in the config_instances variable and replace the ${region} and ${log_stream_name} variables with the values from the current iteration. The templatefile function will use the template file to create the configuration content for each iteration.
To try this out, we can run terraform plan:
terraform plan
This will list the four new agent configurations that will be created:
Terraform will perform the following actions:
# edgedelta_config.multi_confs["instance_1"] will be created
+ resource "edgedelta_config" "multi_confs" {
+ config_content = <<-EOT
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
region: us-west-1
log_group_name: /ecs/microservice
log_stream_name: test-stream-1
EOT
+ id = (known after apply)
+ tag = (known after apply)
}
# edgedelta_config.multi_confs["instance_2"] will be created
+ resource "edgedelta_config" "multi_confs" {
+ config_content = <<-EOT
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
region: us-west-1
log_group_name: /ecs/microservice
log_stream_name: test-stream-2
EOT
+ id = (known after apply)
+ tag = (known after apply)
}
# edgedelta_config.multi_confs["instance_3"] will be created
+ resource "edgedelta_config" "multi_confs" {
+ config_content = <<-EOT
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
region: us-west-2
log_group_name: /ecs/microservice
log_stream_name: test-stream-3
EOT
+ id = (known after apply)
+ tag = (known after apply)
}
# edgedelta_config.multi_confs["instance_4"] will be created
+ resource "edgedelta_config" "multi_confs" {
+ config_content = <<-EOT
version: v2
outputs:
streams:
- name: cloudwatch-test-integration
type: cloudwatch
region: us-west-2
log_group_name: /ecs/microservice
log_stream_name: test-stream-4
EOT
+ id = (known after apply)
+ tag = (known after apply)
}
Plan: 4 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ multi_conf_ids = {
+ instance_1 = (known after apply)
+ instance_2 = (known after apply)
+ instance_3 = (known after apply)
+ instance_4 = (known after apply)
}
Now we can run terraform apply to create the new agent configurations:
terraform apply
When the terraform apply command is run, the provider will create the new agent configurations:
edgedelta_config.multi_confs["instance_2"]: Creating...
edgedelta_config.multi_confs["instance_1"]: Creating...
edgedelta_config.multi_confs["instance_3"]: Creating...
edgedelta_config.multi_confs["instance_4"]: Creating...
edgedelta_config.multi_confs["instance_3"]: Creation complete after 1s [id=333333-4444-5555-6666-777777777777]
edgedelta_config.multi_confs["instance_4"]: Creation complete after 1s [id=444444-5555-6666-7777-888888888888]
edgedelta_config.multi_confs["instance_2"]: Creation complete after 2s [id=222222-3333-4444-5555-666666666666]
edgedelta_config.multi_confs["instance_1"]: Creation complete after 3s [id=111111-2222-3333-4444-555555555555]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
multi_conf_ids = {
"instance_1" = "111111-2222-3333-4444-555555555555"
"instance_2" = "222222-3333-4444-5555-666666666666"
"instance_3" = "333333-4444-5555-6666-777777777777"
"instance_4" = "444444-5555-6666-7777-888888888888"
}
Similar to the previous example, we can use terraform destroy to delete the agent configurations:
terraform destroy
As the previous example, the provider will delete the agent configurations:
edgedelta_config.multi_confs["instance_1"]: Destroying... [id=111111-2222-3333-4444-555555555555]
edgedelta_config.multi_confs["instance_2"]: Destroying... [id=222222-3333-4444-5555-666666666666]
edgedelta_config.multi_confs["instance_4"]: Destroying... [id=444444-5555-6666-7777-888888888888]
edgedelta_config.multi_confs["instance_3"]: Destroying... [id=333333-4444-5555-6666-777777777777]
edgedelta_config.multi_confs["instance_1"]: Destruction complete after 0s
edgedelta_config.multi_confs["instance_4"]: Destruction complete after 0s
edgedelta_config.multi_confs["instance_3"]: Destruction complete after 0s
edgedelta_config.multi_confs["instance_2"]: Destruction complete after 0s
Destroy complete! Resources: 4 destroyed.
Importing Existing Resources
If you already have agent configuration or monitor resources on the Edge Delta side, you can import it into Terraform using the terraform import command. This will import the agent configuration/monitor into the Terraform state file. After importing to the state file, you can use the terraform show command to see the existing resource in HCL format, which can be used to add the resource to your Terraform configuration.
Let's assume we have a monitor with ID 11111111-2222-3333-4444-555555555555 and payload same as payload.json from the example in [creating a monitor resource](#creating-a-monitor-resource) section above, on the Edge Delta side. We can import it into Terraform, to a monitor resource named imported_monitor in our Terraform config using the terraform import command. Let the following be the resource definition we will use to import the monitor:
resource "edgedelta_monitor" "imported_monitor" {
}
Simply run the following command to import the monitor:
terraform import edgedelta_monitor.imported_monitor 11111111-2222-3333-4444-555555555555
Which outputs the following:
edgedelta_monitor.imported_monitor: Importing from ID "11111111-2222-3333-4444-555555555555"...
edgedelta_monitor.imported_monitor: Import prepared!
Prepared edgedelta_monitor for import
edgedelta_monitor.imported_monitor: Refreshing state... [id=11111111-2222-3333-4444-555555555555]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
Now we can use the terraform show command to see the imported monitor in HCL format:
terraform show
The output of terraform show is shown below:
# edgedelta_monitor.imported_monitor:
resource "edgedelta_monitor" "imported_monitor" {
creator = "mail@email.domain"
enabled = true
id = "11111111-2222-3333-4444-555555555555"
monitor_id = "11111111-2222-3333-4444-555555555555"
name = "pattern-check-monitor-remote"
payload = jsonencode(
{
mail_recipients = "test1@example.org"
merge_level = "none"
suppression_window = "12h"
timezone = "Europe/London"
trigger_template = ""
triggers = [
"slack-trigger",
]
}
)
type = "pattern-check"
}
We can directly add this output to our Terraform configuration file. Now that we have imported the monitor, we can use the terraform apply command to synchronize the monitor with the remote one. After that, we can do changes to the resource and run terraform apply, or terraform destroy to delete the resource.
Importing Multiple Resources
Edge Delta Terraform provider also supports importing multiple resources at once. To do this, we can use the same importing command by specifying comma-separated resource IDs. The example command below will import the agent configurations with IDs 11111111, 22222222 and 33333333:
terraform import edgedelta_config.imported_config 11111111,22222222,33333333
Which outputs the following:
edgedelta_config.imported_config: Import prepared!
Prepared edgedelta_config for import
Prepared edgedelta_config for import
Prepared edgedelta_config for import
edgedelta_config.imported_config: Refreshing state... [id=11111111]
edgedelta_config.imported_config-1: Refreshing state... [id=22222222]
edgedelta_config.imported_config-2: Refreshing state... [id=33333333]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
Importing All Resources of a Resource Type
In another use case, we can import all the specific type of resources by using "*" as the resource ID. The example command below will import all the monitors we have remotely on the Edge Delta side:
terraform import edgedelta_monitor.imported_monitor "*"
As in the previous example, remote resources (monitors) will be imported to the state file with names imported_monitor-1, imported_monitor-2, etc.
Conclusion
Now that you’ve seen how the Edge Delta Terraform provider works, you’re ready to start using it. We hope that by automating configurations at scale, you can more gain visibility into every data source in a frictionless manner.
Related Posts
Stay in Touch
