Building an Apstra EVPN-VXLAN fabric with Terraform: lessons from production

I co-developed the Terraform provider for Juniper Apstra while at Juniper, and later used it to build a production EVPN-VXLAN fabric for a multi-gigawatt AI data center. These are the lessons that mattered most once the code met a real data hall.

Split design from deployment

Apstra separates the design catalog (logical devices, interface maps, rack types and templates) from blueprints, which are deployed instances of a design. Mirror that split in your Terraform code: one module or state for the catalog, and one per blueprint. Catalog objects change rarely and are shared; blueprint objects change constantly. Keeping them apart keeps plans small and reviews readable.

Treat resource pools as their own layer

ASN pools, IPv4 pools and VNI pools are shared between blueprints. Define them once, in their own state, and reference them from each blueprint. When two blueprints accidentally define overlapping pools, the problem shows up at deployment time, not plan time.

Make the commit explicit

Changes to a blueprint in Apstra are staged until they are committed. The provider models that commit as its own resource, so a plan can stage many changes and deploy them together:

resource "apstra_datacenter_routing_zone" "prod" {
  blueprint_id = apstra_datacenter_blueprint.dc1.id
  name         = "prod"
}

resource "apstra_blueprint_deployment" "dc1" {
  blueprint_id = apstra_datacenter_blueprint.dc1.id
  comment      = "Deployed by Terraform"
  depends_on   = [apstra_datacenter_routing_zone.prod]
}

This is a simplified fragment. In practice, the deployment resource depends on every blueprint resource, so nothing is committed half-built.

Decide who owns what, and stick to it

The fastest way to break an automated fabric is a well-meaning change in the GUI. The next terraform plan either tries to undo it or fails in a confusing way. Agree up front which objects belong to Terraform and which (if any) operators may change by hand, and make that visible in the code and the runbook.

Pin provider and Apstra versions together

Apstra releases change the API, and provider releases track them. Pin the provider version, record which Apstra versions it was tested against, and upgrade both deliberately in a lab before production.

Validate in a lab first

Every rack type, template and blueprint pattern was built in a lab blueprint before it went near production. With the whole fabric defined as code, standing up the lab copy is a variable change, and the lab finds the problems while they’re still cheap.

The payoff

Once the fabric is code, adding a rack or a virtual network is a reviewed pull request, not a maintenance window spent clicking through a GUI. Configuration is consistent across every device because no one typed it by hand.