This article is a collection of best practices and inspirations we have collected from practioners working with collie.
We're happy to hear from you about your experience - just open an issue on GitHub.
In addition to the conventions described above, the following practices have proven useful for managing complex landing zones construction projects.
Follow Terraform Conventions
HashiCorp publishes module developement guidelines in the official terraform documentation. These conventions enjoy widespread recognition in the terraform community and simplify collaboration.
Additionally, platform teams should consider following a consistent naming convention.
Providing useful documentation for application teams is an essential part of making any platform engineering effort succesful. This is why your kit modules should generate documentation output intended to be read by application teams and other stakeholders like security auditors.
We recommend generating the required
documentation_md output variable in your module with a "heredoc" string. Similarly to the standard terraform module structure splitting configuration into
outputs.tf files, we recommend putting your documentation in a separate
documentation.tf file. This enables you to quickly find and edit how your kit module generates documentation.
Terraform makes a distinction between "root" modules that come with all required
backend configurations and reusable modules that are intended to be called from other modules.
Even though kit modules will be invoked 1:1 from platform modules via terragrunt, platform teams should nonetheless design kit modules as reusable modules. Designing kit modules as reusable modules offers many advantages
- staging: most platform teams want to deploy landing zones to a staging environment to test integration before rolling out core infrastructure changes to production
- separation of concerns:
terragruntis specifically built to overcome terraform limitations and keep configurations DRY. Separating the concerns of defining resources in kit modules and orchestrating terraform executions in platform modules leverages the tools where they are strongest
- community: reusable modules can be shared more easily with the community - and also more easily adopted to jumpstart your own landing zones
Structuring kit modules
Enabling core infrastructure like virtual networks and audit log collection requires orchestrating resources across different cloud tenants. While it's useful to structure kit modules closely resembling the virtual "anatomy" of the cloud resource hierarchy, most platform teams will find it useful to model higher-level capabilities as well, leveraging patterns like dependency inversion and
The following patterns have proven useful in our experience
- structure kit modules according to capabilities they add to your landing zones
- kit modules should follow encapsulation, privileges and volatility boundaries, see Module Creation - Recommended Pattern
- consider blast radius of kit modules - smaller modules are easier to develop and iterate on due to smaller dependency graph
- leverage terragrunt
dependencyto model dependencies between kit modules in favor of
Terragrunt offers many advanced features to keep configuration DRY. However, sometimes a little more repetition is the lesser evil. Terragrunt configurations can get "too clever" quickly, especially when they compose from too many different dynamic configuration sources. We therefore recommend keeping terragrunt trickery to a minimum and stick to basic features like
dependency for as long as possible.
Managing .terraform.lock.hcl files
.terraform.lock.hcl files to lock dependency versions of providers and modules. If members of your cloud foundation team run a variety of operating systems and CPU architectures managing these locks in a way that members of your team won't see depency locking issues can be very cumbersome.
One important trick is to make sure that lock files contain entries for all OS and cpu architectures used in your team. This can be accomplished by running
terraform providers lock across all your platform modules.
collie foundation deploy my-foundation -- providers lock -platform=darwin_amd64 -platform=linux_amd64 -platform=darwin_arm64
-platform commands as appropriate. Don't forget committing the resulting lock file updates back to your repository to share with your team.