AWS organisation as code

Intent

This pattern is a specialised implementation of data-driven Terraform. It realises data-driven Terraform's core aim within AWS account management: making changes quicker and simpler by expressing them as data. It assumes an AWS Organization with several accounts, or one about to be created, managed with Terraform. It applies data-driven Terraform to whatever sits above the systems running in those accounts, such as:

• The accounts themselves.
• The organisational units that group them.
• The guardrails and other policies set on them.
• The single sign-on access people use to reach them.
• The trust that lets pipelines deploy into them.
• The baseline each account starts with.

Each becomes a primitive, as data-driven Terraform calls each kind of resource it manages, with its own collection of entries in the data tier, so every change to the organisation is a change to data. This enables three outcomes:

1. Accounts, guardrails and access are provisioned, changed and audited entirely by creating, editing and reviewing data, so outcomes such as least-privilege access are easier to set up and to prove.
2. Administrators make changes faster, such as creating an account or onboarding a person, and with far less to hold in mind, because each change is an entry checked against a documented interface, rather than configuration traced through consoles.
3. New administrators can work with the organisation's code substantially sooner, because each primitive is a documented data structure they can read in one place, rather than configuration to piece together from consoles.

Motivation

When an AWS Organization is set up by hand in the console, by whoever holds the management account's credentials, it fails in three ways as the organisation grows:

1. Nobody can say how it's set up. Which accounts exist, which guardrails apply to them and who can reach them is spread across consoles, and the organisation itself is the only record.
2. Every change is slow and complex. Creating an account or granting access means working through several consoles by hand and remembering how each was set up, so even routine changes take time, and every new administrator has to be walked through all of it.
3. Changes go unreviewed. A guardrail removed or access granted in the console takes effect at once, with no review and no record of why.

This pattern treats the organisation as one more set of primitives for data-driven Terraform. It uses data-driven Terraform's colocated variant, keeping each primitive's module in the same repository as its data, because one organisation has nothing to version or share with others. Each primitive has its own documented collection of entries, and the primitives follow what the organisation needs, so it can start with accounts and access, and add more as its governance grows. Typical primitives are:

• Accounts, each an entry giving its name, the email address AWS requires for it, and the organisational unit it belongs to, so a new account is one new entry.
• Organisational units, which group the accounts, such as by environment, so policies can be attached to a whole group of accounts at once.
• Organisation policies, such as service control policies, the guardrails that set the most any identity in an account can do, whatever its own permissions say, or tag and backup policies. They attach to organisational units, so every account placed in a unit is covered, including accounts created later.
• Single sign-on access, such as through IAM Identity Centre, as groups of people, permission sets that define what a group can do, and assignments that give a group a permission set in named accounts. People only ever get access through a group, never in their own name, and narrowing what a group can do is a reviewed edit to one permission set's entry, so access can be tightened towards least privilege over time, and shown to an auditor straight from the data.
• Pipeline trust, which lets pipelines deploy into the accounts without long-lived keys, such as the OIDC provider and hub role in hub-and-spoke OIDC across AWS accounts.
• Account baselines, the resources every account starts with, such as the role its pipelines deploy through and the storage that holds its infrastructure state.

Administrators remain the people who change the organisation, but the approach changes how they work. A change is an entry against a documented interface, so it's quicker to make and simpler to reason about, with nothing to trace through consoles. Onboarding a person is one entry adding them to their role's groups, and a new administrator learns how the organisation is set up by reading its data.

Assignments and policies name accounts rather than their IDs, and look the IDs up from wherever the accounts are created, so adding an account never means editing an ID into a policy.

Pipelines check and plan every pull request and apply on merge. They also apply again on a schedule, putting back any change made outside the code. That does two things:

1. It combats configuration drift, where the organisation gradually stops matching its code as people change settings by hand in the console.
2. It shortens how long an unauthorised change stays in place. A guardrail removed or access granted by a malicious actor lasts only until the next scheduled apply, rather than until someone notices it.

Where the organisation runs AWS Control Tower, AWS's managed set-up for a multi-account organisation, this pattern works alongside it rather than around it. Control Tower owns what it manages, and reports any direct change to it as drift, so the primitives it covers are expressed through Control Tower instead:

• Accounts are created through its Account Factory, so each account's entry becomes an account request, as in Account Factory for Terraform, and account baselines become the factory's account customisations.
• Guardrails include Control Tower's own controls, enabled on organisational units through its interfaces, such as the Terraform resource for them, alongside any policies of the organisation's own.
• Single sign-on access stays a primitive of its own where Identity Centre is self-managed, rather than left to Control Tower.

The two variants below differ in how the primitives are divided between configurations, with or without Control Tower.

Applicability

Use this pattern when:

1. You run an AWS Organization with several accounts, or are about to create one.
2. Accounts, guardrails and access should change by pull request, with every change reviewed and recorded.
3. Administrators should be able to make routine changes, such as adding an account or onboarding a person, quickly and without tracing configuration through consoles.
4. An organisation already exists, set up by hand, and needs bringing under code.

Do not use this pattern when:

1. There's a single AWS account, with no organisation to manage.

Structure

Variant A: One configuration

When the organisation has a handful of accounts, and one small team administers all of it, this variant works best.

1. Every primitive lives in one configuration, with one state, applied from the management account, the account that runs the organisation itself.
2. A single review covers any change to the organisation, whether to its structure, its policies or its access.

Variant B: Split by concern

When the organisation grows, or different people own its structure, its access and its account baselines, this variant works best.

1. The organisation's structure and policies, its single sign-on access, and each account's baseline live in configurations of their own, each with its own state.
2. Each configuration looks up what it needs from the others, such as account IDs, so each can change on its own.
3. Concerns that AWS lets the organisation delegate, such as policy management or Identity Centre, can be applied from a delegated administrator account, keeping use of the management account to a minimum.

Participants

Constraints and trade-offs

1. Every constraint of data-driven Terraform still applies, from strict schema discipline to mature state management.
2. The blast radius is wide. The organisation's code can change every account's guardrails and access, so only a few people should be able to approve changes to it, and each change deserves the review of a change to production.
3. The first apply has to run outside the pipeline, because the pipeline can't sign in to AWS until the code has created the trust it signs in with.
4. With AWS Control Tower, the boundary between what it manages and what the data tier manages has to be respected. An account created or moved outside Control Tower drifts from its landing zone, so everything Control Tower owns changes only through Control Tower.

Variant A: One configuration trade-offs

1. The core benefit is simplicity: one configuration to apply, and one place to read how the organisation is set up.
2. The core cost is that every plan reads the whole organisation, so plans slow down as it grows, and a change to one person's access is applied alongside the organisation's guardrails.

Variant B: Split by concern trade-offs

1. The core benefit is smaller plans, each reviewed by the people who own that concern, with less of the work applied from the management account.
2. The core cost is order: a new account has to exist before its access or baseline can be applied, so a change that spans concerns takes more than one apply.

Part of how I deliver

This is how I give an AWS Organization the same review, and the same speed of change, as the rest of your infrastructure. It's one of many patterns I bring through contract delivery, and it can bring an organisation set up by hand under code without rebuilding it.

Related