Automatic client provisioning

Intent

This pattern is a specialised implementation of dynamic Terraform on demand, which is itself a child of data-driven Terraform. It assumes a multi-tenant SaaS platform that stores each client in a mutable record, as in config as data. It fully automates client onboarding: when an operator adds a client, provisioning runs by itself, shifting the work left from DevOps and platform teams to the people who operate the platform. This enables three outcomes:

1. A new client can be onboarded as quickly as the platform can provision it, with no code written.
2. Operators onboard clients themselves, rather than raising a request with a DevOps or platform team.
3. Every client is set up the same way, and a provisioning job changes only the client it was started for.

Motivation

On most multi-tenant platforms, onboarding a client is an engineering task: an operator raises a request, and a DevOps or platform engineer adds the client to the code and deploys it. That fails in three ways:

1. The engineering backlog sets the pace. A client who has signed can't start until an engineer is free, however simple their setup.
2. Specialists become a queue. Every new client takes a DevOps or platform engineer away from improving the platform itself.
3. Manual setup varies. Clients set up by hand, repo by repo, end up different from each other, and nobody can say for certain whether one is complete.

This pattern applies the shift Team Topologies describes, where platform teams provide services that others use without waiting on them. Engineers build the provisioning once, and operators use it, so adding a client becomes routine, business-as-usual work. Terraform then derives what that client needs from the record, one client at a time and from released code only. A failed run goes to a person, and can safely be sent again.

Applicability

Use this pattern when:

1. You onboard clients often, and each needs its own infrastructure.
2. The people who operate the platform, rather than engineers, should be able to bring a client on.
3. Client details live in a mutable record, outside the repos that manage the Terraform, as in dynamic Terraform on demand.

Do not use this pattern when:

1. Clients are few and rarely change, where editing the data in the repo and deploying is simpler.
2. Each client's infrastructure is costly or risky enough to need a person to review its plan.

Structure

Participants

Constraints and trade-offs

1. Reviewing the code no longer tells you everything, because what a deploy builds depends on data that changes without a pull request. Operators' changes need validation of their own, as in config as data, and a record of who changed what, as in immutable audit trail.
2. Applying one client at a time relies on targeted applies, which HashiCorp advises against using routinely, so full deploys still need to bring every client back in line at each release.
3. Runs happen one at a time, so that applies can't conflict. A large batch of new clients takes as long as all of its runs added together.
4. Engineers still own the machinery: one queue, mutex and orchestrator for the whole platform, and a provisioning mode in each repo's deploy workflow. When a run fails, an engineer still has to find the cause and fix it before the run is sent again.

Part of how I deliver

This is how I take client onboarding off the engineering backlog, and hand it to the people who operate your platform. It's one of many patterns I bring through contract delivery, and it builds on the Terraform you already run.

Related