A feature flag is useful when it gives a team a reversible control over exposure. It is dangerous when it becomes an undocumented second configuration system, an authorization shortcut, or a permanent branch nobody owns. This tutorial lays out a safe rollout for a SaaS feature: define the decision, deploy dormant code, target a stable audience, watch the right signals, and remove the flag after the release has settled.
Start with a release decision, not a boolean
Before creating a flag, write the decision it controls: which users receive the new behavior, what the old behavior is, who owns the decision, and what evidence allows the rollout to advance or stop. A switch, a gradual rollout, and an experiment are different operating modes. Microsoft’s feature-management guidance describes these as distinct scenarios; choosing one prevents a simple kill switch from quietly turning into an uncontrolled experiment.
Give the flag a lifecycle before it reaches production. Record its owner, intended removal date, default value, audience rules, dependencies, and the metric that would trigger a rollback. Do not use a flag to conceal a permission decision. If access is contractual or security-sensitive, enforce it with authorization code and policy; a rollout flag may control exposure, but it must not grant access by itself.
- Name the user problem and behavior the flag changes
- Choose switch, rollout, or experiment explicitly
- Define the old-path fallback and the rollback owner
- Set a review or removal date before merging the flag
- Keep entitlements and security decisions outside the exposure flag
Deploy dormant code before exposing it
Keep deployment and exposure as separate actions. First ship code that can run with the flag disabled, alongside any backward-compatible schema or API changes. Verify that the old path still works. Only then change the flag for a controlled audience. This makes a code rollback and an exposure rollback different, deliberate operations rather than one risky release button.
Use a stable contract between the two paths. Both branches should accept the same validated input and produce a response the client understands, or the migration should include compatibility handling at the boundary. If a flag changes a database shape, deploy the additive migration first, support both representations during the transition, and postpone destructive cleanup until no old path can run. Kubernetes Deployments provide controlled replacement and revision rollback for the application artifact, but that does not roll back a separately stored flag value.
- Make the disabled path the safe default for new deployments
- Ship additive database and API changes before switching behavior
- Test old and new paths against the same fixtures
- Keep application rollback and flag rollback independently documented
- Never assume a container revision restores external configuration
Evaluate against a deliberate context
A rollout needs a consistent answer for the same user or tenant. Use an authenticated, stable targeting key and pass only the attributes required for the rule: for example tenant tier, region, or an internal beta marker. OpenFeature describes evaluation context as data used for dynamic evaluation and supports context at global, client, and invocation levels. That is a useful model, but the values still need an ownership and privacy review.
Never let a browser-supplied header or query parameter decide a privileged audience. Derive tenant and user identity from verified authentication, validate attribute types, and define what happens when the flag service is unavailable or the context is incomplete. OpenFeature specifically calls out care with personal data in evaluation context because the provider may handle or persist it; prefer a pseudonymous key and coarse attributes over sending a profile to every evaluator.
- Use a stable targeting key so users do not flip between variants
- Derive identity from the authenticated request context
- Send the minimum attributes needed for targeting
- Document missing-context and flag-provider failure behavior
- Test tenant isolation and do not use flags as authorization
Roll out in observable stages
Start with internal users or a small, explicitly identified cohort. Check that the new path renders, writes correct data, emits expected telemetry, and can be disabled without a redeploy. Then increase exposure in stages only after reviewing error rate, latency, key business outcomes, support reports, and resource consumption. A percentage is not automatically a representative sample: a ten-percent rollout can be unsafe if it concentrates on one large tenant or one unusual workflow.
Keep the measurement window long enough to include the important behavior. A checkout change may need completed orders and support contacts, while an internal dashboard can be evaluated through a shorter operating cycle. Compare the flagged cohort with the control path only when the comparison is meaningful, and record the flag version and evaluation result in telemetry without logging personal data or raw targeting attributes.
- Begin with a cohort you can identify and support
- Increase exposure only after reviewing technical and product signals
- Measure by tenant, route, client version, and relevant workflow
- Track flag state and variant in logs or metrics with privacy-safe labels
- Define a stop condition before each exposure increase
Design failure and rollback paths
A flag service is another dependency. Decide whether a cached value, a local default, or a fail-closed response is appropriate for each feature. A marketing layout may safely fall back to the old experience; a payment or data-migration path may need to stop rather than guess. Put a bounded timeout around remote evaluation and make the default explicit in code and configuration.
Test the rollback as a real operation. Turn the flag off while traffic is active, confirm new requests take the old path, and inspect in-flight work, queued jobs, caches, and data written by the new path. If the new behavior has already changed durable data, switching it off may stop further exposure without undoing the data. Document reconciliation or forward-fix steps instead of promising that every flag is an instant undo button.
- Choose fail-open or fail-closed per feature risk
- Bound flag-evaluation latency and cache stale values deliberately
- Make the kill switch reachable to the on-call owner
- Verify rollback through the same proxy and deployment path as production
- Separate exposure rollback from data repair and code rollback
Remove the flag and verify the final state
A successful rollout ends with less branching, not another permanent setting. Once the new path is the intended behavior, remove the old branch, its tests, targeting rules, dashboards, documentation, and configuration entry. Delete the flag from the provider or registry after the application no longer reads it. Leaving retired flags behind makes future evaluations harder to understand and increases the chance that stale rules affect a later release.
Before cleanup, search the codebase for every flag key and inspect the evaluation call sites. Confirm that all supported client versions, workers, scheduled jobs, and caches use the intended behavior. Keep the rollout evidence and decision record, but do not retain unnecessary user attributes or raw evaluation payloads. The flag’s audit trail should explain what changed without becoming a second store of personal data.
- Set an owner and removal date for every non-permanent flag
- Search code, jobs, clients, dashboards, and configuration before cleanup
- Remove the old branch and delete unused targeting rules
- Verify all supported versions and workers after the final change
- Keep a concise decision record and privacy-safe rollout evidence
Production checklist and failure modes
Feature flags reduce release coupling, but they add runtime state and another way for behavior to vary. They do not replace tests, staged deployment, authorization, observability, or a migration plan. The practical standard is simple: an operator can identify who is exposed, see whether the change is healthy, turn it off safely, and know when the temporary branch will be removed.
Run the rollout in a staging environment with the same evaluation context, cache behavior, and provider failure modes as production. Then exercise the uncomfortable cases: missing flag data, a changed targeting attribute, a tenant crossing cohorts, a provider timeout, a code rollback with a newer flag still stored, and a worker processing data created by the new path. If the expected result is not written down, the flag is not ready to control customer traffic.
- Flag purpose, owner, default, audience, metric, and removal date are documented
- Disabled behavior works on a fresh deployment
- Targeting is stable, authenticated, tenant-safe, and privacy-reviewed
- Flag-provider timeout and unavailable states have tested defaults
- Exposure changes are audited and reversible where the feature permits
- Technical, product, support, and resource signals are monitored by cohort
- Rollback tests cover in-flight work, queues, caches, and durable data
- Expired flags and old code paths are removed promptly
- The runbook explains who may change the flag and what evidence they need