The Red Handle Nobody Pulls: Making Feature Flag Kill Switches Inspection-Ready
Photo: Trevor Butcher from Poland, CC BY 2.0, via Wikimedia Commons
Every office has a fire door that nobody thinks about until the alarm goes off. It is painted, signed, propped open on hot days and occasionally blocked by a stack of printer paper. Feature flags are the software equivalent. Most of the time they are a convenience: a way to ship early, test with a subset of users or turn off a noisy integration without a deploy. The moment you need one as a kill switch, however, it has to behave like safety equipment. It must be easy to find, clearly owned, tested and never blocked by furniture.
A kill switch is not simply a boolean called enableCheckout. It is a promise that a named person can stop a specific piece of behaviour within a known time, from a known place, with a known outcome. If that sentence feels difficult to write for one of your flags, the flag is not ready.
A flag is not a cupboard
The first mistake is treating flags as storage. Teams create them for every branch, every experiment and every awkward dependency, then leave them in the config like old paint tins in a cupboard. The kill switch gets lost among them. A useful rule is to separate flags by purpose. Release flags hide unfinished work. Experiment flags route a percentage of traffic. Operational flags protect the system. Kill switches belong in that last group, and they deserve a different level of care.
Give each kill switch a short name that says what it stops. Not checkout_v2 but stop-card-payments. Not new-router but disable-address-lookup. The name should make sense to someone who is awake at 3am and has never seen the codebase. Put it in the same place as your other operational controls, not in a developer's local notes. If your flag provider has tags, use one called kill-switch. If it does not, make a register. A spreadsheet is not glamorous, but neither is a fire extinguisher.
Name the owner and the expiry date
Every kill switch needs two names: the person who owns it and the date it should be reviewed. The owner is not the team in general. It is a rota or a named engineer with backup. They are responsible for checking that the switch still does what it says, that the flag is documented and that it gets removed when the feature is stable.
The expiry date is not a promise to delete the switch on that day. It is a prompt. Put it in the same calendar as dependency upgrades and certificate renewals. When it comes up, ask three questions. Is the guarded behaviour still risky? Is the switch still tested? Has the feature become so ordinary that the switch is now the risk? A kill switch that nobody has reviewed for a year is not a control. It is a rumour.
Take the furniture away
A kill switch fails most often because something else blocks the path. The flag is off, but a cache still serves the old response. The payment provider is disabled, but a queue keeps retrying. The feature is hidden, but the mobile app has already downloaded a config that ignores the flag. These are the equivalent of stacking chairs in front of a fire door.
Map the dependencies around each switch. Write down what else must be true for the kill switch to take effect. Does it need a cache purge? A deploy? A message to the support team? A database migration that cannot be reversed? If the answer involves more than one step, rehearse it. If it involves a step that only one person knows, fix that before the next release.
Pull the handle in a test
A kill switch that has never been flipped in a test is a hypothesis. Add a test that turns the flag off and asserts the safe behaviour. Keep it small: the checkout button disappears, the background job stops, the third-party call is skipped. The test does not need to prove the whole system is perfect. It needs to prove the switch is connected to the thing you think it is connected to.
Then go further. Run a game day in staging where someone flips the switch without telling the rest of the team. Time how long it takes for the behaviour to stop. Watch what the dashboards do. Check whether the support team can see the change. If the exercise causes confusion, that is useful information. Better to find it on a Tuesday afternoon than during a Saturday incident.
Rehearse the rollback
Rollback is not only about deploying the previous version. For a kill switch, rollback means restoring the guarded behaviour safely, or deciding that the guarded behaviour should stay off. Both directions need a plan. Who decides to turn it back on? What evidence do they need? What happens to the data created while the switch was off? A switch that can stop a feature but cannot restart it cleanly is only half a control.
Write the runbook in plain language. Include the flag name, the expected effect, the dashboards to watch and the person to call. Keep it next to the incident process, not buried in a wiki. A new starter should be able to follow it with a cup of tea and a quiet moment.
A pre-release inspection sheet
Before a release that adds or changes a kill switch, run this short inspection:
- The switch has a name that describes the behaviour it stops.
- A named owner and backup are listed.
- The expiry or review date is in the team calendar.
- The switch is tagged or registered as a kill switch.
- A test flips the switch and checks the safe outcome.
- The runbook lists dependencies such as caches, queues and mobile config.
- Someone has rehearsed the switch in staging this quarter.
- Support and incident channels know the switch exists.
- The flag will be deleted, not just disabled, when it is no longer needed.
None of this is glamorous. It is the software equivalent of checking that the fire door closes, the sign is visible and the corridor is clear. Do it regularly and the red handle stays trustworthy. Skip it and you will eventually find yourself in a corridor full of printer paper, wondering why the door will not open.