Pre-Mortem Drills for Team Decisions Built on AI Output

Most AI-related team mistakes do not come from one dramatic failure. They come from a sequence of reasonable-looking decisions, each leaning on an output that nobody stopped to question. A summary gets pasted into a planning doc. A draft reply gets approved because it reads well. A recommendation from a tool becomes the default because it arrived first. By the time someone notices the assumption underneath, the team has already built on it. This article describes a pre-mortem drill that technical founders and operators can run before acting on AI-assisted output, focusing on the questions that expose weak reasoning early and the records that make the next decision easier to review.
Why a pre-mortem fits AI-assisted decisions
A pre-mortem asks a team to imagine that a decision has already failed, then work backward to explain why. The technique is useful for AI-assisted work because the failure modes are often quiet. An output can be fluent, confident, and wrong in a way that no single reviewer catches on a quick read. Asking the group to assume failure shifts attention from whether the output sounds right to where it could break.
Run the drill before the team commits, not after. Once a recommendation has been implemented, people defend it. Before implementation, the cost of questioning it is low. A useful framing is to ask: if this turns out to be a mistake six weeks from now, what is the most likely reason? Write down the answers without ranking them yet. Surprising answers are often the most valuable.
Setting up the drill in under thirty minutes
Keep the group small, ideally three to six people who touch the decision in different ways: the person who requested the work, the person who will act on it, and someone close to the systems the output will affect. Choose one decision per session. Bundling several decisions produces shallow review.
Start by stating the decision in one sentence and attaching the output that informed it. Then ask each participant to spend five minutes writing failure stories independently before discussing. Independent writing prevents the most confident voice from framing everyone else's thinking. Collect the stories and group them by type: the output misread the input, the output was right for a different situation, the output was right but the action taken from it was wrong, or the output omitted a relevant case.
Finish by assigning a named owner for the decision and a date for checking the outcome. A decision without an owner tends to drift, and an output without a review date tends to become permanent.
Questions that expose weak AI output
Several questions consistently surface problems in AI-assisted outputs. Ask what the output assumes that nobody on the team has verified. Ask which specific fact, number, or claim would have to be wrong for the recommendation to flip. Ask whether the output would look equally plausible if the opposite conclusion were true. If it would, the output may be confident in tone without being grounded in the evidence.
Ask also whether the output has been compared against anything. A single model's answer is one data point. If the decision matters, check whether a different model, a different prompt framing, or a human with domain knowledge reaches a similar conclusion. Disagreement among independent sources is information. It does not automatically mean the team should avoid the decision, but it does mean the disagreement deserves a named reviewer rather than a silent choice of whichever answer looked cleanest.
Finally, ask what the team would lose if the output were wrong and nobody noticed for a month. This question shifts attention from how good the answer looks to how expensive an undetected error would be, which is usually what determines how much review the decision deserves.
A practical checklist for the decision record
Capture the outcome of each drill in a short decision record that any teammate can read later. Include the decision in one sentence, the AI-assisted output that informed it, the failure stories the group generated, the owner, the review date, and the single observation that would make the team reverse course. Keep the record in the same place the team already works rather than in a new tool nobody checks.
Before closing the record, confirm five things: the decision owner is named, the review date is on a calendar someone reads, the source output is linked or quoted in full so it can be reexamined, at least one failure story has a specific mitigation attached, and the reversal trigger is concrete enough that someone could notice it happening. A record missing any of these is still useful, but it will be harder to learn from.
Revisit the records at the review date. Note which failure stories turned out to matter, which did not, and whether any failure happened that nobody had imagined. Over several cycles, these notes reveal patterns: certain kinds of outputs may consistently need more scrutiny, and certain reviewers may consistently catch problems others miss. That accumulated knowledge is the real product of the drill, and it makes the next pre-mortem sharper.
Keeping the habit from becoming theater
A pre-mortem can degrade into a ritual where people write plausible-sounding risks to satisfy the format and then proceed exactly as planned. Guard against this by asking the group, at the end of each session, whether anyone changed the plan because of the drill. If the answer is never, the drill is not doing its job, and the team should either narrow the decisions it covers or make the reversal triggers more concrete.
Keep sessions short and frequent rather than long and rare. A thirty-minute drill on a real decision teaches more than a three-hour workshop on a hypothetical one. Over time, the habit makes it normal for an operator to ask where an AI-assisted answer could fail before the team builds on it, which is the difference between using AI output as a draft and using it as a foundation.