When the Decision Has Already Begun

A recruiter looks at five candidates on a screen.

She can disagree with the ranking. She can open every profile. She can ignore the system’s preferred candidate and choose the person placed fifth.

Nothing about the interface suggests that her judgement is constrained. And perhaps it is not.

The more interesting question is what happened to the other 495 applicants.

By the time the recruiter begins exercising judgement, applications may already have been parsed, normalised, scored, filtered, clustered or quietly removed from view. Certain experiences may have become legible to the system while others lost weight. A threshold may have determined who deserved further attention. A ranking may have made one group cognitively easy to inspect and another practically invisible.

None of this necessarily determines whom the recruiter will choose. That is exactly the problem.

The final decision can remain genuinely human while much of the environment in which it occurs has already been formed elsewhere.

Most discussions of AI-assisted decision-making begin when the machine produces something visible: a recommendation, a score, a classification, a shortlist. We then ask familiar questions. Did the human understand the output? Could they disagree with it? Was override possible? Who was accountable?

Necessary questions. But sometimes they begin too late.

Before asking who made the decision, there is another question worth reconstructing: How did this situation become a decision in this particular form?

That question points to a different object of analysis. I call it the formation of decidability: the process by which an initially indeterminate situation acquires sufficient structure to become actionable as a decision object.

The definition is deliberately narrow. It does not mean that every upstream transformation is a hidden decision, or that the final outcome has been predetermined. It asks us to observe the formation of the object upon which later choice, oversight and responsibility will operate.

Decisions do not arrive fully formed

Reality enters organisations badly.

A fraud team receives millions of transactions, not conveniently bounded fraud decisions. A hospital receives symptoms, histories, measurements and uncertainty, not completed diagnostic objects. A company does not receive five strategic alternatives. It receives competitors, market movements, regulatory constraints, partial numbers, internal pressure, people who disagree about what the numbers mean, and usually a deadline.

Before somebody can decide, the situation has to acquire form. Something becomes salient. Certain evidence becomes relevant. A boundary appears around the problem. Alternatives become imaginable. An actor acquires authority. Eventually the organisation reaches a point at which uncertainty has not disappeared but action becomes possible anyway: we can decide now.

That point has a history.

Humans have always produced such histories. Managers decide what enters a meeting. Analysts select variables. Institutions build categories. Experts compress complexity for other experts. Bureaucracies make some realities administratively visible and others harder to see.

There was never a neutral world in which people encountered pristine decisions and exercised pure judgement over them. AI does not invent mediation.

What changes is how much of that formative work can be performed upstream, at scale, at speed and across layers that the person carrying final responsibility may never inspect. The system may not decide the answer. It can still participate in deciding what will count as the question, what evidence reaches it, which cases are visible and which alternatives remain cognitively available.

This is why the useful boundary is not simply machine versus human. It is between the formation of the decision field and the exercise of judgement inside it.

Choice architecture is an important neighbouring literature here. Thaler, Sunstein and Balz show that people choose inside environments whose features shape behaviour (Thaler, Sunstein & Balz, 2013). Formation of decidability is not a replacement for that account. The analytical move is earlier and somewhat different: before asking how an environment structures a choice, ask how the situation acquired the form of a choice object at all.

The distinction also prevents a conceptual inflation that would make the term useless. Formation, narrowing and intervention should not be collapsed. First, an indeterminate situation is formed into something an organisation recognises as decidable. Then the field may narrow as alternatives, interpretations or evidential paths become less available. Only then does the familiar question of human intervention arise: what can a person still contest, recover, reverse or redirect?

An AI system may affect one of these processes without controlling the others. That is more precise than saying that AI influences decisions. Almost everything influences decisions. The harder task is locating where influence enters the architecture.

Freedom can survive after the field has narrowed

This is also why the phenomenon cannot be reduced to automation bias.

Automation-bias research examines an important failure mode: people can over-rely on automated advice, reduce independent information seeking or accept outputs they should challenge. Systematic reviews show that these effects are shaped by context, task demands and verification complexity (Goddard, Roudsari & Wyatt, 2012; Lyell & Coiera, 2017). Experimental work also suggests that interaction design can reduce over-reliance by forcing more deliberate engagement, although not without trade-offs (Bucinca, Malaya & Gajos, 2021).

Now imagine the opposite case: an unusually sceptical reviewer.

She distrusts the model. She verifies its output. She frequently rejects its preferred recommendation. She insists on independent judgement. We would probably describe this as strong human oversight, and perhaps it is.

Still, there may be something she cannot challenge.

She can reject what the system recommends. She cannot inspect what the system never allowed to become a recommendation. She can question the ranking. She may not be able to recover evidence that disappeared before the ranking was constructed. She can resist the answer. She cannot challenge what she never sees.

That is a structurally different problem.

It means an organisation can preserve substantial human autonomy at the final decision point while allowing the environment around that autonomy to become progressively more constructed. This can happen without deception, manipulation or any conscious attempt to remove human agency. Often the opposite is intended.

Systems are introduced to reduce cognitive burden. Interfaces are simplified. Evidence is summarised. Irrelevant cases are filtered. Decision-makers are protected from information overload. All sensible. Yet every simplification contains prior judgement about what can safely disappear.

Usually this is harmless. Sometimes it is not.

The governance question therefore cannot be whether filtering or compression exist. Complex institutions cannot function without them. The question is whether consequential transformations remain sufficiently visible and reconstructable when somebody later needs to understand how a decision field took its final shape.

There is another complication. Formal decision authority and formative influence may sit in different parts of the system. The person signing the decision may be a manager. The threshold that generated the case may have been configured by another team. A taxonomy may come from a vendor. The evidence displayed on screen may depend on retrieval logic. The order of alternatives may come from a ranking model. Escalation may be constrained by workflow policy.

Nobody individually made the field. It emerged across layers. Responsibility, however, often lands at the end.

The visible authority can therefore be concentrated while formative influence is distributed. That asymmetry is one reason apparently simple questions such as who decided become surprisingly difficult under scrutiny.

Human oversight has a temporal problem

We often talk about human oversight as if it were something a system either has or does not have. There is a human reviewer. There is an override. There is an escalation route. Therefore humans remain in control.

Sometimes that is true. But control has a temporal dimension.

Suppose a reviewer receives an automated recommendation at 16:57. The operational batch executes at 17:00. Rejecting the recommendation requires opening another system, locating the case, providing a written justification and obtaining a second approval.

The reviewer possesses formal authority. The three-minute window is also real.

In principle, intervention exists. Operationally, its usefulness is close to disappearing.

This is the crude version. The more interesting version occurs upstream. A reviewer may possess significant authority over every case in the queue while having almost no visibility into why those cases entered the queue, what never reached it, or which alternatives became difficult to recover before the review began.

The familiar phrase human in the loop hides this surprisingly well. Which part of the loop? When does the human enter? What remains open at that moment?

Can evidence still be challenged? Can alternatives be recovered? Can escalation alter the trajectory rather than simply document disagreement? Can the person change what happens next, or only record that they would have preferred something else?

Meaningful-human-control scholarship already rejects simple human presence as sufficient and asks for stronger relations between system behaviour, human reasons and traceable responsibility (Santoni de Sio & van den Hoven, 2018). The present argument is adjacent to that work. It asks an upstream question: what state has the decision field reached by the time those control capacities are supposed to operate?

The regulatory architecture points in a similar direction without using this vocabulary. For high-risk AI systems, the EU AI Act places record-keeping, transparency and effective human oversight next to one another as requirements (Regulation (EU) 2024/1689, Arts. 12-14). That combination matters. Oversight depends not only on a person’s nominal presence but on information, timing and the practical ability to interpret, disregard, override or intervene.

NIST’s AI Risk Management Framework likewise treats AI risk as socio-technical and organisational rather than as a property of a model in isolation (Tabassi, 2023). A human role written into a process does not by itself establish that the surrounding system leaves that person in a position to challenge what matters.

A person can therefore be fully present and arrive too late. A person can possess authority while the practical intervention horizon has become narrow. And a person can make the final choice after many of the conditions shaping that choice have stabilised elsewhere.

This does not mean the decision is no longer human. It means human decision may be an insufficient description of the system.

The record usually preserves the survivor

The difficulty becomes clearer when somebody tries to reconstruct a decision later.

Organisations are usually good at preserving final artefacts. The score remains. The approval remains. The model output remains. Perhaps there is a rationale and a timestamp showing that a human reviewed the case.

Viewed months later, the record can look impressively complete.

Yet a complete record of the outcome is not necessarily a complete record of the decision.

What did the reviewer actually see at the time? Which evidence had already been compressed? Which alternatives had disappeared? Which threshold caused this case to become visible? Was the explanation present before the judgement or generated afterwards? Which system version produced the classification? Could escalation still alter the outcome? What uncertainty existed then that the final record no longer carries?

These questions are awkward because organisations naturally preserve what survives. The selected alternative leaves a strong trace. Discarded possibilities leave weaker ones. Things that never became candidates may leave none.

Over time, the decision starts to look more inevitable than it actually was. The organisation remembers the answer better than the field of uncertainty from which the answer emerged.

This is not merely a documentation problem. It is a problem of institutional self-knowledge.

Bowker and Star’s work on classification is useful here because classifications are not merely labels attached to an already neutral world; they become parts of information infrastructure and can make some distinctions durable while rendering others less visible (Bowker & Star, 1999). James C. Scott’s analysis of institutional legibility operates at a different scale, but it offers a parallel warning about what happens when complex realities are made administratively manageable through simplification (Scott, 1998).

Neither lineage is identical to formation of decidability. Classification is not decidability, and legibility is not decidability. But both make it harder to pretend that the object presented for decision arrived untouched from reality.

When governance improves but reflexive capacity does not

Imagine an organisation becoming increasingly sophisticated at AI governance.

High-risk outputs receive human confirmation. Reviewers are trained. Models are documented. Appeal routes exist. Responsibilities are assigned. Decisions are logged. The governance programme improves year after year.

At the same time, more of the cognitive environment surrounding those decisions becomes technically mediated. Systems prioritise which cases deserve attention. Models summarise evidence. Agents generate candidate responses. Ranking systems influence sequence. Retrieval systems determine which historical material appears relevant. Interfaces make some paths obvious and others cumbersome. Previous automated outputs become inputs to later processes.

Nothing dramatic happens. Nobody replaces the humans. Formal oversight may actually increase.

Yet the organisation can become less capable of observing how its own conditions of judgement are being produced.

By reflexive capacity I mean something practical rather than philosophical: an organisation’s ability to observe, reconstruct and, when necessary, intervene upon the conditions under which its own decisions become possible.

An institution can know who approved a decision and still not know how the field presented to that approver acquired its shape. It can govern the output without being able to reconstruct the upstream transformations. It can audit the visible choice while remaining partially blind to the production of the choice environment.

Existing controls do not become useless because of this. The point is narrower. They may describe only part of the architecture.

The failure mode is subtle precisely because conventional indicators can all move in the right direction: more documentation, more review, more oversight, more accountable humans – and yet less institutional visibility into how judgement itself is being conditioned.

There is a temptation to jump from here directly to democratic theory. That move should be resisted for now. The organisational problem is already substantial enough to research empirically.

Still, the direction is worth noticing. Public institutions also depend on mechanisms that determine what becomes visible, classifiable, urgent or actionable. Which signals become risks? Which populations become administratively legible? Which anomalies become policy problems? Which alternatives appear serious enough to evaluate?

These are not yet decisions in the conventional sense. They help determine what an institution will later be capable of deciding about.

That deserves a separate analysis. For the present argument, the narrower point is enough: AI-assisted systems do not need to take final authority away from humans in order to reshape institutional judgement. They can participate earlier, in forming what counts as the object of judgement.

Before asking who decided

None of this requires a return to supposedly unmediated human decision-making. There is nowhere to return to.

Filtering is necessary. Summarisation is necessary. Categories are necessary. Decision support can improve consistency, surface weak signals and reduce cognitive burden.

The relevant distinction is not mediation versus no mediation. It is between mediation that remains governable and mediation whose consequences become difficult to observe from the place where responsibility is exercised.

That changes the starting point of analysis.

When an AI-assisted decision is challenged, we usually ask: Who approved it? Who owned the model? Could the recommendation have been overridden? Was the human properly informed?

Those questions remain necessary. But in some cases another reconstruction has to come first.

How did this situation become a decision in this particular form?

What became salient? What became evidence? What became an option? What became irrelevant? What became urgent? What quietly ceased to be available? And at what point did those transformations become difficult to reverse?

Only then does the familiar question – who decided? – recover its full meaning.

Because the human may still make the final choice.

The deeper issue is whether the conditions under which that choice became possible were themselves sufficiently open to observation and judgement.

Decision freedom does not have to disappear suddenly.
It can narrow much earlier.
While the human is still there.