Start with what the climb adds
Omnine describes a ladder with five rungs, and it opens at the Skeptic. A Skeptic has seen the demos and is not convinced, and the tools have not yet earned a place in their real work. An Orchestrator sits at the other end and runs many AI systems as one instrument, with other people's work depending on the whole. Between them are the Sampler, the Driver and the Operator, and none of the five is a score, a percentage or a promise about how long any rung takes.
Almost everything written about that climb is a list of what gets added. More tools, more hours, more fluency, more of the work handed across. All of that is real, and none of it is the subject here. This article is about the other direction: what the person at the bottom of the ladder does that the person at the top may have stopped doing, without anybody deciding that it should stop.
The claim is narrow, so it is worth stating plainly. Being a Skeptic is a stage, and the stage is one you are meant to leave. But a Skeptic is not only a person who has not started. A Skeptic is also a person running a set of habits, and some of those habits were doing work that nothing else was doing. When the habits fall away because the doubt that justified them has been answered, the work they did can fall away with them.
There is no name for the habits in what follows, and that is deliberate. Giving the set a name would make it feel like a possession, something you either have or have been graduated out of. It is closer to a small number of ordinary acts, and the useful way to hold them is as acts.
The ledger a Skeptic keeps
Picture a person in an unnamed role who does not trust the new tool, and consider what that distrust makes them do in a week. This is an illustration of a pattern, built from no particular person or day.
They read the output slowly, because they expect it to be wrong somewhere and they would like to find where. They keep doing the task their own way beside it, at least some of the time, so they have something to compare against. They remember the last time it failed, and the memory changes how closely they look. They keep the part of the job that requires judgment, because they are not yet willing to hand judgment over. And they tend to know, in detail, what a correct answer looks like in their field, because that knowledge is the instrument they use to test the tool.
Taken together that is a ledger. On one side is what the tool produced. On the other side is what the person independently believes. The Skeptic is checking the two against each other every time, and the checking costs effort, which is why it feels like friction and why it is the first thing to go once the friction is no longer justified by the doubt.
Notice what that ledger needs in order to work. It needs a second source of knowledge that does not come from the tool. It needs attention that stays on the output instead of sliding past it. And it needs a live memory of the tool being wrong. The climb up the ladder, done well, supplies the reasons to trust the tool. It does not automatically supply any of those three requirements, and in some conditions it works against them.
What happens to attention when the checking stops
One review of this problem is Parasuraman and Manzey (2010), who reviewed empirical studies of complacency and automation bias in human use of automated and decision support systems and proposed an integrated model to explain both. Two things in their account are worth separating, because they fail in different places.
Complacency, in their account, occurs under multiple-task load, when manual tasks compete with the automated task for the operator's attention (Parasuraman & Manzey, 2010). Read that against the top of the ladder. An Orchestrator is by definition the person with the most going on at once: many systems, many threads, other people waiting. The condition that produces complacency is the condition an Orchestrator works in.
Automation bias is a related failure, and it shows up as both omission and commission errors when decision aids are imperfect (Parasuraman & Manzey, 2010). The review's wording of this is worth taking at its word and no further: the errors are tied to imperfection in the aid. A perfect system would not produce them. No system you will use this month is perfect, and that is the whole reason a Skeptic was checking it.
Two further findings from the same review deserve to be set down carefully, because they remove the comforting answer. Complacency occurs in both naive and expert participants and cannot be overcome with simple practice (Parasuraman & Manzey, 2010). Automation bias also occurs in both naive and expert participants, and the review reports that it cannot be prevented by training or instructions (Parasuraman & Manzey, 2010). The comforting answer is that experience cures the problem. For these two phenomena, in the studies reviewed, it did not.
It is worth being exact about the limits. This was a review of research on automation and decision aids in general, and it predates the language systems most people now use at work. It does not measure anyone's climb up a ladder. What it supports is a more modest and still useful statement: the effects were found in experts as well as novices, and they were tied to how attention is spent. If the pattern carries over, then becoming experienced with a tool is not by itself protection against over-trusting it.
The part of a skill nobody counts
The pilots are the clearest place to see what is lost, because airline cockpits are the oldest environment in which skilled people supervise automation for a living, and because somebody measured it.
Casner et al. (2014) asked 16 airline pilots to fly routine and nonroutine scenarios in a Boeing 747-400 simulator, varying the level of automation they used, grading their performance and asking what they were thinking about as they flew. The result is not the one the worry predicts. The pilots' instrument scanning and manual control skills were mostly intact, even when the pilots reported that they were infrequently practiced (Casner et al., 2014).
The trouble was elsewhere. When the pilots were asked to perform manually the cognitive tasks that manual flight requires, such as tracking the aircraft's position without a map display, deciding which navigational steps come next, and recognizing instrument system failures, the researchers observed more frequent and significant problems (Casner et al., 2014). Performance on those cognitive tasks was associated with how often the pilots engaged in task-unrelated thought when cockpit automation was used (Casner et al., 2014). The authors concluded that retention of the cognitive skills needed for manual flying may depend on how actively the pilots stay engaged in supervising the automation (Casner et al., 2014).
Hold two features of that result against the ladder. First, the part that survived was the part you could see: hands on the controls, eyes on the instruments. The part that eroded was the part that does the Skeptic's job, which is knowing where you are, knowing what comes next, and noticing that something has failed. Second, the thing associated with the erosion was where the mind went, not how many hours had passed. A pilot who stayed engaged in supervising kept more of it.
The study has limits that matter. It involved 16 pilots in a simulator, in one aircraft type, and the association with task-unrelated thought is an association, not a demonstration that wandering attention caused the problems. Cockpits are also not offices. But the shape is the one that matters here: the visible half of a skill held, and the half that audits the machine did not hold as well.
Fluent in the drill, thin outside it
A second pilot study adds a different failure, and it is closer to how most people learn a tool. Casner et al. (2012) presented 18 airline pilots with three abnormal events that are required during periodic training and testing. Each event was presented under the familiar circumstances used in training and also under less predictable circumstances, as it might occur in flight.
When the events came in the routine way seen during training, the pilots gave appropriate responses and showed little variability. When the same events were presented unexpectedly, the responses were less appropriate and varied greatly from pilot to pilot (Casner et al., 2012). The authors suggested that the training and testing practices may produce rote-memorized skills that are specific to the training situation and that generalize modestly to other situations, and they recommended that pilots practice recognizing the event and choosing the response, not only the response (Casner et al., 2012).
Set that beside a person climbing from Sampler to Driver to Operator. Every rung brings more practice with the tool behaving the way it behaves when you are learning it: the prompts that work, the outputs that look right, the failures you have already met. That is a rehearsed circumstance. What it does not give you is practice at recognizing a failure that arrives in an unfamiliar shape, which is the one thing the Skeptic was never rehearsed out of, because the Skeptic never believed the rehearsal was the real thing.
The distinction in that study is the one the closing question of this article turns on. Responding well to something you recognize is one skill. Recognizing it in the first place, when it does not announce itself, is another, and the first can look from the outside exactly like the second until the day it does not.
Where the Skeptic is wrong
None of this makes the Skeptic right, and an article that ended there would be flattering a stance it has no evidence for. The doubt has its own failure, and it is documented.
Dietvorst et al. (2015) ran five studies in which participants saw an algorithm make forecasts, saw a human make forecasts, saw both, or saw neither, and then decided whether to tie their incentives to the future predictions of the algorithm or the human. Participants who saw the algorithm perform were less confident in it and less likely to choose it over an inferior human forecaster, and this held even among those who saw the algorithm outperform the human (Dietvorst et al., 2015). The authors explain this by saying that people more quickly lose confidence in algorithmic than human forecasters after seeing them make the same mistake (Dietvorst et al., 2015).
That is a ledger with a thumb on the scale. A Skeptic who remembers every miss and discounts the hits is not testing the tool, only confirming a conclusion reached in advance. The habits described earlier are worth keeping only if they are used as a check, a way of finding out how good the tool is, rather than as a verdict. The same studies carry a limit of their own: they concern forecasting, with participants making incentivized choices, and not the broad range of open-ended work people hand to AI systems today.
So the picture is two-sided. The Skeptic over-weights failure and the Orchestrator can under-weight it. The useful thing to carry up the ladder is not the doubt, which the climb is right to retire, but the machinery of checking, which can be kept running even after the doubt has gone. A checker that runs without needing to be suspicious is a quieter thing than a Skeptic, and it is harder to maintain, because there is no feeling of unease prompting it.
Objections worth taking seriously
An Orchestrator has systems for this, and a person watching is the weak link. This is a fair point and part of it is right. Automated tests, review steps and a second system checking the first are exactly what a well run operation builds. But a check is only as good as the person who decided what it should look for. The review reports that automation bias results in both omission and commission errors when decision aids are imperfect (Parasuraman & Manzey, 2010). A checking system someone else built, or you built last year, can be the next aid you stop questioning. The habit that remains valuable is the one that asks whether the checks themselves still cover what the work has become.
The aviation evidence does not transfer to knowledge work. Mostly this is correct and I would not want to overstate the carry over. Pilots are trained under unusually strict conditions, the stakes are different, and the tasks are narrower than an Orchestrator's day. What transfers is a mechanism, not a rate. In the 2014 study the visible half of the skill was mostly intact while the cognitive half showed more problems (Casner et al., 2014), and in the 2012 study responses practiced under familiar circumstances did not carry over well to unexpected ones (Casner et al., 2012). Whether the same thing happens when someone supervises a set of writing, analysis or scheduling systems is an empirical question that these studies do not answer.
Slowing down to check erases the gain from climbing at all. It can, if the checking is applied evenly to everything, and that is the wrong way to do it. The point of knowing where the tool has failed before is that your checking can be uneven: heavy on the places where the failures live and light elsewhere. A Skeptic checks everything because they know nothing yet. An Orchestrator who kept the Skeptic's habits would check selectively, because they have the Skeptic's memory of where it went wrong and the Operator's knowledge of where it tends to be right.
Some people were never Skeptics, so they have nothing to forget. Someone may have come to these tools through a job that handed them an Orchestrator's responsibilities in a month. That is possible, and in that case the claim is not that they forgot anything. It is that no one ever gave them the ledger in the first place, and the ladder does not stop at the stage where they started.
What carries up
The summary is a short list, and the items are acts rather than attitudes.
Keep a second source of knowledge about what a correct answer looks like in your own field, one that does not pass through the tool. The Skeptic has this by default, because the Skeptic is still doing the work. The person who has handed the work across has to keep it deliberately, through some of the same tasks done by hand and some of the outputs rebuilt from scratch.
Keep your attention on the output, especially when you are busy. The review reports that complacency occurs under multiple-task load, when manual tasks compete with the automated task for the operator's attention (Parasuraman & Manzey, 2010). The cockpit study points the same way, since its authors concluded that retention of the cognitive skills may depend on how actively pilots remain engaged in supervising (Casner et al., 2014).
Practice recognizing, not only responding. Make the failure arrive in an unfamiliar shape on purpose, by asking a colleague to plant an error in something you rely on, or by running a task through the pipeline where you already know the right answer and looking at what comes out. In that study the pilots gave appropriate, low-variability responses in the routine presentation and less appropriate, highly variable ones when the events were unexpected (Casner et al., 2012).
And keep the memory of failure even in both directions. Write down when the tool was wrong, and write down when it was right and you doubted it. Dietvorst et al. (2015) found that people lose confidence in an algorithm faster than in a human after the same mistake, so the ledger needs the other column filled in on purpose.
If you want something to run in an hour, take the last piece of work an AI system produced that someone else relied on. Without opening the output, write down what you expected the answer to be and the three places it would most likely go wrong. Then compare. The difference between your expectation and the output is the size of the second source you still hold, and a blank page is useful information too.
An ending without a lesson
The ladder is real and the climb is worth making. A person who runs many systems as one instrument has gained something a Skeptic does not have, and nobody should be talked back down it.
What the Skeptic had, in the days of doubt, was a reason to keep looking. The climb removes the reason and leaves the looking up to you. The research here does not say that experience protects against over-trust, and in the cases reviewed it did not. It says that the part of a skill that audits the machine is the part that goes quiet, and that it goes quiet while everything visible still works. Holding on to it takes no suspicion. It takes a decision, made once and renewed, that the person at the top of the ladder will still be able to tell the difference between recognizing an answer and knowing it.