oversight

taken together, it appears that for most domains LLMs will continue to look like a cracked intern: quick and effective in the hands of an adult but not given run of the place. most firms will not be able to adopt fully autonomous AI, not for problems of skill issue or lagging technology diffusion but rather for structural reasons seemingly endemic to current architectures.

Both interruptions and auto-approvals increase with experience. This apparent contradiction reflects a shift in users’ oversight strategy. New users are more likely to approve each action before it’s taken, and therefore rarely need to interrupt Claude mid-execution. Experienced users are more likely to let Claude work autonomously, stepping in when something goes wrong or needs redirection. The higher interrupt rate may also reflect active monitoring by users who have more honed instincts for when their intervention is needed. We expect the per-turn interrupt rate to eventually plateau as users settle into a stable oversight style, and indeed the curve may already be flattening among the most experienced users (though widening confidence intervals at higher session counts make this difficult to confirm).

Respondents found the task less meaningful when they thought that the slogans were AI-generated (Figure 1). Compared to the human-label group, the AI-label group saw a drop in task meaning of about 0.07 standard deviations. Respondents in the AI-label condition were also less willing to contribute a slogan of their own by 3.4 percentage points. Because 26% of participants provided a slogan—and some were remarkably creative—the estimated effect is substantial: a 13% decrease relative to the baseline.

The important ironies of the classic approach to automation lie in the expectations of the system designers, and in the nature of the tasks left for the human operators to carry out.

And so generating code is not necessarily the part that you need to worry about the most. Given enough context, agents can write the implementation and run the tests and inspect failure and revise code for us. We need to get to a place where we feel like there is enough of human taste encoded in the environment that we can trust what is being built, so that our human attention can be focused on the places where it’s needed most.

a cloud software factory is just an automation around the core loop of development, from triage → spec → implement → review → verify → ship → monitor, where at every step a mix of agents and humans moves the process forward.

The core insight is this: AI agents don’t save you time by finishing your work. They expose how much work was always possible but never attempted. Five constraints now govern how much of that backlog any organization can actually capture: judgment, planning, coordination, evaluation, and absorption. Understanding those five constraints tells you exactly which new roles are being created and why.

Agents doing judgement-heavy work need a way to learn from the people whose judgement they are trying to approximate. 1. Principles beat rules, because rules overfit and principles transfer. 2. Agents need to learn how to learn, or feedback turns into brittle exceptions. 3. The feedback loop has to live where the team already works, or people stop participating.

When an agent does work repeatedly, the prompt starts to become the thing you review. If those instructions determine production behaviour, they should live in a repo, with version history, review, and rollbacks. The daily learning agent does not directly change production behaviour. It opens a PR showing what feedback it reviewed, what principle it thinks should change, and the exact diff to the skill file. A human reviews it like any other change.

long-horizon agents assume requirements all exist upfront. They are fundamentally against human in the loop. And they don't have true judgment for what humans like. That—is why I don't like them.