oversight

Respondents found the task less meaningful when they thought that the slogans were AI-generated (Figure 1). Compared to the human-label group, the AI-label group saw a drop in task meaning of about 0.07 standard deviations. Respondents in the AI-label condition were also less willing to contribute a slogan of their own by 3.4 percentage points. Because 26% of participants provided a slogan—and some were remarkably creative—the estimated effect is substantial: a 13% decrease relative to the baseline.

The important ironies of the classic approach to automation lie in the expectations of the system designers, and in the nature of the tasks left for the human operators to carry out.

And so generating code is not necessarily the part that you need to worry about the most. Given enough context, agents can write the implementation and run the tests and inspect failure and revise code for us. We need to get to a place where we feel like there is enough of human taste encoded in the environment that we can trust what is being built, so that our human attention can be focused on the places where it’s needed most.

a cloud software factory is just an automation around the core loop of development, from triage → spec → implement → review → verify → ship → monitor, where at every step a mix of agents and humans moves the process forward.

The core insight is this: AI agents don’t save you time by finishing your work. They expose how much work was always possible but never attempted. Five constraints now govern how much of that backlog any organization can actually capture: judgment, planning, coordination, evaluation, and absorption. Understanding those five constraints tells you exactly which new roles are being created and why.

Agents doing judgement-heavy work need a way to learn from the people whose judgement they are trying to approximate. 1. Principles beat rules, because rules overfit and principles transfer. 2. Agents need to learn how to learn, or feedback turns into brittle exceptions. 3. The feedback loop has to live where the team already works, or people stop participating.

When an agent does work repeatedly, the prompt starts to become the thing you review. If those instructions determine production behaviour, they should live in a repo, with version history, review, and rollbacks. The daily learning agent does not directly change production behaviour. It opens a PR showing what feedback it reviewed, what principle it thinks should change, and the exact diff to the skill file. A human reviews it like any other change.

long-horizon agents assume requirements all exist upfront. They are fundamentally against human in the loop. And they don't have true judgment for what humans like. That—is why I don't like them.