Mark Reveley

Papers

10 quotes from Research / Papers, newest first.

These results show that a model’s capabilities are compatible, generalizable and can carry over to other harnesses. Providers do report to optimize some models for their own coding environments: OpenAI, for example, describes GPT-5-Codex as optimized for agentic software engineering in Codex.12 Yet we observe that a shared provider does not guarantee the best pairing. The practical question remains which harness delivers the best balance of cost and task success for a given model and workload.

HarnessTax: How Much Does the Harness Matter for Coding Agents? · UC Berkeley · Arena

Modern retrieval agents expose many configuration choices—LLM, retriever, number of documents, number of hops, and synthesis strategy—each shaping both
answer quality and serving cost. Today, these pipelines are typically hand-tuned
once per workload, leaving substantial per-query optimization untapped. We formulate Query2Conf : given a natural-language query and either an accuracy or
a budget target, select from a predefined pipeline catalog the configuration that
minimizes cost (or maximizes accuracy) at inference time. We propose BRANE
which uses an LLM to convert each query into workload-specific characteristics,
then trains a lightweight per-configuration predictor that estimates whether the
pipeline will answer the query correctly. At inference time, BRANE selects the
configuration that maximizes predicted correctness penalized by cost, exposing a
tunable cost–quality tradeoff without retraining. Across MuSiQue, BrowseCompPlus, and FinanceBench, BRANE consistently pushes the cost–quality Pareto
frontier, matches the best fixed configuration’s accuracy at up to 89% lower cost,
and outperforms LLM-routing, rule-based, and fine-tuned Qwen3-4B baselines.
These results show that per-query configuration of the full retrieval pipeline is a
practical alternative to static workload-level tuning.

Natural Language Query to Configuration for Retrieval Agents · UC Berkeley · University of Washington · Microsoft Azure Research - Systems · 26 May 2026

We leverage probabilistic programming principles to tackle complex tasks, and utilize differentiable and classical programming paradigms with their respective strengths. The framework introduces a set of polymorphic,
compositional, and self-referential operations for multi-modal data that connects multi-step generative processes and aligns their outputs with user objectives in complex workflows. As a result, we can
transition between the capabilities of various foundation models with in-context learning capabilities
and specialized, fine-tuned models or solvers proficient in addressing specific problems. Through
these operations based on in-context learning our framework enables the creation and evaluation of
explainable computational graphs. Finally, we introduce a quality measure and its empirical score
for evaluating these computational graphs, and propose a benchmark that compares various stateof-the-art LLMs across a set of complex workflows.

SymbolicAI: A framework for logic-based approaches combining generative models and solvers · hn · ExtensityAI · Johannes Kepler University · RICAM · Amazon Devices · 1 February 2024 · Primitives notebook

By enforcing capability-based security policies,
CaMeL effectively prevents unintended data
flows and actions. Even with a compromised
cloud storage file containing prompt injections
targeting the Quarantined LLM, the capabilities system acts as a robust and granular control mechanism, preventing data exfiltration, as
well as other prohibited actions. This approach
provides a secure environment for LLM agents,
preventing harm by design, without requiring
modifications to the underlying LLM itself, and
offering a significantly more fine-grained and
robust defense compared to isolation and/or
adversarial training strategies alone.

Defeating Prompt Injections by Design · Google · Google DeepMind · ETH Zurich · 24 March 2025

Despite appearances, perhaps the internal representations of neural networks implicitly realize symbolic structure. In support of this hypothesis, we show that the vector representations of a variety of neural networks can be closely approximated with symbolic structures: we can replace the network's entire representation-generating process with a closed-form equation instantiating a symbolic structure, and the network's behavior remains largely unchanged.

The Emergent Symbolic Structure of Artificial Neural Networks · hn · Yale University · Johns Hopkins University · New York University · Microsoft Research · 30 August 2026

Accepting program modifications
demanded by changing external circumstances to be an essential part of programming, it is argued that the primary aim of
programming is to have the programmers
build a theory of the way the matters at
hand may be supported by the execution
of a program. Such a view leads to a
notion of program life that depends on
the continued support of the program by
programmers having its theory. Further,
on this view the notion of a programming method, understood as a set of rules of
procedure to be followed by the programmer, is based on invalid assumptions and so has to be rejected.

Programming as Theory Building · Peter Naur · 1985

We argue that coding agents have crossed a threshold of capability at which traditional human code review is no longer a necessary component of a software quality pipeline. Our argument rests on two claims: every stated goal of code review can be served by agents at lower cost and higher throughput; the naive integration in which agents write code and humans remain the mandatory reviewers is a dead end because it neither provides meaningful assurance nor scales with AI-assisted throughput.

The End of Code Review: Coding Agents Supersede Human Inspection · Martin Monperrus · 11 June 2026

We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.

How AI Impacts Skill Formation · Anthropic · 28 January 2026