context

a turn by turn context awareness enables a context engineering mindset. mastering llms is all about mastering malloc().

And so generating code is not necessarily the part that you need to worry about the most. Given enough context, agents can write the implementation and run the tests and inspect failure and revise code for us. We need to get to a place where we feel like there is enough of human taste encoded in the environment that we can trust what is being built, so that our human attention can be focused on the places where it’s needed most.

Global workspace theory,4 or GWT, describes the human brain as a collection of specialized modules sharing data through a central, limited capacity bottleneck. Most machine learning architectures already satisfy the first three indicators5 of this theory. However, they consistently fail to implement GWT-4, the requirement for temporal persistence. To address this, we use the framework of computational self-availability, or CSA. This describes a system where internal processing is available as an input to the system itself.

I get pushback that falling inference prices will eventually bring us back to sending everything through the largest models. But I’m not so sure: those same gains will benefit the K3s and Qwens, and as we continue to develop better harnesses it will be easier to provide weaker (but still great) models with sufficient context to perform well.