← Back to timeline
The Operating Layer Behind My Agents

2026-07-08

The Operating Layer Behind My Agents

A practical tour of the operating layer behind my agents: coding agents, context, planning, memory, skills, and evaluation.

The Operating Layer Behind My Agents

As of July 2026, this is roughly how I work.

I default to Codex as my superapp (use the desktop app), dictate 75%+ of my prompts, transcribe every meeting, and keep pushing more of my world into my second brain as raw markdown.

The operating layer gives each of those things a place: I set intent, feed agents real context, let them act, and turn repeated work into durable instructions. These are the rules behind that setup.

1. Make a coding agent your default control plane

I do not know who needs to hear this, but you have to use a coding agent now. I have been screaming this from the rooftops since January 2026. You are a laggard if you are not using one.

The term coding agent undersells it. This is a general productivity tool. I use one for writing, research, planning, analysis, prototyping, reviews, slides, and random automation. Basically anything with text, files, or a decision surface.

Codex is my go-to, but Claude Code and Cursor are acceptable alternatives. By harness, I mean the environment that gives the agent access to your terminal, files, tools, browser, and instructions (harness is the nervous system to the model’s brain). My harness stays model-agnostic.

Use the full-powered option. In Claude’s case, that means Claude Code, not Chat or Cowork. I want the version that can work across my system, not a watered-down surface that runs in a sandbox.

The few things I do not use a coding agent for:

  • Fast search: Search needs to be extremely quick and accurate. This excludes deep research. I use Perplexity, and my threads are saved to my second brain through the Perplexity CLI I created with Printing Press.
  • Voice mode on the move: I use ChatGPT Voice Mode when I am driving or walking.
  • Quick design revs: I still use Claude Design or Figma Make for some early design iterations. Lately, I have been generating mocks with GPT-ImageGen and having the coding agent implement them.

2. Feed it real context

Yap to the model

I talk to the model a lot.

Voice gets more context out of me and involves less self-editing. Typed prompts are usually the sanitized version of my thinking. Voice is the raw thought and agents filter the noise.

I use AirPods today, but I have considered a gooseneck mic and the DJI Mic Mini. That would let me dictate more quietly in an office and probably push voice use closer to 90%.

I used to lean hard on Wispr Flow. Now I use Codex’s built-in dictation because it is free and works across my computer. If you want a local-first version of this category, Muesli is interesting.

On mobile, I use the app’s built-in dictation in ChatGPT or Perplexity. For general use, I use Apple’s built-in dictation. It is not great right now, but I keep hoping the next iOS update improves it.

Transcribe every meeting

I transcribe every single meeting. I used to view meetings as a complete waste of time. Now I speak in the meeting for my agents. The transcript becomes source material for agents to act on, and my system can generate action items from it.

I personally use Granola, but you can use open-source options or build your own. Anarlog is strong if you care more about file ownership. Meetily is a solid local option.

How my system uses transcripts:

  1. Record all meetings.
  2. A cron pulls new transcripts every 30 minutes into /raw/meetings/transcripts/YYYY-MM-DD-meeting-name with YAML frontmatter for title, date, participants, and type.
  3. The system extracts summaries, action items, and wiki entries for people, companies, and ideas. Each derived artifact links back to the original transcript.
  4. Every new thread can navigate the derived artifacts or return to the raw transcript as source data.

Give the agent access to your brain

Everyone I see using agents is building specialized workflows that model companies will eventually cannibalize. The moat is in the context.

We are finally at a point where a second brain can exist with little friction. Notes, docs, transcripts, code, bookmarks, messages, old plans, screen captures, and session logs all become useful when you can route them cleanly. I use tools like Familiar because the extra context matters.

In my experience, most “AI does not work” complaints are context failures.

I put everything into /raw/. Raw data is the evidence layer. Summaries, action items, people pages, and wiki entries are derived working context that link back to it. The full inventory is in the appendix. The point is to capture enough of my world to build an exobrain of myself.

3. Run an intent -> plan -> action loop

Start with a plan

On anything non-trivial, which is almost everything, I start in plan mode. I want the agent to say what it is going to do, what it will touch, and how it will verify the result. Otherwise it wanders. This is the easiest way I know to get better work out of the same model.

You can use the built-in plan feature in your harness. My current recommendation:

  1. Use Compound Engineering’s /ce-plan with the best model you have available, such as Fable, GPT-5.6 Sol, or Opus, on high reasoning. Optionally use /ce-brainstorm first.
  2. Use Matt Pocock’s /grill-me to relentlessly interview you about the plan until it is concrete.
  3. Use Compound Engineering’s /ce-work to implement the plan with implementation subagents.

I store every plan in one /plans/ folder. Plans have YAML frontmatter for title, status, and creation date, which lets me view them on a kanban board. Some plans span days or weeks.

I do not read the plan. The plan is for the agent, not for me. I might ask leading questions to improve it, but I do not need to know every detail. A CEO does not read the design spec. They review the outcome, ask questions, and redirect it when needed.

Give the agent access to your hands

Agents are useful when they can act on your behalf, so let them take action.

Permissions: Full access. Dangerously skip permissions. Always. I do not want to manually approve every API or CLI command. I want the agent to run.

Software access: I give agents access to as much of my software as possible, with clear ownership boundaries.

  • Important external actions, such as sending an email or buying something, still require my approval until I have enough trust in that workflow to let it cook.
  • The coding agent on my computer, such as Codex, is an extension of me. It has access to my computer and accounts. My OpenClaw and Hermes agents run on a separate Mac Mini and are my employees. They use their own email, accounts, and identities.
  • Agents get the credentials they need through a dedicated 1Password vault. I use the CLI on my personal computer and a service-account token for OpenClaw and Hermes.
  • Access comes through MCP, CLI, and API. I prefer CLIs and build a lot of custom ones with Printing Press. I still think MCP will become the golden child.

Never read: Do not bother reading the code, the plan, or most agent responses. Give the agent intent and a goal, then let it run. I skim the artifact: a markdown doc, HTML page, prototype, or product. I give feedback while using it. Spend your time playing with the outcome, asking questions, and giving feedback instead of verifying every intermediate step. Code is now free, so get to the end state faster.

4. Package repeated work into skills and instruction files

A skill is a reusable AI package that tells the model how to do a specific task consistently. It can contain instructions, examples, rules, markdown, scripts, or configuration metadata. A weekly-report skill, travel-planning skill, code-review checklist, and data-cleaning workflow are all skills.

Skills are one of the most important primitives of agents. Anyone who uses a coding agent should package repeatable work.

Research flows. Writing flows. Review flows. Weekly ingest. Handoffs. Planning. Extraction. Whatever repeats should become a reusable skill.

I have an absurd number of skills: 343 as of 2026-07-14. Some favorites:

  • stop-slop: coupled with my VOICE.md, it removes AI slop from writing.
  • goalcraft: turns a messy goal into a paste-ready /goal prompt.
  • slides: builds decks in any format or template.
  • today: generates a daily briefing from my sources.
  • autoplan: builds plans in the background from raw data.
  • adversarial-review: spins up a fresh agent in a different harness or model to review docs, plans, and code.
  • voice-of-customer: uses customer digital twins to simulate feedback.

My rules for skills:

  1. Create most of your own skills. They should fit your system and your risk tolerance.
  2. Use a skill-creator skill to create new skills consistently.
  3. Use a skill resolver to help the agent find the right skill.
  4. Give subagent personas specialized groups of skills.
  5. Put all skills in .agents and symlink that folder everywhere.

I still review skills from other people before importing them. I start with reputable sources such as skills.sh, run a security audit on the repository, check GitHub stars as a loose signal, and have my agent create a tailored version when needed. I take skills as-is only from providers I already trust.

A resolver routes messy human intent to the right action, entity, or instruction file. My root AGENTS.md files are lightweight routers. They give the agent just enough context to decide where to look next. I keep deeper instructions in nested AGENTS.md files so the agent progressively discloses context as it enters the relevant part of the system. I have 848 AGENTS.md files as of 2026-07-08. That may be excessive. It is much better than putting every instruction in the root file.

A subagent persona is a scoped role with dedicated instructions, tools, and skills. I have personas such as product designer, software engineer, architect, product manager, video producer, and user researcher. My orchestrator spins them up for bounded work.

The .agents folder is Git-version-controlled. It contains personas, skills, crons, plugins, and more. I symlink the skills folder into other harnesses. For visibility, I also symlink an unhidden agents folder.

I also keep a few purpose-specific markdown helpers:

  1. AGENTS.md: a README and resolver for agents.
  2. VISION.md: the long-term concept for a product or idea.
  3. DESIGN.md: design tokens, system rules, rationale, and usage for an app.
  4. VOICE.md: the voice and tone router for writing.
  5. MEMORY.md: durable facts, preferences, and memory routing.

I use /goal for bounded work and VISION.md for longer-term direction. VOICE.md first asks who the reader is, what they need, what context they carry, and what would make them trust the writing. Then it routes to the right writing style. MEMORY.md is still a work in progress across short-, medium-, and long-term memory.

5. Optimize time and money with subagents

Subagents make the system faster when the work can be split cleanly. They also keep the main agent’s context focused on the actual decision instead of every search result, failed attempt, and implementation detail.

I use four functions. They can be separate agents, or one agent can cover several of them when the task is small.

  1. Orchestrator: Reads the request, estimates ambiguity and effort, identifies parallel work, and decides whether the task needs a plan or workers.
  2. Planner: Turns an ambiguous request into bounded work. It defines the outcome, assumptions, constraints, and how the result will be checked. I use the strongest model here when the decision is novel or expensive to reverse.
  3. Workers: Execute bounded tasks. A worker might research a narrow question, inspect a code path, make a change, or draft a section. The model, effort, and harness should match the job.
  4. Verification: Tests, reviews, or inspects the artifact before it comes back to me. This is how I skip reading every plan, tool call, and intermediate response while maintaining quality.

A typical request starts as something I dictate while walking or driving. The orchestrator checks ambiguity, expected effort, task category, and whether any work can happen in parallel. If ambiguity could change the outcome, it goes through planning. Otherwise, it executes.

The planner gives workers a bounded contract. Workers return an artifact, not a stream of thoughts. Verification checks that artifact against the original request, existing patterns, tests, or a review rubric. I react to the result. I step in when judgment, taste, strategy, or an external consequence needs me.

This does not need to become a distributed-systems science project. Subagents help with independent, bounded work. They waste money when the work is sequential, shallow, or requires one writer to hold the entire argument. Coordination itself costs tokens and time.

Spend tokens where judgment is

Here are some cost optimization strategies:

RuleWhat I do
Route model and effort deliberatelyUse light models for fixed-shape work, mid-tier models for synthesis and voice, and expensive reasoning for planning or real trade-offs.
Load the minimum relevant contextStart with a small router, then load the instructions, files, and history the task actually needs.
Ignore noiseExclude build artifacts, dependencies, logs, and irrelevant folders before agents start searching.
Compress outputsAsk tools and workers for the relevant excerpt, table, diff, or decision instead of dumping full files into the main context.
Isolate subagent contextLet workers carry their own search and implementation history. Return the finding or artifact to the orchestrator.
Cache repeated contextReuse stable instructions, voice rules, and reference material when the harness supports it.
Measure before tuningLog task class, model, input tokens, output tokens, cached tokens, latency, and whether the result was actually useful.

Cost optimization without outcome data becomes penny-pinching around the edges. I care about the cost of a useful result, not the cheapest possible token.

6. Keep one durable command surface

One of my favorite things is a pinned, long-lived Chief of Staff thread. I talk to one thread, and it spins out separate threads with subagents when the work deserves it.

It remembers important long-term context and simplifies how I work. I do not want to think about threads or context windows all day.

How it works in practice:

  1. Reactive messaging: I send a message. The Chief of Staff thread is my main chat surface and spins out work as needed.
  2. Proactive messaging: A heartbeat runs every 30 minutes, checks which skills need to run, drafts unanswered Slack or Gmail replies, updates me on crons, and can surface a daily or weekly brief.

I have officially switched from CLI to app. The Codex app is my default AI surface. Before that, I used Claude Code, Codex, and Cursor through cmux, and before that I used Obsidian with a terminal extension.

My personal preference:

Claude Code CLI > Codex CLI Codex App >> Claude Code App

Features I use in the Codex app as of July 2026:

  • Dictation that completely replaces Wispr Flow for me.
  • An in-app browser that runs custom apps side-by-side with my agent and lets me annotate them.
  • A terminal with CMD-J, where I run general command-line work, Claude Code, and other agents.
  • Computer and Browser Use for GUI execution.
  • Durable threads that spin out other threads through the Chief of Staff pattern.
  • Primitives such as /goal.
  • Automations for cloud-based crons, although I still prefer LaunchAgents on my Mac.
  • Chronicle is cool, but I prefer Familiar.

That is why it has become my default surface.

7. Make it autonomous and portable

I use Apple LaunchAgents as cron-style background jobs far more than built-in Routines in Claude Code or Automations in Codex.

I want ingestion, sync, summaries, maintenance, and repeatable work happening before I sit down. A lot of the best AI work should happen in the background, write to files, and wait for you.

That is why I store as much as I can in markdown and keep building a knowledge graph around it. Meeting transcripts. Notes. Calls. Emails. Messages. Reading history. Podcasts. Screen captures. Session logs. All of it compounds.

If you want a public starting point for the knowledge-graph part, Karpathy’s LLM Wiki is still the cleanest reference. If you want a more built-out version, gbrain is worth looking at.

I build on standard primitives and avoid getting locked into one harness. Skills, AGENTS.md, subagents, /goal, hooks, MCP, and plugins are the foundation. Higher-level workflows can sit on top of them, or you can use what the model companies give you, such as /loop in Claude Code, Automations in Codex, or Routines in Claude Code.

I have seen people build systems that model companies will absorb six months later. Use the primitive when it already exists. Your durable system stays portable while the underlying harness gets better.

I also want agents to run the app, test their work, fix issues as they go, and leave feedback at the end of a session. A night-shift loop can take a bounded queue of work, leave artifacts and traces behind, and let the next agent continue if it fails partway through.

8. Build applications on top of the system

Once you have this system, what do you build with it? A lot of people can stay busy with agents without producing anything. They are token-maxxing instead of building.

At Everpure, I am building agentic IT operations for storage. The product watches, recommends, and acts on storage needs so admins can focus on what matters.

I am also building personal apps for fun and productivity:

  • Mise: a meal planner that determines what to cook, orders groceries, and monitors pantry levels.
  • Self-learning todo list: a custom todo app that is slowly becoming my superapp. It aggregates action items, acts on them, and flags what I should do.
  • PRM: a personal relationship manager that works with my people graph.
  • Daily briefing: sends me a daily brief of what I need to know. I may have it printed automatically every morning.
  • Communications Triage: a kanban board to triage email and Slack.

There is no excuse not to build. Models are good enough to do anything today. Anything extra is a bonus.

I want agents doing everything except work that is new and hard: strategy, taste, judgment, and ambiguous trade-offs. That is where I want my attention.

My system architecture

Here is the shape of the system as of 2026-07-14:

  • 00-goals: direction, priorities, and decisions.
  • 01-raw: immutable source evidence.
  • 02-wiki: compiled knowledge and current understanding.
  • 03-people: personal CRM and relationship context.
  • 04-projects: active, time-bound work.
  • 05-areas: ongoing responsibilities.
  • 06-resources: reference material.
  • 07-archive: completed or historical material.
  • .agents: skills, personas, crons, plugins, and operating instructions.
  • apps: custom software and CLIs.
  • identity: values, voice, and personal context.
  • system: schema, memory, audits, and operating model.

Top-level stats as of 2026-07-14:

  • 343 SKILL.md files under .agents/skills/.
  • 13 agent definition files under .agents/agents/.
  • 650 files under .agents/crons/.
  • 210,466 tracked files in the monorepo.

My current stack

  • Codex: default surface and superapp.
  • Claude Code: backup models and terminal-first work.
  • Cursor: still in the mix when I want an IDE surface.
  • Granola: meeting capture and transcripts.
  • Obsidian: markdown notes and personal knowledge base.
  • Matter: saved reading that later enters the system.
  • Obsidian Web Clipper: web pages to markdown.
  • Familiar: screen context and memory.
  • OpenClaw and Hermes on a Mac Mini: background employee mode.
  • QMD: minimal CLI semantic search for markdown.

Start here

Do not try to build the entire system at once.

  1. Pick one coding agent and make it your default control plane.
  2. Give it one durable source of context, starting with meeting transcripts, notes, or both.
  3. Turn one repeated workflow into a skill, then let the system improve from there.

Then give your agent this article and ask it to implement the parts that fit your life. My exobrain starter is a decent starting point.

What I am working on next

  • dreaming v2
  • proactive action based on context
  • a digital twin people can talk to instead of me; I already have a Delphi, but it is not good enough
  • auto portfolio trading and rebalancing
  • orchestration v2
  • cost optimization v2
  • multiplayer brain for teams and companies
  • a self-improving harness

Appendix: what lives in /raw

The point of /raw is to capture the evidence layer cleanly enough that agents can use it later. Today it includes:

  • bookmarks from browsers over decades
  • calendar history
  • call data
  • every stored contact
  • raw email threads
  • health data from Apple, Eight Sleep, labs, and more
  • journals
  • social data from LinkedIn and X
  • transcribed meetings
  • text messages and WhatsApp
  • scratch notes from Apple Notes
  • geolocation data
  • podcast listening data
  • purchases, including data for a custom home-inventory app
  • saved articles and books
  • interesting GitHub repositories
  • screen captures from Familiar and potentially Codex Chronicle
  • Perplexity and Google searches
  • coding-agent session logs across Codex, Claude Code, and Cursor
  • Slack DMs and mentions
  • a swipe file
  • YouTube and Netflix watching history
  • wealth and financial data