A polished gold sphere at the junction of branching channels in a matte black metal routing plate
Post

Why you should build your own software factory

Why you should build your own software factory

Epistemic Confidence: As with all things AI this area is in incredible flux. A year ago I would have held a drastically different position on this topic because agentic coding was in such a different place. So it’s not unlikely that in some low number of months everything I wrote here is outdated.

Everyone seems to be building their own software factory. It’s been pointed out that this can be a waste of time and yet another way to procrastinate on the actual thing you want to build (article, HN discussion). As the article points out the key is that you shouldn’t start with one as your side project. If your first step when embarking on a new project is to first build a software factory, you are likely gonna waste a bunch of time. However, I think it can be a great exercise and to some degree it seems to be a natural step in mature software development process that builds on agentic coding tools.

As all my software development has moved away from touching the code directly and instead prompting/discussing requirements and architecture choices with agents I don’t want to repeat the same guidelines and request the same process steps all the time. Like everyone else, I’ve addressed this through a patchwork of CLAUDE.md/AGENTS.md statements and skill files which I largely see as macros for prompts. For simpler tasks like CRUD features my work increasingly becomes being the glue between these macros.

I will tell the agent to read a Linear ticket and ask me any questions about the scope. Then I might ask it to generate a few options for comps (usually I have Claude shell out to Codex for this and have it use its fabulous imagegen skill). Then we discuss architecture and it writes a spec. Then I have a series of review skills, etc. You get the gist of this.

For me, my software factory, Kaizen2K, emerged as a way to formalize and automate the routing I do between different skills and process steps. Is GUI work required and the ticket has no comps or insufficient comps? Create comps and let me choose between the options or give feedback for another on prompts. Is concurrency critical for this working correctly? Review the draft with the Leslie Lamport skill.

The obvious win here is that I can focus on hard and consequential decisions. The formalized structure of the factory also forces me to think more explicitly about my own decision process. How am I making a given decision? What data am I looking at to make the decision? Is there something else I should be considering as well?

If a task cannot be picked up by the factory it begs the question why not? Is there a skill or decision guideline I could build that will allow the factory to pick this up in the future? If you are always in manual prompt mode anyways, the shortcoming is less obvious because there is no mode change. The pain of the work not going into the factory to begin with or having to be pulled out of it is a strong signal to automate more.

It took very little effort to systematically collect all kinds of data. For example:

  • How often do I reject work at a given stage and what are common reasons?
  • What token cost is associated with each step?
  • Where do agents spend most time? Running tests? Editing code? Waiting on me?

Data like this has been proven very valuable at continuously tweaking the system. For example we (Claude and me) found that I almost always requested to address review findings the agent rejected as “pre-existing” or “outside of scope”. Quick prompt-adjustment but time saved! Similarly there were quite a few cases where I needed to click “retry” even though the jobs were able to be mechanically retried.

Build Your Own For Yourself!

Your software factory - the way I describe it - automates the glue between your skills and your prompts. I think building a factory that is universal and works for other teams is likely futile or will lead to inferior results at higher cost. It’s inherently linked to how you build software and that’s good! Thanks to coding agents it’s cheap to build now. My own factory, Kaizen2K, went from 0 to web GUI, worker pool and being able to deliver very simple CRUD work in significantly less than an hour of my attention. Everything since then (still likely under 5 hours active attention) has been tuning that to some degree has been opinionated and will not work for everyone. There is no value in doing more work to generalize something that anyone can build for themselves in very little time. The flip side is that this also can be a great way to standardize workflows across a team if the team builds a shared factory.

The Factory as the Future of Cross-Functional Collaboration

I still cannot run all the work through Kaizen2K that I would like to. Some things still get stuck. But it gets smoother every day. Next week I want to open the factory up to other developers on my team to have their own view and process their tickets through a shared worker pool. However, in the long run I want to bring the rest of our cross-functional team on board. If there are non-technical scope questions during initial discovery of the ticket the product manager should be able to answer the question directly in Kaizen2K. Maybe eventually Kaizen2K will replace our Linear setup entirely.

Bonus: How my Factory, Kaizen2K works

To go with the theme this section will be the only section entirely written by AI with me only having given feedback on it. Enjoy!

Kaizen2K has two halves: a server that tracks the work and decides what happens next, and runners on my machines that execute it. The server is one Rust application built with Axum, with a web board and background workers in the same process. Postgres holds the tickets, job queue, specs, questions, review results, and metrics. There is no separate queue service.

Kaizen2K architecture: the web board and iPhone app handle human decisions, and Linear supplies ticket context. A Rust server coordinates the workflow using Postgres. Local runners return results from Paseo agent sessions and mechanical git jobs.

In Build mode, a Linear ticket moves through scope review, a technical spec, implementation, code review, CI on main, and acceptance on staging. The routing is ordinary Rust code: given a ticket’s stored facts, it decides the next action. Agents classify the work and propose answers; rules determine whether to request comps, add a concurrency review, or ask me to decide. Automated reviews include another model family to catch different blind spots.

Each agent job packages a versioned skill, ticket context, model settings, and an expected JSON result. A runner claims it and starts a Claude or Codex session through Paseo, using my subscriptions. Implementation keeps a persistent git worktree per ticket, so questions and revisions can resume against the same files. The runner handles commits and pushes mechanically. Review gates inspect the committed work before it can reach main; risky changes or unresolved findings bring the diff to me.

The web board and native iPhone app let me answer questions, pick comps, and approve work with its supporting evidence. Those answers become stored inputs that let the pipeline continue. Jobs have leases renewed by heartbeats, so a vanished runner can be detected and its work retried. Workflow state survives server restarts.

The runners also extract timing, token use, tool calls, and test results from session transcripts. Together with recorded reviews and gate decisions, that gives me the feedback to keep changing the skills and routing rules.

All rights reserved by the author.