When you bring an AI agent into your codebase, you quickly hit two extremes. Option one is babysitting the model on every ticket and reading every line it writes. That's safe, but while you nurse one task, the rest of your backlog sits frozen. Option two sounds nicer: hand the agent your repo keys and let it grab tickets and push straight to production.
On real projects, that kind of autopilot turns into a lottery. Developer Cole Medin unpacked this in a recent video and open-sourced the ai-software-factory framework. In his view, a code factory automates the grunt work between tickets while keeping humans on planning at the start and sign-off at the end—across five distinct models.
The Dark Factory Illusion and the Cost of Vague Specs
The highest level of autonomy is the Dark Factory. You drop in a rough two-paragraph idea, and agents split it into subtasks, write code, review each other's work, and deploy straight to users.
Medin tested this pipeline on livestreams. Models ran builds cleanly and AI reviewers caught obvious flaws, yet small bugs still slipped into the app. The bottleneck is human: writing 100% unambiguous specs is nearly impossible, so any gap turns into flawed logic in production. The only place a Dark Factory justifies burning hundreds of dollars on endless debugging is throwaway prototypes you wouldn't mind scrapping tomorrow.
That leads to a two-axis rule for choosing a setup: how much control you want to keep, and whether automated tests can actually prove the agent got it right. The subtler the product logic, the riskier it is to remove humans.
Fleet Factory and PR Factory: Two Pragmatic Bridges
When a task has a crisp, measurable finish line, autopilot works reliably. That's the Fleet Factory, named after Spotify's internal Honk agent fleet. Without manual review, the pipeline handles routine work backed by strict tests: migrations, type annotations, and code cleanup. Through Honk, Spotify has merged over 2.5 million service PRs, Airbnb migrated 3,500 test files in six weeks, and the Bun runtime was rewritten from Zig to Rust.
Once tasks allow room for interpretation, the magic fades. A recent study of open-source repos found maintainers rejected roughly half of agent-submitted PRs even when all tests passed, because the code didn't fit the project's architecture.
For commercial systems, it's safer to start with a PR Factory, like Stripe Minions. Agents build tasks in the background and open one pull request per task with a checklist, while a developer reviews the code by hand. The bottleneck shifts to the reviewer: when twenty pull requests land before lunch, the tech lead spends all day reading diffs.
Release Factory and a Self-Improving Loop from Production
The Release Factory—the core of ai-software-factory—breaks out of review paralysis. You and an agent bundle 10 to 20 tasks into a single release work order. The factory lives in a separate folder outside the target repo and drives the build externally through Archon and strict rules.
The pipeline maps dependencies and runs builds in parallel. Planning, coding, review, and user testing each run in fresh sessions with clean context so the model doesn't get confused by stale logs or rubber-stamp its own work. A dispatcher agent at the top makes sure branches don't clash on shared files.
Finished branches merge into a release branch, pass end-to-end tests, and deploy to staging. A tech lead only needs 15 minutes once a day to click through staging and approve one combined pull request. Of course, if your tests are just box-checking, broken code will look healthy on staging.
The fifth level, the Self-Improving Factory, closes the loop with live users. Production monitoring like Sentry, PostHog, or Langfuse feeds into an alert triage agent that turns real failures into new tickets—so real usage becomes the next task. Just remember to filter out noise in Sentry, or agents will burn cycles fixing random client browser quirks.
You can also dial control per module: inspect payments line by line in a PR Factory, sign off on standard features on staging in a Release Factory, and hand boring migrations straight to a fleet.

