A guide to cloud software factories for engineering leaders

Software development is about to undergo yet another major change: the shift from interactive coding agents to cloud software factories. These factories are systems that automate major parts of the SDLC. They minimize human variability and maximize software output, while improving security and compliance. They allow organizations to directly measure the ROI and business value of coding agents and improve them over time. In short, factories fix the shortcomings of the current interactive agents, which tend to be expensive, hard to tie to ROI, and difficult to govern.

This document gives an overview of cloud software factories for engineering leaders: what they are, why they matter, a tour of the key components, and how to deploy them in your organization.

Preface

Every C-suite executive is thinking about how to make their organizations agentic and make AI actually deliver business value. For software development, they want to: measurably increase velocity, deliver value to customers, and stay at the forefront of the market, all while maintaining a reasonable cost to ship (cost being humans + tokens).

Initially, there was a rush to put AI tools in the hands of every developer - first Copilot, then Cursor, then Claude Code. Organizations created leaderboards to celebrate token consumption. They tracked tool adoption to encourage usage. This made sense, given the power of a novel technology coming to the market and needing teams to try it.

Now I hear over and over again that every engineer is using interactive coding agents; but it’s still murky if the cost of tokens plus humans has real ROI. On top of that, the sprawl of these tools is a nightmare to control and govern.

You could try to layer controls over the current interactive toolset, but there’s a better solution: the cloud software factory.

Introducing cloud software factories

For folks who aren’t familiar with the “factory” framing, a cloud software factory is just an automation around the core loop of development, from triage → spec → implement → review → verify → ship → monitor, where at every step a mix of agents and humans moves the process forward.

Cloud factories are becoming increasingly popular (they are a variant of an agentic “loop”), so much so that one of the main tracks at the mainstage of the AI Engineer World Fair was dedicated entirely to factory engineering.

Here are two reasons they are catching on: First, agents have recently become good enough that true automation is possible. That is, rather than having a human interactively use a coding agent like Claude Code or Codex in their terminal or desktop, it’s now possible for agents to automatically do some subset of engineering work on their own, all the way from issue to shipping.

An agent can triage an issue, come up with a proposal, implement a fix, verify it, ship it, and monitor it all on its own. Of course, this isn’t possible yet for all issues, but it doesn’t need to be in order to be valuable. What matters is that it’s possible for some issues, and that over time the percentage of issues feasible to address this way rises. Today about 20-30% of issues fall into this bucket, but this will rapidly increase in the next year. Even for the 70% of issues that aren’t fully automatable today, most of them are semi-automatable; the factory can do important parts of the SDLC like review, verification and monitoring.

The second reason is that extensive interactive agent use has created a number of problems that I’m hearing from engineering leaders again and again, from cost controls to governance to security. The root problem is that interactive agents have human operators, and those humans all use them in very different ways, which don’t always maximize business value and can create risk.

For instance, it’s easy for a human to always use the most expensive model, even when they don’t need to, for a given task. Or for a human to inadvertently create a security issue by installing MCPs that have too much access. Or for different humans to have different skill levels in prompting agents.

As I noted in the beginning of this article, the idea behind a factory approach is to set up a system that minimizes human variability and maximizes output, with controls that ensure security and compliance. It creates a system where you can measure the ROI of agents and tie them to business value.

You can also address these issues with non-factory approaches by trying to standardize every developer’s local setup, but in reality it’s much easier and more effective to create control by moving all development to a centralized cloud factory. Cloud factories should not be thought of as adding another tool that developers have to learn – rather, they work with existing tools and infra, stitching them together into automations. There’s a very similar logic here to the rise of the cloud 20 years ago – when you centralize in the cloud, you get control, standardization, visibility, and so on. Cloud Factories are the SaaSified version of interactive agents.

The cloud factory workflow

Think of a cloud factory as an automation loop around the SDLC, with the following steps where agents do most of the work, and humans guide as necessary:

1. New issue comes into the system, either via a human or a monitoring agent
2. Triage agent runs and tries to understand and repro issue
   1. If it determines the task is automatable → hand it to the Implementation agent
   2. If it needs specs because of scope → have the spec agent iterate with a human to come up with a spec
   3. If it’s ambiguous → get human input and re-run, or just decide to park the issue for now
3. [If necessary] Spec agent runs
   1. Human reviews specs and then passes to implementation agent
4. Implementation agent writes code
5. Code review agent reviews code
6. Verification agent does computer-use or other verification
7. Human reviews code and verification output
   1. If necessary, go back to step 2, 3, 4 or 5
8. CI / CD
9. Ship it
10. Monitor agent runs and creates issues if need be completing the loop

This is the basic flow, but you can imagine variations – the point is just that a factory is an automation loop on top of the SDLC.

The components of a cloud software factory

Cloud software factories are infrastructure, similar to CI/CD. As such, they have a few core components that you will want to set up, either by building them directly or partnering with a vendor (more on this below).

Cloud runtimes and sandboxes

In order to automate the SDLC, the first step is moving agents off of developers’ laptops. You can’t create reliable automations around machines that might be asleep or turned off, and you want to standardize the environment where agents run, so individual laptops are not suitable.

Typical solutions are

1. Run an agent in an existing cloud development environment if your team has this set up
2. Dockerize your environment and run it on a docker host
3. Use Kubernetes or another cloud orchestration platform to run your environments

In addition to needing an environment, you’ll also need a host, which is just a computer to run the environment in. This could be in AWS, GCS, or a hosting platform built specifically for agents like Modal or Daytona. It could be in a public cloud, or, for most enterprises, on your own infrastructure.

Within this runtime, you’ll also want to have a coding agent like Claude Code, Codex, Cursor, OpenCode, Warp or another harness that is able to make changes to the codebase.

For the coding agent to work well, it will also need access to the tools that coding agents rely on, like MCP, other CLIs perhaps, and likely an integration with your source code forge (e.g. Github or Gitlab). Typical MCPs might be for your task tracking system (e.g. Jira), your communication system (e.g. Slack or Teams). You’ll need a system for managing who the agent acts as – agent authentication – and what access the agent has – agent authorization. You can see how this starts to get complex.

Orchestration layer

Once you have a place to run agents in the cloud, you need a system that actually launches those agents in response to a trigger. Triggers could be direct from a human (e.g. a user sends a slack message or opens a new ticket), or they could be more automated (e.g. a schedule).

For software factories in particular, there is a specific workflow around the SDLC that you probably want to implement that looks like triage → spec → implement → review → verify → ship → monitor, as I described above. But you’ll want to build your system so that the workflows it executes are actually more flexible. For instance, you may want to add an agent that does canarying or one that does dead code cleanup, and so on. The orchestrator should be flexible enough to handle custom agents and custom workflows.

This orchestration layer also should give you a global view of all agents, both active and historical, and what stage every issue is at. It should provide a way of seeing the state of every agent, including actually joining an agent session in case a human needs to interact and steer a running agent. Think of this as your factory control room – it’s where you go to see how the factory is running, what the bottlenecks are, what needs attention, and so on.

Integrations

If you’re building a factory, the best way to interface with it is where work is already happening, rather than adding a new destination. Your team is already putting ideas and issues into Slack and Teams; those ideas are often things that ought to go directly into the factory. They are opening issues in Jira and Linear. They are responding to customer emails, using support tools, and so on. They are using terminals and IDEs on their desktops and responding to PR comments in Github and Gitlab. A factory should accept input from any source and be able to communicate with developers where they already are.

For this to work well, a factory needs to support integrations into the most common tools. These integrations should support adding work items to the factory, checking on their status, and, crucially, changing the operation of the factory itself. E.g. If someone notices a code review agent giving advice that doesn’t match the conventions of the repo, they should be able to instruct the agent directly in the code review tool how to adjust its behavior in the future.

Human-in-the-loop

Not all development is automatable. Currently the Warp team automates 20-30% of all PRs. Some complex tasks are better done interactively right now. Sometimes agents will get stuck and work needs to move from the cloud to a local workbench. Sometimes you want to directly test an agent's output and not rely on automated verification.

For this to work well, you need a few primitives in your factory:

1. Steering: the ability to join a live session and iterate with an agent
2. Handoff: the ability to transfer an agent session and associated context from the cloud to local and back
3. Notifications: the ability for an agent to notify a human that it needs help

These primitives are not trivial to build but are very important if you want developers to adopt a factory workflow; without them work gets stuck or done incorrectly and the factory creates friction.

Multi-harness and multi-model

The best agent today is not the best agent tomorrow. The landscape changes weekly, and you’ll want to build your factory in such a way that it’s able to use the agent and model that gives the best cost/quality/speed tradeoff down the road.

This means harness flexibility: whether to run Claude Code, Codex, Cursor, OpenCode, Warp, etc. at the harness layer, and possibly mix and match harnesses across tasks.

It also means model routing: within a given harness, is it using the right model for the task, where “right” means the least expensive, fastest model that meets the required quality bar. You might want to use open weight or frontier, and have routing even within a given task, to optimize for cost and quality.

Adding support for multiple harnesses creates more complexity, but provides you with the most flexibility going forward. Anchoring on a single harness or model is high-risk from a cost-control perspective (you get locked in from a model vendor), an availability perspective (model providers are struggling to even get two nines of reliability), and a geo-political risk perspective (e.g. export controls).

Measurement, evals and continual improvement

The promise of the factory approach is that it creates the foundation for measuring and improving the ROI of software development over time. This only works though if you measure your factory throughput and efficiency, where

factory efficiency = (shipped product) / (token cost).

In order to measure, you must first do all the above, centralizing and standardizing in the cloud. But once you’ve done that, you can start to approach software development more like COGS than R&D, and tune your factory over time.

To tune the factory and improve efficiency, you need evals and experiments, which allow you to benchmark and test different inputs into the factory and see how they impact the output. Typical inputs to test are model and harness mix, MCP and Skill availability and contents, and so on – anything that might impact how the agents work.

You also will want self-improvement loops: think of these as agents that observe how the factory is functioning and suggest improvements to make it work better. E.g. Can your factory become more cost efficient in how it triages issues? Can it become better at computer use to verify fixes?

Finally, you’ll want memory, a way of agents learning from interactions and prior conversations. This memory should be explicit – supporting instructions like “remember that we always use terraform for infra changes”-- and implicit, where the agent itself extracts relevant information from interactions and resurfaces it into context at the right time. You will want to make sure that your company owns and stores this memory, otherwise it’s a big lock-in risk.

Factory-as-code

The best technical approach to setting up a factory is through code. After all, it’s infrastructure, and we have learned that there are benefits to defining infra-as-code over the years. That means all of your factory config should be defined in files and be version controlled. This has the added benefit that coding agents themselves can act on these files to update them.

Build vs. buy

For most companies, it doesn’t make sense to build all this factory infrastructure. There’s a ton to develop and maintain over time, and all the effort of building and maintenance takes away from your team’s core focus on solving customer problems.

The exception is if you are truly a large software organization (think Stripe or Uber). Then, building might make sense because the factory needs to be very tightly tied to existing complex developer infrastructure.

You may find that folks on your engineering team want to build this infrastructure, and even that they can build a prototype of it quickly. This is indeed possible, but not recommended, because there is an explosion of complexity in going from the 20% of a demo to the 100% of a fully implemented system. Most companies should not be building custom CI/CD infrastructure (e.g. Github Actions), so you probably should not be building custom factory infra either.

Instead, most companies should look to partner with a vendor that provides the infrastructure for you. This is what Warp does: working closely with companies that want to start down the automation path and set up a solid foundation for their internal factories. If not Warp, there are other vendors with solid solutions as well, although none are as flexible.

If you do partner with a vendor, you should be on the lookout for various flags:

1. Single model or single harness: this is a form of lock-in that will make it harder for your factory to improve over time and give the model provider leverage over prices.
2. Data ownership: make sure that whatever vendor you use doesn’t capture or train on your factory data. All the data flowing through the factory should be owned and housed by your company.
3. Compute inflexibility: you probably want a variety of different hosting solutions, including self-hosting.
4. Forced token reselling: vendors should allow you to bring your own inference endpoints (e.g. Bedrock, Vertex and Azure) and generally not be in the business of selling or re-selling tokens. Folks in the token selling business are going to have a conflict of interest when it comes to optimizing your costs, so it’s better to pick a vendor that is agnostic to whether you buy inference through them.

If you are looking for more info on how to stand up your own cloud factory, we’d be excited to explore this with you – you can set up time for an initial chat here, or explore setting up a factory yourself here.

Illustrations by Dan Roam.