AI Is a Multiplier, Not a Signature

When I put this team together, I wanted experienced engineers who could take a problem and run with it. AI was part of that decision. I expected it to multiply what a small team could do.

A year in, I can see it in what a senior engineer can build in a week. Exploring an approach, producing something we can try, and working through the changes afterwards all take less time than they did. We can get to a working version while the idea is still taking shape.

The internal dashboard I built took days. It brought together views of delivery, risk and the people doing the work. I could build a tool around a problem I was trying to solve while still carrying the rest of the job.

The first version mostly helped me. Nobody else opened it. Later, giving it a view of goal status for stand-up gave the team a reason to use it. AI helped me get to a real tool quickly. Finding a useful place for it in the team's day took longer.

Help with the rest of the job

I also use the assistant for the reading and preparation around a decision. I need to understand what already exists, find the relevant context and work out which question is still unanswered. It can bring those pieces together so I have somewhere to start.

I want links back to the files or records it used. I can then open the part that matters and check it myself. A short explanation or diagram can help me work through an approach too. I can ask the assistant to challenge an assumption or point out a question I have missed, while I am still figuring out what I think.

For repeatable work, I have been writing down the instructions: what to look at, what to do with it, and what needs to come back to a person. The assistant can follow those instructions, and I can inspect the result. Someone else can also read the procedure and take over the task. That has been useful in a job that keeps pulling me between engineering, planning and running the company.

From the prototype to real use

Our product started with a vibe-coded prototype. Once we had design partners and real data, we could see how much there was still to learn. We shortened our planning cycles so we could try things, check the result and adjust sooner.

That is where I find the speed most useful. We can get something in front of us and learn from it. Before we rely on it for customers, I still need an engineer to read the implementation and work through what the demo did not exercise: how it handles failures, protects data and fits the rest of the system.

I don't assume code is poor quality because AI wrote it. It can be useful, well-tested production code. I want to judge the actual change. Does it meet the requirement? Does it belong in this codebase? Can someone else understand it well enough to change it later?

AI can help with that work too: propose tests, explain unfamiliar code, suggest a simpler implementation or work through a failure. The engineer still needs to check the reasoning. If the assistant writes both the implementation and its tests, I want someone to check that they have not simply repeated the same mistaken assumption.

That is what I wanted the team's experience for. An engineer who can frame a problem and recognise when an approach needs changing has more room to work with these tools. My part is to explain the outcome and constraints, then answer when they find a gap. If I leave that question waiting, the code can move quickly and the work can still get stuck with me.

854 pull requests in thirty days

In one thirty-day stretch, our team of roughly ten engineers merged 854 pull requests across our repositories. AI had become part of how we built, and that was the volume of changes moving through the team.

Those were not 854 features or production releases, and the count does not tell me how much of the code was AI-written. They were changes we needed to understand, test and integrate. I had wanted more capacity to build; now I needed the work around each change to keep up.

Getting the changes through

One practical change came from an engineer who added a check that fails pull requests over four hundred lines, to push the coding agent towards smaller, stacked changes. It gives the agent a boundary and gives a person a more manageable change to examine.

We also built a review service using several models to look at correctness, security, tests and the reach of a change. It raises findings for the engineer to investigate and address. That lets us use AI to help work through the increased volume as well as produce it.

As we grow, I want more of that help available to everyone. An agent should be able to find our existing interfaces and patterns, run our tests and build, and work in an environment where it cannot affect production. Shared instructions and clear access limits would save each engineer having to set that up again. I also want to see how much review and rework it takes to get a change finished as we try different models.

The person submitting the change still needs to explain what it does and why, and be willing to merge it with their name on it. A model finding can point them to a problem; they have to decide whether the fix actually deals with it.

The next step is getting that work into production. We build for finance, so I need to be able to show how a change got there and who took responsibility for it. We have been reshaping review and release around independent human approval before production. We are still putting the full process into the tooling.

The small-change check and the review service give us specific ways to handle the volume now. I want to keep the speed we gained in building, and make it easier for an engineer to carry a change all the way through to something a customer can use.

A few useful reads

• Simon Willison, Not all AI-assisted programming is vibe coding (but vibe coding rocks). On the difference between exploring with generated code and taking responsibility for code you understand.
• Birgitta Böckeler, The role of developer skills in agentic coding. Examples of the framing, steering and correction involved in working well with an agent.
• GitHub, Does GitHub Copilot improve code quality?. A vendor-run controlled study finding quality gains on a bounded API task; it does not establish long-term production quality.
• Stripe, Minions: one-shot, end-to-end coding agents, part 2. How Stripe combines coding agents with repository context, automated checks and human review.
• Uber, Running a Software Factory Efficiently at Uber Scale. A firsthand account of operating and evaluating agents across development, review and maintenance.
• DORA, Balancing AI tensions: moving from adoption to effective use. Research on the benefits engineers report and the verification work that accompanies faster generation.
• Anthropic, How AI is transforming work at Anthropic. An internal study of coding, code understanding and work engineers previously lacked time to attempt; its productivity estimates are self-reported.