Agentic Development Is Accelerating Delivery. Infrastructure Optimization Has to Keep Up.

AI-assisted and agentic development is changing how quickly engineering teams can build and ship software. Features get implemented faster, services appear faster, infrastructure changes can be generated alongside application code, and engineers can move from an idea to production with much less friction than a few years ago.

That is a real productivity gain, but it creates a second-order problem: the speed at which we create and change infrastructure can exceed the speed at which we go back and question whether that infrastructure is still sized correctly. The result is usually not spectacular waste or an obviously abandoned cluster. More often, production, staging, and development environments quietly accumulate excess capacity over time.

Infrastructure decisions are often conservative for good reasons

When a service starts struggling under load, the safe engineering decision is usually to give it more room. Replicas go from six to ten, a database moves from four vCPUs to eight, a workload gets more memory, a node pool gets a higher minimum size, or a queue worker gets more concurrency.

Those decisions are often completely reasonable when they are made. Reliability matters more than squeezing a few percent out of the cloud bill during an incident or a growth phase.

The problem is what happens afterwards. Three months later, application code may have changed, queries may have been optimized, a memory leak may have been fixed, traffic patterns may have shifted, or parts of the workload may have moved elsewhere. The infrastructure configuration often stays exactly where it was because nobody has an operational reason to revisit it.

There is rarely a ticket saying: “In six weeks, check whether we still need those four extra replicas.”

As engineering velocity increases, teams make more of these decisions and make them faster. The number of infrastructure assumptions that deserve another look grows with them.

Agentic development can amplify safe defaults

An agent asked to deploy a new service will generally optimize for producing something that works. A configuration like this may be perfectly reasonable as a starting point:

resources:
  requests:
    cpu: "2"
    memory: "4Gi"

replicas: 4

The deployment succeeds, the service behaves correctly, and the team moves on to the next feature.

A month later, the actual workload might show roughly 620m CPU at p95, 1.8 GiB of memory at p95, and little evidence that four permanently available replicas are necessary under normal load. Nothing is broken, so there is no operational pressure to revisit the configuration.

One service running with extra headroom is not particularly interesting. Forty or eighty services doing the same thing can become a meaningful percentage of the cloud bill.

This is not a new problem created by AI. Engineers have deliberately over-provisioned systems for decades. What has changed is the speed at which we can create software and infrastructure, while the feedback loop for reviewing those decisions often remains manual and infrequent.

Production infrastructure can drift just as easily as development

Production is where the most expensive assumptions tend to persist because teams are understandably cautious about changing something that already works.

Imagine a production service configured with 12 replicas, a 2-core CPU request, and 6 GiB of requested memory. That configuration may have been introduced during a period of unusually high traffic or while the service was substantially less efficient.

Thirty days later, the workload might look more like this:

CPU p50:        240m
CPU p95:        720m
Memory p95:     2.4 GiB
Typical replicas required: 5–7
Configured replicas:       12

The conclusion should not automatically be “cut everything in half.” There may be burst traffic, latency requirements, failure-domain considerations, or scheduled workloads that justify the capacity.

But there is enough evidence to ask why the configuration still looks this way.

That distinction matters. Optimization should begin with investigation, not automated downsizing.

Staging often inherits production economics without production traffic

Staging environments are another common source of quiet over-scaling because they are frequently designed to resemble production. That is useful when engineers need realistic integration testing, performance testing, or release validation, but similarity can easily turn into unnecessary parity.

Suppose production runs 20 application replicas and a large managed database because it serves millions of requests. Staging may run eight replicas and a database only one size smaller, despite receiving less than 5% of production traffic.

The architecture looks familiar and safe, but the economics are poor.

The same pattern can appear in Kubernetes node pools, database replicas, Redis clusters, search infrastructure, message brokers, observability stacks, and load balancers. Configuration is copied because consistency is operationally convenient, even when workload characteristics are completely different.

A useful question for every non-production environment is therefore not whether it looks like production, but which parts actually need production-like capacity.

Development infrastructure tends to become permanent

Development environments often accumulate for a different reason: nobody has a strong incentive to optimize infrastructure that is comparatively cheap on an individual-resource basis.

A development database costing €180 per month does not usually trigger a discussion. Neither does a €90 Redis instance, a few idle workers, or a mostly empty Kubernetes node.

Across several teams, however, the numbers compound quickly.

Consider an organization with eight development environments, each costing roughly:

Kubernetes capacity:   €550/month
Database:              €260/month
Cache and queues:      €130/month
Observability:         €110/month
Other services:        €150/month

Total per environment: €1,200/month

Eight environments put the annualized cost at roughly €115,000.

If those environments are active primarily during working hours and many of their resources run below 10–15% utilization, the question is no longer whether each individual resource is expensive. The question is whether the environment design still matches how development actually happens.

Autoscaling does not automatically mean optimized infrastructure

Autoscaling can make infrastructure look dynamic while preserving old assumptions underneath.

Consider a service configured with:

minReplicas: 8
maxReplicas: 40
targetCPUUtilization: 60%

An engineering improvement later reduces CPU consumption by 45%. Under normal traffic, four replicas would now be sufficient, but the autoscaler will never get there because the minimum is still eight.

The application became substantially more efficient, yet the infrastructure cost barely moved.

The same thing happens with node-pool minimums, memory requests, database tiers, queue worker counts, scheduled scaling, and reserved capacity. Infrastructure settings encode assumptions about workload behavior, and those assumptions do not update themselves when the application changes.

A useful optimization practice is therefore to revisit scaling configuration whenever workload behavior changes materially, not only when infrastructure is failing.

Look for configuration that has outlived its reason

One of the most useful questions in infrastructure optimization is simply: why is this configured this way?

If the answer is clear and still valid, there may be nothing to do. If nobody knows, or the explanation refers to a problem that was fixed six months ago, the configuration deserves another look.

This is where cost, utilization, infrastructure history, and deployment history become much more useful together than they are individually.

Suppose Kubernetes compute spend increases from €18,000 to €24,500 per month. The important question is not only which nodes are expensive, but what changed around the same time.

Maybe a new service was deployed with oversized requests. Maybe minimum node-pool capacity increased. Maybe staging was expanded to reproduce a production issue and never scaled back down. Maybe a database resize solved an incident and became permanent.

Cost changes become much easier to act on when they can be connected to engineering decisions.

Avoid rightsizing from averages

Average utilization is useful for finding candidates, but it is a poor basis for making production changes on its own.

A workload requesting 2 CPU and 4 GiB of memory may average only 420m CPU and 1.7 GiB over seven days. That looks dramatically oversized until you discover that p95 utilization reaches 1.4 CPU and 3.2 GiB, with a daily batch workload reaching 1.8 CPU and 3.6 GiB.

The recommendation is no longer to reduce capacity by 70 or 80 percent. A smaller first change may be appropriate, or there may be good reasons not to change it at all.

Good rightsizing decisions need several signals: p50, p95 and peak utilization, throttling, OOM events, traffic patterns, scheduled workloads, latency requirements, recent deployments, and the consequences of getting the recommendation wrong.

Low average utilization is evidence worth investigating. It is not proof that infrastructure is wasteful.

Measure infrastructure against useful output

The total cloud bill also needs context.

If revenue doubles while infrastructure cost grows by 30%, the system may be becoming substantially more efficient. If customer traffic stays mostly flat while infrastructure spend doubles, the story is different.

Useful efficiency metrics depend on the workload, but I like ratios such as:

• cloud cost per active customer
• infrastructure cost per production request
• database cost per transaction
• compute cost per background job
• observability cost per service
• development and staging cost relative to production

For AI systems, similar thinking applies to cost per inference, cost per million tokens, cost per training run, or cost per processed dataset.

The important part is connecting infrastructure consumption to something useful the system actually produces. Without that denominator, a 20% increase in the cloud bill does not tell you whether infrastructure is becoming less efficient.

Build an optimization loop alongside the delivery loop

Teams increasingly have sophisticated software-delivery loops, but infrastructure optimization is still often treated as a quarterly exercise.

I think it needs a lightweight continuous loop of its own: detect meaningful changes in cost or utilization, correlate them with deployments and configuration changes, identify suspiciously inefficient workloads, estimate the financial impact, validate operational risk, implement the highest-value changes, and then verify whether the expected savings actually appeared.

Verification matters because theoretical savings and realized savings are not always the same. If a change was expected to save €2,000 per month and the bill moved by €200, that difference contains useful information about the architecture or the original assumption.

The goal should also not be to optimize everything. A €50 monthly saving that requires redesigning a stateful production workload is probably irrelevant. A service costing €3,000 per month because it still carries scaling assumptions from a year ago may deserve attention immediately.

Infrastructure optimization is ultimately a prioritization problem.

Make infrastructure efficiency part of operational excellence

One of the best ways to prevent infrastructure drift is to make efficiency part of the normal operational excellence process rather than treating it as a separate cost-cutting exercise.

Most teams already review reliability indicators such as error rates, latency, saturation, failed jobs, queue depth, database health, and incident trends. I would extend that same review to include signals that show when infrastructure is consistently running outside an efficient range — both because it is over-utilized and becoming a reliability risk, and because it is significantly under-utilized and carrying unnecessary cost.

The exact metrics depend on the platform, but a useful recurring review might include:

• CPU and memory p50, p95, and peak utilization versus requests and limits
• Kubernetes replica counts and how often workloads actually approach their configured minimums and maximums
• node-pool utilization, scheduling pressure, and unused allocatable capacity
• database CPU, memory, connection usage, storage growth, and IOPS against the provisioned tier
• queue depth, worker utilization, and concurrency
• autoscaling events and whether scaling boundaries are routinely hit — or never approached
• development and staging utilization compared with their production equivalents
• cost per request, transaction, customer, job, or other meaningful unit of output

The important part is reviewing these metrics against an expected operating range, not just waiting for something to break. A database running continuously above 85% CPU deserves attention, but so does one provisioned for 16 vCPUs that rarely exceeds 15%. A Kubernetes service repeatedly hitting its maximum replica count may need more capacity, while another sitting permanently at minReplicas: 10 with 20% utilization deserves a different conversation.

I would make this a regular engineering review — monthly for relatively stable systems and more frequently for fast-moving environments. The output should be a small set of concrete actions with owners: resize this database, lower this node-pool minimum, investigate this memory trend, change this autoscaling target, or deliberately leave this workload alone because the additional headroom is justified.

That creates an important shift in mindset. Infrastructure efficiency stops being something the team looks at when finance complains about the bill and becomes part of operating the system well.

Faster software needs faster infrastructure feedback

Agentic development does not automatically make infrastructure inefficient. What it changes is the rate at which software, workloads, and the assumptions behind them can evolve.

A team can now create services faster, modify applications faster, scale systems faster, and provision infrastructure faster. Production, staging, and development environments can therefore drift away from what they actually need more quickly as well.

I do not think the answer is to slow developers down or introduce more approval gates. The better answer is to make infrastructure feedback keep pace with engineering velocity.

As software delivery accelerates, teams need to get better at continuously asking whether yesterday’s infrastructure assumptions still make sense today.