Why does generated code need more observability than hand-written code?

Because debugging normally uses a mental model that someone built while writing the system, and with generated code nobody built one. The information that used to sit in an engineer's memory has to be recorded in production instead, which means the telemetry has to carry the reasoning the author would otherwise supply from recall.

What should you instrument first in a generated codebase?

Start from the business guarantees rather than the endpoints: anything that moves money, any permission or tenancy decision, anything that writes or deletes customer data, and any number a customer acts on. Those are the paths where a silent failure is expensive, and they are worth full request-level detail before anything else gets a dashboard.

Why are model-generated log lines usually not enough?

Because a log line is a prediction about which questions will matter later, and a model does not know which of your failures are costly. It emits reasonable-looking messages at reasonable-looking places, generally without the identifiers that let you follow one request across services, so the volume rises while the number of questions you can answer does not.

How do you test whether your observability is actually good?

Write down three real questions you would need answered during an incident, then try to answer them from the existing telemetry while timing yourself. If any of the three requires adding a log line and waiting for it to reproduce, the instrumentation has failed the only test that matters.

What is the single most valuable field to add to logs?

A request or trace identifier generated where the request first enters the system and carried through every service, queue, and background job the request touches. Without it you have a set of separate events, and with it you can reconstruct one user's actual path, which is the difference between guessing and reading.

Does adding more logging make debugging easier?

Not by itself, and past a certain volume it makes things worse, because search cost rises and the important line hides among the routine ones. What helps is structured events with consistent field names and high-cardinality identifiers, so you can ask a question rather than scroll.

How does observability relate to the tests and evals in CI?

They cover different halves of the same problem: checks catch the failures you predicted before release, and telemetry catches the ones you did not predict after it. A team that has invested heavily in one and not the other usually finds out during an incident that nobody thought to ask about, which is exactly the category checks cannot cover.

What is a silent failure and how do you find one?

A failure that produces no error, no alert, and no complaint, such as a nightly job that stopped running, a retry that hides an exception, or a filter that quietly returns fewer rows than it should. You find them by asking what could be wrong right now with nobody noticing, then instrumenting those specific paths with expectations rather than waiting for a report.

Should the model write its own instrumentation?

It can produce the mechanical parts, but the decision about what is worth recording has to come from someone who knows which failures cost the business money. We treat the observability requirements as part of the specification, written before the code, so the instrumentation reflects the actual risk rather than a generic pattern.

How much telemetry is enough for a small team?

Enough to answer, without new deploys, what happened to one specific user request and whether each critical guarantee held. That is usually a trace identifier, structured events on the guarantee paths, and alerts tied to business outcomes rather than to CPU, which is far less than a full platform and covers most real incidents.

What changes about on-call when an agent wrote the service?

The first responder cannot ask the author anything, so the runbook and the telemetry have to provide the answers the author used to give. We write the failure modes and the recovery steps as part of delivery, because an on-call engineer discovering an unfamiliar service at 2am is the most expensive place to learn it.

Why observability matters more when a machine wrote the code: Reveneau

There is an input to debugging that never appears in any diagram, and it is the one that matters most.

Someone remembers.

An engineer looks at a latency graph and a thought comes without effort: that queue backed up the same way in March, and it turned out to be the retry loop in the payment adapter. That thought is not in the code. It is not in the logs. It exists in the memory of a person who wrote the thing, was there when it broke, and now has a summary understanding of how the system behaves under stress.

When a model writes the code, that input is gone. The code is still there. The mental model was never built, because nobody built the code.

This is the part of AI-assisted development that gets discussed least and costs the most, and it shows up months after the delivery date, in a production incident, at an hour nobody enjoys. Here is what actually changes, and what to do about it.

1. Debugging has always depended on memory

Watch a senior engineer work through an incident and count how much of it comes from the screen.

They check one graph. They skip four others without looking, because they already know those are never the cause. They go straight to a specific service, not because the evidence pointed there but because the pattern of the symptom is familiar. Then they find it in eleven minutes and everyone assumes the tooling was good.

The tooling was ordinary. The engineer was carrying a model built over two years of writing that code and being woken by it.

Now remove the writing. The code arrives correct-looking, uniformly formatted, and unfamiliar. The person on call has read it once, in review, four months ago. Their guess about the location of the bug is worth almost nothing, so they do the thing you do when expectations fail: they read logs from the beginning. That is the expensive path, and it is now the default path.

Stack Overflow's 2025 developer survey gives a related number. Among developers using AI tools, 45.2 percent said debugging AI-generated code is more time-consuming than writing it themselves, and 66 percent named output that is "almost right, but not quite" as their biggest frustration. Almost-right code is exactly the kind that fails in production rather than in CI, which is to say it fails where only telemetry can see it.

The practical conclusion: the understanding that used to come at no cost now has to be paid for, and observability is how you pay for it.

2. A log line is a prediction, and the model cannot make it

This is the mechanism, and it explains why generated code tends to be under-instrumented even when it appears to log plenty.

Every log line encodes a guess about the future. Someone decided that when this thing happens, a human will one day want to know, and these are the fields they will need. That decision requires knowing which of your failures are expensive. A wrong invoice total might end a customer relationship. A slow dashboard is nobody's problem. The code cannot tell the difference and neither can a model reading it.

So what you get is plausible instrumentation. Log statements in reasonable places, at reasonable levels, with reasonable messages. What is missing is specific and consistent: the identifier that lets you follow one request across four services, the field that says which tenant this was, the value of the input that caused the branch, the timing around the external call that actually failed.

You end up with more log volume and fewer questions you can answer, which is the worst combination, because the cost is visible on the bill and the deficiency is invisible until an incident.

The fix is to treat observability requirements as part of the specification rather than as something added afterwards. Before the code exists, write down what has to be reconstructable from production data. That is the same discipline as writing specs an agent can verify, applied to the runtime instead of the test suite.

3. Instrument the guarantees, not the endpoints

The common approach is to instrument by structure: a dashboard per service, a latency chart per endpoint, an error rate per route. It produces a lot of screens and answers very few questions.

Start from the business risk instead. Ask which failures would be serious for the business, then make each one observable.

Money moved, or should have moved and did not.
A permission decision, with the inputs that produced it.
Data written, updated, or deleted, and by whom.
A number a customer will act on.
A scheduled job that has to run, with an expectation that it did.

Each of these gets enough detail to reconstruct the event afterwards without deploying anything new. That last clause is the whole standard. If answering a real question requires adding a log line and waiting for the problem to happen again, the instrumentation has already failed.

The test is easy to run and uncomfortable to fail. Write three questions an incident would force you to answer. Then use the existing telemetry and try to answer them, with a timer. Most teams discover the answer to at least one is "we would add logging and wait", and they discover it on a Tuesday afternoon rather than during an outage, which is the point of doing it deliberately.

4. One identifier, carried everywhere

If you take one thing from this, take this one, because it is cheap and it changes how every investigation works.

Generate an identifier where the request enters your system, attach it to the request, and carry it through every service call, every queue message, every background job, and every log line the work touches. Put it in the response header so support can quote it from a customer report.

Without it, you have separate events. With it, you can follow one request from start to end. The difference is that the first requires you to guess which events belong together, using timestamps and guesswork, and the second lets you ask the system directly.

This matters more in generated codebases for a structural reason. Generated services tend to be consistent in style and inconsistent in convention across sessions, so the naming of context fields changes unless you fix it in the specification. Fix the field names once, check them in CI, and the names stop changing. Related work on making the checks themselves reliable is in evals in CI for coding agents.

5. Go looking for the silent failures

The obvious failures already send you alerts. Nobody needs help with a service returning 500s.

The dangerous category is different: the nightly reconciliation that stopped running six weeks ago, the retry that catches an exception and returns an empty list, the filter that silently excludes records because a field was null, the webhook handler that acknowledges and then drops. These produce no error, no alert, and no complaint, until a customer notices something that has been wrong for a month.

Generated code often produces this failure mode, and the reason is understandable, with no bad intent behind it. Broad exception handling that returns a safe default is a reasonable-looking pattern, it appears throughout training data, and it passes review because each instance looks careful. In aggregate it produces a system that hides its own problems.

So audit for it directly. Search for the places where an error is caught and not re-raised, and ask what a caller would see. Add a counter to each one. A hidden exception that increments a metric is no longer silent, and that is a small change with a large effect on how much of your system you can actually see. The broader review pattern is in how to review AI-generated code.

6. Write the runbook while the context still exists

On-call has always relied on being able to find the person who wrote it. Generated services break that quietly, because there is no such person and the reviewer has moved on.

So the delivery has to include it: what this service does, what it depends on, the three ways it has failed or plausibly could, what the alerts mean, and how to roll back. Two pages, written while somebody still has the context, is worth more than a full architecture document written a year later by someone reconstructing it.

We treat this as part of delivery rather than as documentation debt, for the same reason we treat the checks that way. Anything left to be written later is written by a person who no longer knows.

What this costs, honestly

Instrumenting properly is work, it happens before the payoff, and it is the first item cut when a delivery date moves. It also does not feel urgent, because a system with poor observability behaves identically to a well-observed one until the moment something goes wrong.

The argument for funding it is simple enough to say in a planning meeting. If you are producing code faster than anyone can build a mental model of it, production is the only place where you can see what the system actually does. What you can see there limits how fast you can fix anything.

Thanks to the on-call engineers who told us about their worst nights and let us ask what they wished had been in the logs. The answer was almost never a missing dashboard. It was a missing identifier, and an hour spent proving which events belonged to the same request.

Code used to come with someone who understood it. Now it comes with whatever you decided to record.

Related guide: Making AI agents reliable.

Sources

Stack Overflow 2025 Developer Survey, AI section: 45.2 percent of developers report debugging AI-generated code is more time-consuming than writing it themselves; 66 percent name "almost right, but not quite" as their biggest frustration with AI tools.

Why observability matters more when a machine wrote the code