Instrument Your Work

You shipped the feature. Six months later someone asks what it did. You reconstruct it from memory: the ticket, the PR, the rough sense that it went well. That reconstruction is the problem. Not the forgetting. The fact that you never built a way to know in the first place.

Most people conflate two different things: knowing what they built and knowing what it changed. These are not the same. A shipped feature is an output. Revenue unlocked, activation rate moved, latency cut in half. Those are outcomes. Technical health metrics (error rates, latency, uptime) tell you the system is working. Business metrics tell you the work mattered. Most instrumentation stops at the first kind and calls it done.

Two axes clarify the gap. The first is metric type: are you measuring the health of what you built, or the behavior it was meant to change? A dashboard showing p95 latency tells you the cache layer is running. A dashboard showing how often the workflow completes tells you whether the feature was worth building. Both matter. Only the second one tells you something about the decision to build it. The second axis is time horizon: how does your work look at ship time versus a quarter later? Most impact is invisible at ship. It shows up in the accumulation. Closing the loop closes the gap between those two moments.

Define what success looks like in numbers before you ship, not after. “Shipped billing v2, drove $400K in new ARR by Q3.” That sentence is only possible if you decided what to measure before the feature went out.

Every metric should be an argument. “Strong growth across the board” doesn’t change a decision. “Median wait fell from 15 seconds to 8 seconds while volume grew 8x” does. If flipping the number wouldn’t flip your recommendation, the metric is decoration. Cut it.

Instrument the failures too. “Built team notifications, usage plateaued at 12 DAU, recommended sunsetting it” is more valuable than most wins. It tells you something about what not to build next quarter.

“How would I know this is working?” should be answerable before the feature ships. If you can’t answer it then, you won’t be able to answer it later.

Closing the loop builds product sense that books and frameworks can’t give you. You develop it by being right and wrong about your own work, repeatedly, with the data to know which is which.

Before you ship, ask these. What business metric moves if this works? What would tell me it failed? If someone asked me in six months what this did, what number would I point to? After you ship: can I answer what it changed without reconstructing from memory?

One edge case: not everything warrants instrumentation. A proof of concept, a one-off migration, exploratory tooling, something that runs once. The signal is whether anyone would make a decision based on this data. If no, skip it. The goal isn’t to measure everything. It’s to never be in a position where impact is invisible on purpose.

Anyone can list what they built. The person who knows what it changed is the one whose judgment compounds.