Blog/Why Runtime Signals Beat Static Analysis
PerformanceEngineeringStatic Analysis

Why Runtime Signals Beat Static Analysis

Static analysis has served the industry for decades. But as systems grow more complex and agentic-generated code becomes the norm, the gap between "looks correct" and "runs correctly" is widening fast.

AL

Aurora Labs Engineering

Engineering

April 22, 20268 min read

Static analysis is one of the most productive investments a software team can make. Tools like Clang-Tidy, Coverity, SonarQube, and CodeQL have prevented billions of dollars in production failures. We are not here to argue against them.

We are here to argue that they are not enough — and to explain precisely where they fail.

What Static Analysis Actually Does

Static analysis reasons about code structure. It reads source files, builds a representation of the program, and applies rules to that representation. The rules can be simple (do not dereference a null pointer) or complex (does this path through the call graph always initialize this variable?).

It is a fundamentally syntactic and semantic activity. The analysis operates on text — or at best, on a model of what the text implies. It does not execute anything.

This is both its strength and its hard limit.

The Execution Gap

Here is the fundamental problem: correctness is a runtime property, not a source property.

A function that looks safe at rest can behave catastrophically under load. A module that passes all static checks can violate timing constraints on specific hardware. Code that satisfies every rule in your analyzer can exhaust a CPU's instruction pipeline in a way that costs 40% of your latency budget.

None of these failures are visible in source code. All of them are visible in execution traces.

Consider a real-world example. A firmware team introduces a change to a signal-processing pipeline — refactoring a tight loop to improve readability. Every static check passes. Tests pass. Code review approves.

In production, on the target ECU, the refactored loop now misses a real-time deadline by 3 milliseconds. The compiled binary's memory access pattern changed. The hardware prefetcher cannot keep up. Static analysis had no way to know.

LOCI would have flagged this before merge — because LCLM saw the timing deviation in the execution trace.

Three Classes of Bugs Static Analysis Cannot Catch

1. Emergent Performance Degradation

Performance is not a property of a function. It is a property of a function executing on specific hardware, with specific memory layout, under specific load, in interaction with specific OS scheduler behavior.

Static analyzers model none of this. They can flag O(n²) complexity in a loop. They cannot tell you that your 3% increase in cache misses will push p99 latency past your SLA at peak traffic.

Execution signals can.

2. Power and Thermal Anomalies

In embedded and automotive systems, power consumption is a first-class constraint. A firmware change that draws 15% more power during initialization can cause thermal throttling, battery drain, or safety-mode activation on the target device.

No static analyzer reasons about power. LCLM — trained on real CPU and GPU execution traces — learns power signatures as part of its execution model.

3. Control Flow Integrity Under Real Workloads

CFI tools can verify that control flow follows the expected call graph at the binary level. But the envelope of expected behavior — which execution paths activate under which conditions — is something that only emerges from observing the program run.

A code change that passes CFI checks at rest can introduce an unexpected hot path under production load that violates the execution model of the broader system.

How LOCI Fills the Gap

LOCI does not replace static analysis. It operates where static analysis stops.

The architecture is straightforward:

  1. LCLM generates an execution model for each module — a representation of how the compiled code behaves on real hardware, derived from training on billions of execution traces.
  2. Each commit is evaluated against the module's baseline execution model. LOCI computes a delta: what changed in timing, stack, power, CFI, and throughput.
  3. If the delta exceeds threshold, the commit is flagged. The developer sees the specific signals that deviated, not a vague warning.

The result is a quality gate that operates at the execution layer — not the source layer. It catches what static analysis misses because it reasons about a fundamentally different model of the program.

The Practical Workflow

For teams already running static analysis (and you should be), LOCI integrates as a complementary check in the same pipeline:

  • Static analysis runs first — catching source-level bugs early and cheaply.
  • LOCI runs at PR time — catching execution-level regressions before merge.
  • Neither replaces the other. Together, they cover the full quality surface.

The teams that operate most safely are the ones that understand both tools and deploy both layers. Source correctness and execution correctness are different properties. They require different instruments.

The Honest Bottom Line

Static analysis tells you whether your code is plausibly correct. Execution signals tell you whether your code actually behaves correctly — on real hardware, under real load, in the real execution environment.

As AI agents write more of your codebase and code velocity climbs, the gap between "plausibly correct" and "actually correct" becomes the gap between shipping safely and shipping incidents.

That is the gap LOCI is built to close.