Eliminating Allocation Bottlenecks: V8's Mutable Heap Number Optimization

Introduction

In the ongoing quest to make JavaScript run faster, the V8 team continuously analyzes performance benchmarks to identify and eliminate hidden slowdowns. Recently, a deep dive into the JetStream2 benchmark suite revealed a surprising performance cliff in the async-fs benchmark. By introducing a targeted optimization—mutable heap numbers—V8 achieved a remarkable 2.5x speedup on that specific benchmark, contributing to an overall score improvement. This article explores the technical details behind the bottleneck and how a creative change to V8's internal number representation made such a difference.

The Problem: Seed Allocation in Math.random

The async-fs benchmark simulates an asynchronous file system in JavaScript. While its core logic involves I/O operations, profiling unexpectedly pointed to a different hotspot: the custom Math.random implementation used for deterministic test runs. The relevant code looks like this:

let seed;
Math.random = (function() {
  return function () {
    seed = ((seed + 0x7ed55d16) + (seed << 12)) & 0xffffffff;
    seed = ((seed ^ 0xc761c23c) ^ (seed >>> 19)) & 0xffffffff;
    seed = ((seed + 0x165667b1) + (seed << 5)) & 0xffffffff;
    seed = ((seed + 0xd3a2646c) ^ (seed << 9)) & 0xffffffff;
    seed = ((seed + 0xfd7046c5) + (seed << 3)) & 0xffffffff;
    seed = ((seed ^ 0xb55a4f09) ^ (seed >>> 16)) & 0xffffffff;
    return (seed & 0xfffffff) / 0x10000000;
  };
})();

The variable seed is read and written on every call. Initially a simple number, it becomes larger than 2^31 quickly and thus cannot fit into V8's Small Integer (SMI) range. Instead, it must be stored as a HeapNumber—an immutable 64-bit floating-point value allocated on the garbage-collected heap. Each time the function updates seed, a new HeapNumber must be created, even though only a new numeric value is needed.

Understanding V8's ScriptContext and Number Tagging
To appreciate the bottleneck, we need to understand how V8 stores variables like seed. Variables declared in script scope live inside a ScriptContext. Internally, a ScriptContext is an array of V8's tagged values. On a typical 64-bit V8 build, each entry occupies only 32 bits of memory (compressed representation). The least significant bit acts as a tag:

Tag 0: 31-bit signed integer (SMI) stored directly, with the actual value left-shifted by one bit.
Tag 1: A compressed pointer to a heap object, with the pointer incremented by one.

For numbers, if the value fits in 31 bits (including integers up to 2^31-1), it is stored as an SMI directly in the ScriptContext slot. Otherwise, the slot contains a pointer to an immutable HeapNumber on the heap that holds the full 64-bit IEEE 754 double.
In the case of seed, the number quickly exceeds SMI range, so it becomes a pointer to a HeapNumber. Because HeapNumbers are immutable, every assignment to seed forces V8 to allocate a brand new HeapNumber, populate it with the updated value, and store the pointer back into the ScriptContext slot.

The Performance Cliff
Profiling Math.random in the async-fs benchmark revealed two major issues:

Excessive HeapNumber allocation: Each call to Math.random produces a new HeapNumber for the updated seed. The allocation rate is high, putting pressure on the garbage collector and causing frequent minor collections.
Pointer chasing overhead: Reading seed requires dereferencing the pointer to the HeapNumber, adding an extra memory indirection compared to reading an SMI directly.

These two factors combined created a significant performance bottleneck, despite the benchmark's main focus being asynchronous I/O.

The Solution: Mutable Heap Numbers
The V8 team realized that the immutability of HeapNumbers was the root cause. While immutability helps with sharing and optimization in many scenarios, in this hot loop, it was wasteful. The team introduced a new internal optimization: mutable heap numbers. Under this scheme, when a number exceeds SMI range and is assigned repeatedly to the same variable (here, a ScriptContext slot), V8 can reuse the existing HeapNumber object and simply overwrite its 64-bit value in-place.
This change eliminates allocation entirely for such patterns. The ScriptContext slot continues to point to the same HeapNumber object; only the double value inside it changes. No new objects are created, no garbage is generated, and the memory indirection is preserved (though it remains minimal compared to allocation).

Impact on JetStream2
The mutable heap number optimization yielded a dramatic 2.5x speedup on the async-fs benchmark. Because the benchmark exercises Math.random heavily, the removal of allocation overhead directly translated into faster iteration. The overall JetStream2 score also saw a noticeable improvement, confirming that such patterns—though inspired by a benchmark—occur in real-world JavaScript code that uses custom PRNGs, simulation loops, or numeric state machines.

Real-World Implications
While the async-fs benchmark triggered this particular optimization, the fix benefits any JavaScript code that repeatedly assigns a non-SMI number to the same variable in a tight loop. Common examples include:

Custom random number generators (seeds and state variables).
Animation loops that compute fractional positions or colors.
Physics or game loops updating floating-point coordinates.
Any algorithm that maintains a large numeric state and modifies it frequently.

V8's mutable heap numbers essentially allow these patterns to run at near-optimal speed without triggering garbage collection churn. Developers do not need to change their code; the optimization is transparent and automatic.

Conclusion
By rethinking the assumption that V8's internal heap numbers must be immutable, the team unlocked a significant performance gain for common numeric patterns. This story illustrates how focusing on edge cases in benchmarks can lead to optimizations that improve everyday code as well. The mutable heap number optimization is now part of V8, silently boosting performance whenever a JavaScript developer writes a hot loop that mutates large numbers. As V8 continues to evolve, we can expect more such creative solutions that remove obstacles to faster execution.