Every developer who has integrated session-replay and behavioral analytics tools (Hotjar, FullStory, Smartlook, or local equivalents) remembers the moment the sysadmin walks over wide-eyed with a disk-usage graph in Grafana. Within a couple of weeks, the database or S3 object store balloons to hundreds of gigabytes, and cloud infrastructure bills start climbing exponentially.
A team's first instinctive reaction is to set a hard time-to-live in the config file for all incoming events:
retention_ttl_days: 30 # Delete absolutely all records older than one month
It looks like the problem is solved: disk appetite is stabilized, and old data is purged on schedule. In reality, this flat approach proves both ruinously expensive for the budget and crippling for technical support.
Let's examine the engineering experience of the Broux team, who measured the real footprint of hundreds of thousands of sessions, and see how a well-designed two-tier retention policy works.
The Flat TTL Trap: How "Delete Everything After 30 Days" Breaks Incident Investigation
Why is a single retention window a classic architectural trap? It suffers from two mutually exclusive flaws:
On one hand, it is excessively wasteful. The lion's share of disk space (up to 97%) is consumed by heavy DOM-mutation streams. Yet let's ask a simple question: when do engineers or product managers actually watch a recording of a customer session?
According to Broux's stats, 90% of views happen within the first 72 hours after an incident or a support ticket. Two weeks later, the odds of anyone opening an old screen recording drop practically to zero. Keeping hundreds of gigabytes of heavy video-like streams on fast storage for a full month is an unaffordable luxury.
On the other hand, this policy is destructively stingy. Along with layout mutations, lightweight metadata goes under the knife: error stack traces, browser console logs, network response codes, and user click paths.
Imagine a real-world scenario: developers track down an elusive intermittent bug in March, fix it, and move on. In May, a major release ships and accidentally brings back the old bug (a regression). An engineer opens monitoring, yet with a 30-day TTL the system treats the error as brand new:
- The history of the failure's trend and frequency is lost;
- The list of customers who hit the bug two months ago is gone;
- The link to historical commits is severed.
The developer starts the investigation from scratch, burning hours on something already analyzed back in spring.
Three Layers of a User Session: Where the Gigabytes of DOM Mutations Hide
To design efficient storage, you need to look inside a user session. Modern tools build recording on top of the open rrweb library standard. The client script captures a full DOM snapshot when a tab opens and then continuously writes a differential event stream.
Session data splits cleanly into three layers with a massive gap in volume:
- Layer 1: Layout Event Stream:
Includes serialized DOM node mutations, mouse cursor movements quantized every 50 milliseconds, scroll events, form inputs (with private data masked), and inlined CSS stylesheets.
- Median compressed session size is around 220 KB;
- One in ten sessions exceeds 1 MB;
- On dynamic SPAs with infinite catalog scrolling, session size easily shoots past 20 MB. This layer alone accounts for up to 97% of total disk storage.
- Layer 2: Technical Layer (Diagnostics):
Includes
console.logandconsole.errormessages, XHR/Fetch network calls with status codes, and JavaScript exception stack traces.- Across 100,000 sessions, the technical layer took only 1.3 GB—45x smaller than the first layer!
- An interesting paradox: developers often assume console logs flood the database. In practice, 90% of this layer's volume (1.2 GB out of 1.3 GB) came from network headers and request URLs, whereas console logs took a modest 78 MB.
- Layer 3: Derived Analytical Data: Classified error groups, rage-click detectors, dead clicks, and client profiles. Across 100,000 sessions, this layer weighs only around 100 MB—hundreds of times smaller than recordings! Yet it delivers the bulk of the value for long-term application stability analysis.
The Heatmap Paradox: Why Layout Snapshots Are Three Times Heavier Than Console Logs
You might think a click heatmap would be nearly weightless: just a pair of numeric (X, Y) coordinates on screen. Click coordinates from hundreds of thousands of users really do take less than half a gigabyte.
Yet a heatmap cannot be rendered in a vacuum: it needs a background canvas—a visual snapshot of the page.
Here engineers run into the dynamic nature of the modern web. Sites change constantly: new banners roll out, A/B tests run, layouts adapt to smartphones and 4K monitors. Overlaying clicks onto an outdated snapshot leaves dots floating in empty space:
- At Broux, snapshots were split by device class (Mobile, Tablet, Desktop);
- For each "URL + device" pair, the database maintains a sliding pool of 3 reference snapshots;
- Snapshots are ranked by a scoring algorithm: overlapping modal windows, active loading spinners, or unloaded blocks lower a canvas's score;
- Snapshots older than a week receive a time penalty, so a fresh snapshot automatically displaces an old one after a release rolls out.
Still, a single high-quality layout reference with inlined styles weighs between 100 KB and 1 MB. On an e-commerce store with 5,000 unique URLs, the snapshot pool grows to 12,000 items and weighs 2.5 GB. Altogether, heatmap canvases took 4.3 GB—three times as much as the entire technical diagnostics layer!
Two-Tier Retention in Practice: How to Save 90% of Disk Space Without Losing Analytics
Once they understood this anatomy, the engineers moved to a split-retention architecture grounded in fundamental system design patterns:
- Short-Lived Bucket (TTL 14–30 days): Holds the "heavyweights": raw rrweb DOM-mutation streams and the sliding pool of heatmap canvases. They answer the question: "What exactly did the user see at the moment of the error?" That question is only relevant while an incident is still fresh. Rotating this data frees up to 95% of disk space.
- Long-Term Bucket (TTL 6–12 months or indefinite): Holds the technical layer: error logs, stack traces, network failures, user profiles, and UX anomalies. They answer the question: "Has this failure happened before, and which customers did it affect?"
Because the lightweight layer is two orders of magnitude smaller than video-like recordings, keeping it long-term increases total storage by only 2–3%, while preserving the full diagnostic picture for analytics and regression hunting.
This differentiation proves there is no place for blunt, flat policies in infrastructure management. Understanding the nature and value of every byte lets you slash disk costs several times over without sacrificing incident-investigation quality.

![Node.JS [ru]](/api/digests/it_development/daily/20261105/assets/sources/we-use-js.jpg?v=dd82e88a7e726190393fcef7d544ee35baed63c81bf03d147365326a808ee3c1)