Isotherm

Detecting a redesign without being told: DOM structure fingerprinting

The SDK hashes the DOM skeleton with runs of identical siblings folded to `tag*`, so a new column forks the heatmap version but a 25th table row does not.

A heatmap that averages clicks from before and after a redesign is worse than no heatmap, because it still renders. You get a picture of a page that never existed, and nothing about it looks broken. The button that everyone clicked has moved 200px left, half your clicks land on empty background, and the map quietly tells you the users are confused.

The obvious fixes all put the burden on a human. Put a version string in the snippet and have the client bump it on deploy. Add a "start new version" button in the dashboard. Both fail identically: someone forgets, the data keeps flowing, and the failure is silent. The one thing you can be sure of is that the person shipping a CSS change at 6pm on a Friday is not thinking about your heatmap tool.

So the page declares its own identity. Isotherm's SDK computes a fingerprint of the DOM structure on every pageview and sends it with each batch of events; the server files events under a page_versions row keyed by that fingerprint. New structure, new row, on the deploy that caused it. Nobody declares anything.

The whole thing is 139 lines in sdk/src/fingerprint.ts, maybe 80 of them code. The interesting part is not the hashing. It is deciding what counts as "the same page".

The 25th row problem

Naively serializing the DOM into a string and hashing it produces a fingerprint that changes constantly. An orders table with 24 rows and the same table with 25 rows would hash differently. Every new customer order forks the heatmap. Within a week you have 400 versions of /admin/orders, each with three clicks in it, and the tool is useless.

The fix is that runs of siblings with the same signature collapse into one entry. Here is the actual walk:

function serialize(el: Element, depth: number): string {
  if (depth > MAX_DEPTH) return '';

  const children = Array.from(el.children)
    .filter((c) => !IGNORED_TAGS.has(c.tagName.toLowerCase()));

  const parts: string[] = [];
  let index = 0;

  while (index < children.length) {
    const child = children[index];
    const sig = signature(child);

    // How many siblings in a row share this signature?
    let run = 1;
    while (index + run < children.length && signature(children[index + run]) === sig) {
      run++;
    }

    const inner = serialize(child, depth + 1);
    const body = inner ? `(${inner})` : '';

    // "*" means "one or more" — deliberately not the count.
    parts.push(run > 1 ? `${sig}${body}*` : `${sig}${body}`);

    index += run;
  }

  return parts.join(',');
}

* means "one or more", not "24". That comment is load-bearing. The temptation is to emit the count, or a bucketed count ("1", "few", "many"), because more information feels safer. It isn't: a bucket boundary is just a fork waiting to happen when a table crosses it. A table is a table.

Note what the descent does: serialize(child, ...) runs on the first element of the run and nothing else. The subtree of rows two through twenty-four never enters the string. This is an assumption, not a proof, and it is worth being blunt about it, because signature() is child-blind — it is tag, stable classes, role, data-hm-id, and nothing below the element. Two <tr>s with the same classes have the same signature whether or not their insides match. So a table where the last row carries an extra <td> with an "Edit / Delete" action group, or a full-width colspan summary row that happens to share the row class, folds into a single tr* and the odd one out is dropped from the fingerprint entirely. Adding that action column to every row changes the fingerprint. Adding it to one row that already looked like its neighbours does not.

I kept it anyway. Walking every sibling of every run turns a cheap tree walk into something you can feel on a page with a thousand rows, and it does it on the hot path of a tracker whose entire job is to not be noticed. But it is a real hole, and it is the mirror image of the failure mode I chose — more on that below.

MAX_DEPTH is 12. Past that, structure stops meaning anything — you are down in the guts of a component library and a click doesn't care.

What a signature is

The other half of the problem is that a DOM node carries a lot of stuff that has nothing to do with layout. A signature keeps only what survives a page reload:

function signature(el: Element): string {
  const tag = el.tagName.toLowerCase();
  const role = el.getAttribute('role');
  const marked = el.getAttribute('data-hm-id');

  let sig = tag + stableClasses(el);
  if (role) sig += `[${role}]`;
  if (marked) sig += `#${marked}`;

  return sig;
}

No text. No id — half of them are radix-:r3h: or input-8821 and change on every render. No inline styles. role is in because landmark roles (navigation, main, dialog) are exactly the anchors a redesign moves around.

Classes are the hard case, because they are simultaneously the most useful structural signal and the noisiest. stableClasses filters twice. First, anything that looks machine-generated:

const VOLATILE_CLASS = /^(sc-|css-|jsx-|_|[a-z]+-[0-9a-f]{5,})|[0-9a-f]{6,}$/i;

That's styled-components (sc-bdVaJa), emotion (css-1x2y3z), Next's jsx-1234, CSS-module hashes, Tailwind JIT output. These rotate when the build rotates. Keeping them would fork the version on a dependency bump that changed nothing visible.

Second, state classes:

.filter((c) => !/^(is-|has-|active|open|hover|focus|selected|disabled)/.test(c))

This one bit me before I wrote it. .active on the current nav item means the fingerprint of /orders differs from the fingerprint of /customers — fine, they're different pages. But .open on a dropdown means the fingerprint depends on what the visitor happened to be doing with their mouse when we took it. Two visitors on the same page, one with a menu open, and you have two layout versions of the same screen. State is not structure.

Finally the kept classes are .sort()ed, because class="btn primary" and class="primary btn" are the same element and Vue and React will happily produce both.

There is one escape hatch. data-hm-id is folded into the signature verbatim, so a site owner can pin an element's identity across a refactor that changes its tag and all its classes. It is the only lever a human has inside the fingerprinting path, and it is opt-in per element rather than per page — which is the right shape for it, because you only reach for it when you already know something specific. The other levers live after the fact, in the dashboard.

FNV-1a, and why not SHA-256

The serialized string for a real admin page runs a few kilobytes. It gets hashed with FNV-1a:

function hash(input: string): string {
  let h = 0x811c9dc5;
  for (let i = 0; i < input.length; i++) {
    h ^= input.charCodeAt(i);
    h = Math.imul(h, 0x01000193);
  }
  let g = 0x811c9dc5;
  for (let i = input.length - 1; i >= 0; i--) {
    g ^= input.charCodeAt(i);
    g = Math.imul(g, 0x01000193);
  }

  const hex = (n: number) => (n >>> 0).toString(16).padStart(8, '0');
  return hex(h) + hex(g);
}

Two 32-bit passes, one forward and one over the reversed string, concatenated to a 16-hex-char output. The reverse pass is there because a single 32-bit FNV has a birthday bound around 65k distinct inputs, which is not comfortable when a busy multi-tenant install might see that many layouts. 64 bits pushes the collision point out to billions.

crypto.subtle.digest was the road not taken, and it's worth saying why, because "just use SHA-256" is the reflex. Three reasons. It's async, so the fingerprint would have to be awaited before the first batch could be sent, and the fingerprint sits on the hot path of a tracker that must not delay anything. It's unavailable on insecure origins, which is exactly where people run their staging admin panel. And there is no adversary here: nobody gains anything by crafting a DOM that collides with another DOM. The property needed is stability — same layout, same string, on every visitor's machine, with no coordination. Two Math.imul loops give you that in microseconds, in the same 5.5 kB gzipped bundle as everything else, with zero API surface.

If a fingerprint throws for any reason, fingerprint() returns '0'.repeat(16) and the tracker carries on. A layout version is a nice-to-have. Losing the clicks is not.

Where this is wrong

Mostly it over-forks. That's the honest failure mode, and it's structural, not a bug I can fix with a better regex.

A feature flag that shows an extra panel to 10% of users produces two fingerprints. So does an A/B test. So does a role-gated admin nav where managers see one more menu item than agents. In every one of those cases the SDK is technically right — the DOM really is different — and semantically wrong, because you think of them as one page and you want one heatmap.

And then there is the run fold, which fails the other way, and that one I like a lot less. An under-fork is a layout change the fingerprint never sees, so no new version appears and the clicks pile into the old map — the exact confidently-false picture this whole thing exists to prevent. It is narrow: the differing element has to be a sibling in a run whose members all carry the same tag and stable classes, so the change hides below a signature that didn't move. A per-row action column added to a table under a .row class is the case I have actually hit. Widening signature() to include a cheap hash of the child tags would close it, at the cost of making the fingerprint sensitive to things I spent this whole file trying to make it ignore. I haven't found a version of that trade I want.

Over-forking, meanwhile, is at least visible: you open the dashboard, you see two versions of /orders that appeared on the same day, and you go look. So the detection is deliberately twitchy and the interpretation is human. From VersionController:

$version->update(['merged_into_id' => $target->id]);

That's the whole merge. A pointer. Nothing is rewritten — events keep the page_version_id they were recorded against, and the dashboard follows the pointer when it aggregates. Which means a merge is reversible (unmerge nulls the column), and a misclick cannot destroy a quarter of data.

Two things guard it. The target has to resolve to a real version on the same page and not be the version itself (PageVersion::where('page_id', $version->page_id)->find($data['into'])), which stops both a typo'd id and the degenerate self-merge. Then the cycle check: memberIds() walks the whole subtree that already rolls up into the target, so you are blocked from merging into any of your own descendants, not just a direct A→B→A pair. Without it, effectiveId() would chase the loop until it hit its hop limit and land somewhere arbitrary.

Related