What Filtering Out Prerelease OS Devices Cost Me — Separating Telemetry Tracks Instead of Dropping Them

There is no public API that tells you a device is on a beta

The first wall I hit while rebuilding was that iOS gives you no reliable way to ask this question.

Platform.Version in React Native returns a version string like "27.0" on iOS and a number like 36 — the API level — on Android. Same property, different types. Comparing it without normalizing first breaks Android silently, which is exactly the kind of bug that survives review.

iOS beta builds are known to carry a lowercase suffix on the build number, but there's no stable public API that hands you that build number. Any check built on it inherits the fragility of however you scraped it.

So I switched to inference. Keep a baseline of the newest major version I know has shipped generally, and treat anything above it as prerelease. Inference is sometimes wrong. Making it wrong in a controlled direction became the actual design problem.

// src/telemetry/osTrack.ts
import { Platform } from "react-native";
 
export type OsTrack = "ga" | "prerelease" | "unknown";
 
export type GaBaseline = { ios: number; android: number; updatedAt: string };
 
/**
 * Returns the device OS major version as a number.
 * iOS     : Platform.Version is a string such as "27.0"
 * Android : Platform.Version is a number such as 36 (API level)
 * Same property, different types — normalize before comparing.
 */
export function parseOsMajor(): number | null {
  const raw = Platform.Version;
  if (typeof raw === "number") return raw;
  const major = Number.parseInt(String(raw), 10);
  return Number.isFinite(major) ? major : null;
}
 
export function resolveOsTrack(baseline: GaBaseline | null): OsTrack {
  const major = parseOsMajor();
 
  // Never default to "prerelease" when inputs are missing.
  // The moment the baseline goes stale, that default empties your GA track.
  if (major === null || baseline === null) return "unknown";
 
  const knownGa = Platform.OS === "ios" ? baseline.ios : baseline.android;
  return major > knownGa ? "prerelease" : "ga";
}

Giving "can't tell" its own value is the part of this function I spent the most time on. With a boolean, uncertainty has to land somewhere. Land it on prerelease and your production numbers thin out; land it on ga and beta anomalies contaminate the metric you page on. A third value costs almost nothing and saves you later.

Ship the baseline remotely, and fail backward quietly

Baking the baseline into the binary means every user stays classified as prerelease until you ship an update on GA day. With review time, that's several days — the exact days you most want the data.

I serve a small JSON file and fetch it at startup. If the fetch fails, fall back to cache; if the cache is missing or malformed, fall back to a bundled value.

// src/telemetry/gaBaseline.ts
import AsyncStorage from "@react-native-async-storage/async-storage";
import type { GaBaseline } from "./osTrack";
 
const CACHE_KEY = "telemetry.gaBaseline.v1";
const ENDPOINT = "https://config.example.com/ga-baseline.json";
const TIMEOUT_MS = 2500;
 
// What was generally available at build time. The last resort when
// both the network and the cache let us down.
const BUNDLED: GaBaseline = { ios: 26, android: 36, updatedAt: "2026-07-01" };
 
function isValid(v: unknown): v is GaBaseline {
  const b = v as Partial<GaBaseline> | null;
  return (
    !!b &&
    typeof b.ios === "number" &&
    typeof b.android === "number" &&
    b.ios > 0 &&
    b.android > 0
  );
}
 
export async function loadGaBaseline(): Promise<GaBaseline> {
  let fallback: GaBaseline = BUNDLED;
  try {
    const cached = await AsyncStorage.getItem(CACHE_KEY);
    const parsed = cached ? JSON.parse(cached) : null;
    if (isValid(parsed)) fallback = parsed;
  } catch {
    // A corrupt cache must never block startup
  }
 
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), TIMEOUT_MS);
  try {
    const res = await fetch(ENDPOINT, { signal: controller.signal });
    if (!res.ok) throw new Error(`ga-baseline http ${res.status}`);
    const next: unknown = await res.json();
    if (!isValid(next)) throw new Error("ga-baseline shape mismatch");
    await AsyncStorage.setItem(CACHE_KEY, JSON.stringify(next));
    return next;
  } catch {
    return fallback;
  } finally {
    clearTimeout(timer);
  }
}

I capped the timeout at 2,500 ms to keep this off the startup critical path. A telemetry init that blocks first render turns into a launch-time regression, which is its own kind of self-inflicted wound — the same budget thinking I wrote up in When Your App's Launch Got Quietly Slower Release After Release applies directly here. I start reporting as unknown and overwrite the attributes once the baseline resolves.

The updatedAt field isn't decoration. Forgetting to bump the baseline is the single most likely operational failure in this design, and I surface a warning when the value is more than 90 days old.

Stop dropping. Add a dimension instead.

The rebuild itself was anticlimactically small. I deleted the skip branch and added one attribute.

// src/telemetry/index.ts
import crashlytics from "@react-native-firebase/crashlytics";
import analytics from "@react-native-firebase/analytics";
import { loadGaBaseline } from "./gaBaseline";
import { parseOsMajor, resolveOsTrack, type OsTrack } from "./osTrack";
 
let currentTrack: OsTrack = "unknown";
 
export function getOsTrack(): OsTrack {
  return currentTrack;
}
 
export async function initTelemetry(): Promise<void> {
  // Events fired before resolution are still worth keeping — send them as unknown.
  await analytics().setUserProperty("os_track", currentTrack);
 
  const baseline = await loadGaBaseline();
  currentTrack = resolveOsTrack(baseline);
 
  const staleDays =
    (Date.now() - new Date(baseline.updatedAt).getTime()) / 86_400_000;
 
  await crashlytics().setAttributes({
    os_track: currentTrack,
    os_major: String(parseOsMajor() ?? "unknown"),
    ga_baseline: `${baseline.ios}/${baseline.android}`,
    baseline_stale_days: String(Math.floor(staleDays)),
  });
 
  await analytics().setUserProperty("os_track", currentTrack);
}

Recording baseline_stale_days on the crash report means that when you later suspect the classifier itself, the evidence for that suspicion is sitting right there in the report.

I attached the analytics side as a user property rather than an event parameter on purpose. Event parameters would force me to rebuild funnels and audiences I had already defined. A user property layers a filter on top of reports that keep working.

Split the thresholds and the notification channels

Separating the numbers is only half of it. The other half is deciding how you react to each one.

Loosening prerelease to 98.0% isn't resignation. With a small denominator, a 99.5% threshold trips on a single crash. Alerts that trip constantly get ignored, and an ignored monitor is indistinguishable from no monitor.

unknown gets no threshold because its share is itself the signal: when it grows, the classification is broken, not the app. I watch 5% as the line. Reading it alongside baseline_stale_days usually tells me within a minute whether the cause is a stale baseline or flaky connectivity in a particular region.

In the release gate, blockers and fixes are different lists

I brought the same separation into the check that decides whether a phased rollout advances. Prerelease numbers never hold a release. They also never pass by unnoticed.

// scripts/releaseGate.ts
export type TrackStats = {
  track: "ga" | "prerelease" | "unknown";
  sessions: number;
  crashFreeRate: number; // 0.0 – 1.0
};
 
export type GateResult = {
  pass: boolean;
  blockers: string[];
  warnings: string[];
};
 
const GA_MIN_SESSIONS = 500;
const GA_MIN_CRASH_FREE = 0.995;
const PRE_MIN_SESSIONS = 50;
const PRE_MIN_CRASH_FREE = 0.98;
const UNKNOWN_MAX_SHARE = 0.05;
 
const pct = (v: number) => `${(v * 100).toFixed(2)}%`;
 
export function evaluateGate(stats: TrackStats[]): GateResult {
  const blockers: string[] = [];
  const warnings: string[] = [];
 
  const byTrack = (t: TrackStats["track"]) => stats.find((s) => s.track === t);
  const total = stats.reduce((sum, s) => sum + s.sessions, 0);
 
  const ga = byTrack("ga");
  if (!ga || ga.sessions < GA_MIN_SESSIONS) {
    blockers.push(
      `GA track has ${ga?.sessions ?? 0} sessions — not enough to decide on (need ${GA_MIN_SESSIONS})`
    );
  } else if (ga.crashFreeRate < GA_MIN_CRASH_FREE) {
    blockers.push(
      `GA crash-free rate ${pct(ga.crashFreeRate)} is below ${pct(GA_MIN_CRASH_FREE)}`
    );
  }
 
  // Prerelease never blocks, but always lands in the fix queue
  const pre = byTrack("prerelease");
  if (pre && pre.sessions >= PRE_MIN_SESSIONS && pre.crashFreeRate < PRE_MIN_CRASH_FREE) {
    warnings.push(
      `prerelease crash-free rate ${pct(pre.crashFreeRate)} — fix before the OS ships`
    );
  }
 
  const unknown = byTrack("unknown");
  if (unknown && total > 0 && unknown.sessions / total > UNKNOWN_MAX_SHARE) {
    warnings.push(
      `unknown is ${pct(unknown.sessions / total)} of sessions — check baseline delivery`
    );
  }
 
  return { pass: blockers.length === 0, blockers, warnings };
}

Returning warnings while excluding them from pass is the whole point of the function. Put "stop the release" and "fix this soon" in one list and whoever is on call will collapse them into one meaning — usually the more permissive one.

Warnings also need a destination. Mine go straight onto the next build's milestone. A warning with nowhere to go stops being read by the third occurrence.

Doubting the baseline delivery itself

After running this design through a full cycle, I read my own implementation again. The separation was working. What I had barely questioned was the JSON the separation rests on.

So I pulled the classification logic out on its own and measured what happens when delivery breaks. On Node v22.23.2 I built an input set of 10,000 devices with iOS majors spread from 24 to 27, pinned the reference date to release day, and let the resolver classify them. The population is a modeled shape of how devices move on that day, not a distribution of real users.

B happens because isValid only checks the types of ios and android and never looks at updatedAt. A JSON payload missing the date gets accepted, new Date(undefined) yields NaN, and baseline_stale_days carries the string "NaN" into every crash report. The 90-day warning I wrote about above catches F, where I simply forgot to update — but it stays quiet for the more dangerous case where the date is absent entirely. The attribute I added for monitoring turned out to be very good at hiding a hole in that monitoring.

D is the premise of this entire article coming back around. prerelease share: 0.00%. I deleted the filtering branch, and one wrong number in a config file reproduced the same blind spot. unknown stays at 0.00% too, so the unknown-share check from the previous section has nothing to say.

C runs the other way: the GA share thins out to 19.00%. Four fifths of the people actually running the shipped OS get judged against 98.0%, the threshold I deliberately loosened for beta devices.

I fed both to the gate, and one of them said nothing

I passed C and D straight into evaluateGate from the previous section.

D does stop the release — for the wrong reason. Beta crashes merge into GA, 99.71% drops to 99.19%, and whoever is on call goes looking for a GA regression that does not exist. The cause sits in a config file, and nothing on screen points there.

C says nothing at all. GA sessions come to 1,900, above the 500 floor; the rate is 99.85%, above its threshold; prerelease sits at 99.25%, above the looser 98.0%. On the numbers alone it looks healthy. It looks healthy precisely because four fifths of users are wearing the wrong label.

A baseline is accepted on how it moved since last time, not on whether it arrived. A shipping release advances the major by one per cycle. Writing that obvious property into validation stops both the rollback and the fat-fingered value at the door.

// src/telemetry/gaBaseline.ts (validation, replaced)
export type BaselineLoad = {
  baseline: GaBaseline;
  rejected: "rollback" | "jump" | "malformed" | null;
};
 
function isValid(v: unknown): v is GaBaseline {
  const b = v as Partial<GaBaseline> | null;
  if (!b || typeof b.ios !== "number" || typeof b.android !== "number") return false;
  if (b.ios <= 0 || b.android <= 0) return false;
  // Validate updatedAt here, or staleDays turns into NaN downstream
  if (typeof b.updatedAt !== "string") return false;
  return Number.isFinite(Date.parse(b.updatedAt));
}
 
function accept(next: GaBaseline, prev: GaBaseline): BaselineLoad["rejected"] {
  if (next.ios < prev.ios || next.android < prev.android) return "rollback";
  // Shipping releases move one major at a time; +2 or more is a delivery accident
  if (next.ios > prev.ios + 1 || next.android > prev.android + 1) return "jump";
  return null;
}
 
export function reconcile(remote: unknown, prev: GaBaseline): BaselineLoad {
  if (!isValid(remote)) return { baseline: prev, rejected: "malformed" };
  const rejected = accept(remote, prev);
  return rejected ? { baseline: prev, rejected } : { baseline: remote, rejected: null };
}
 
export function staleDays(baseline: GaBaseline): number | null {
  const t = Date.parse(baseline.updatedAt);
  return Number.isFinite(t) ? Math.floor((Date.now() - t) / 86_400_000) : null;
}

I ran the same 10,000-device input back through with this validation in place. B, C and D are all rejected, the previous value of ios=26 holds, the prerelease share reads 20.00%, and baseline_stale_days comes out as 75. Scenario A — the legitimate bump — passes through untouched and 27 is accepted.

Rejection does not happen quietly. rejected travels to both the crash attributes and the gate.

// scripts/releaseGate.ts (additions)
const GA_MIN_SHARE = 0.5;
 
export type GateContext = {
  baselineRejected: string | null;
  betaWindowOpen: boolean; // is a beta for the next major currently distributed
};
 
export function evaluateTrackSanity(stats: TrackStats[], ctx: GateContext): string[] {
  const warnings: string[] = [];
  const total = stats.reduce((sum, s) => sum + s.sessions, 0);
  const ga = stats.find((s) => s.track === "ga");
  const pre = stats.find((s) => s.track === "prerelease");
 
  if (ctx.baselineRejected) {
    warnings.push(`Baseline delivery rejected (${ctx.baselineRejected}) — classifying on the previous value`);
  }
  if (total > 0 && ga && ga.sessions / total < GA_MIN_SHARE) {
    warnings.push(`GA track is only ${pct(ga.sessions / total)} of sessions — baseline may have advanced too far`);
  }
  if (ctx.betaWindowOpen && (!pre || pre.sessions === 0)) {
    warnings.push("Zero prerelease sessions during a beta window — separation may not be working");
  }
  return warnings;
}

With those three checks added, C — the one that had stayed silent — reported GA track is only 19.00% of sessions, and D, now rejected upstream, reported Baseline delivery rejected (jump). The zero-prerelease-during-a-beta-window check is the last net, for the day something slips past rejection anyway.

What I went back and forth on was strictness: reject too eagerly and you block the legitimate swap on release day. The compromise I settled on is that rejection means "hold the previous value and tell a human," never "ignore." Classification stays put; the message always lands. I may have had this backwards from the start — the thing worth doubting was never the resolver, but the values flowing into it.

Three things that ran opposite to my expectations

The first crashes to surface on prerelease devices weren't in code touching new OS features. They were in screens I hadn't opened in years. New-feature code gets written carefully because you know you don't know it. The dangerous code is whatever you stopped questioning.

Second: the noise disappeared once I separated the tracks. What had actually annoyed me back when I was filtering wasn't the existence of beta devices — it was that their reports arrived mixed into the same channel as everything else. Changing the destination turned the same volume of information into something I could act on. Filtering had never matched the shape of the problem.

Third: I forgot to update the baseline. On the first GA day under this design, the JSON sat unchanged for roughly half a day while users upgrading to the shipped version kept getting counted as prerelease. The unknown fallback and baseline_stale_days made the retroactive correction straightforward. Building the system to absorb my own forgetfulness turned out to be the most valuable decision in the whole rewrite.

Pairing os_track with build provenance makes triage faster still — the stamping approach in Which Generation Introduced This Bug? combines cleanly with this one. With both attributes on the crash report, "which OS generation, which generated code" becomes a single filter instead of two investigations.

Before release day, and on it

iOS 27 and iPadOS 27 ship on September 14. If you have apps live on the App Store, this order has made the day calmer for me:

1. Implement os_track resolution and attachment, then verify on a real device that all three values — including unknown — reach your crash attributes
2. Serve the baseline JSON and bump one value to confirm the device-side classification actually changes (this is the only way to test the update path)
3. Add the rejection rules — monotonic, one major at a time, updatedAt validated — then deliberately publish a rollback and a fat-fingered value to watch them get turned away
4. Split the alert destinations for ga and prerelease, and drop prerelease to a daily digest
5. Wire an evaluateGate-equivalent check into your phased rollout review, rendering blockers and warnings in separate sections
6. On the day itself, treat "swap the baseline JSON" and "check the GA share afterward" as one task, not two

Deleting the filter branch takes under an hour. What takes real time is deciding how you'll respond to the separated numbers. Skip that and you've only added visible noise.

If you want one place to start, delete updatedAt from your own baseline JSON once and see what lands in baseline_stale_days. That is where I found the hole in my own monitoring.