TestFlight External Testing or Phased Release: Where a Solo Developer's Time Actually Pays Off

Keep only purchases, permissions, and device layout in pre-release testing

Once you decide to run external testing less often, the next question is what you still verify beforehand. I keep three things.

The purchase flow. Buy, restore, and the state transition after cancellation — all three get one real round trip on a physical device. This is the one area I refuse to hand to a phased rollout. If purchases are broken when you ship to 5% of users, that 5% contains people who paid and did not get what they paid for. A crash you can apologize for in the next update. A failed payment is a different category of damage.

Permission dialogs. Photos, notifications, and tracking each appear exactly once per install. Get the order wrong and that device can never verify the correct order again. Skipping the effort of wiping a test device is how a wrong ordering reaches production.

Device-specific layout. Only in the update right after a new screen size appears, I walk the main screens on a device or simulator. This is the one area where more devices genuinely help, so external testers do have an edge. But your tester pool's device mix was already skewed the moment you recruited it, and there is no guarantee the model you care about is in there. If you want to check a specific model, running the simulator is faster.

Everything outside those three goes to the phased rollout and crash monitoring. How I structure the receiving side is in Running Crash-Free Rate as a Budget: An SLO Design Note for Deciding Where to Invest Across 6 Indie Apps.

Stop checking every morning whether the build reached external distribution

I have not abandoned external testing, so there are still releases that use it. The annoying part was having to open App Store Connect just to learn what state a build was in. With six apps, that check alone eats about ten minutes every morning.

I keep a script that queries the App Store Connect API and prints, in one pass, whether each build is processing, waiting on review, or ready to distribute. Below is the portion that does the actual determination. It runs on Node.js 20 or later and depends only on jsonwebtoken.

// check-build-state.mjs
// Usage: node check-build-state.mjs <APP_ID>
// Setup: create an API key under "Users and Access > Integrations" in App Store
//        Connect, store the .p8 somewhere safe, and pass the path via env vars.
import { readFileSync } from "node:fs";
import jwt from "jsonwebtoken";
 
const KEY_ID = process.env.ASC_KEY_ID;        // e.g. YOUR_KEY_ID
const ISSUER_ID = process.env.ASC_ISSUER_ID;  // e.g. YOUR_ISSUER_ID
const KEY_PATH = process.env.ASC_KEY_PATH;    // e.g. /secure/AuthKey_YOUR_KEY_ID.p8
const APP_ID = process.argv[2];
 
if (!KEY_ID || !ISSUER_ID || !KEY_PATH || !APP_ID) {
  console.error("ASC_KEY_ID / ASC_ISSUER_ID / ASC_KEY_PATH and APP_ID are required");
  process.exit(2);
}
 
// Keep the token under 20 minutes. Beyond that the API answers with 401.
function makeToken() {
  const privateKey = readFileSync(KEY_PATH, "utf8");
  return jwt.sign({}, privateKey, {
    algorithm: "ES256",
    expiresIn: "15m",
    issuer: ISSUER_ID,
    audience: "appstoreconnect-v1",
    header: { alg: "ES256", kid: KEY_ID, typ: "JWT" },
  });
}
 
async function api(path, token) {
  const res = await fetch(`https://api.appstoreconnect.apple.com${path}`, {
    headers: { Authorization: `Bearer ${token}` },
  });
  if (res.status === 401) throw new Error("Auth failed. Check that KEY_ID matches the .p8");
  if (res.status === 429) throw new Error("Rate limited. Space out the runs");
  if (!res.ok) throw new Error(`${path} returned ${res.status}`);
  return res.json();
}
 
// Print processing state, external distribution state, and expiry for the last 5 builds.
async function main() {
  const token = makeToken();
  const query = [
    `filter[app]=${APP_ID}`,
    "sort=-uploadedDate",
    "limit=5",
    "include=buildBetaDetail,preReleaseVersion",
    "fields[builds]=version,processingState,expired,buildBetaDetail,preReleaseVersion",
    "fields[buildBetaDetails]=externalBuildState,internalBuildState",
    "fields[preReleaseVersions]=version",
  ].join("&");
 
  const body = await api(`/v1/builds?${query}`, token);
  const included = new Map((body.included ?? []).map((x) => [`${x.type}:${x.id}`, x]));
 
  for (const build of body.data) {
    const detailRef = build.relationships?.buildBetaDetail?.data;
    const versionRef = build.relationships?.preReleaseVersion?.data;
    const detail = detailRef ? included.get(`buildBetaDetails:${detailRef.id}`) : null;
    const version = versionRef ? included.get(`preReleaseVersions:${versionRef.id}`) : null;
 
    const marketing = version?.attributes?.version ?? "?";
    const buildNo = build.attributes.version;
    const processing = build.attributes.processingState;      // PROCESSING / VALID / FAILED / INVALID
    const external = detail?.attributes?.externalBuildState;  // e.g. READY_FOR_BETA_SUBMISSION
    const expired = build.attributes.expired ? " [expired]" : "";
 
    console.log(`${marketing} (${buildNo}) processing=${processing} external=${external ?? "-"}${expired}`);
  }
}
 
main().catch((err) => {
  console.error(`Could not read build state: ${err.message}`);
  process.exit(1);
});

Output is five lines per app, shaped like this.

1.8.0 (412) processing=VALID external=WAITING_FOR_BETA_REVIEW
1.7.9 (409) processing=VALID external=READY_FOR_BETA_TESTING
1.7.9 (408) processing=VALID external=READY_FOR_BETA_TESTING [expired]

The part that took me the longest to get right was the combination of include and fields. Adding include=buildBetaDetail is not enough on its own — without fields[buildBetaDetails], the related resource can come back with its attributes stripped and externalBuildState reads as undefined. Related resources arrive flattened in the included array, so I rebuild them into a Map keyed by type:id. Once the shape settled, adding more apps stopped requiring any change to the logic.

To sweep several apps at once, a small shell wrapper is enough.

#!/usr/bin/env bash
set -euo pipefail
for APP_ID in 1111111111 2222222222 3333333333; do
  echo "--- ${APP_ID} ---"
  node check-build-state.mjs "${APP_ID}" || echo "  (could not read)"
done

The || echo alongside set -euo pipefail is deliberate: if one app fails, I still want the rest. Halting on the first failure just sends me back to opening App Store Connect.

Decide once each morning whether the rollout continues

Handing risk to a phased rollout means the receiving side needs its own routine. The thing that matters most there, in my experience, is not checking too often.

Right after distribution begins, you want to refresh the numbers constantly. But day-one figures sit on a tiny denominator, and a single crash swings the rate. React to a swing and you will halt updates that were never actually broken, which makes the next release feel dangerous.

I settled on looking once, in the morning, and picking one of three outcomes on the spot.

If two of the three lean toward halting, I halt. If only one does, I continue. This coarse rule left me with less hesitation than a set of precise thresholds ever did. Precise thresholds create their own maintenance work — you end up revising the thresholds instead of shipping.

The recovery path after a halt is laid out in three layers in A Phased Release Strategy for Rork Apps.

Two adjustments for apps built with Rork

The split above transfers to generated code, with two assumptions that change.

First, regeneration rewrites parts of the codebase. The build your testers examined and the build produced by a later generation pass are not guaranteed to share the code they looked at. If you are going to run external testing, freeze generation before you distribute, or the review loses its meaning. I leave a build untouched overnight before it goes to testers.

Second, over-the-air updates overlap awkwardly with beta testing. On React Native based setups you can swap the JavaScript side without a store review. That speeds up recovery after a halted rollout, and it also means tester devices can pick up a bundle you never intended them to see. Keeping external testing on its own update channel avoids a confusing week later.

Rork Max, which emits native Apple code, has no such escape hatch. I keep the same overall split, but I weight the three pre-release checks (purchases, permissions, device layout) a little more heavily. With one fewer way to roll back, more of the verification moves to the front.

If a build behaves on TestFlight and falls over after release, the separation of causes is covered in App Crashes After App Store Release But Not on TestFlight.

What to decide today

Take your next release and sort its changes into the first table. If the middle row — anything touching other people's account state or the migration of existing users — comes out empty, that release can skip external testing and start at 5% of a phased rollout. If it is not empty, external testing is worth the days it costs.

Narrowing the decision to one question converts hesitation directly into release speed. Moving external testing from "something I do every time" to "something I do when the release calls for it" is what let me pull my update interval back to where it had been.