Cutting App Store Review Replies From 32 Minutes a Day to 8 — App Store Connect API × Gemini × Cloudflare Workers in Production

Google Play Developer API Setup

Creating a Service Account

The Google Play Developer API uses Google Cloud service account authentication. Steps:

1. Create or select a project in Google Cloud Console
2. Enable the Google Play Android Developer API under APIs & Services → Library
3. Create a service account under IAM & Admin → Service Accounts
4. Download a JSON key for that service account
5. In Google Play Console, go to Settings → API access, link the service account, and grant Reply to reviews permission

Note that both steps 4 and 5 are required — the Google Cloud service account and the Play Console permission grant are separate systems.

JWT Authentication for Cloudflare Workers

Google Play uses RS256 (RSA-SHA256) rather than ES256. There's also an extra step: your JWT assertion needs to be exchanged for an OAuth2 access token before calling the API.

// src/googleplay/auth.ts
// Google Play API access token via RS256 JWT → OAuth2 exchange
 
interface GoogleServiceAccount {
  client_email: string;
  private_key: string;
}
 
async function generateGoogleAccessToken(
  serviceAccount: GoogleServiceAccount
): Promise<string> {
  const now = Math.floor(Date.now() / 1000);
 
  const header = { alg: "RS256", typ: "JWT" };
  const payload = {
    iss: serviceAccount.client_email,
    scope: "https://www.googleapis.com/auth/androidpublisher",
    aud: "https://oauth2.googleapis.com/token",
    iat: now,
    exp: now + 3600,
  };
 
  const encodeBase64Url = (obj: object): string =>
    btoa(JSON.stringify(obj))
      .replace(/\+/g, "-")
      .replace(/\//g, "_")
      .replace(/=/g, "");
 
  const header64 = encodeBase64Url(header);
  const payload64 = encodeBase64Url(payload);
  const signingInput = `${header64}.${payload64}`;
 
  const keyBuffer = pemRsaToBuffer(serviceAccount.private_key);
  const cryptoKey = await crypto.subtle.importKey(
    "pkcs8",
    keyBuffer,
    { name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" },
    false,
    ["sign"]
  );
 
  const encoder = new TextEncoder();
  const signature = await crypto.subtle.sign(
    "RSASSA-PKCS1-v1_5",
    cryptoKey,
    encoder.encode(signingInput)
  );
 
  const sig64 = btoa(String.fromCharCode(...new Uint8Array(signature)))
    .replace(/\+/g, "-")
    .replace(/\//g, "_")
    .replace(/=/g, "");
 
  const jwtAssertion = `${signingInput}.${sig64}`;
 
  const tokenResponse = await fetch("https://oauth2.googleapis.com/token", {
    method: "POST",
    headers: { "Content-Type": "application/x-www-form-urlencoded" },
    body: `grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion=${jwtAssertion}`,
  });
 
  if (!tokenResponse.ok) {
    const err = await tokenResponse.text();
    throw new Error(`Google OAuth2 token exchange failed: ${err}`);
  }
 
  const tokenData = (await tokenResponse.json()) as { access_token: string };
  return tokenData.access_token;
}
 
function pemRsaToBuffer(pem: string): ArrayBuffer {
  const base64 = pem
    .replace(/-----BEGIN PRIVATE KEY-----/g, "")
    .replace(/-----END PRIVATE KEY-----/g, "")
    .replace(/\n/g, "");
 
  const binary = atob(base64);
  const buffer = new ArrayBuffer(binary.length);
  const view = new Uint8Array(buffer);
  for (let i = 0; i < binary.length; i++) {
    view[i] = binary.charCodeAt(i);
  }
  return buffer;
}
 
export { generateGoogleAccessToken };

Gemini API: Sentiment Analysis and Response Generation

This is the core of the system. The key design choices here are using responseMimeType: "application/json" for structured output, keeping temperature low for consistency, and building a fallback for the rare cases when JSON parsing fails.

// src/ai/reviewResponder.ts
 
interface ReviewAnalysis {
  sentiment: "positive" | "negative" | "neutral" | "mixed";
  mainIssue: string | null;
  suggestedResponse: string;
  confidence: number;      // 0.0–1.0: how confident Gemini is in the response quality
  shouldAutoSend: boolean; // true only when confidence ≥ 0.85 and content is clearly safe
}
 
async function analyzeAndRespond(
  review: {
    rating: number;
    title: string;
    body: string;
    territory: string;
  },
  appContext: {
    name: string;
    category: string;
    recentUpdates: string[];
    knownIssues: string[];
  },
  geminiApiKey: string
): Promise<ReviewAnalysis> {
  const responseLanguage = getResponseLanguage(review.territory);
 
  const prompt = `
You are the developer of an app called "${appContext.name}" responding to a user review.
 
## App Context
- App name: ${appContext.name}
- Category: ${appContext.category}
- Recent updates: ${appContext.recentUpdates.join(", ") || "None"}
- Known issues: ${appContext.knownIssues.length > 0 ? appContext.knownIssues.join(", ") : "None"}
 
## Review to respond to
- Rating: ${review.rating} star(s)
- Title: ${review.title || "(no title)"}
- Body: ${review.body}
- Territory: ${review.territory}
 
## Instructions
1. Classify the sentiment: positive / negative / neutral / mixed
2. Summarize the main issue in one sentence if there is one (null otherwise)
3. Write a developer response with these principles:
   - Language: ${responseLanguage}
   - Length: 80–200 words
   - Tone: genuine, warm, never defensive or salesy
   - Positive review → thank them, share what's coming next
   - Bug report → acknowledge the issue, invite them to contact support@appname.com
   - Feature request → thank them, say you've noted it
   - Emotional review → absorb the frustration without becoming defensive
4. Set confidence (0.0–1.0) based on how appropriate and safe the response is
5. Set shouldAutoSend to true only if confidence ≥ 0.85 AND there's nothing ambiguous or potentially inflammatory
 
Respond in JSON only:
{
  "sentiment": "positive" | "negative" | "neutral" | "mixed",
  "mainIssue": "one-sentence summary or null",
  "suggestedResponse": "the response text",
  "confidence": 0.0–1.0,
  "shouldAutoSend": true | false
}
`;
 
  const response = await fetch(
    `https://generativelanguage.googleapis.com/v1beta/models/gemini-3.6-flash:generateContent?key=${geminiApiKey}`,
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        contents: [{ parts: [{ text: prompt }] }],
        generationConfig: {
          responseMimeType: "application/json",
          temperature: 0.3,
          maxOutputTokens: 512,
        },
      }),
    }
  );
 
  if (!response.ok) {
    throw new Error(`Gemini API error: ${response.status}`);
  }
 
  const data = (await response.json()) as {
    candidates: Array<{
      content: { parts: Array<{ text: string }> };
    }>;
  };
 
  const text = data.candidates[0]?.content.parts[0]?.text;
  if (!text) throw new Error("Gemini API returned empty response");
 
  try {
    return JSON.parse(text) as ReviewAnalysis;
  } catch {
    // Fallback: safe default that requires manual review
    return {
      sentiment: "neutral",
      mainIssue: null,
      suggestedResponse:
        "Thank you for taking the time to share your feedback. We're always working to improve the app and your input means a lot to us.",
      confidence: 0.3,
      shouldAutoSend: false, // Never auto-send when parsing fails
    };
  }
}
 
function getResponseLanguage(territory: string): string {
  const map: Record<string, string> = {
    JPN: "Japanese",
    USA: "English",
    GBR: "English",
    AUS: "English",
    CAN: "English",
    DEU: "German",
    FRA: "French",
    KOR: "Korean",
    CHN: "Simplified Chinese",
    TWN: "Traditional Chinese",
  };
  return map[territory] ?? "English";
}
 
export { analyzeAndRespond };

Cloudflare Workers: Putting It All Together

Here's the full Worker that ties everything together, with Cloudflare KV for state management and a REST API for the Rork dashboard.

// src/worker.ts
 
interface Env {
  APPSTORE_ISSUER_ID: string;
  APPSTORE_KEY_ID: string;
  APPSTORE_PRIVATE_KEY: string;
  GOOGLE_SERVICE_ACCOUNT_JSON: string;
  GEMINI_API_KEY: string;
  APP_ID_IOS: string;
  PACKAGE_NAME_ANDROID: string;
  APP_NAME: string;
  APP_CATEGORY: string;
  KV_REVIEWS: KVNamespace;
}
 
export default {
  // Cron trigger — set "0 * * * *" in wrangler.toml for hourly runs
  async scheduled(_event: ScheduledEvent, env: Env, ctx: ExecutionContext) {
    ctx.waitUntil(processNewReviews(env));
  },
 
  // REST API for the Rork dashboard
  async fetch(request: Request, env: Env) {
    const url = new URL(request.url);
    const corsHeaders = {
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Methods": "GET, POST, OPTIONS",
      "Access-Control-Allow-Headers": "Content-Type, Authorization",
    };
 
    if (request.method === "OPTIONS") {
      return new Response(null, { headers: corsHeaders });
    }
 
    if (url.pathname === "/api/reviews" && request.method === "GET") {
      const reviews = await getPendingReviews(env);
      return Response.json(reviews, { headers: corsHeaders });
    }
 
    if (url.pathname === "/api/approve" && request.method === "POST") {
      const { reviewId, platform, response: responseText } =
        (await request.json()) as {
          reviewId: string;
          platform: "ios" | "android";
          response: string;
        };
 
      await sendReviewResponse(reviewId, platform, responseText, env);
      await env.KV_REVIEWS.delete(`pending:${reviewId}`);
      await env.KV_REVIEWS.put(`approved:${reviewId}`, "1", {
        expirationTtl: 60 * 60 * 24 * 30,
      });
 
      return Response.json({ success: true }, { headers: corsHeaders });
    }
 
    if (url.pathname === "/api/dismiss" && request.method === "POST") {
      const { reviewId } = (await request.json()) as { reviewId: string };
      await env.KV_REVIEWS.delete(`pending:${reviewId}`);
      await env.KV_REVIEWS.put(`dismissed:${reviewId}`, "1", {
        expirationTtl: 60 * 60 * 24 * 30,
      });
      return Response.json({ success: true }, { headers: corsHeaders });
    }
 
    return new Response("Not Found", { status: 404 });
  },
};
 
async function processNewReviews(env: Env) {
  const { generateAppStoreToken } = await import("./appstore/auth");
  const { fetchReviews } = await import("./appstore/reviews");
  const { analyzeAndRespond } = await import("./ai/reviewResponder");
 
  let currentToken = await generateAppStoreToken({
    issuerId: env.APPSTORE_ISSUER_ID,
    keyId: env.APPSTORE_KEY_ID,
    privateKey: env.APPSTORE_PRIVATE_KEY,
  });
 
  const iosReviews = await fetchReviews(env.APP_ID_IOS, currentToken, 30);
  const appContext = {
    name: env.APP_NAME,
    category: env.APP_CATEGORY,
    recentUpdates: ["Latest performance improvements", "UI refresh"],
    knownIssues: [],
  };
 
  let count = 0;
 
  for (const review of iosReviews) {
    const [pending, approved, dismissed] = await Promise.all([
      env.KV_REVIEWS.get(`pending:${review.id}`),
      env.KV_REVIEWS.get(`approved:${review.id}`),
      env.KV_REVIEWS.get(`dismissed:${review.id}`),
    ]);
    if (pending || approved || dismissed) continue;
 
    // Refresh token every 10 reviews to avoid expiry mid-batch
    if (count > 0 && count % 10 === 0) {
      currentToken = await generateAppStoreToken({
        issuerId: env.APPSTORE_ISSUER_ID,
        keyId: env.APPSTORE_KEY_ID,
        privateKey: env.APPSTORE_PRIVATE_KEY,
      });
    }
 
    try {
      const analysis = await analyzeAndRespond(
        review,
        appContext,
        env.GEMINI_API_KEY
      );
 
      await env.KV_REVIEWS.put(
        `pending:${review.id}`,
        JSON.stringify({ review, analysis, platform: "ios" }),
        { expirationTtl: 60 * 60 * 24 * 7 }
      );
 
      if (analysis.shouldAutoSend && analysis.confidence >= 0.85) {
        await sendReviewResponse(
          review.id,
          "ios",
          analysis.suggestedResponse,
          env
        );
        await env.KV_REVIEWS.delete(`pending:${review.id}`);
        await env.KV_REVIEWS.put(`approved:${review.id}`, "auto", {
          expirationTtl: 60 * 60 * 24 * 30,
        });
      }
 
      count++;
    } catch (error) {
      // Log the error but keep processing — one failure shouldn't stop the batch
      console.error(`Failed to process review ${review.id}:`, error);
    }
  }
}
 
async function getPendingReviews(env: Env) {
  const list = await env.KV_REVIEWS.list({ prefix: "pending:" });
  const reviews = await Promise.all(
    list.keys.map((key) => env.KV_REVIEWS.get(key.name, "json"))
  );
  return reviews.filter(Boolean);
}
 
async function sendReviewResponse(
  reviewId: string,
  platform: "ios" | "android",
  responseText: string,
  env: Env
) {
  if (platform === "ios") {
    const { generateAppStoreToken } = await import("./appstore/auth");
    const token = await generateAppStoreToken({
      issuerId: env.APPSTORE_ISSUER_ID,
      keyId: env.APPSTORE_KEY_ID,
      privateKey: env.APPSTORE_PRIVATE_KEY,
    });
 
    const res = await fetch(
      "https://api.appstoreconnect.apple.com/v1/customerReviewResponses",
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${token}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          data: {
            type: "customerReviewResponses",
            attributes: { responseBody: responseText },
            relationships: {
              review: {
                data: { type: "customerReviews", id: reviewId },
              },
            },
          },
        }),
      }
    );
 
    if (!res.ok) {
      const errText = await res.text();
      throw new Error(`App Store response failed: ${res.status} ${errText}`);
    }
  }
 
  if (platform === "android") {
    const { generateGoogleAccessToken } = await import("./googleplay/auth");
    const serviceAccount = JSON.parse(env.GOOGLE_SERVICE_ACCOUNT_JSON);
    const accessToken = await generateGoogleAccessToken(serviceAccount);
 
    const res = await fetch(
      `https://androidpublisher.googleapis.com/androidpublisher/v3/applications/${env.PACKAGE_NAME_ANDROID}/reviews/${reviewId}:reply`,
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${accessToken}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({ replyText: responseText }),
      }
    );
 
    if (!res.ok) {
      const errText = await res.text();
      throw new Error(`Google Play response failed: ${res.status} ${errText}`);
    }
  }
}

Building the Rork Management Dashboard

With the backend running, the Rork dashboard is the piece that makes daily use practical. This is a personal-use app — no App Store submission needed. Install it on your device via Rork Companion.

Use this prompt in Rork:

Create a Review Management Dashboard app with:

## Main Screen
- Three tabs: "Needs Review", "Approved", "Auto-Sent"
- Review cards showing: star rating (★ display), review body (max 3 lines),
  sentiment badge (Positive=green / Negative=red / Neutral=gray),
  territory code, date
- Swipe left to dismiss, swipe right to approve and send

## Review Detail Screen
- Full review text
- AI-generated response in an editable TextInput
- "Approve & Send" button (green, primary) and "Dismiss" button (red, destructive)
- Confidence score as percentage badge

## API Configuration
- Base URL: https://your-worker.workers.dev (stored as WORKER_URL env var)
- GET /api/reviews — fetch pending reviews
- POST /api/approve — send {reviewId, platform, response}
- POST /api/dismiss — send {reviewId}

## Design
- Dark mode support
- Sentiment-based color coding throughout
- FlatList with pull-to-refresh
- Empty state illustration for each tab

Set the Worker URL as an environment variable in Rork and the dashboard is live. If you want to structure the backend routing and JWT auth more rigorously, the walkthrough in Building a Production-Ready REST API with Rork, Hono.js & Cloudflare Workers works as a direct foundation for this system.

Common Mistakes and Pitfalls

Three problems that actually happened during development of this system.

1. App Store Connect JWT Expires Mid-Batch

App Store Connect API JWTs are valid for 20 minutes maximum. Processing 30+ reviews in a single batch can easily exceed that. The failure mode is subtle — you get a 401 halfway through, with no obvious indication that it's a token expiry issue rather than a permissions problem.

// ❌ Generate once and reuse indefinitely
const token = await generateAppStoreToken(config);
for (const review of largeReviewList) {
  await processReview(review, token); // 401 error partway through
}
 
// ✅ Refresh every 10 reviews
let token = await generateAppStoreToken(config);
for (let i = 0; i < reviews.length; i++) {
  if (i > 0 && i % 10 === 0) {
    token = await generateAppStoreToken(config); // Refresh before expiry
  }
  await processReview(reviews[i], token);
}

Setting the JWT expiry to 19 minutes (instead of 20) adds a buffer that prevents edge-case failures when requests take slightly longer than expected.

2. Gemini Occasionally Returns Malformed JSON

Even with responseMimeType: "application/json" set, complex prompts occasionally produce output that fails JSON parsing. The fix is always the same: wrap JSON.parse() in a try/catch and return a safe fallback.

The critical rule in the fallback: never set shouldAutoSend: true. When you can't trust the output, you want human eyes on it before anything goes out.

try {
  return JSON.parse(text) as ReviewAnalysis;
} catch {
  return {
    sentiment: "neutral",
    mainIssue: null,
    suggestedResponse: "Thank you for your feedback — we're always working to improve the app.",
    confidence: 0.3,
    shouldAutoSend: false, // Non-negotiable: manual review when uncertain
  };
}

Keeping temperature at 0.3 significantly reduces the frequency of malformed output. Above 0.7, you'll see it often enough to be annoying.

3. Google Play Developer API Has Up to 24-Hour Permission Propagation

After creating a service account in Google Cloud and granting permissions in Google Play Console, the API may still return 403 errors for up to 24 hours. This isn't a code problem — it's propagation delay between the two Google systems.

If you've verified both the service account creation (Cloud Console) and the permission grant (Play Console) but still get 403s, wait a full day before concluding there's something wrong. I spent half a day debugging what turned out to be a waiting problem.

4. KV Key Design Matters More Than It Seems

The state machine here has four states: pending:, approved:, dismissed:, auto: (auto-sent). Designing the key namespace upfront makes list() prefix filtering fast and prevents confusion when you're debugging state transitions. Without a clear schema, you end up with orphaned keys and unclear state.

One practical tip: set consistent expirationTtl values across all keys. 30-day expiry for final states (approved/dismissed) and 7-day expiry for pending items works well. It keeps the KV namespace from growing indefinitely.

Seven Operational Traps That Aren't in the Docs

Here's the part you don't get from Apple's or Google's reference material — the judgment calls that only show up after you've shipped a system like this to production. I'll share the ones that bit me on day one and the edge cases that took a month of running before they surfaced.

1. JWT clock skew causes intermittent 401s on Workers

App Store Connect's JWT requires exp within 20 minutes of the current time. I generate the JWT at the Cloudflare Workers edge, but Workers' server clock and Apple's auth server's clock can drift by a few hundred milliseconds to a few seconds. If you compute iat directly from Date.now(), you'll occasionally generate a token that Apple sees as "issued in the future" — and it returns 401. Setting iat = Math.floor(Date.now() / 1000) - 5 (intentionally backdating by 5 seconds) eliminated the 401s entirely across three months of production runs.

2. Google Play API quotas aren't really documented

The docs say "standard quota," but empirically you'll hit quotaExceeded once you exceed roughly 60 requests per minute. The catch: quota is per service account, not per app. If you're polling reviews across five apps from the same service account in parallel, you'll burn through quota fast. The fix is to stagger cron triggers per app by 20 seconds — express it in your cron expressions rather than relying on rate limiting at the request level.

3. Don't trust Apple/Google's locale field alone for language detection

reviewerNickname or locale may be ja, but the body might be written in English (a Japanese user living abroad, for instance). I use Intl.Segmenter to tally hiragana/katakana/kanji ratios in the body. If they're under 30%, I reply in the body's actual language rather than the declared locale. This eliminated the "replied to a Japanese user in English" mistakes that happen with naive locale-based routing.

4. Force a three-part structure for 1–2 star replies

I embed three of my best historical responses in the few-shot prompt and force response_template: "apology → concrete_action → gratitude" via responseSchema. Even a lightweight Flash-tier model produces stable tone with this constraint. Dropping temperature to 0.4 also helps. The default of 1.0 occasionally slips in over-apologetic phrasing like "We deeply regret causing such fatal inconvenience" that feels off in English-speaking markets.

5. Confidence threshold 0.85 is the sweet spot in my experience

At 0.90, auto-send rate drops below 20% and the approval queue piles up. At 0.80, you reach 60% auto-send but get one or two "I really didn't want to send that" replies per month. At 0.85, I land at 40–45% auto-send with effectively zero misfires. Your app's category will shift the sweet spot, so my recommendation is to start at 0.95 for the first two weeks and step down by 0.05 weekly until you find your tolerance.

6. Use a 7-day TTL on pending: keys

I ran without TTL initially and accumulated about 1,200 unhandled pending: keys over three months. They resurfaced en masse during an unrelated app update and caused real confusion. Setting expirationTtl: 60 * 60 * 24 * 7 means anything left untouched for a week gets silently dropped. Important items get promoted to an important: prefix that persists.

7. Carry reply-language continuity across re-reviews from the same user

If a user updates their review later, replying in a different language than last time feels jarring. I keep replied_lang:{authorName} in KV with a 90-day TTL and include "you previously replied to this user in Japanese" in the Gemini prompt. It's a tiny thing, but anecdotally my re-review rate (the rate at which a user appends to their original review) went up roughly 1.4x after adding this.

Six Weeks of Walking the Confidence Threshold From 0.95 Down to 0.85

Trap 5 says "start at 0.95 and drop 0.05 per week." Here are the actual numbers from doing exactly that. Japanese-language reviews only, 280-340 new reviews per week. "Weekend backlog" is the count of pending: keys still neither approved nor rejected as of Friday night.

Three things stand out.

First, the 0.90 to 0.85 step buys 12 points of auto-send rate at zero cost in bad sends. That stretch was free headroom. Put differently: the two weeks I spent sitting at 0.90 weren't buying safety, they were just adding to my own approval workload.

Second, the first bad sends only appear at 0.80 — two of them. One reply referenced a version number that didn't exist; the other answered an English review in Japanese that was oddly over-polite. Both source reviews were under 15 characters. Short reviews can come back with high confidence even when Gemini has no real context, so I added a length guard that bypasses the threshold entirely.

// Length guard runs before the confidence check
const isTooShort = review.body.trim().length < 20;
const canAutoSend = !isTooShort && analysis.confidence >= CONFIDENCE_THRESHOLD;

Third, the backlog peaked at 61 in the 0.95 weeks — accumulating faster than I could clear it. Tightening the threshold didn't make the system safer; it just moved the bottleneck onto me. And a reply that sits for five days has lost most of its value anyway. Sending late is often worse than not sending at all.

Decide your rollback rule up front, too. Mine is: if a single bad send happens in any two-week window, the threshold goes back up 0.05. Tolerate the first one and the next signal you get is a falling rating.

Three Topics That Never Reach the Confidence Check

Whenever I talk about thresholds, someone asks whether everything above 0.85 can just go out. In my setup it can't. Before confidence is evaluated at all, there's a layer that kills auto-send based on what the review is actually about.

The lesson came from a refund complaint. Gemini produced a warm, well-structured reply with a confidence of 0.91. Nothing wrong with the writing. But refund decisions belong to Apple and Google, and a public developer reply that sounds like a commitment is not mine to make. Confidence measures whether the sentence is good. It says nothing about whether the topic is one I should be speaking on at all. I had been conflating the two.

Three areas are hard-blocked.

Plain keyword matching turned out to be enough. Handing this judgment back to the LLM defeats the purpose: when it misses a case you want blocked, nothing tells you it missed. Ambiguity should fall toward blocking. That's the whole design.

// src/ai/hardBlock.ts
// Runs before the confidence check. Anything ambiguous falls to manual.
 
type BlockReason = "refund" | "privacy" | "safety" | null;
 
const BLOCK_PATTERNS: Record<Exclude<BlockReason, null>, RegExp> = {
  refund: /refund|money back|charged twice|double charge|返金|払い戻し/i,
  privacy: /privacy|delete my (data|account)|GDPR|CCPA|personal (data|info)/i,
  safety: /child|children|minor|self-harm|medical|diagnos|medication/i,
};
 
export function detectHardBlock(review: {
  title: string;
  body: string;
}): BlockReason {
  const text = `${review.title} ${review.body}`;
  for (const [reason, pattern] of Object.entries(BLOCK_PATTERNS)) {
    if (pattern.test(text)) return reason as Exclude<BlockReason, null>;
  }
  return null;
}
 
// Confidence is only consulted after this gate passes
export function decideAutoSend(
  review: { title: string; body: string },
  analysis: { confidence: number; shouldAutoSend: boolean },
  threshold = 0.85
): { autoSend: boolean; blockedBy: BlockReason } {
  const blockedBy = detectHardBlock(review);
  if (blockedBy) return { autoSend: false, blockedBy };
  return {
    autoSend: analysis.shouldAutoSend && analysis.confidence >= threshold,
    blockedBy: null,
  };
}

The volume this catches is small against total review count. But reading back through the blockedBy queue at month end, what's sitting there is consistently the set of replies that would have created work for me had they gone out automatically. Writing these twenty lines paid off more than another round of threshold tuning.

The threshold decides how much work you hand off. The hard block decides where you stop handing it off at all. Tune only the first, and the workload drops while the boundary of what you're accountable for quietly blurs.

I Finally Measured the Language Detector Instead of Guessing at It

In the traps section above I mentioned that I stopped trusting locale and started deciding the reply language from the character mix in the review body. Re-reading this article, I realised something uncomfortable: when I first picked 30% as the cutoff, I never actually measured anything. It felt about right, the obvious mistakes stopped, and I left it there.

So I sat down and measured it. I wrote out eleven review bodies in the shapes people actually send, and compared two ways of counting.

• Naive: Japanese characters as a share of all characters in the body
• Word-aware: split with Intl.Segmenter, then count words containing Japanese characters

The more effort a Japanese user puts into pasting a log, the more likely a naive character ratio is to answer them in English. A body as ordinary as "it crashes on open, I see Fatal Exception: NSInvalidArgumentException" scores 0.19 and falls below the 30% line. Someone who pasted a whole stack trace scored 0.05. The people taking the most care to tell me what broke were the ones getting a reply in the wrong language.

Counting by word instead of by character moves those same bodies to 0.70 and 0.20. That rescues a body with one exception name in it. Long logs and stack traces still sit around 0.20.

My next instinct was to lower the threshold, so I swept it in 0.05 steps. Whatever single line I chose, two of the eleven came out wrong. Japanese with a long log (0.21 and 0.20) and English with one Japanese phrase in it (0.50 both times) overlap on the ratio axis. One line cannot separate them — that is the part I only learned by measuring.

What needed fixing was not where the line sat, but how many lines there were.

// src/ai/replyLanguage.ts
// Decide the reply language from the character mix. Send the ambiguous band to manual.
 
const JP_CHAR = /[\u3040-\u309F\u30A0-\u30FF\u4E00-\u9FFF]/u;
 
/** Share of word-like segments containing Japanese characters (not a character ratio) */
export function japaneseWordRatio(text: string): number {
  const segmenter = new Intl.Segmenter("ja", { granularity: "word" });
  const words = [...segmenter.segment(text)].filter((s) => s.isWordLike);
  if (words.length === 0) return 0;
  const jp = words.filter((w) => JP_CHAR.test(w.segment)).length;
  return jp / words.length;
}
 
export type LanguageRoute =
  | { kind: "auto"; language: "ja" | "body" }
  | { kind: "manual"; ratio: number };
 
const HIGH = 0.6;  // above this, reply in Japanese
const LOW = 0.15;  // below this, reply in the language of the body
 
export function routeReplyLanguage(text: string): LanguageRoute {
  const ratio = japaneseWordRatio(text);
  if (ratio >= HIGH) return { kind: "auto", language: "ja" };
  if (ratio <= LOW) return { kind: "auto", language: "body" };
  return { kind: "manual", ratio };  // 0.15-0.60 never reaches auto-send
}

With the band in place, those eleven bodies split six automatic and five manual, and nothing on the automatic side came out wrong. Close to half going manual looks expensive until you see who lands there: the person who pasted a log, and the person who dropped one Japanese phrase into an English review. Both are reviews I'd rather read myself before answering. What I wanted to cut was never the effort — it was the rudeness.

If Intl.Segmenter isn't available in your runtime, keep the naive count and widen the band instead. You can't sharpen the judgment, so you widen the range where you decline to judge.

Don't let something unjudgeable pass as if it had been judged. It's the same line I drew for the hard-block topics, applied to language. Before you go back to nudging a threshold from 0.30 to 0.20, run ten real review bodies through your own detector and look at the ratios side by side. My 30% was a number I had never measured.

Design Choices That Matter When You Run Several Apps in Parallel

This section is for indie developers running multiple apps in parallel. Across the apps I maintain, combined new reviews land somewhere around 1,200–1,800 per month. A naive single-app implementation breaks down here, so a few specific design choices became necessary.

Use Durable Objects to isolate per-app state

Vanilla Workers hits read-modify-write race conditions on KV when multiple apps process reviews concurrently. Routing through a Durable Object keyed by app ID serializes review processing per app and keeps the pending:/processed: state consistent. Costs land under $0.20/month for typical indie-scale write volumes.

Two-dimensional cron staggering: app × language

My production cron fires on */15 * * * *. Inside the handler, I split apps into five groups via (minute % 5) and split into Japanese / English / other-language fetches via (minute % 3). This spreads API consumption evenly and made quotaExceeded errors disappear from my logs.

Couple to a Crashlytics × Claude in Chrome auto-fix pipeline

I recently built a separate pipeline that pulls the latest Crashlytics crashes and uses Claude in Chrome to drive Xcode through fix patches. Cross-referencing this system with that one via crashHash lets "crashes on launch" reviews branch automatically into three reply patterns:

Replying "could you tell me which screen crashed?" to a crash report when you already know about the crash from Crashlytics is rude — you're asking the user to do diagnostic work you've already done. Once I wired in the Crashlytics correlation and could lead with "v2.1.0 already ships the fix," the rate at which 1–2 star reviews get edited up to 5 stars went up roughly 1.8x compared to before.

Keeping the Handcraft Feel in AI-Drafted Replies

This is less about technology and more about posture. The moment you hand drafting over to AI, some amount of warmth starts draining out of the replies. Preserving that warmth turned out to be as practical a problem for me as tuning the confidence threshold.

A few concrete habits I've ended up with:

First, the historical replies I embed in the few-shot prompt are personally curated. I only include five replies that I feel still carry the right temperature. Anything I wrote on autopilot gets excluded. This alone changes Gemini's output noticeably.

Second, even on auto-sent replies, I run a small post-processing pass to normalize tone signals: standardize emoji choices (a 🙏 to match the app's tone, for instance), normalize Japanese wave dashes, fix smart-quote inconsistencies. It's five lines of regex at the end of the Worker, but the cumulative effect on brand voice is real.

Third, I added a deliberate 3-second delay on the approval button in the Rork dashboard. The button stays grayed out for three seconds after the screen renders, then activates. This is a safety I built against my own reflexes — once you fall into tapping "approve" instinctively, AI's mistakes go out the door unedited. Three seconds is small, but it produces just enough friction for "wait, this one isn't quite right" to surface. Over three months, those three seconds stopped 11 replies from going out: six referencing a version number that didn't exist, three replying to the wrong review, two simply overwrought in tone.

For me, designing the technical layer alongside these posture-level details is how I keep the work honest toward the users on the other end.

Case Study: Turning a 1-Star Review Into a 5-Star Reply in 24 Hours

To close, here's a concrete example from production. In March 2026, one of my wallpaper apps received this 1-star review: "After updating to iOS 17 the app crashes immediately on launch. I'd like a refund." With permission and personal details removed, here's what the pipeline did:

1. Cloudflare Workers cron fetched new reviews from App Store Connect API on the 15-minute tick
2. Gemini classified the review as sentiment: negative, category: crash_report, urgency: high
3. The Crashlytics API was queried by appVersion + iOS 17 and the matching crash was found (first detected within the last 24 hours)
4. KV indicated that a fix patch was already applied to an unreleased build (Claude in Chrome had driven the Xcode fix the previous day)
5. Gemini received the context "crash is recognized; v2.4.1 ships the fix; submitting to App Review within 24 hours" and drafted a reply
6. Confidence came back at 0.78 — below the auto-send threshold — so the reply landed in my phone's notification queue

From notification to approval-tap took about 90 seconds: open the screen, wait the deliberate 3-second delay, read the draft, tap approve. The reply itself was straightforward — acknowledged the crash, gave a concrete release window, apologized for the inconvenience. The next day, the user updated their rating to 5 stars and appended "thanks for the quick response."

What I took away from this case was that, more than getting the technical content right, what matters is communicating "I see your problem" quickly. AI-drafted replies make that speed possible. My role, I've come to think, is to add the final layer of warmth and approve.

Three Months of Production Results

Some honest data from running this system:

Time saved: Daily review handling dropped from ~32 minutes to ~8 minutes. The biggest gains came from non-English reviews (no translation overhead) and bug-report responses (no need to write the same "we'll look into this, please contact support" message manually).

Auto-send rate: With confidence threshold at 0.85, roughly 40–45% of reviews get auto-sent. Emotionally charged one-star reviews and complex bug reports consistently fall below the threshold and land in the manual queue — which is exactly the right behavior.

API costs: At 30 reviews per day, Gemini API costs run about $1–2 per month. The prompt sits around 1,000 tokens and the output around 300, so even a tenfold increase in review volume keeps you in the same order of magnitude. Cost will not be the thing that stops you from shipping this.

A note on model IDs (added August 2026): The numbers in this article were measured on gemini-2.0-flash, which was shut down on June 1, 2026. The code above now targets gemini-3.6-flash, the officially recommended replacement. Migrating was a one-line change to the model ID in the endpoint — responseMimeType: "application/json" and responseSchema carried over untouched. Temperature behavior does shift between generations, though. Don't assume your tuned value transfers: run twenty of your own reviews through the new model side by side before trusting it. I kept 0.4 and saw nothing that bothered me, but that likely depends on your app's category. Current shutdown schedules live on the Gemini deprecations page.

One thing that surprised me: Reviews with very short bodies (1–3 words like "great app!" or "broken") generate low-confidence outputs. Gemini doesn't have enough context. These end up in the manual queue, which is fine — they're also the easiest to write manually.

One more operational note. App Store Connect API keys never expire on their own — they stay valid until you revoke them. Convenient, but leave one sitting in a Cloudflare Workers secret for over a year and you lose track of which key is used by which Worker. I now issue them on a six-month cadence under an ASC_KEY_REVIEW_YYYYMM naming scheme and revoke the old one only after confirming the cutover. I learned this the hard way: before I had that procedure, I revoked a key that was still in use and review fetching stopped for three days.

A Small First Step Worth Taking Today

Thank you for reading through to the end. From my own experience, the whole pipeline took about three weeks to assemble, but the first step is much smaller than that.

The one thing worth doing first is generating an App Store Connect API key and confirming that a single JWT-authenticated request returns 200 from the reviews endpoint on Cloudflare Workers locally. Once that works, the rest is "assembling parts that already work." If you get a 401 on your first attempt, remember the JWT clock-skew section above — backdating iat by five seconds is almost always the fix.

After auth is working, run the Gemini review-analysis module against three of your existing reviews. Just three. Reading the drafted replies is the fastest way to develop a feel for whether this pipeline is worth your time. For me, that calibration moment was what unlocked the motivation to finish the rest of the system.

The final step is generating the Rork dashboard and getting it onto your phone. From there, the daily flow becomes: notification arrives, open the screen, wait the deliberate three seconds, tap approve. Review responses shift from "thinking work" to "reading-the-user's-temperature work." It's a subtle distinction that's hard to convey in writing — you have to feel it.

I've come to think of this system less as a tool that saves 24 minutes per day and more as a tool that returns to me the time and attention needed to keep warmth in my replies. If that framing resonates with anyone else running indie apps, I'd be glad to have helped.