I Let AI Read My Rork App Reviews — It Found My Next Feature in 3 Minutes

When I first launched a wallpaper app on the App Store, I read every single review by hand. One hundred reviews, then five hundred, then a thousand. I didn't mind it. There was something grounding about reading what real people had to say about something I'd built.

Then the app kept growing, and Google Play alone started bringing in over 2,000 new reviews a month. Reading them all stopped being feasible.

"Just focus on the important ones," I told myself. But that raised a harder question: which ones are important? Prioritizing one-star reviews introduced negative bias. Skimming only five-star reviews hid real problems. A few particularly emotional complaints once pulled me into a six-month feature development cycle that barely moved the needle on ratings.

The answer I eventually landed on: let AI read the reviews, then show me the numbers.

Using Claude API to automatically analyze reviews and score feature priorities, I can now see "what I should build next" in about three minutes. This article walks through building that system inside a Rork app, with working code at every step.

Why "Loud Users" Distort Your Roadmap

Every indie developer I know has been burned by the same pattern. People who are frustrated tend to find it easier to write a review than people who are satisfied. The result is a feedback feed that skews toward problems — and within problems, toward whoever feels most strongly about them.

I spent six months building dark mode for my wallpaper app because the requests kept coming up. After shipping it, ratings barely moved. When I ran the analysis system I'll describe here, I found that dark mode requests represented 8% of all reviews. "App is slow to load" covered 23% — but those complaints were written calmly, without emotional intensity, so they didn't stand out when I was reading manually.

What AI analysis does well is separate emotional intensity from actual frequency. A quietly mentioned issue mentioned by 23% of users matters more than an angry request from 8%, even if the angry request feels louder. That's the core insight this system is built on.

Architecture Overview

The system has four layers:

Data collection layer: App Store Connect API and Google Play Developer API pull reviews automatically on a schedule
AI analysis layer: Claude API classifies each review by category, scores sentiment, and extracts themes
Scoring layer: A weighted formula combining frequency, sentiment intensity, and low-rating rates produces priority scores
Visualization layer: A dashboard inside the Rork app surfaces the results

The backend runs on Node.js with Cloudflare Workers. The Rork app fetches data via REST API. Review collection runs as a daily batch job, and results are cached in Cloudflare KV.

One constraint on the Google Play side is worth knowing up front. According to the official documentation, the reviews endpoint only returns reviews created or modified in roughly the last week, and star ratings without a comment are not included. GET requests are also limited to 200 per hour per app. In practice that means you have to pull daily and store the results yourself — a month of history can't be reconstructed later. For older reviews, download the CSV from Play Console.

Step 1: Pulling Reviews from App Store Connect API

App Store Connect API uses JWT authentication. You can generate API keys directly from the App Store Connect website — no Xcode needed.

Here's a Cloudflare Worker that handles token generation and review fetching:

// workers/fetch-reviews.js
import jwt from '@tsndr/cloudflare-worker-jwt';
 
async function generateAppStoreToken(privateKey, keyId, issuerId) {
  const now = Math.floor(Date.now() / 1000);
  const payload = {
    iss: issuerId,
    iat: now,
    exp: now + 1200, // valid for 20 minutes
    aud: 'appstoreconnect-v1',
  };
 
  const token = await jwt.sign(payload, privateKey, {
    algorithm: 'ES256',
    header: { kid: keyId, typ: 'JWT' }
  });
 
  return token;
}
 
async function fetchAppReviews(appId, token, limit = 200) {
  const url =
    `https://api.appstoreconnect.apple.com/v1/apps/${appId}/customerReviews` +
    `?sort=-createdDate&limit=${limit}&filter[rating]=1,2,3,4,5`;
 
  const response = await fetch(url, {
    headers: {
      Authorization: `Bearer ${token}`,
      'Content-Type': 'application/json',
    },
  });
 
  if (!response.ok) {
    throw new Error(`App Store API error: ${response.status}`);
  }
 
  const data = await response.json();
 
  return data.data.map(review => ({
    id: review.id,
    rating: review.attributes.rating,
    title: review.attributes.title || '',
    body: review.attributes.body,
    date: review.attributes.createdDate,
    territory: review.attributes.territory,
  }));
}
 
export default {
  async fetch(request, env) {
    try {
      const token = await generateAppStoreToken(
        env.APP_STORE_PRIVATE_KEY,
        env.APP_STORE_KEY_ID,
        env.APP_STORE_ISSUER_ID
      );
 
      const reviews = await fetchAppReviews(env.APP_ID, token);
 
      // Cache in KV for 35 days (a 30-day summary can't be built from 24 hours of data)
      await env.REVIEW_CACHE.put(
        `reviews_${new Date().toISOString().split('T')[0]}`,
        JSON.stringify(reviews),
        { expirationTtl: 60 * 60 * 24 * 35 }
      );
 
      return new Response(
        JSON.stringify({ count: reviews.length, reviews }),
        { headers: { 'Content-Type': 'application/json' } }
      );
    } catch (error) {
      return new Response(
        JSON.stringify({ error: error.message }),
        { status: 500, headers: { 'Content-Type': 'application/json' } }
      );
    }
  },
};

One important gotcha: App Store Connect API has rate limits, and a 429 means you've hit one. The remaining quota is reported in the X-Rate-Limit response header (check Apple's documentation for the exact format and limits). When I first implemented pagination to backfill older reviews, I forgot to check data.links.next before firing the next request and hit a temporary block. Always respect the pagination links and add a short delay between pages when backfilling.

Another slip of mine: I had set the KV expiry to 24 hours while the dashboard claimed to show "the last 30 days." It was really summarizing one day. I only noticed because the sample sizes looked oddly small. The code above now uses 35 days.

Step 2: Analyzing Reviews with Claude API

This is the heart of the system. The key design choice: don't send reviews one at a time. Batch 20–30 reviews per request to keep costs under control.

// workers/analyze-reviews.js
const CLAUDE_API_URL = 'https://api.anthropic.com/v1/messages';
 
const CATEGORIES = [
  'UI/UX',
  'Performance',
  'Feature Request',
  'Crash/Bug',
  'Pricing',
  'Content Quality',
  'Competitor Comparison',
  'Other',
];
 
async function analyzeReviewsBatch(reviews, apiKey) {
  const batchSize = 20;
  const results = [];
 
  for (let i = 0; i < reviews.length; i += batchSize) {
    const batch = reviews.slice(i, i + batchSize);
    const batchText = batch
      .map(
        (r, idx) =>
          `[${idx}] Rating: ${r.rating} stars\nTitle: ${r.title}\nBody: ${r.body}`
      )
      .join('\n---\n');
 
    const response = await fetch(CLAUDE_API_URL, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'x-api-key': apiKey,
        'anthropic-version': '2023-06-01',
      },
      body: JSON.stringify({
        model: 'claude-haiku-4-5-20251001', // Haiku for cost efficiency
        max_tokens: 2000,
        messages: [
          {
            role: 'user',
            content: `Analyze the following app reviews. For each, return a JSON array with:
 
Categories: ${CATEGORIES.join(', ')}
 
Fields to return:
- index: review number from [N]
- category: single best-fit category
- sentiment: score from -1.0 (most negative) to +1.0 (most positive)
- intensity: emotional intensity from 0.0 to 1.0 (strength of feeling)
- feature_request: one-sentence summary if a specific feature is requested, null otherwise
- key_phrase: 3–5 word phrase capturing the review's core point
 
Return only the JSON array. No explanation text.
 
Reviews:
${batchText}`,
          },
        ],
      }),
    });
 
    const data = await response.json();
    const analysisText = data.content[0].text;
 
    try {
      const batchResults = JSON.parse(analysisText);
      results.push(
        ...batchResults.map(r => ({
          ...r,
          review_id: batch[r.index].id,
          rating: batch[r.index].rating,
          date: batch[r.index].date,
        }))
      );
    } catch (e) {
      console.error('JSON parse error for batch:', e);
    }
 
    // Rate limit buffer: 0.5s between batches
    await new Promise(resolve => setTimeout(resolve, 500));
  }
 
  return results;
}

Why Haiku, not Sonnet? Sentiment classification and category labeling are instruction-following tasks, not reasoning tasks. Haiku handles them well enough for this job at about one-third the price. Per Anthropic's published pricing (as of October 2026), per million tokens, Haiku 4.5 is $1 input / $5 output and Sonnet 4.5 / 4.6 is $3 / $15.

Here is an estimate for 3,000 reviews a month, assuming 220 input and 120 output tokens per review plus prompt overhead — roughly 680k input and 360k output tokens.

Setup	Monthly estimate	Notes
Haiku 4.5 (standard)	about $2.5	$0.68 input + $1.80 output
Haiku 4.5 (Batch API)	about $1.2	50% discount per the docs; results arrive asynchronously
Sonnet 4.6 (standard)	about $7.4	About 3x for the same job

The token counts are assumptions, not measurements — check the usage field on your own data once. If a daily batch is fine for you, the Batch API is the cheapest route.

You Sent 20 Reviews and Got 17 Back

A few weeks into running this, I noticed something odd: feeding the same month of reviews through the pipeline twice produced different rankings. "Feature requests" and "Performance" kept trading places. Nothing in the code is random. The input was identical.

The cause is at the bottom of the function above.

} catch (e) {
  console.error('JSON parse error for batch:', e);
}

When a batch fails to parse, twenty reviews disappear in exchange for one log line. The caller sees a successful return. The scoring function computes percentages against whatever arrived, so nothing downstream can tell that anything went missing.

What makes this worse than a plain bug is that the loss isn't random.

With max_tokens: 2000 and twenty reviews times six fields, each review costs roughly 90–120 output tokens. Twenty reviews lands at 1,800–2,400 — right on the boundary. What pushes a batch over is the length of feature_request, and that field gets long when someone took the time to describe exactly what they wanted. The most informative reviews are the ones most likely to truncate the batch they're sitting in.

Truncation also always hits the tail of the array. If you queue reviews newest-first, the tail is the oldest reviews, so the loss skews in time as well.

There's a second failure mode that survives a count check:

review_id: batch[r.index].id

If the model returns an index one position off, batch[r.index] is undefined and reading .id throws. That map sits inside the try, so the exception lands in the same catch — one misnumbered item takes all twenty reviews with it.

Symptom	What's actually happening	How to confirm it
Rankings shift between identical runs	Whole batches drop, and a different batch drops each time	Log input count against `results.length` on every run
"Feature requests" is suspiciously underrepresented	Long `feature_request` values inflate output and trigger truncation	Count batches where `stop_reason` is `max_tokens`
Older reviews vanish consistently	Truncation cuts the tail of the array, which is the oldest half	Plot the dates of the missing `review_id` values
Twenty reviews disappear at once	A bad `index` makes `batch[r.index]` undefined and the map throws	Temporarily remove the range check and let the exception surface
Counts match but results look shuffled	The same `index` came back twice and the later one overwrote the earlier	Count duplicate `index` values per batch

The fix isn't a smarter prompt. It's reconciliation: compare what you sent against what came back, and re-fetch only the gap.

// workers/analyze-reviews.js (with reconciliation)
 
// Size the budget from measured output, on the high side
const TOKENS_PER_REVIEW = 140;
 
// The model sometimes wraps JSON in a code fence. Strip it before parsing
function stripFence(text) {
  return text.trim().replace(/^\x60{3}(?:json)?\s*/i, '').replace(/\s*\x60{3}$/, '');
}
 
// Validate ranges. Review text comes from strangers, so don't trust the output blindly either
function sanitize(item) {
  const sentiment = Number(item.sentiment);
  const intensity = Number(item.intensity);
  if (!Number.isFinite(sentiment) || !Number.isFinite(intensity)) return null;
  return {
    ...item,
    category: CATEGORIES.includes(item.category) ? item.category : 'Other',
    sentiment: Math.max(-1, Math.min(1, sentiment)),
    intensity: Math.max(0, Math.min(1, intensity)),
  };
}
 
async function callClaude(batch, apiKey) {
  const batchText = batch.map((r, idx) =>
    `[${idx}] Rating:${r.rating}\nTitle:${r.title}\nBody:${r.body}`
  ).join('\n---\n');
 
  const response = await fetch(CLAUDE_API_URL, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-api-key': apiKey,
      'anthropic-version': '2023-06-01'
    },
    body: JSON.stringify({
      model: 'claude-haiku-4-5-20251001',
      // Never a fixed number. Scale it with the batch
      max_tokens: Math.min(8000, batch.length * TOKENS_PER_REVIEW + 200),
      messages: [{ role: 'user', content: buildPrompt(batchText) }]
    })
  });
 
  if (!response.ok) throw new Error('HTTP ' + response.status);
 
  const data = await response.json();
 
  // Check this before attempting to parse. Truncation is visible here
  if (data.stop_reason === 'max_tokens') throw new Error('truncated');
 
  const parsed = JSON.parse(stripFence(data.content[0].text));
  if (!Array.isArray(parsed)) throw new Error('not an array');
  return parsed;
}
 
// Resolve one batch until every review is accounted for
async function resolveBatch(batch, apiKey, depth = 0) {
  const resolved = new Map(); // batch-local index -> analysis
 
  try {
    for (const item of await callClaude(batch, apiKey)) {
      const i = Number(item.index);
      // Drop out-of-range and duplicate indices here, before they can throw
      if (!Number.isInteger(i) || i < 0 || i >= batch.length) continue;
      if (resolved.has(i)) continue;
      const clean = sanitize(item);
      if (!clean) continue; // a corrupted number counts as unresolved and gets re-fetched
      resolved.set(i, {
        ...clean,
        review_id: batch[i].id,
        rating: batch[i].rating,
        date: batch[i].date
      });
    }
  } catch (e) {
    // Don't swallow it. Fall through to the retry below
  }
 
  const missing = batch.map((_, i) => i).filter(i => !resolved.has(i));
  if (missing.length === 0) {
    return { results: [...resolved.values()], failed: [] };
  }
 
  // If a single review still fails, record it and move on
  if (batch.length === 1 || depth >= 4) {
    return {
      results: [...resolved.values()],
      failed: missing.map(i => batch[i])
    };
  }
 
  const targets = missing.map(i => batch[i]);
  const mid = Math.ceil(targets.length / 2);
  const head = await resolveBatch(targets.slice(0, mid), apiKey, depth + 1);
  const tail = await resolveBatch(targets.slice(mid), apiKey, depth + 1);
 
  return {
    results: [...resolved.values(), ...head.results, ...tail.results],
    failed: [...head.failed, ...tail.failed]
  };
}
 
async function analyzeReviewsBatch(reviews, apiKey) {
  const batchSize = 20;
  const results = [];
  const failed = [];
 
  for (let i = 0; i < reviews.length; i += batchSize) {
    const r = await resolveBatch(reviews.slice(i, i + batchSize), apiKey);
    results.push(...r.results);
    failed.push(...r.failed);
    await new Promise(resolve => setTimeout(resolve, 500));
  }
 
  return {
    results,
    failed,
    lossRate: reviews.length === 0 ? 0 : failed.length / reviews.length
  };
}

Changing the return type from an array to { results, failed, lossRate } is deliberate. It makes the loss rate impossible for the caller to ignore.

sanitize is there for a reason. Review text is written by strangers and goes straight into your prompt, so something like "ignore your previous instructions" can show up. Wrap each review in <review> tags and say explicitly that the tag contents are data, not instructions. On the output side, a category allow-list and numeric ranges keep a bad response down to one misclassified review.

Also, if a batch fails with a 429 or 5xx, splitting it in half sends even more requests in the middle of a rate limit. In production, wait out the retry-after seconds and resend the same batch for HTTP 429 / HTTP 5xx, and reserve splitting for the case where a response came back but is incomplete.

const { results, failed, lossRate } = await analyzeReviewsBatch(reviews, apiKey);
 
if (lossRate > 0.03) {
  // Don't publish a ranking. Past 3%, the lower categories reorder
  return { status: 'degraded', lossRate, analyzed: results.length, dropped: failed.length };
}

Three percent is a practical threshold, not a derived one — it's roughly where the gap between my top two categories stopped being wider than the noise. With a smaller review volume, tighten it.

Splitting and re-fetching does mean more API calls, but only for the parts that failed, and it's still Haiku. My monthly bill barely moved. That's a poor trade to refuse when the alternative is a ranking you can't trust.

The dashboard now prints the loss rate next to the scores. Building this taught me that a ranked list arrives looking authoritative no matter how carelessly it was assembled, and I'd rather see the caveat than remember it.

Step 3: Priority Scoring Algorithm

Raw AI output isn't enough on its own. Adding a scoring layer separates "act now" from "consider later."

// utils/scoring.js
 
function calculatePriorityScore(allResults) {
  // Leave favorable reviews (sentiment 0.2 or higher) out of the "what to fix first" score
  const analysisResults = allResults.filter(r => r.sentiment < 0.2);
  const categoryStats = {};
 
  for (const result of analysisResults) {
    const cat = result.category;
    if (!categoryStats[cat]) {
      categoryStats[cat] = {
        count: 0,
        sentiment_sum: 0,
        intensity_sum: 0,
        feature_requests: [],
        low_rating_count: 0,
      };
    }
 
    categoryStats[cat].count++;
    categoryStats[cat].sentiment_sum += result.sentiment;
    categoryStats[cat].intensity_sum += result.intensity;
    if (result.rating <= 3) categoryStats[cat].low_rating_count++;
    if (result.feature_request) {
      categoryStats[cat].feature_requests.push(result.feature_request);
    }
  }
 
  const totalReviews = analysisResults.length;
  const maxCount = Math.max(...Object.values(categoryStats).map(st => st.count));
  const scores = [];
 
  for (const [category, stats] of Object.entries(categoryStats)) {
    const avgSentiment = stats.sentiment_sum / stats.count;
    const avgIntensity = stats.intensity_sum / stats.count;
    const share = stats.count / totalReviews; // share of the pool (for display)
    const frequency = stats.count / maxCount; // relative to the biggest category = 1.0 (for scoring)
    const negativity = stats.low_rating_count / stats.count;
 
    // Priority = frequency (40) + negativity (30) + intensity (20) + low-rating rate (10)
    const priorityScore =
      frequency * 40 +
      (1 - (avgSentiment + 1) / 2) * 30 +
      avgIntensity * 20 +
      negativity * 10;
 
    scores.push({
      category,
      priority_score: Math.round(priorityScore * 10) / 10,
      review_count: stats.count,
      share: Math.round(share * 100),
      avg_sentiment: Math.round(avgSentiment * 100) / 100,
      avg_intensity: Math.round(avgIntensity * 100) / 100,
      low_rating_rate: Math.round(negativity * 100),
      top_feature_requests: deduplicateRequests(stats.feature_requests).slice(0, 5),
    });
  }
 
  return scores.sort((a, b) => b.priority_score - a.priority_score);
}
 
function deduplicateRequests(requests) {
  const unique = [];
  const seen = new Set();
  for (const req of requests) {
    const key = req.toLowerCase().replace(/[^a-z0-9]/g, '');
    if (!seen.has(key)) {
      seen.add(key);
      unique.push(req);
    }
  }
  return unique;
}

The weight order matters: frequency (40 points) should outweigh negativity (30 points). A calm complaint mentioned by 23% of users is more actionable than an angry one from 3%, even if the angry one feels more urgent when you read it manually.

There is a trap here that I missed for a long time. My first version used frequency as a share of all reviews. Split across eight categories, even the biggest one rarely exceeds 0.2–0.3, so the "40 points" was really worth about 10.

Two made-up categories (compared only against each other)	Frequency	Avg sentiment	Intensity	Low-rating rate	Old total	Total with relative frequency
A: slow to launch	23%	-0.6	0.8	70%	56.2	87.0
B: rare but furious	5%	-0.9	0.9	100%	58.5	65.2

With the old formula, B — 5% of reviews — outranks A at 23% (these are straight calculations from the formula above). Making frequency relative to the biggest category puts A back on top. I also tried mixing in a category full of praise, and it climbed the ranking on frequency alone, which is why the function now drops favorable reviews first.

Check a scoring weight by how many points it actually moves in your data, not by the number written in the formula. I now print each term's real contribution once, every time I change the weights.

Here's a before/after from my own experience running this on a wallpaper app:

Before (manual impression):

Dark mode requests seem common
Some crash reports coming in
A few people unhappy about pricing

After (scoring output):

Performance: 67.4 (frequency 23%, negative intensity 0.82)
Crash/Bug: 52.1 (frequency 14%, low-rating rate 68%)
Feature Request: 38.9 (frequency 31%, negative intensity 0.31)
Pricing: 22.3 (frequency 8%, negative intensity 0.54)

I spent six months chasing dark mode. The data said fix performance first. I fixed performance — average rating went from 4.1 to 4.4 the following month.

Step 4: Dashboard in the Rork App

Expose the analysis via a Cloudflare Workers endpoint and build the dashboard inside your Rork app:

// screens/FeedbackDashboard.js
import React, { useState, useEffect } from 'react';
import {
  View, Text, ScrollView, ActivityIndicator,
  StyleSheet, TouchableOpacity
} from 'react-native';
 
const API_BASE = 'https://your-worker.your-domain.workers.dev';
 
export default function FeedbackDashboard() {
  const [scores, setScores] = useState([]);
  const [loading, setLoading] = useState(true);
  const [selectedCategory, setSelectedCategory] = useState(null);
 
  useEffect(() => { fetchAnalysis(); }, []);
 
  async function fetchAnalysis() {
    try {
      const res = await fetch(`${API_BASE}/api/feedback-summary`);
      const data = await res.json();
      setScores(data.scores);
    } catch (err) {
      console.error('Fetch error:', err);
    } finally {
      setLoading(false);
    }
  }
 
  if (loading) return <ActivityIndicator style={styles.loader} />;
 
  return (
    <ScrollView style={styles.container}>
      <Text style={styles.title}>Feedback Priority</Text>
      <Text style={styles.subtitle}>Review analysis — last 30 days</Text>
 
      {scores.map((item, index) => (
        <TouchableOpacity
          key={item.category}
          style={[styles.card, index === 0 && styles.topCard]}
          onPress={() =>
            setSelectedCategory(
              selectedCategory === item.category ? null : item.category
            )
          }
        >
          <View style={styles.cardHeader}>
            <Text style={styles.rank}>#{index + 1}</Text>
            <Text style={styles.category}>{item.category}</Text>
            <View style={[styles.badge, { backgroundColor: scoreColor(item.priority_score) }]}>
              <Text style={styles.badgeText}>{item.priority_score}</Text>
            </View>
          </View>
 
          <View style={styles.barBg}>
            <View style={[styles.bar, {
              width: `${item.priority_score}%`,
              backgroundColor: scoreColor(item.priority_score),
            }]} />
          </View>
 
          <Text style={styles.meta}>
            {item.review_count} reviews · {item.low_rating_rate}% low-rated
          </Text>
 
          {selectedCategory === item.category && (
            <View style={styles.requests}>
              <Text style={styles.requestsTitle}>Top issues / requests</Text>
              {item.top_feature_requests.map((req, i) => (
                <Text key={i} style={styles.requestItem}>• {req}</Text>
              ))}
            </View>
          )}
        </TouchableOpacity>
      ))}
    </ScrollView>
  );
}
 
function scoreColor(score) {
  if (score >= 60) return '#EF4444';
  if (score >= 40) return '#F97316';
  if (score >= 20) return '#EAB308';
  return '#22C55E';
}
 
const styles = StyleSheet.create({
  container: { flex: 1, backgroundColor: '#F8F9FA', padding: 16 },
  loader: { flex: 1, justifyContent: 'center' },
  title: { fontSize: 22, fontWeight: '700', color: '#1A1A2E', marginBottom: 4 },
  subtitle: { fontSize: 13, color: '#6B7280', marginBottom: 20 },
  card: {
    backgroundColor: '#FFF', borderRadius: 12, padding: 16,
    marginBottom: 12, shadowColor: '#000',
    shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.08,
    shadowRadius: 4, elevation: 2,
  },
  topCard: { borderLeftWidth: 4, borderLeftColor: '#EF4444' },
  cardHeader: { flexDirection: 'row', alignItems: 'center', marginBottom: 8 },
  rank: { fontSize: 14, color: '#9CA3AF', width: 28 },
  category: { flex: 1, fontSize: 16, fontWeight: '600', color: '#1A1A2E' },
  badge: { paddingHorizontal: 10, paddingVertical: 4, borderRadius: 12 },
  badgeText: { color: '#FFF', fontSize: 13, fontWeight: '700' },
  barBg: { height: 6, backgroundColor: '#F3F4F6', borderRadius: 3, marginBottom: 8, overflow: 'hidden' },
  bar: { height: '100%', borderRadius: 3 },
  meta: { fontSize: 12, color: '#6B7280' },
  requests: { marginTop: 12, paddingTop: 12, borderTopWidth: 1, borderTopColor: '#F3F4F6' },
  requestsTitle: { fontSize: 12, fontWeight: '600', color: '#374151', marginBottom: 8 },
  requestItem: { fontSize: 13, color: '#4B5563', marginBottom: 4, lineHeight: 20 },
});

Three Pitfalls I Hit in Production

Pitfall 1: API costs spiking unexpectedly

My first version used Claude Sonnet, one review per API call. At 1,000 reviews a day (30,000 a month) and the same assumptions as the estimate above, that comes to roughly $70 a month on Sonnet 4.6. The fix: switch to Haiku (about one-third the price) and batch 20–30 reviews per request. If you don't need real-time results, the Batch API takes another 50% off.

Pitfall 2: Negative reviews dominating the scores

Early in the implementation, a single -0.9 sentiment review carried the same weight as ten +0.7 reviews. The fix was in the scoring formula: frequency (40 points) has to outweigh negativity (30 points). Ten calm complaints at 0.5 intensity matter more than one furious complaint at 1.0 intensity.

Pitfall 3: Classification accuracy dropping on multilingual reviews

App Store reviews for apps with Japanese and international users often mix Japanese, English, and Korean. Claude handles multilingual input well, but category definitions written only in Japanese reduced accuracy on English reviews.

The fix: add English keyword hints to category names in the prompt. "Performance (slow, lag, loading, crash, freeze)" resolves the language ambiguity quickly.

What Changes After You Have the Numbers

The thing that took me longest to accept is that listening to users and implementing what users literally asked for are two different activities. Intuition about what people want does sharpen with time, but intuition and data don't always agree — and when they disagree, I've learned to take the data seriously.

A score is a starting question, not an answer. "Launch performance, 23%" is a good reason to go find out why the app feels slow; it doesn't tell you what to change. From there I go back and read ten raw reviews in that category by hand. What this system reduced wasn't the amount of reading — it was the hesitation about which reviews to read. The final call about what to build is still mine. What's gone is the part where I read an emotionally charged review at midnight and convinced myself that whatever bothered the loudest user this week was the most urgent problem in the app.

The feedback loop gets tighter too. After shipping a performance fix, you can run the analysis again two weeks later and see whether the scores actually moved. That feedback-to-data loop is what makes the difference between shipping improvements that matter and shipping improvements that feel good to ship.

Get your App Store Connect API key set up, pull your last month of reviews, and run the analysis. The first time you see the scored categories, there's usually a moment of "oh — that's what was actually going on." Just check that the input count matches the analyzed count before you let that moment land. The ranking can wait until the arithmetic holds.

Trust a ranking only when it can travel with its loss rate. That was the most expensive lesson in this build.

If you're running into App Store Connect API authentication issues specifically, the App Store Connect API authentication guide from Apple covers the JWT format in detail.