I Let AI Read My Rork App Reviews — It Found My Next Feature in 3 Minutes

When I first launched a wallpaper app on the App Store, I read every single review by hand. One hundred reviews, then five hundred, then a thousand. I didn't mind it. There was something grounding about reading what real people had to say about something I'd built.

Then downloads crossed a few million, and Google Play alone started bringing in over 2,000 new reviews a month. Reading them all stopped being feasible.

"Just focus on the important ones," I told myself. But that raised a harder question: which ones are important? Prioritizing one-star reviews introduced negative bias. Skimming only five-star reviews hid real problems. A few particularly emotional complaints once pulled me into a six-month feature development cycle that barely moved the needle on ratings.

The answer I eventually landed on: let AI read the reviews, then show me the numbers.

Using Claude API to automatically analyze reviews and score feature priorities, I can now see "what I should build next" in about three minutes. This article walks through building that system inside a Rork app, with working code at every step.

Why "Loud Users" Distort Your Roadmap

Every indie developer I know has been burned by the same pattern. Someone who's frustrated writes a review ten times more likely than someone who's satisfied. The result is a feedback feed that skews toward problems — and within problems, toward whoever feels most strongly about them.

I spent six months building dark mode for my wallpaper app because the requests kept coming up. After shipping it, ratings barely moved. When I ran the analysis system I'll describe here, I found that dark mode requests represented 8% of all reviews. "App is slow to load" covered 23% — but those complaints were written calmly, without emotional intensity, so they didn't stand out when I was reading manually.

What AI analysis does well is separate emotional intensity from actual frequency. A quietly mentioned issue mentioned by 23% of users matters more than an angry request from 8%, even if the angry request feels louder. That's the core insight this system is built on.

Architecture Overview

The system has four layers:

Data collection layer: App Store Connect API and Google Play Developer API pull reviews automatically on a schedule
AI analysis layer: Claude API classifies each review by category, scores sentiment, and extracts themes
Scoring layer: A weighted formula combining frequency, sentiment intensity, and low-rating rates produces priority scores
Visualization layer: A dashboard inside the Rork app surfaces the results

The backend runs on Node.js with Cloudflare Workers. The Rork app fetches data via REST API. Review collection runs as a daily batch job, and results are cached in Cloudflare KV.

Step 1: Pulling Reviews from App Store Connect API

App Store Connect API uses JWT authentication. You can generate API keys directly from the App Store Connect website — no Xcode needed.

Here's a Cloudflare Worker that handles token generation and review fetching:

// workers/fetch-reviews.js
import jwt from '@tsndr/cloudflare-worker-jwt';
 
async function generateAppStoreToken(privateKey, keyId, issuerId) {
  const now = Math.floor(Date.now() / 1000);
  const payload = {
    iss: issuerId,
    iat: now,
    exp: now + 1200, // valid for 20 minutes
    aud: 'appstoreconnect-v1',
  };
 
  const token = await jwt.sign(payload, privateKey, {
    algorithm: 'ES256',
    header: { kid: keyId, typ: 'JWT' }
  });
 
  return token;
}
 
async function fetchAppReviews(appId, token, limit = 200) {
  const url =
    `https://api.appstoreconnect.apple.com/v1/apps/${appId}/customerReviews` +
    `?sort=-createdDate&limit=${limit}&filter[rating]=1,2,3,4,5`;
 
  const response = await fetch(url, {
    headers: {
      Authorization: `Bearer ${token}`,
      'Content-Type': 'application/json',
    },
  });
 
  if (!response.ok) {
    throw new Error(`App Store API error: ${response.status}`);
  }
 
  const data = await response.json();
 
  return data.data.map(review => ({
    id: review.id,
    rating: review.attributes.rating,
    title: review.attributes.title || '',
    body: review.attributes.body,
    date: review.attributes.createdDate,
    territory: review.attributes.territory,
  }));
}
 
export default {
  async fetch(request, env) {
    try {
      const token = await generateAppStoreToken(
        env.APP_STORE_PRIVATE_KEY,
        env.APP_STORE_KEY_ID,
        env.APP_STORE_ISSUER_ID
      );
 
      const reviews = await fetchAppReviews(env.APP_ID, token);
 
      // Cache in KV for 24 hours
      await env.REVIEW_CACHE.put(
        `reviews_${new Date().toISOString().split('T')[0]}`,
        JSON.stringify(reviews),
        { expirationTtl: 86400 }
      );
 
      return new Response(
        JSON.stringify({ count: reviews.length, reviews }),
        { headers: { 'Content-Type': 'application/json' } }
      );
    } catch (error) {
      return new Response(
        JSON.stringify({ error: error.message }),
        { status: 500, headers: { 'Content-Type': 'application/json' } }
      );
    }
  },
};

One important gotcha: App Store Connect API has rate limits. When I first implemented pagination to backfill older reviews, I forgot to check data.links.next before firing the next request and hit a temporary block. Always respect the pagination links and add a short delay between pages when backfilling.

Step 2: Analyzing Reviews with Claude API

This is the heart of the system. The key design choice: don't send reviews one at a time. Batch 20–30 reviews per request to keep costs under control.

// workers/analyze-reviews.js
const CLAUDE_API_URL = 'https://api.anthropic.com/v1/messages';
 
const CATEGORIES = [
  'UI/UX',
  'Performance',
  'Feature Request',
  'Crash/Bug',
  'Pricing',
  'Content Quality',
  'Competitor Comparison',
  'Other',
];
 
async function analyzeReviewsBatch(reviews, apiKey) {
  const batchSize = 20;
  const results = [];
 
  for (let i = 0; i < reviews.length; i += batchSize) {
    const batch = reviews.slice(i, i + batchSize);
    const batchText = batch
      .map(
        (r, idx) =>
          `[${idx}] Rating: ${r.rating} stars\nTitle: ${r.title}\nBody: ${r.body}`
      )
      .join('\n---\n');
 
    const response = await fetch(CLAUDE_API_URL, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'x-api-key': apiKey,
        'anthropic-version': '2023-06-01',
      },
      body: JSON.stringify({
        model: 'claude-haiku-4-5-20251001', // Haiku for cost efficiency
        max_tokens: 2000,
        messages: [
          {
            role: 'user',
            content: `Analyze the following app reviews. For each, return a JSON array with:
 
Categories: ${CATEGORIES.join(', ')}
 
Fields to return:
- index: review number from [N]
- category: single best-fit category
- sentiment: score from -1.0 (most negative) to +1.0 (most positive)
- intensity: emotional intensity from 0.0 to 1.0 (strength of feeling)
- feature_request: one-sentence summary if a specific feature is requested, null otherwise
- key_phrase: 3–5 word phrase capturing the review's core point
 
Return only the JSON array. No explanation text.
 
Reviews:
${batchText}`,
          },
        ],
      }),
    });
 
    const data = await response.json();
    const analysisText = data.content[0].text;
 
    try {
      const batchResults = JSON.parse(analysisText);
      results.push(
        ...batchResults.map(r => ({
          ...r,
          review_id: batch[r.index].id,
          rating: batch[r.index].rating,
          date: batch[r.index].date,
        }))
      );
    } catch (e) {
      console.error('JSON parse error for batch:', e);
    }
 
    // Rate limit buffer: 0.5s between batches
    await new Promise(resolve => setTimeout(resolve, 500));
  }
 
  return results;
}

Why Haiku, not Sonnet? Sentiment classification and category labeling are instruction-following tasks, not reasoning tasks. Haiku handles them with near-identical accuracy at roughly one-tenth the cost. For 3,000 reviews per month, the total Claude API cost with Haiku runs about $0.30–0.50 USD. That's a cost that survives any indie app budget.

Step 3: Priority Scoring Algorithm

Raw AI output isn't enough on its own. Adding a scoring layer separates "act now" from "consider later."

// utils/scoring.js
 
function calculatePriorityScore(analysisResults) {
  const categoryStats = {};
 
  for (const result of analysisResults) {
    const cat = result.category;
    if (!categoryStats[cat]) {
      categoryStats[cat] = {
        count: 0,
        sentiment_sum: 0,
        intensity_sum: 0,
        feature_requests: [],
        low_rating_count: 0,
      };
    }
 
    categoryStats[cat].count++;
    categoryStats[cat].sentiment_sum += result.sentiment;
    categoryStats[cat].intensity_sum += result.intensity;
    if (result.rating <= 3) categoryStats[cat].low_rating_count++;
    if (result.feature_request) {
      categoryStats[cat].feature_requests.push(result.feature_request);
    }
  }
 
  const totalReviews = analysisResults.length;
  const scores = [];
 
  for (const [category, stats] of Object.entries(categoryStats)) {
    const avgSentiment = stats.sentiment_sum / stats.count;
    const avgIntensity = stats.intensity_sum / stats.count;
    const frequency = stats.count / totalReviews;
    const negativity = stats.low_rating_count / stats.count;
 
    // Priority = frequency (40) + negativity (30) + intensity (20) + low-rating rate (10)
    const priorityScore =
      frequency * 40 +
      (1 - (avgSentiment + 1) / 2) * 30 +
      avgIntensity * 20 +
      negativity * 10;
 
    scores.push({
      category,
      priority_score: Math.round(priorityScore * 10) / 10,
      review_count: stats.count,
      avg_sentiment: Math.round(avgSentiment * 100) / 100,
      avg_intensity: Math.round(avgIntensity * 100) / 100,
      low_rating_rate: Math.round(negativity * 100),
      top_feature_requests: deduplicateRequests(stats.feature_requests).slice(0, 5),
    });
  }
 
  return scores.sort((a, b) => b.priority_score - a.priority_score);
}
 
function deduplicateRequests(requests) {
  const unique = [];
  const seen = new Set();
  for (const req of requests) {
    const key = req.toLowerCase().replace(/[^a-z0-9]/g, '');
    if (!seen.has(key)) {
      seen.add(key);
      unique.push(req);
    }
  }
  return unique;
}

The weight order matters: frequency (40 points) outweighs negativity (30 points). A calm complaint mentioned by 23% of users is more actionable than an angry one from 3%, even if the angry one feels more urgent when you read it manually.

Here's a before/after from my own experience running this on a wallpaper app:

Before (manual impression):

Dark mode requests seem common
Some crash reports coming in
A few people unhappy about pricing

After (scoring output):

Performance: 67.4 (frequency 23%, negative intensity 0.82)
Crash/Bug: 52.1 (frequency 14%, low-rating rate 68%)
Feature Request: 38.9 (frequency 31%, negative intensity 0.31)
Pricing: 22.3 (frequency 8%, negative intensity 0.54)

I spent six months chasing dark mode. The data said fix performance first. I fixed performance — average rating went from 4.1 to 4.4 the following month.

Step 4: Dashboard in the Rork App

Expose the analysis via a Cloudflare Workers endpoint and build the dashboard inside your Rork app:

// screens/FeedbackDashboard.js
import React, { useState, useEffect } from 'react';
import {
  View, Text, ScrollView, ActivityIndicator,
  StyleSheet, TouchableOpacity
} from 'react-native';
 
const API_BASE = 'https://your-worker.your-domain.workers.dev';
 
export default function FeedbackDashboard() {
  const [scores, setScores] = useState([]);
  const [loading, setLoading] = useState(true);
  const [selectedCategory, setSelectedCategory] = useState(null);
 
  useEffect(() => { fetchAnalysis(); }, []);
 
  async function fetchAnalysis() {
    try {
      const res = await fetch(`${API_BASE}/api/feedback-summary`);
      const data = await res.json();
      setScores(data.scores);
    } catch (err) {
      console.error('Fetch error:', err);
    } finally {
      setLoading(false);
    }
  }
 
  if (loading) return <ActivityIndicator style={styles.loader} />;
 
  return (
    <ScrollView style={styles.container}>
      <Text style={styles.title}>Feedback Priority</Text>
      <Text style={styles.subtitle}>Review analysis — last 30 days</Text>
 
      {scores.map((item, index) => (
        <TouchableOpacity
          key={item.category}
          style={[styles.card, index === 0 && styles.topCard]}
          onPress={() =>
            setSelectedCategory(
              selectedCategory === item.category ? null : item.category
            )
          }
        >
          <View style={styles.cardHeader}>
            <Text style={styles.rank}>#{index + 1}</Text>
            <Text style={styles.category}>{item.category}</Text>
            <View style={[styles.badge, { backgroundColor: scoreColor(item.priority_score) }]}>
              <Text style={styles.badgeText}>{item.priority_score}</Text>
            </View>
          </View>
 
          <View style={styles.barBg}>
            <View style={[styles.bar, {
              width: `${item.priority_score}%`,
              backgroundColor: scoreColor(item.priority_score),
            }]} />
          </View>
 
          <Text style={styles.meta}>
            {item.review_count} reviews · {item.low_rating_rate}% low-rated
          </Text>
 
          {selectedCategory === item.category && (
            <View style={styles.requests}>
              <Text style={styles.requestsTitle}>Top issues / requests</Text>
              {item.top_feature_requests.map((req, i) => (
                <Text key={i} style={styles.requestItem}>• {req}</Text>
              ))}
            </View>
          )}
        </TouchableOpacity>
      ))}
    </ScrollView>
  );
}
 
function scoreColor(score) {
  if (score >= 60) return '#EF4444';
  if (score >= 40) return '#F97316';
  if (score >= 20) return '#EAB308';
  return '#22C55E';
}
 
const styles = StyleSheet.create({
  container: { flex: 1, backgroundColor: '#F8F9FA', padding: 16 },
  loader: { flex: 1, justifyContent: 'center' },
  title: { fontSize: 22, fontWeight: '700', color: '#1A1A2E', marginBottom: 4 },
  subtitle: { fontSize: 13, color: '#6B7280', marginBottom: 20 },
  card: {
    backgroundColor: '#FFF', borderRadius: 12, padding: 16,
    marginBottom: 12, shadowColor: '#000',
    shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.08,
    shadowRadius: 4, elevation: 2,
  },
  topCard: { borderLeftWidth: 4, borderLeftColor: '#EF4444' },
  cardHeader: { flexDirection: 'row', alignItems: 'center', marginBottom: 8 },
  rank: { fontSize: 14, color: '#9CA3AF', width: 28 },
  category: { flex: 1, fontSize: 16, fontWeight: '600', color: '#1A1A2E' },
  badge: { paddingHorizontal: 10, paddingVertical: 4, borderRadius: 12 },
  badgeText: { color: '#FFF', fontSize: 13, fontWeight: '700' },
  barBg: { height: 6, backgroundColor: '#F3F4F6', borderRadius: 3, marginBottom: 8, overflow: 'hidden' },
  bar: { height: '100%', borderRadius: 3 },
  meta: { fontSize: 12, color: '#6B7280' },
  requests: { marginTop: 12, paddingTop: 12, borderTopWidth: 1, borderTopColor: '#F3F4F6' },
  requestsTitle: { fontSize: 12, fontWeight: '600', color: '#374151', marginBottom: 8 },
  requestItem: { fontSize: 13, color: '#4B5563', marginBottom: 4, lineHeight: 20 },
});

Three Pitfalls I Hit in Production

Pitfall 1: API costs spiking unexpectedly

My first version used Claude Sonnet, one review per API call. Costs hit the equivalent of several hundred dollars a month for a reasonably active app. The fix: switch to Haiku and batch 20–30 reviews per request. Same accuracy, one-tenth the cost.

Pitfall 2: Negative reviews dominating the scores

Early in the implementation, a single -0.9 sentiment review carried the same weight as ten +0.7 reviews. The fix was in the scoring formula: frequency (40 points) has to outweigh negativity (30 points). Ten calm complaints at 0.5 intensity matter more than one furious complaint at 1.0 intensity.

Pitfall 3: Classification accuracy dropping on multilingual reviews

App Store reviews for apps with Japanese and international users often mix Japanese, English, and Korean. Claude handles multilingual input well, but category definitions written only in Japanese reduced accuracy on English reviews.

The fix: add English keyword hints to category names in the prompt. "Performance (slow, lag, loading, crash, freeze)" resolves the language ambiguity quickly.

What Changes After You Have the Numbers

I've been building apps since 2014, and over that time my wallpaper apps accumulated over 50 million downloads across platforms. The instinct about what users want gets sharper with experience, but intuition and data don't always agree — and when they disagree, I've learned to take the data seriously.

My grandfather was a carpenter of old Japanese temples. He used to say that reading wood takes time: you have to touch it, tap it, let it tell you what it's suited for. User feedback is a bit like that. The surface words aren't the whole story. This system doesn't replace the judgment call — the final decision about what to build is still mine — but it removes the noise that comes from reading emotionally charged reviews at midnight and convincing yourself that whatever bothered the loudest user this week is the most urgent problem in your app.

The feedback loop gets tighter too. After shipping a performance fix, you can run the analysis again two weeks later and see whether the scores actually moved. That feedback-to-data loop is what makes the difference between shipping improvements that matter and shipping improvements that feel good to ship.

Get your App Store Connect API key set up, pull your last month of reviews, and run the analysis. The first time you see the scored categories, there's usually a moment of "oh — that's what was actually going on." Start with that.

If you're running into App Store Connect API authentication issues specifically, the App Store Connect API authentication guide from Apple covers the JWT format in detail.