Rork × LiveKit: Production Voice Agent Infrastructure for AI-Powered Apps

Chapter 3: The Token API

Connecting from the client to LiveKit requires a JWT. Embed "which room" and "how many seconds allowed" into that token.

Why Quota Checks Belong Here

It might seem simpler to do quota checks inside the agent. But then the user's room connects, and the agent silently refuses to respond. That reads as "the app connected but nothing happens" — a sure path to a one-star review. Reject the connection at token issuance.

Hono + Cloudflare Workers

// server/token-api/src/index.ts
import { Hono } from "hono";
import { cors } from "hono/cors";
import { AccessToken } from "livekit-server-sdk";
import { createClient } from "@supabase/supabase-js";
 
type Env = {
  LIVEKIT_API_KEY: string;
  LIVEKIT_API_SECRET: string;
  SUPABASE_URL: string;
  SUPABASE_SERVICE_KEY: string;
};
 
const app = new Hono<{ Bindings: Env }>();
app.use("*", cors({ origin: ["https://your-app.example"] }));
 
app.post("/token", async (c) => {
  const { authorization } = c.req.header();
  if (!authorization?.startsWith("Bearer ")) {
    return c.json({ error: "unauthorized" }, 401);
  }
 
  // 1. Verify user with Supabase
  const supabase = createClient(c.env.SUPABASE_URL, c.env.SUPABASE_SERVICE_KEY);
  const {
    data: { user },
    error,
  } = await supabase.auth.getUser(authorization.replace("Bearer ", ""));
  if (error || !user) return c.json({ error: "invalid token" }, 401);
 
  // 2. Check remaining seconds this month
  const { data: usage } = await supabase
    .from("voice_usage")
    .select("seconds_used, seconds_limit")
    .eq("user_id", user.id)
    .eq("period", new Date().toISOString().slice(0, 7)) // YYYY-MM
    .single();
 
  const remaining = (usage?.seconds_limit ?? 600) - (usage?.seconds_used ?? 0);
  if (remaining <= 30) {
    return c.json({ error: "quota_exceeded", remaining }, 402);
  }
 
  // 3. Mint a LiveKit JWT
  const roomName = `voice-${user.id}-${Date.now()}`;
  const at = new AccessToken(c.env.LIVEKIT_API_KEY, c.env.LIVEKIT_API_SECRET, {
    identity: user.id,
    ttl: Math.min(remaining, 1800), // 30 minutes or remaining, whichever is smaller
    metadata: JSON.stringify({ plan: user.app_metadata?.plan ?? "free" }),
  });
  at.addGrant({
    roomJoin: true,
    room: roomName,
    canPublish: true,
    canSubscribe: true,
  });
 
  return c.json({
    token: await at.toJwt(),
    url: "wss://your-project.livekit.cloud",
    room: roomName,
    secondsRemaining: remaining,
  });
});
 
export default app;

Expected success response:

{
  "token": "eyJhbGci...",
  "url": "wss://your-project.livekit.cloud",
  "room": "voice-abc-1713775200000",
  "secondsRemaining": 480
}

A Common Mistake: Setting the TTL Too High

If you ship with ttl: 86400 (one day), a user can reuse the same token repeatedly and bypass your quota. Rule of thumb: one token per call, 30 minutes maximum.

Chapter 4: The React Native Client from Rork

Now the client. When prompting Rork, give it something concrete like "a voice call app using LiveKit React Native, with a record button that starts a conversation and a stop button that ends it." Rork will scaffold the screen structure; the logic below is what we plug in.

Dependencies

In the Rork-generated project:

npx expo install @livekit/react-native @livekit/react-native-webrtc
npx expo install expo-av expo-permissions

Remember the microphone permission in app.config.ts. Forgetting this means iOS silently fails to acquire the audio session and the app stays mute forever.

// app.config.ts (excerpt)
export default {
  expo: {
    // ...
    ios: {
      infoPlist: {
        NSMicrophoneUsageDescription: "We use the microphone so you can have a voice conversation with the AI assistant.",
      },
    },
    android: {
      permissions: ["RECORD_AUDIO", "MODIFY_AUDIO_SETTINGS"],
    },
    plugins: ["@livekit/react-native-expo-plugin", "@config-plugins/react-native-webrtc"],
  },
};

LiveKit Client Wrapper

Keeping the connection logic out of the UI makes reconnect handling centralizable and tests actually writable.

// lib/livekit.ts
import { Room, RoomEvent, ConnectionState, Track, LocalAudioTrack } from "livekit-client";
 
export type VoiceConnectionState = "idle" | "connecting" | "connected" | "disconnected" | "error";
 
export class VoiceClient {
  private room: Room;
  private listeners = new Set<(s: VoiceConnectionState) => void>();
  private state: VoiceConnectionState = "idle";
 
  constructor() {
    this.room = new Room({
      adaptiveStream: true,
      dynacast: true,
      publishDefaults: {
        audioPreset: {
          maxBitrate: 32_000, // 32 kbps is plenty for voice-only
        },
      },
    });
 
    this.room.on(RoomEvent.ConnectionStateChanged, (cs) => {
      if (cs === ConnectionState.Connected) this.setState("connected");
      if (cs === ConnectionState.Disconnected) this.setState("disconnected");
      if (cs === ConnectionState.Reconnecting) this.setState("connecting");
    });
 
    this.room.on(RoomEvent.ConnectionQualityChanged, (quality, participant) => {
      if (quality === "poor") {
        console.warn("connection quality degraded", participant.identity);
      }
    });
  }
 
  async connect(url: string, token: string) {
    try {
      this.setState("connecting");
      await this.room.connect(url, token);
      const track = await LocalAudioTrack.createMicrophoneTrack({
        echoCancellation: true,
        noiseSuppression: true,
        autoGainControl: true,
      });
      await this.room.localParticipant.publishTrack(track);
    } catch (e) {
      this.setState("error");
      throw e;
    }
  }
 
  async disconnect() {
    await this.room.disconnect();
    this.setState("idle");
  }
 
  onState(cb: (s: VoiceConnectionState) => void) {
    this.listeners.add(cb);
    return () => this.listeners.delete(cb);
  }
 
  private setState(s: VoiceConnectionState) {
    this.state = s;
    this.listeners.forEach((l) => l(s));
  }
}

The Call Screen

Prompt Rork for "connecting, active, and error states with a large central button and subtle animation" and it will produce the scaffolding. The minimal logic looks like this.

// app/(tabs)/voice.tsx
import { useEffect, useRef, useState } from "react";
import { View, Text, Pressable, ActivityIndicator, StyleSheet } from "react-native";
import { registerGlobals } from "@livekit/react-native";
import { VoiceClient, VoiceConnectionState } from "../../lib/livekit";
import { requestToken } from "../../lib/api";
 
registerGlobals();
 
export default function VoiceScreen() {
  const [state, setState] = useState<VoiceConnectionState>("idle");
  const [remaining, setRemaining] = useState<number | null>(null);
  const [error, setError] = useState<string | null>(null);
  const clientRef = useRef<VoiceClient | null>(null);
 
  useEffect(() => {
    const client = new VoiceClient();
    clientRef.current = client;
    const unsub = client.onState(setState);
    return () => {
      unsub();
      client.disconnect();
    };
  }, []);
 
  const handleStart = async () => {
    try {
      setError(null);
      const token = await requestToken();
      if ("error" in token) {
        if (token.error === "quota_exceeded") {
          setError("You've used your free minutes this month. Please upgrade to continue.");
        } else {
          setError("We couldn't connect. Please try again shortly.");
        }
        return;
      }
      setRemaining(token.secondsRemaining);
      await clientRef.current?.connect(token.url, token.token);
    } catch (e) {
      setError((e as Error).message);
    }
  };
 
  const handleStop = async () => {
    await clientRef.current?.disconnect();
  };
 
  const isActive = state === "connected" || state === "connecting";
 
  return (
    <View style={styles.container}>
      {state === "connecting" && <ActivityIndicator size="large" />}
      {state === "connected" && (
        <Text style={styles.status}>Talking — {remaining}s remaining</Text>
      )}
      {error && <Text style={styles.error}>{error}</Text>}
 
      <Pressable
        onPress={isActive ? handleStop : handleStart}
        style={[styles.button, isActive && styles.buttonActive]}
      >
        <Text style={styles.buttonText}>{isActive ? "End call" : "Start call"}</Text>
      </Pressable>
    </View>
  );
}
 
const styles = StyleSheet.create({
  container: { flex: 1, justifyContent: "center", alignItems: "center", padding: 24 },
  status: { fontSize: 18, marginBottom: 24 },
  error: { color: "#c00", marginBottom: 12, textAlign: "center" },
  button: {
    width: 200,
    height: 200,
    borderRadius: 100,
    backgroundColor: "#4a90e2",
    justifyContent: "center",
    alignItems: "center",
  },
  buttonActive: { backgroundColor: "#e2574a" },
  buttonText: { color: "white", fontSize: 20, fontWeight: "600" },
});

Expected behavior: tapping the button flips state to connecting, then connected, and the agent plays its greeting.

Chapter 5: Production Pitfalls

Here's what actually bit me in production, in the order it happened.

Pitfall 1: No Audio on iOS

Symptom: the app connects, but you can't hear the agent, and the mic doesn't pick up anything either.

Cause: iOS needs AVAudioSession in playAndRecord category before LiveKit can open the mic. @livekit/react-native sets this up when you call registerGlobals() — forgetting that call fails silently.

Fix: call registerGlobals() at the top of the root component. If Rork's generated code doesn't include it, add it manually.

Pitfall 2: Works on TestFlight, Crashes in Production

Symptom: the build is fine in dev and TestFlight, then crashes right after connect on the App Store release.

Cause: Hermes aggressive optimization occasionally mangles a WebAssembly-adjacent helper that LiveKit uses internally.

Fix: in metro.config.js, preserve function names to keep the bundled runtime addressable by Hermes.

// metro.config.js
const { getDefaultConfig } = require("expo/metro-config");
const config = getDefaultConfig(__dirname);
config.transformer.minifierConfig = {
  keep_fnames: true,
  mangle: { keep_fnames: true },
};
module.exports = config;

Pitfall 3: Call Time Tracks Incorrectly

Symptom: the user talked for five minutes, but only 30 seconds are recorded.

Cause: if you fire the "call ended" event only from the client, a force-quit or crashed app will never emit it, and you lose the usage.

Fix: trust the server. LiveKit sends a room_finished webhook. Log from there.

// server/token-api/src/webhook.ts
app.post("/livekit-webhook", async (c) => {
  const event = await c.req.json();
  if (event.event === "room_finished") {
    const userId = event.room.name.split("-")[1];
    const duration = event.room.duration; // seconds
    const supabase = createClient(c.env.SUPABASE_URL, c.env.SUPABASE_SERVICE_KEY);
    await supabase.rpc("add_voice_seconds", { uid: userId, secs: duration });
  }
  return c.json({ ok: true });
});

Server-side truth means crashes and force-quits bill correctly.

Pitfall 4: Sluggish Response Times

Symptom: the user finishes speaking and the agent takes two to three seconds to respond.

Cause: serial STT → LLM → TTS stacks up each step's latency. Full gpt-4o first-token can push past 800ms on its own.

Fix, in descending impact order:

• Swap STT for gpt-4o-transcribe, which is faster than whisper-1
• Use gpt-4o-mini or Groq-hosted Llama 3.3 for the LLM
• Use a streaming-native TTS provider (Cartesia, ElevenLabs Turbo v2) that emits the first audio chunk inside 200ms
• For the truly latency-sensitive, bolt OpenAI Realtime API or Gemini Live API directly into LiveKit Agents — both have official plugins

Pitfall 5: App Store Rejection

Symptom: 4.0 rejection, cited "unclear microphone usage."

Cause: I had NSMicrophoneUsageDescription set to a vague "The app uses the microphone."

Fix: say exactly what it's for. "To hold a voice conversation with the AI assistant." Apple rejects ambiguous permission copy on purpose.

Chapter 6: Designing the Business Model

Shipping this in production only makes sense if you attach pricing to call minutes. For solo developers targeting ¥300,000+/month, I recommend three tiers.

Example Plan Design

• Free: 10 minutes (600 seconds) / month. Trial grease to reduce initial churn.
• Standard: ¥980/month, 3 hours (10,800 seconds). The sweet spot for most users.
• Pro: ¥2,480/month, 15 hours (54,000 seconds). For heavy users.

Stripe's metered billing is overkill at first. "Flat monthly + bolt-on packs when you hit the cap" covers the 90% case.

Server-Side Usage Update

-- Stored procedure in Supabase
CREATE OR REPLACE FUNCTION add_voice_seconds(uid uuid, secs int)
RETURNS void AS $$
DECLARE
  cur_period text := to_char(now(), 'YYYY-MM');
  user_limit int;
BEGIN
  SELECT CASE
    WHEN plan = 'pro' THEN 54000
    WHEN plan = 'standard' THEN 10800
    ELSE 600
  END INTO user_limit
  FROM users WHERE id = uid;
 
  INSERT INTO voice_usage (user_id, period, seconds_used, seconds_limit)
  VALUES (uid, cur_period, secs, user_limit)
  ON CONFLICT (user_id, period)
  DO UPDATE SET seconds_used = voice_usage.seconds_used + EXCLUDED.seconds_used;
END;
$$ LANGUAGE plpgsql;

Reconnect UX That Saves Churn

Apps that show "Call disconnected" when the network flickers get uninstalled. Instead:

1. On disconnect, show "Reconnecting…" and let LiveKit try its built-in auto-reconnect (5-second window)
2. If recovery fails after 5 seconds, say "Please check your connection and try again" — warm, not scolding
3. Cache the last 3–4 conversation turns locally so reconnect preserves context and doesn't feel like a cold start

These three changes moved my 30-day retention from 18% to 34%.

Chapter 7: Unit Economics and Break-Even

The scariest scenario in a voice product is "more users = bigger losses." Know your cost structure up front.

Cost per Minute (April 2026 ballpark)

• STT (gpt-4o-transcribe): ~$0.006/min
• LLM (gpt-4o-mini, ~2K tokens combined): ~$0.003/min
• TTS (OpenAI tts-1): ~$0.015/min
• LiveKit Cloud (audio-only): ~$0.002/min
• Total: ~$0.026/min (~¥4/min)

A Pro user (¥2,480/month, 900-minute cap) who actually uses the full quota costs you ¥3,600/month at those rates. You go underwater. Mitigations:

• Cap the cap: Pro is 15 hours, period; bursts are add-on packs
• Cheaper LLM: gpt-4o-mini → Gemini 2.5 Flash-Lite cuts this by ~3x
• Cheaper TTS: Cartesia Sonic at ~$0.005/min is perceptually indistinguishable for conversational use

That brings you down to ~¥2/min and keeps Pro at 60%+ gross margin.

Monitoring

Build a daily view so "underwater users" are visible.

CREATE VIEW voice_cost_monitor AS
SELECT
  u.email,
  vu.seconds_used,
  ROUND(vu.seconds_used * 0.0000667::numeric, 2) AS cost_usd,
  p.monthly_revenue_usd,
  CASE WHEN vu.seconds_used * 0.0000667 > p.monthly_revenue_usd
       THEN 'LOSS' ELSE 'OK' END AS status
FROM voice_usage vu
JOIN users u ON u.id = vu.user_id
JOIN plans p ON p.id = u.plan_id
WHERE vu.period = to_char(now(), 'YYYY-MM');

Check status = 'LOSS' users daily. Most are unintentional heavy users; a friendly email with usage tips usually resolves it.

Chapter 7.5: Observability — What You'll Actually Watch at 2 AM

The moment you have paying users, you'll find yourself waking up to support emails like "the app just stopped talking halfway through a call." Without the right traces, you'll have no idea whether the STT dropped, the LLM rate-limited you, or LiveKit rotated a media server. Invest one afternoon in observability up front — it pays back the first time production flinches.

The Three Signals That Matter

Not every metric is worth tracking. For a voice agent, these three tell you 90% of what's happening.

• End-of-turn latency: the time between "user stops speaking" and "agent starts speaking." Target under 1,200ms at p95. Above 2,000ms, people hang up.
• Mid-call disconnect rate: the fraction of calls that end before the user tapped stop. A healthy number is under 4%. Climbing past 8% usually points to a mobile-network regression, not your code.
• Per-call cost: the dollar cost of each call, broken down by provider. Spikes usually mean a runaway tool call loop or a TTS provider quietly switching to a higher-tier voice.

Wire It into Supabase in Under 50 Lines

A single call_events table plus two RPC calls is enough to answer the questions above.

CREATE TABLE call_events (
  id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id uuid REFERENCES users(id),
  room text NOT NULL,
  event text NOT NULL, -- 'turn_start', 'turn_end', 'error', 'disconnect'
  latency_ms int,
  cost_usd numeric(10, 6),
  metadata jsonb,
  created_at timestamptz DEFAULT now()
);
 
CREATE INDEX ON call_events (user_id, created_at DESC);
CREATE INDEX ON call_events (room);

Emit events from the agent process. Python makes this trivial.

# server/agent/telemetry.py
from typing import Optional
import httpx
 
SUPABASE_URL = "https://xxxxx.supabase.co"
SERVICE_KEY = "YOUR_SUPABASE_SERVICE_KEY"
 
async def emit_event(
    user_id: str,
    room: str,
    event: str,
    latency_ms: Optional[int] = None,
    cost_usd: Optional[float] = None,
    metadata: Optional[dict] = None,
):
    payload = {
        "user_id": user_id,
        "room": room,
        "event": event,
        "latency_ms": latency_ms,
        "cost_usd": cost_usd,
        "metadata": metadata or {},
    }
    async with httpx.AsyncClient(timeout=2.0) as client:
        try:
            await client.post(
                f"{SUPABASE_URL}/rest/v1/call_events",
                headers={
                    "apikey": SERVICE_KEY,
                    "Authorization": f"Bearer {SERVICE_KEY}",
                    "Content-Type": "application/json",
                    "Prefer": "return=minimal",
                },
                json=payload,
            )
        except httpx.TimeoutException:
            # Never let telemetry take down the agent
            pass

The two-second timeout matters. Telemetry must be strictly best-effort; blocking the audio loop to log something is worse than losing the log entry.

The Dashboard Query I Run Every Morning

SELECT
  date_trunc('hour', created_at) AS hour,
  COUNT(*) FILTER (WHERE event = 'turn_end') AS turns,
  ROUND(AVG(latency_ms) FILTER (WHERE event = 'turn_end'))::int AS avg_latency_ms,
  ROUND(PERCENTILE_CONT(0.95) WITHIN GROUP (ORDER BY latency_ms)
    FILTER (WHERE event = 'turn_end'))::int AS p95_latency_ms,
  COUNT(*) FILTER (WHERE event = 'error') AS errors,
  ROUND(SUM(cost_usd)::numeric, 2) AS cost_usd
FROM call_events
WHERE created_at > now() - interval '24 hours'
GROUP BY 1
ORDER BY 1 DESC;

When p95 latency crosses 2,000ms for more than two consecutive hours, I know to investigate before users start complaining.

Why Client-Side Metrics Lie

You may be tempted to ship the end-of-turn latency from the phone instead of the agent. Don't. Client-side latency numbers include the round-trip to LiveKit plus audio buffering, and they vary wildly by network. The agent-side measurement — between "VAD said 'user stopped'" and "TTS started outputting" — is the number that reflects your stack. That's what you optimize.

One Rule About Error Logs

Log the error, then swallow it, then continue. The user is still on the line. They'd rather hear "sorry, could you repeat that?" than hear nothing at all. In the Python VoiceAssistant subclass, override on_error to speak a graceful apology and continue listening. I learned this after shipping a version that silently died on the first rate-limit error, leaving users on a frozen call.

Chapter 8: What to Do Next

What you've gained here is the minimum map for a production voice agent. Where you grow from here depends on your goals.

Direction 1: Deepen Conversation Quality

The next stop is LiveKit Agents' function_calling. The pattern looks like this: define a set of tool functions the LLM can invoke mid-conversation, and the voice assistant will transparently pause, resolve the tool call, and continue speaking with the result embedded in the response. I use this for calendar booking ("can you schedule a 30-minute sync with Sato-san next Tuesday?"), RAG lookups against a knowledge base, and fetching the user's latest subscription status from Stripe. The trick is keeping each tool's execution under 300ms, because anything longer leaves the user in an awkward pause. Pre-warm your database connections and cache frequently used embeddings.

Direction 2: Scale Geographically

A single Fly.io region serves one continent comfortably. Past ~500 concurrent calls, or if you have a meaningful user base outside your home region, split workers across Tokyo, Singapore, and Virginia (or whichever trio covers your users). LiveKit Cloud routes users to the nearest media server automatically; you only need to make sure the agent worker is reachable from that server with low latency. Fly.io's fly scale count 2 --region nrt,sjc,iad does exactly this in one line.

Direction 3: Bring Your Own LLM

LiveKit Agents accepts arbitrary LLM plugins. If your product hinges on a fine-tuned model or an in-house inference stack, write a plugin that implements the LLM interface and swap it in. This also lets you bypass OpenAI's rate limits entirely when you scale past their production tier. The integration point is small — roughly 50 lines of adapter code between your inference endpoint and LiveKit's ChatContext.

What I Actually Shipped

Let me close with a concrete story, because abstract architecture talk only goes so far.

I used this stack to ship a voice-memo-with-AI-summary app and crossed ¥400,000 MRR three months after release. The real walls weren't technical — they were UX and unit economics. If you hit a wall while implementing any of the above, find me on X at @dolice. Happy to swap war stories.

The biggest lesson: ship a working voice call to ten real users before optimizing anything. Real users surface problems — background noise, weird accents, accidental barge-ins — that never appear in your test sessions. Everything in this guide came from watching real calls fail in specific ways and patching each failure one at a time.

For related implementation notes, see Rork × OpenAI Realtime API: Building a Voice Assistant App and Rork AI App Monetization Strategy: The Complete Guide.