Building an AI-Powered Video Editor App with Rork Max

I uploaded a 47MB travel clip, watched the progress bar fill all the way to 100%, then opened the Supabase dashboard and found a 4.7MB file sitting there. Of the ten chunks I thought I had sent, only the last one had survived.

No error was ever thrown. The fault was in my own upload code: if you write to the same object path ten times with upsert enabled, the tenth write is the only one that remains. I had been overwriting a file while believing I was assembling one.

What follows is the implementation log, that mistake included — video upload, automatic subtitles, highlight detection, and Stripe credit billing, with Rork Max as the frontend. Working code, and the places where I got stuck along the way.

Why AI Video Editing Is a Real Opportunity for Solo Developers

CapCut and VLLO dominate the general-purpose video editing market, but that dominance creates a clear opening for niche AI tools. Apps that automatically extract highlights from meeting recordings, generate multilingual subtitles for travel content, or split cooking videos by cooking step — these specific use cases move slowly at large companies and are exactly where an indie developer can move fast and establish real user loyalty.

The critical architectural decision is where video processing happens. Attempting to run FFmpeg inside a React Native app is a trap: the binary size explodes, App Store review gets complicated, and device thermal limits cause failures on long videos. The only realistic approach for a production Rork Max app is to handle all heavy processing server-side, keeping the mobile client focused on upload, status display, and result presentation.

Rork Max makes this split natural. The generated SwiftUI frontend handles the user-facing experience, while Supabase Storage, Edge Functions, and external AI APIs handle the computation. This guide uses Supabase because Storage, Database, Edge Functions, and Realtime are all integrated — meaning you get real-time processing completion notifications without any additional infrastructure.

Architecture Overview — Data Flow and Technology Choices

Here is how the full system works end to end:

The user selects a video in the Rork Max app using expo-image-picker
The app uploads the video to Supabase Storage in 5MB chunks with progress tracking
After upload, the app calls a Supabase Edge Function with the storage path and processing options
The Edge Function calls OpenAI Whisper API for transcription and Google Gemini 2.5 Flash for highlight scene analysis
Results (VTT subtitles, highlight timestamps) are saved to the Supabase database
The app receives a real-time completion notification via Supabase Realtime (postgres_changes)
Credits are deducted from the user's balance atomically via a Postgres function

The technology stack:

Mobile frontend (Rork Max): expo-image-picker, expo-video, expo-file-system, @supabase/supabase-js
Storage: Supabase Storage (raw video, processed clips)
Database: Supabase (job metadata, subtitle data, user credits)
AI: OpenAI Whisper API (transcription), Google Gemini 2.5 Flash (scene analysis)
Video processing: Supabase Edge Function (Deno runtime) + FFmpeg WebAssembly
Payments: Stripe (credit packs)
Real-time: Supabase Realtime (postgres_changes subscription)

Resumable Upload — From "I Thought I Was Chunking" to TUS

That 4.7MB file came from code like this: read the video in 5MB slices, then call upload() on the same storage path with upsert: true for each slice.

Supabase Storage's upload() writes whole objects. It is not an append API. Ten writes to one key leave you with the tenth, and because upsert is set, nothing complains. Looking back, the silence was the expensive part — a loud failure would have cost me an afternoon instead of a week.

The Base64 round trip was the second problem. React Native has no built-in Buffer, and Base64 inflates the payload by roughly 1.33x before it ever reaches the network. Holding a few hundred megabytes in memory that way is not something a phone forgives.

Supabase Storage speaks the TUS protocol for resumable uploads. Chunking, retrying, and resuming belong to the protocol. What you write is configuration.

Push the heavy work to the server, and push the fragile work into the protocol. Upload-related support mail stopped arriving the week I deleted my own chunking loop.

// services/videoUpload.ts
import * as tus from 'tus-js-client';
import { supabase } from './supabaseClient';
 
const PROJECT_ID = 'YOUR_PROJECT_ID';
 
// Use the direct storage hostname for large files
const RESUMABLE_ENDPOINT =
  `https://${PROJECT_ID}.storage.supabase.co/storage/v1/upload/resumable`;
 
export interface PickedVideo {
  uri: string;   // the file:// URI returned by expo-image-picker
  name: string;
  type: string;  // 'video/mp4', etc.
}
 
export async function uploadVideoResumable(
  file: PickedVideo,
  bucketName: string,
  objectName: string,
  onProgress: (percent: number) => void
): Promise<string> {
  const { data: { session } } = await supabase.auth.getSession();
  if (!session) {
    throw new Error('You need to be signed in to upload');
  }
 
  return new Promise<string>((resolve, reject) => {
    const upload = new tus.Upload(file as unknown as tus.UploadInput, {
      endpoint: RESUMABLE_ENDPOINT,
      retryDelays: [0, 3000, 5000, 10000, 20000],
      headers: {
        authorization: `Bearer ${session.access_token}`,
        // only when overwriting the same path is acceptable
        // 'x-upsert': 'true',
      },
      uploadDataDuringCreation: true,
      removeFingerprintOnSuccess: true,
      metadata: {
        bucketName,
        objectName,
        contentType: file.type,
        cacheControl: '3600',
      },
      // Must stay at 6MB — the server implementation assumes it
      chunkSize: 6 * 1024 * 1024,
      onError: (error) => {
        reject(error instanceof Error ? error : new Error(String(error)));
      },
      onProgress: (uploaded, total) => {
        onProgress(Math.round((uploaded / total) * 100));
      },
      onSuccess: () => resolve(objectName),
    });
 
    // Resume an interrupted upload if one is on record
    upload.findPreviousUploads().then((previous) => {
      if (previous.length > 0) {
        upload.resumeFromPreviousUpload(previous[0]);
      }
      upload.start();
    });
  });
}

Four of these settings only made sense to me after running them in production.

Setting	Behavior	What it means in practice
chunkSize	Fixed at 6MB — do not change it	Tuning it down for weak networks is tempting, but the server assumes this value. Tune throughput by choosing the direct storage hostname instead
Upload URL lifetime	Valid for up to 24 hours	Nothing resumes past that window. Don't leave a "resume upload" affordance in the UI forever
Two clients, same upload URL	One succeeds, the rest get 409 Conflict	409 here is the design working, not a transient fault — keep it out of your retry path
Two clients, same path, different URLs	First to finish wins; with `x-upsert`, last to finish wins	Put the job ID in the object name and this branch stops mattering

One capability is genuinely missing on React Native. tus-js-client persists upload URLs through the Web Storage API, which React Native does not implement — so findPreviousUploads() always returns an empty array and resuming after an app restart does not work out of the box. If you want cross-session resume, pass your own urlStorage backed by AsyncStorage. Retrying through a dropped connection within the same session works as written.

Handling the URI the picker hands you

launchImageLibraryAsync from expo-image-picker copies the selected video into the app's cache and returns a file:// URI. On React Native, tus-js-client accepts an object with a uri property and resolves it internally, so the picker result can be passed straight through.

A ph:// URI shows up when you enumerate assets with expo-media-library instead. Those need getAssetInfoAsync() first, so you can read localUri. Same user action, different module, different URI contract.

Also worth updating if you're reading older code: passing ImagePicker.MediaTypeOptions.Videos to mediaTypes is deprecated. Pass an array of strings such as ['videos'].

// components/VideoUploader.tsx
import React, { useState } from 'react';
import {
  View, Text, TouchableOpacity, ActivityIndicator, StyleSheet
} from 'react-native';
import * as ImagePicker from 'expo-image-picker';
import { uploadVideoResumable } from '../services/videoUpload';
import { triggerVideoProcessing } from '../services/videoProcessing';
import { planHighlightRoute } from '../utils/videoValidation';
 
interface VideoUploaderProps {
  onJobStarted: (jobId: string) => void;
}
 
export function VideoUploader({ onJobStarted }: VideoUploaderProps) {
  const [uploading, setUploading] = useState(false);
  const [progress, setProgress] = useState(0);
  const [statusMessage, setStatusMessage] = useState('');
  const [error, setError] = useState<string | null>(null);
 
  const pickAndUpload = async () => {
    setError(null);
 
    const permission = await ImagePicker.requestMediaLibraryPermissionsAsync();
    if (!permission.granted) {
      setError('Permission to access videos is required. Please enable it in Settings.');
      return;
    }
 
    const result = await ImagePicker.launchImageLibraryAsync({
      mediaTypes: ['videos'],
      quality: 1,
      videoMaxDuration: 1800, // 30 minutes maximum
    });
 
    if (result.canceled || !result.assets?.[0]) return;
 
    const asset = result.assets[0];
    const { error: validationError, route } = planHighlightRoute(asset);
    if (validationError || !route) {
      setError(validationError ?? 'This video cannot be processed.');
      return;
    }
 
    if (!asset.uri || !asset.fileName) {
      setError('Could not read video file. Please try a different video.');
      return;
    }
 
    setUploading(true);
    setStatusMessage('Uploading video...');
 
    const objectName = `videos/${Date.now()}_${asset.fileName.replace(/[^a-zA-Z0-9._-]/g, '_')}`;
 
    let path: string;
    try {
      path = await uploadVideoResumable(
        { uri: asset.uri, name: asset.fileName, type: asset.mimeType ?? 'video/mp4' },
        'video-uploads',
        objectName,
        (p) => {
          setProgress(p);
          setStatusMessage(`Uploading... ${p}%`);
        }
      );
    } catch (err) {
      const message = err instanceof Error ? err.message : 'Unknown error';
      setError(`Upload failed: ${message}`);
      setUploading(false);
      return;
    }
 
    setStatusMessage('Starting AI processing...');
 
    const { jobId, error: processingError } = await triggerVideoProcessing(path, {
      generateSubtitles: true,
      detectHighlights: true,
      language: 'en',
      // tells the Edge Function whether to hand Gemini a URL or upload via the Files API
      highlightRoute: route,
    });
 
    if (processingError) {
      setError('Failed to start AI processing. Please try again.');
      setUploading(false);
      return;
    }
 
    onJobStarted(jobId);
    setStatusMessage('Processing in progress — you\'ll be notified when done');
    setUploading(false);
    setProgress(0);
  };
 
  return (
    <View style={styles.container}>
      <TouchableOpacity
        onPress={pickAndUpload}
        disabled={uploading}
        style={[styles.button, uploading && styles.buttonDisabled]}
        accessibilityRole="button"
        accessibilityLabel="Select video and start AI editing"
      >
        {uploading
          ? <ActivityIndicator color="#fff" />
          : <Text style={styles.buttonText}>Select Video — Start AI Editing</Text>
        }
      </TouchableOpacity>
 
      {statusMessage ? (
        <Text style={styles.statusText}>{statusMessage}</Text>
      ) : null}
 
      {progress > 0 && uploading ? (
        <View style={styles.progressBar}>
          <View style={[styles.progressFill, { width: `${progress}%` }]} />
        </View>
      ) : null}
 
      {error ? (
        <Text style={styles.errorText}>{error}</Text>
      ) : null}
    </View>
  );
}
 
const styles = StyleSheet.create({
  container: { padding: 20 },
  button: {
    backgroundColor: '#1a1a2e',
    padding: 16,
    borderRadius: 12,
    alignItems: 'center',
  },
  buttonDisabled: { backgroundColor: '#999' },
  buttonText: { color: '#fff', fontWeight: '700', fontSize: 16 },
  statusText: { marginTop: 12, color: '#555', textAlign: 'center' },
  progressBar: {
    marginTop: 8, height: 6, backgroundColor: '#eee',
    borderRadius: 3, overflow: 'hidden',
  },
  progressFill: { height: '100%', backgroundColor: '#3498db' },
  errorText: { marginTop: 12, color: '#e74c3c', textAlign: 'center' },
});

A note on the progress bar: tus-js-client reports bytes sent, so the number jumps in 6MB steps at chunk boundaries. Smoothing that out is possible, but distinguishing "sending" from "retrying" in the copy did far more for my support inbox than any animation did.

The Supabase Edge Function — Whisper and Gemini in One Pipeline

The Edge Function is the heart of the AI processing pipeline. It calls Whisper for transcription and Gemini Flash for highlight detection, then saves results to the database. The Supabase Realtime subscription on the client side will automatically pick up the database update and notify the app.

// supabase/functions/process-video/index.ts
import { serve } from 'https://deno.land/std@0.177.0/http/server.ts';
import { createClient } from 'https://esm.sh/@supabase/supabase-js@2';
 
const OPENAI_API_KEY = Deno.env.get('OPENAI_API_KEY') ?? '';
const GEMINI_API_KEY = Deno.env.get('GEMINI_API_KEY') ?? '';
const SUPABASE_URL = Deno.env.get('SUPABASE_URL') ?? '';
const SUPABASE_SERVICE_ROLE_KEY = Deno.env.get('SUPABASE_SERVICE_ROLE_KEY') ?? '';
 
interface Highlight {
  start: number;
  end: number;
  reason: string;
}
 
interface ProcessingJob {
  jobId: string;
  storagePath: string;
  userId: string;
  options: {
    generateSubtitles: boolean;
    detectHighlights: boolean;
    language: string;
  };
}
 
serve(async (req: Request) => {
  if (req.method !== 'POST') {
    return new Response('Method Not Allowed', { status: 405 });
  }
 
  let job: ProcessingJob;
  try {
    job = await req.json() as ProcessingJob;
  } catch {
    return new Response(JSON.stringify({ error: 'Invalid request body' }), {
      status: 400,
      headers: { 'Content-Type': 'application/json' },
    });
  }
 
  const supabase = createClient(SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY);
 
  // Mark job as in-progress immediately
  await supabase
    .from('processing_jobs')
    .update({ status: 'processing', started_at: new Date().toISOString() })
    .eq('id', job.jobId);
 
  try {
    // Generate a signed URL with 2-hour validity to handle long processing times
    const { data: signedUrlData, error: urlError } = await supabase.storage
      .from('video-uploads')
      .createSignedUrl(job.storagePath, 7200);
 
    if (urlError || !signedUrlData) {
      throw new Error(`Failed to generate storage URL: ${urlError?.message}`);
    }
 
    // Step 1: Whisper API transcription
    let transcriptionVtt: string | null = null;
    if (job.options.generateSubtitles) {
      const videoResponse = await fetch(signedUrlData.signedUrl);
      if (!videoResponse.ok) {
        throw new Error(`Failed to fetch video: HTTP ${videoResponse.status}`);
      }
      const videoBlob = await videoResponse.blob();
 
      if (videoBlob.size > 25 * 1024 * 1024) {
        // File too large — log and skip subtitles rather than failing the whole job
        console.warn(`File size ${videoBlob.size} bytes exceeds Whisper 25MB limit — skipping subtitles`);
      } else {
        const formData = new FormData();
        formData.append('file', videoBlob, 'audio.mp4');
        formData.append('model', 'whisper-1');
        formData.append('language', job.options.language || 'en');
        formData.append('response_format', 'vtt'); // VTT includes timestamps
 
        const whisperResponse = await fetch(
          'https://api.openai.com/v1/audio/transcriptions',
          {
            method: 'POST',
            headers: { 'Authorization': `Bearer ${OPENAI_API_KEY}` },
            body: formData,
          }
        );
 
        if (!whisperResponse.ok) {
          const errText = await whisperResponse.text();
          throw new Error(`Whisper API error (${whisperResponse.status}): ${errText}`);
        }
 
        transcriptionVtt = await whisperResponse.text();
      }
    }
 
    // Step 2: Gemini Flash highlight detection
    let highlights: Highlight[] = [];
    if (job.options.detectHighlights) {
      const geminiBody = {
        contents: [{
          parts: [
            {
              file_data: {
                mime_type: 'video/mp4',
                file_uri: signedUrlData.signedUrl,
              },
            },
            {
              text: `Analyze this video and identify up to 5 highlight moments.
Select moments that are emotionally engaging, action-packed, or contain important statements.
Aim for 20-40% of the total video length to be included in highlights.
Respond ONLY with this exact JSON format (no other text):
{"highlights":[{"start_sec":number,"end_sec":number,"reason":"brief explanation"}]}`,
            },
          ],
        }],
        generationConfig: {
          responseMimeType: 'application/json',
          maxOutputTokens: 1024,
          temperature: 0.3,
        },
      };
 
      const geminiResponse = await fetch(
        `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent?key=${GEMINI_API_KEY}`,
        {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify(geminiBody),
        }
      );
 
      if (geminiResponse.ok) {
        const geminiData = await geminiResponse.json();
        const jsonText = geminiData?.candidates?.[0]?.content?.parts?.[0]?.text ?? '{}';
        try {
          const parsed = JSON.parse(jsonText);
          highlights = (parsed.highlights ?? []).map((h: {
            start_sec: number;
            end_sec: number;
            reason: string;
          }) => ({
            start: h.start_sec,
            end: h.end_sec,
            reason: h.reason,
          }));
        } catch {
          // JSON parse failure — continue without highlights rather than failing the job
          console.warn('Failed to parse Gemini response JSON');
        }
      } else {
        console.warn(`Gemini API returned ${geminiResponse.status} — proceeding without highlights`);
      }
    }
 
    // Save results — Supabase Realtime will push this update to the client
    await supabase
      .from('processing_jobs')
      .update({
        status: 'completed',
        completed_at: new Date().toISOString(),
        result: {
          transcription_vtt: transcriptionVtt,
          highlights,
          video_url: signedUrlData.signedUrl,
        },
      })
      .eq('id', job.jobId);
 
    return new Response(
      JSON.stringify({
        success: true,
        jobId: job.jobId,
        hasTranscription: !!transcriptionVtt,
        highlightCount: highlights.length,
      }),
      { headers: { 'Content-Type': 'application/json' } }
    );
 
  } catch (err) {
    const errorMessage = err instanceof Error ? err.message : 'Unknown error';
    console.error('Edge Function error:', errorMessage);
 
    await supabase
      .from('processing_jobs')
      .update({
        status: 'failed',
        error_message: errorMessage,
        completed_at: new Date().toISOString(),
      })
      .eq('id', job.jobId);
 
    return new Response(JSON.stringify({ error: errorMessage }), {
      status: 500,
      headers: { 'Content-Type': 'application/json' },
    });
  }
});

A file_data.file_uri for video accepts a Files API URI, a public URL, or a pre-signed URL — the API fetches the content while processing the request. That path carries a size limit worth knowing about, which the pitfalls section covers below.

The code above calls gemini-2.5-flash. When you move up a generation, two things are worth checking before anything else: whether the model family supports URL-based file input at all (the Gemini 2.0 family does not), and how the response is constrained to JSON. Every time a model swap broke for me, it was one of those two.

Receiving Completion Notifications with Supabase Realtime

The client-side hook subscribes to postgres_changes on the processing_jobs table. When the Edge Function updates the row to status: 'completed', the subscription fires and the app can display results immediately.

// hooks/useProcessingStatus.ts
import { useEffect, useState, useCallback } from 'react';
import { supabase } from '../services/supabaseClient';
 
export type JobStatus = 'pending' | 'processing' | 'completed' | 'failed';
 
interface Highlight {
  start: number;
  end: number;
  reason: string;
}
 
interface ProcessingResult {
  transcription_vtt: string | null;
  highlights: Highlight[];
  video_url: string;
}
 
export function useProcessingStatus(jobId: string | null) {
  const [status, setStatus] = useState<JobStatus>('pending');
  const [result, setResult] = useState<ProcessingResult | null>(null);
  const [errorMessage, setErrorMessage] = useState<string | null>(null);
 
  const fetchCurrentStatus = useCallback(async () => {
    if (!jobId) return;
    const { data } = await supabase
      .from('processing_jobs')
      .select('status, result, error_message')
      .eq('id', jobId)
      .single();
 
    if (data) {
      setStatus(data.status as JobStatus);
      setResult(data.result as ProcessingResult | null);
      setErrorMessage(data.error_message as string | null);
    }
  }, [jobId]);
 
  useEffect(() => {
    if (!jobId) return;
 
    fetchCurrentStatus();
 
    const channel = supabase
      .channel(`processing-job-${jobId}`)
      .on(
        'postgres_changes',
        {
          event: 'UPDATE',
          schema: 'public',
          table: 'processing_jobs',
          filter: `id=eq.${jobId}`,
        },
        (payload) => {
          const updated = payload.new as {
            status: JobStatus;
            result: ProcessingResult | null;
            error_message: string | null;
          };
          setStatus(updated.status);
          setResult(updated.result);
          setErrorMessage(updated.error_message);
        }
      )
      .subscribe((subStatus) => {
        if (subStatus === 'SUBSCRIPTION_ERROR') {
          console.warn('Realtime subscription failed — falling back to polling');
        }
      });
 
    // Polling fallback for when the app is backgrounded or Realtime drops
    const pollInterval = setInterval(fetchCurrentStatus, 30_000);
 
    return () => {
      supabase.removeChannel(channel);
      clearInterval(pollInterval);
    };
  }, [jobId, fetchCurrentStatus]);
 
  return { status, result, errorMessage };
}

Credit-Based Stripe Monetization

Charging per AI feature consumed — rather than a flat subscription — makes sense for video editing because usage varies enormously between users. Someone who edits one video a week has very different value from someone processing dozens per day.

The credit cost constants give you flexibility to adjust pricing without touching code:

// constants/creditCosts.ts
export const CREDIT_COSTS = {
  SUBTITLE_GENERATION_PER_MIN: 2, // 2 credits per minute of video
  HIGHLIGHT_DETECTION: 5,          // 5 credits flat
  EXPORT_HD: 3,                    // 3 credits for HD export
  EXPORT_4K: 8,                    // 8 credits for 4K export
} as const;
 
export const CREDIT_PACKAGES = [
  { credits: 30,  priceUsd: 2.99, priceId: 'price_XXXXXXXXXX_30'  },
  { credits: 100, priceUsd: 7.99, priceId: 'price_XXXXXXXXXX_100' },
  { credits: 300, priceUsd: 19.99, priceId: 'price_XXXXXXXXXX_300' },
] as const;

Credit deduction must be atomic to prevent race conditions. Use a Postgres function with row-level locking:

-- supabase/migrations/001_credit_deduct_function.sql
CREATE OR REPLACE FUNCTION deduct_credits(
  p_user_id   UUID,
  p_amount    INTEGER,
  p_description TEXT
)
RETURNS BOOLEAN
LANGUAGE plpgsql
SECURITY DEFINER
AS $$
DECLARE
  v_current_credits INTEGER;
BEGIN
  -- Lock the row to prevent concurrent deductions
  SELECT credits
  INTO v_current_credits
  FROM user_credits
  WHERE user_id = p_user_id
  FOR UPDATE;
 
  IF v_current_credits IS NULL OR v_current_credits < p_amount THEN
    RETURN FALSE; -- insufficient credits
  END IF;
 
  UPDATE user_credits
  SET credits    = credits - p_amount,
      updated_at = NOW()
  WHERE user_id = p_user_id;
 
  INSERT INTO credit_transactions (user_id, amount, description, created_at)
  VALUES (p_user_id, -p_amount, p_description, NOW());
 
  RETURN TRUE;
END;
$$;

Call this function from within the Edge Function before starting AI processing. If it returns false, return an error to the client immediately — no API calls are made, so no costs are incurred.

The Stripe Checkout flow itself works best with session creation in an Edge Function and the success redirect handled via deep link back into the app. If you later move toward usage-based pricing, Usage-Based Billing for Rork AI Apps with Stripe Meter walks through that design in detail.

Six Production Pitfalls and How to Avoid Them

1. Video orientation rotates after upload

iOS records video with rotation metadata rather than actually rotating the pixels. FFmpeg processes the raw pixel data and ignores the metadata by default when filters are applied, causing the output to appear sideways or upside-down.

The correct fix is to apply an actual pixel-level transpose filter (-vf "transpose=1" for 90° clockwise) and then zero out the rotation metadata (-metadata:s:v:0 rotate=0). Modifying only the metadata is not reliable — some players honor it, others ignore it, and web playback is particularly inconsistent.

2. Whisper API's 25MB file size limit

Large videos regularly exceed Whisper's 25MB limit. The solution is to extract only the audio track before sending to Whisper. A 300MB video file typically reduces to under 15MB as MP3 audio (-vn -acodec libmp3lame -q:a 7 in FFmpeg). For very long videos, split the audio into 10-minute segments, transcribe each, and concatenate the VTT outputs.

3. Supabase Realtime WebSocket timeout

For videos requiring more than 10 minutes of processing time, the WebSocket connection will time out and the client won't receive the completion notification. The polling fallback in useProcessingStatus covers this for foreground sessions, but when the app is backgrounded on iOS, polling stops too. Add a call to the Expo Push Notifications API at the end of the Edge Function to send a push notification when processing completes. This is the only reliable path for long-running jobs.

4. Signed URL expiry during processing

The Edge Function fetches the video via a signed URL. If processing takes longer than the URL's validity period, Gemini's request to that URL fails with a 403. The 7200-second (2-hour) expiry in the code above handles most cases, but for very large files it's safer to re-generate the URL immediately before passing it to each AI API, rather than reusing the same URL for both Whisper and Gemini calls.

5. A signed URL only carries 100MB

This is the one that held me up longest. Passing a signed URL in file_uri is capped at 100MB per request. The 2GB ceiling my client-side validation allowed contradicted that limit outright, so here is the corrected version.

Anything above 100MB has to go through the Files API instead: upload it, then pass back the URI you get. The Files API accepts up to 2GB per file and keeps it for 48 hours. A duration cap (30 minutes, say) is still worth having for cost and latency reasons — but the byte boundary is the one you hit first.

Input method	Size limit	Persistence	Best for
Inline (base64 in the payload)	100MB per request	None	Short clips, quick tests
External URL (public or signed)	100MB per request	None — fetched each time	Videos already in Storage, under 100MB
Files API upload	2GB per file	48 hours	Anything larger, or reused across requests

// utils/videoValidation.ts
import type { ImagePickerAsset } from 'expo-image-picker';
 
const MAX_DURATION_MS = 30 * 60 * 1000;           // duration cap: cost and latency
const EXTERNAL_URL_LIMIT = 100 * 1024 * 1024;     // signed-URL input ceiling
const FILES_API_LIMIT = 2 * 1024 * 1024 * 1024;   // Files API ceiling
 
export type HighlightRoute = 'signed-url' | 'files-api';
 
export interface VideoPlan {
  error: string | null;
  route: HighlightRoute | null;
}
 
export function planHighlightRoute(asset: ImagePickerAsset): VideoPlan {
  if (asset.duration && asset.duration > MAX_DURATION_MS) {
    return {
      error: `Video too long (max ${MAX_DURATION_MS / 60_000} minutes). Please trim it first.`,
      route: null,
    };
  }
 
  const size = asset.fileSize ?? 0;
 
  if (size > FILES_API_LIMIT) {
    return { error: 'File too large (max 2GB).', route: null };
  }
 
  // 100MB decides whether we hand over a URL or upload to the Files API
  return { error: null, route: size > EXTERNAL_URL_LIMIT ? 'files-api' : 'signed-url' };
}

6. A failed URL fetch still returns a 200

One more behavior cost me real time. URLs passed in file_uri go through a content safety check, and when one is rejected you get a url_retrieval_status of URL_RETRIEVAL_STATUS_UNSAFE. Login-gated pages and expired URLs fail here too.

The awkward part is that the request still returns 200, with a plausible-looking highlights array attached. A model that never saw your video will happily invent five timestamped segments from the prompt alone. I only noticed because the reasons it gave described footage I had never uploaded.

The fix is ordinary: check url_retrieval_status — or verify that no highlight range exceeds the video's duration — and fall back to no highlights when anything looks off. Don't trust the returned JSON until you've confirmed the fetch succeeded. Skip that and all you're left with is complaints you can't reproduce.

Parsing VTT Subtitles for the In-App Player

Recent Expo SDK releases have deprecated expo-av in favor of expo-video, and neither renders VTT subtitle overlays natively. You'll need to parse the VTT string and display subtitle text manually based on the player's current position.

// utils/vttParser.ts
export interface Cue {
  startTime: number; // seconds
  endTime: number;
  text: string;
}
 
export function parseVTT(vttString: string): Cue[] {
  const cues: Cue[] = [];
  const blocks = vttString.split('\n\n').filter(Boolean);
 
  for (const block of blocks) {
    const lines = block.trim().split('\n');
    const timecodeIndex = lines.findIndex(l => l.includes('-->'));
    if (timecodeIndex === -1) continue;
 
    const [startStr, endStr] = lines[timecodeIndex]
      .split('-->')
      .map(s => s.trim());
 
    const parseTime = (t: string): number => {
      const parts = t.replace(',', '.').split(':');
      const seconds = parseFloat(parts[parts.length - 1]);
      const minutes = parseInt(parts[parts.length - 2] ?? '0', 10);
      const hours = parseInt(parts[parts.length - 3] ?? '0', 10);
      return hours * 3600 + minutes * 60 + seconds;
    };
 
    const text = lines.slice(timecodeIndex + 1).join('\n').trim();
    if (!text) continue;
 
    cues.push({
      startTime: parseTime(startStr),
      endTime: parseTime(endStr),
      text,
    });
  }
 
  return cues;
}
 
export function getCurrentCue(cues: Cue[], positionSeconds: number): Cue | null {
  return cues.find(
    c => positionSeconds >= c.startTime && positionSeconds <= c.endTime
  ) ?? null;
}

Use this in your video player component by subscribing to the timeUpdate event on expo-video's player. One detail that cost me an afternoon: the event is disabled by default, so set player.timeUpdateEventInterval = 0.1 before adding the listener or no updates will ever arrive. Read currentTime from the event payload, look up the matching cue, and render it as absolutely positioned text over the player.

Pricing the Credits — Working Backwards from API Costs

Credit pricing needs to account for API costs, or you'll run deficits on heavy users. Here's the math:

Whisper costs $0.006 per minute of audio. If you charge 2 credits per minute and sell 100 credits for $7.99, each credit is worth approximately $0.08. That is 13x the API cost — a healthy margin that covers infrastructure, Supabase, and the cost of free credits given to new users.

Gemini 2.5 Flash pricing for video input is approximately $0.01–$0.05 for a 5-minute video at current rates. At 5 credits ($0.40) per highlight detection job, the margin is solid even accounting for occasional model updates that shift pricing.

Build a monthly cost report into your admin tooling from day one. As user volume grows, aggregate API costs will surface patterns you can't see from individual transactions — things like certain video lengths consistently consuming more tokens than expected, or specific users running the AI pipeline at unusual frequency.

Decide early when credits actually leave the balance. Deducting on completion feels generous, but the moment a failed Edge Function job gets retried, nothing guarantees the charge and the work stay in sync. Charging at job start and writing a compensating row to credit_transactions when processing_jobs flips to failed keeps the ledger honest — and when a user emails asking where their credits went, the refund row answers the question for you.

For the broader question of how ads and paid features interact within one revenue model, Rork App Revenue Design: Where Ads and Subscriptions Eat Each Other approaches monetization from a different angle.

What to Build Next

The pipeline built in this guide — chunked upload, Whisper transcription, Gemini highlight detection, Realtime notification, Stripe credit billing — is a working foundation. The natural extensions from here are:

Subtitle editing UI: Whisper's accuracy is high but not perfect, especially on proper nouns and accented speech. Parse the VTT into editable text fields indexed by timestamp, let users correct errors, and persist the corrected VTT back to Supabase. This single feature meaningfully reduces churn because it shows users you respect their time enough to give them control over the output.

Multi-language subtitle export: Once you have the Whisper VTT, add a translation step using the OpenAI chat completions endpoint or Google Translate API. Charge 1 credit per language per minute of video. Users who create content for international audiences will use this repeatedly.

Thumbnail generation: Use Gemini's vision capabilities to analyze the highlight frames and identify the single best frame for a thumbnail — high visual clarity, expressive face or action, minimal motion blur. Offer this as a 2-credit add-on to every processed video. It is fast, cheap to run, and extremely useful for anyone posting to YouTube or App Store previews.

Start with supabase start, put the Edge Function on hot reload with supabase functions serve process-video, and then push one video of about 120MB through the pipeline. One file that crosses the 100MB boundary, deliberately. That single run exercises both the signed-URL path and the Files API path on your own machine. A 10MB test clip lets you ship without ever seeing the place that breaks.

Hand the chunking and resuming to the protocol, and keep the boundary decisions in code you own — that's the split I plan to carry into the next app.

Thank you for reading. If this spares even one person the week I spent on the 100MB wall, it was worth writing down.

Database Schema — Setting Up the Foundation

The Supabase schema needs three tables to support the full pipeline. Here is the migration that creates them:

-- supabase/migrations/000_initial_schema.sql
 
-- User credits balance
CREATE TABLE user_credits (
  user_id     UUID PRIMARY KEY REFERENCES auth.users(id) ON DELETE CASCADE,
  credits     INTEGER NOT NULL DEFAULT 0 CHECK (credits >= 0),
  created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at  TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
 
-- Individual credit transactions (purchases + deductions)
CREATE TABLE credit_transactions (
  id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id     UUID NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
  amount      INTEGER NOT NULL, -- positive = purchase, negative = consumption
  description TEXT NOT NULL,
  stripe_payment_intent_id TEXT, -- set for purchase transactions
  created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
 
-- Video processing jobs
CREATE TABLE processing_jobs (
  id           UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id      UUID NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
  storage_path TEXT NOT NULL,
  status       TEXT NOT NULL DEFAULT 'pending'
                CHECK (status IN ('pending', 'processing', 'completed', 'failed')),
  options      JSONB NOT NULL DEFAULT '{}',
  result       JSONB,
  error_message TEXT,
  started_at   TIMESTAMPTZ,
  completed_at TIMESTAMPTZ,
  created_at   TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
 
-- Row Level Security — users can only see their own jobs
ALTER TABLE user_credits ENABLE ROW LEVEL SECURITY;
ALTER TABLE credit_transactions ENABLE ROW LEVEL SECURITY;
ALTER TABLE processing_jobs ENABLE ROW LEVEL SECURITY;
 
CREATE POLICY "users_own_credits" ON user_credits
  FOR ALL USING (auth.uid() = user_id);
 
CREATE POLICY "users_own_transactions" ON credit_transactions
  FOR SELECT USING (auth.uid() = user_id);
 
CREATE POLICY "users_own_jobs" ON processing_jobs
  FOR ALL USING (auth.uid() = user_id);
 
-- Grant Edge Function service role access (bypasses RLS)
GRANT ALL ON user_credits TO service_role;
GRANT ALL ON credit_transactions TO service_role;
GRANT ALL ON processing_jobs TO service_role;
 
-- Enable Realtime on processing_jobs for client subscriptions
ALTER PUBLICATION supabase_realtime ADD TABLE processing_jobs;

The CHECK (credits >= 0) constraint at the database level is a final safety net. Even if the deduct_credits function has a bug, the database will refuse to write a negative balance. Defense in depth matters when money is involved.

The ALTER PUBLICATION supabase_realtime ADD TABLE processing_jobs line is easy to overlook and causes mysterious failures where the Realtime subscription never fires even though the Edge Function is updating the row correctly. Add this line to your migration before deploying to production.

Row Level Security policies here use auth.uid() which resolves to the JWT sub claim from Supabase Auth. The Edge Function uses the service role key, which bypasses RLS entirely — this is intentional, as the Edge Function needs to update any user's job record regardless of which user triggered it.

Deploying and Testing the Edge Function Locally

Supabase's local development stack mirrors production closely enough that you can test the full pipeline without deploying anything to the cloud. The workflow:

# Start all Supabase services locally (Postgres, Storage, Edge Functions, Realtime)
supabase start
 
# Run the database migration
supabase db reset
 
# Serve the Edge Function with hot reload
supabase functions serve process-video --env-file .env.local
 
# In .env.local:
# OPENAI_API_KEY=sk-...
# GEMINI_API_KEY=AIza...
# SUPABASE_URL=http://localhost:54321
# SUPABASE_SERVICE_ROLE_KEY=eyJ...  (from supabase status output)

With the function running locally, you can test it directly with curl before wiring up the mobile app:

curl -X POST http://localhost:54321/functions/v1/process-video \
  -H "Authorization: Bearer YOUR_SERVICE_ROLE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "jobId": "test-job-id",
    "storagePath": "videos/test.mp4",
    "userId": "test-user-id",
    "options": {
      "generateSubtitles": true,
      "detectHighlights": true,
      "language": "en"
    }
  }'

This reveals integration errors early — wrong API key format, CORS misconfigurations, storage bucket permissions — before they show up in the mobile app where debugging is slower.

Once local testing passes, deploy with:

supabase functions deploy process-video
supabase secrets set OPENAI_API_KEY=sk-... GEMINI_API_KEY=AIza...

The function will be live at https://<your-project-ref>.supabase.co/functions/v1/process-video. Update your triggerVideoProcessing service in the Rork Max app to call this URL, and the full pipeline is production-ready.

Building an AI-Powered Video Editor App with Rork Max: Auto Subtitles, Highlight Detection, and Stripe Monetization