The Three-Minute Video That Kept Failing — Moving Rork's Gemini Uploads Off the Worker

Hand out the destination, not the pipe

Resumable upload in the Files API happens in two stages.

Stage one sends metadata only, using the start command, and the response comes back with an X-Goog-Upload-URL header. Stage two sends the actual bytes to that URL.

The useful property is in that URL. The destination you get back carries its own session token embedded in the URL itself. Stage two needs no API key at all.

Which means only stage one has to live on the server. The key stays put, the device receives a single-use destination, and the video goes from the phone straight to Google without touching Cloudflare.

// ✅ The Worker only issues a destination. No video bytes pass through it.
interface Env {
  GEMINI_API_KEY: string;
  SESSIONS: KVNamespace;
}
 
const MAX_BYTES = 2 * 1024 * 1024 * 1024; // Files API caps a single file at 2GB
 
async function issueUploadUrl(request: Request, env: Env): Promise<Response> {
  const { sizeBytes, mimeType, userId } = await request.json<{
    sizeBytes: number;
    mimeType: string;
    userId: string;
  }>();
 
  // Reject before issuing. Letting someone upload 2GB and then saying no wastes their data plan.
  if (!Number.isInteger(sizeBytes) || sizeBytes <= 0 || sizeBytes > MAX_BYTES) {
    return Response.json({ error: 'invalid_size' }, { status: 400 });
  }
  if (!['video/mp4', 'video/quicktime'].includes(mimeType)) {
    return Response.json({ error: 'unsupported_type' }, { status: 415 });
  }
 
  // Issuing a destination is effectively reserving an analysis, so meter it per user
  const quotaKey = `upload_quota:${userId}:${new Date().toISOString().slice(0, 10)}`;
  const used = Number((await env.SESSIONS.get(quotaKey)) ?? '0');
  if (used >= 20) {
    return Response.json({ error: 'daily_quota_exceeded' }, { status: 429 });
  }
 
  const initResponse = await fetch(
    `https://generativelanguage.googleapis.com/upload/v1beta/files?key=${env.GEMINI_API_KEY}`,
    {
      method: 'POST',
      headers: {
        'X-Goog-Upload-Protocol': 'resumable',
        'X-Goog-Upload-Command': 'start',
        'X-Goog-Upload-Header-Content-Length': String(sizeBytes),
        'X-Goog-Upload-Header-Content-Type': mimeType,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ file: { display_name: `v_${Date.now()}` } }),
    }
  );
 
  const uploadUrl = initResponse.headers.get('X-Goog-Upload-URL');
  if (!uploadUrl) {
    return Response.json({ error: 'init_failed' }, { status: 502 });
  }
 
  await env.SESSIONS.put(quotaKey, String(used + 1), { expirationTtl: 86400 });
 
  // The session token lives in this URL. The API key never leaves the Worker.
  return Response.json({ uploadUrl, sizeBytes });
}

Taking sizeBytes from the client may look sloppy. It is self-reported, and nothing stops someone from lying.

But a lie here only hurts the liar. Send more bytes than declared and Google rejects them; send fewer and the finalize never completes. What the server is protecting is the key and the issuance count, not the honesty of a number. I made peace with that distinction.

The per-user daily quota exists because issuing a destination is, in practice, a reservation for one analysis. Stopping someone here means the ceiling gets hit before the analysis is ever dispatched.

Sending straight from the device

The app side gets simpler: request a destination, then send the file to it.

The classic React Native trap here is FormData. Reach for a Blob the way you would in a browser and nothing works — React Native wants an object with uri, type, and name.

Except this time it is not multipart at all. Stage two of a resumable upload just wants raw bytes in the body. Expo's upload task will stream a file straight from disk, which means the video never enters the JS heap on the device either. That turns out to matter as much on the phone as it did on the Worker.

// app/lib/uploadVideo.ts
import * as FileSystem from 'expo-file-system/legacy'; // SDK 54 and later: see below
import * as ImagePicker from 'expo-image-picker';
 
const API_BASE = 'https://your-worker.example.workers.dev';
 
export type UploadResult = { fileUri: string; sizeBytes: number };
 
export async function pickAndUploadVideo(
  userId: string,
  onProgress?: (ratio: number) => void
): Promise<UploadResult | null> {
  const permission = await ImagePicker.requestMediaLibraryPermissionsAsync();
  if (!permission.granted) return null;
 
  const picked = await ImagePicker.launchImageLibraryAsync({
    mediaTypes: ['videos'],
    quality: 1,
  });
  if (picked.canceled || !picked.assets[0]) return null;
 
  const asset = picked.assets[0];
  const info = await FileSystem.getInfoAsync(asset.uri);
  if (!info.exists) return null;
 
  const sizeBytes = info.size ?? 0;
  const mimeType = asset.mimeType ?? 'video/mp4';
 
  // Step 1: ask the server for a destination
  const issued = await fetch(`${API_BASE}/api/upload-url`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ sizeBytes, mimeType, userId }),
  });
  if (!issued.ok) {
    const { error } = await issued.json();
    throw new Error(`Could not get an upload destination (${error})`);
  }
  const { uploadUrl } = await issued.json();
 
  // Step 2: device to Google, directly. The Worker is not in this path.
  const task = FileSystem.createUploadTask(
    uploadUrl,
    asset.uri,
    {
      httpMethod: 'POST',
      uploadType: FileSystem.FileSystemUploadType.BINARY_CONTENT,
      headers: {
        'Content-Length': String(sizeBytes),
        'X-Goog-Upload-Offset': '0',
        'X-Goog-Upload-Command': 'upload, finalize',
      },
    },
    (p) => onProgress?.(p.totalBytesSent / p.totalBytesExpectedToSend)
  );
 
  const response = await task.uploadAsync();
  if (!response || response.status >= 300) {
    throw new Error(`Upload failed (${response?.status})`);
  }
 
  const body = JSON.parse(response.body) as {
    file: { uri: string; state: string };
  };
  return { fileUri: body.file.uri, sizeBytes };
}

BINARY_CONTENT is the load-bearing choice. Switch it to MULTIPART and boundary strings wrap the payload, and Google can no longer read it as a video. It fails quietly, which cost me an evening.

Honest progress reporting came free with the change. Relaying through the Worker, the progress bar hit 100% the moment the upload to Cloudflare finished — and then the user stared at a full bar for another 40 seconds on a three-minute video while Cloudflare pushed the bytes to Google. Now the bar tracks the last byte.

The morning the function was gone

That code needs a footnote.

Expo SDK 54 rebuilt expo-file-system. The new API that shipped as expo-file-system/next back in SDK 52 was promoted to stable, the next entry point disappeared, and the old API moved to expo-file-system/legacy. Which means: leave your import as expo-file-system, bump the SDK, and FileSystem.createUploadTask is simply not there anymore. If you're lucky the types catch it. In plain JavaScript you get undefined is not a function at the moment the user picks a video, and it took me a few round trips through the logs to realize the failure wasn't in the upload — it was in the thing that was supposed to start the upload.

That's why the import above points at expo-file-system/legacy. It works. But the legacy upload helpers are deprecated as of SDK 54, and the recommended path now is expo/fetch.

The new File class implements the Blob interface, so it drops straight into an expo/fetch body. Since the second leg of a resumable upload is nothing but raw bytes, that's arguably a better fit than the helper ever was.

// app/lib/uploadVideoModern.ts — the SDK 54+ shape
import { fetch } from 'expo/fetch';
import { File } from 'expo-file-system';
 
export async function putVideo(uploadUrl: string, localUri: string) {
  const file = new File(localUri);
 
  // File is a Blob, so it goes in the body directly and never lands in the JS heap
  const res = await fetch(uploadUrl, {
    method: 'POST',
    headers: {
      'Content-Length': String(file.size),
      'X-Goog-Upload-Offset': '0',
      'X-Goog-Upload-Command': 'upload, finalize',
    },
    body: file,
  });
 
  if (!res.ok) throw new Error(`upload_failed_${res.status}`);
  return (await res.json()) as { file: { uri: string; state: string } };
}

Shorter. No FileSystemUploadType to specify, so no way to confuse BINARY_CONTENT with MULTIPART.

There is a cost. expo/fetch gives you no upload progress callback — the very progress I was congratulating myself about one paragraph ago. On a thin connection, a three-minute video means roughly 40 seconds of silence.

I'm still on legacy for now. Uploading a video is the longest wait this app asks of anyone, and a progress bar that appears frozen there costs me more than one deprecation warning in the build log does. Deprecated is not a deadline; it's notice that a destination exists. I'd rather move once I can render progress myself.

If I were starting fresh, I'd probably go the other way. For anything where progress doesn't need showing — syncing a settings file, shipping a log bundle — expo/fetch is the simpler choice from day one.

How long ACTIVE takes scales with the video

A finished upload is not a usable file. The Files API marks freshly uploaded content as PROCESSING, and generateContent cannot reference it until the state turns ACTIVE.

My original code waited in a fixed loop: 2 seconds, ten times. Twenty seconds total.

Fine for 28 seconds of footage. Not fine for three minutes. This was the second reason short clips felt reliable — a fixed loop manufactures confidence on small inputs and quietly collapses on large ones.

If the wait scales with the video, the budget should too.

// workers/src/waitForActive.ts
type FileState = 'PROCESSING' | 'ACTIVE' | 'FAILED';
 
export async function waitForActive(
  fileName: string,
  durationSec: number,
  env: Env
): Promise<void> {
  // Measured around one sixth of the video's length; allow 3x headroom, floor at 15s
  const budgetMs = Math.max(15_000, Math.ceil(durationSec / 6) * 3 * 1000);
  const deadline = Date.now() + budgetMs;
 
  let waitMs = 1_000;
  while (Date.now() < deadline) {
    const res = await fetch(
      `https://generativelanguage.googleapis.com/v1beta/files/${fileName}?key=${env.GEMINI_API_KEY}`
    );
    const { state } = await res.json<{ state: FileState }>();
 
    if (state === 'ACTIVE') return;
    if (state === 'FAILED') throw new Error('file_processing_failed');
 
    await new Promise((r) => setTimeout(r, waitMs));
    waitMs = Math.min(waitMs * 1.6, 8_000); // back off, cap at 8s
  }
  throw new Error(`file_not_active_within_${budgetMs}ms`);
}

Backing off is partly a cost decision. Polling is not billed, but it spends Worker CPU time and subrequests. One-second polling across three minutes means 180 calls, which walks you toward the subrequest ceiling for no benefit. Starting at one second and capping at eight brought a three-minute wait down to roughly 30 calls.

Checking for FAILED explicitly matters more than it looks. Corrupt files and unsupported codecs land there. If you only watch the clock, a heavy video and a broken video produce the same error message — and one of them gets better if you wait while the other never will. Blur that line and you have built a retry button that can be pressed forever.

A fileUri expires in 48 hours

Files uploaded through the Files API are deleted automatically after 48 hours. While they exist you can read metadata, but you cannot download them back.

I filed that away as irrelevant. The analysis runs seconds after the upload; who needs 48 hours?

Then I added a re-ask feature — open a past result, ask a different question about the same footage. I stored the fileUri on the device and reused it. Everything worked beautifully until the next morning, when history entries started returning 403.

A fileUri is not an identifier. It is a 48-hour lease. Conflate the two and you ship something that works perfectly through launch day and starts generating support mail on day two.

So I split what gets stored.

The KV TTL is 47 hours rather than 48 on purpose. A matching TTL creates a window where KV still has the reference but Google has already dropped the file. Giving up an hour moves the expiry decision entirely to my side of the line — better to treat it as gone than to fetch it and catch a 403.

Inside the window, follow-up questions skip the upload entirely. Outside it, the app quietly re-uploads from the copy still sitting on the device. Either way the user just asked again.

There's a second ceiling that bites well before the 48 hours does: the Files API stores 20GB per project. Two gigabytes for any single file, twenty for everything you have.

Going direct made that number much harder to see. When the Worker carried the bytes, a day's volume was one log query away. Now the Worker knows only how many destinations it handed out.

At 96MB a video, roughly two hundred uploads fill the project. For an indie app that's not an abstract number — it's one decent week of attention after launch. And the failure surfaces during the start call, so from the user's side the symptom is "I pressed upload and nothing happened," which is the hardest kind of broken to explain.

So the issuer now tallies declared bytes alongside the count. It's a few lines on top of issueUploadUrl.

// Track declared bytes, not just issuances, against the 20GB project ceiling
const bytesKey = `upload_bytes:${new Date().toISOString().slice(0, 10)}`;
const bytesUsed = Number((await env.SESSIONS.get(bytesKey)) ?? '0');
 
const SOFT_CEILING = 14 * 1024 * 1024 * 1024; // start sweeping at 14 of the 20
if (bytesUsed + sizeBytes > SOFT_CEILING) {
  // Don't block the issuance — clean up first
  await sweepExpiredFiles(env);
}
await env.SESSIONS.put(bytesKey, String(bytesUsed + sizeBytes), { expirationTtl: 172800 });

sweepExpiredFiles walks files.list and calls files.delete on anything already analyzed, reclaiming space without waiting out the 48 hours. The KV total is a sum of self-reported sizes, so it isn't an accurate balance. I treat it as a signal that it's time to sweep, nothing more.

A limit you route around is a limit you stop seeing. That was the most useful thing the rewiring taught me. Shortening a path makes it faster, and it also quietly retires whatever was doing the counting along the way. Writing down what the middle was measuring, before removing the middle, would have saved me this particular detour.

Resolution and frame rate decide the bill

With the path sorted out, cost was next.

Video token consumption turned out to be far more legible than I assumed. By default the API samples one frame per second and spends roughly 300 tokens per second of video. Drop media_resolution to low and each frame costs 66 tokens, which — with audio at about 32 tokens per second and some metadata — lands near 100 tokens per second of video.

That is a 3x spread. A model with a 1M context window can take about an hour of video at default resolution, or three hours at low.

I had started writing my own frame-decimation code. I did not need it. videoMetadata accepts an fps argument, and startOffset / endOffset will clip a range server-side.

// workers/src/analyze.ts
type Intent = 'quick_summary' | 'form_coaching' | 'lecture_notes';
 
interface VideoPlan {
  model: string;
  mediaResolution: 'MEDIA_RESOLUTION_LOW' | 'MEDIA_RESOLUTION_MEDIUM';
  fps: number;
}
 
// Branch on what you need to see — not on how long the video is
function planFor(intent: Intent): VideoPlan {
  switch (intent) {
    case 'quick_summary':
      // Knowing what is on screen is enough; static scenes survive 0.5fps
      return { model: 'gemini-flash-latest', mediaResolution: 'MEDIA_RESOLUTION_LOW', fps: 0.5 };
    case 'lecture_notes':
      // Only needs to catch the moment a slide changes
      return { model: 'gemini-flash-latest', mediaResolution: 'MEDIA_RESOLUTION_LOW', fps: 1 };
    case 'form_coaching':
      // Joint angles live in single frames. Decimate here and the conclusion changes.
      return { model: 'gemini-pro-latest', mediaResolution: 'MEDIA_RESOLUTION_MEDIUM', fps: 4 };
  }
}
 
export async function analyze(
  fileUri: string,
  intent: Intent,
  clip: { startSec?: number; endSec?: number },
  env: Env
) {
  const plan = planFor(intent);
 
  const body = {
    contents: [
      {
        parts: [
          {
            fileData: { fileUri, mimeType: 'video/mp4' },
            videoMetadata: {
              fps: plan.fps,
              ...(clip.startSec !== undefined && { startOffset: `${clip.startSec}s` }),
              ...(clip.endSec !== undefined && { endOffset: `${clip.endSec}s` }),
            },
          },
          { text: PROMPTS[intent] },
        ],
      },
    ],
    generationConfig: {
      temperature: 0.2,
      maxOutputTokens: 2048,
      mediaResolution: plan.mediaResolution,
    },
  };
 
  const res = await fetch(
    `https://generativelanguage.googleapis.com/v1beta/models/${plan.model}:generateContent?key=${env.GEMINI_API_KEY}`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(body),
    }
  );
  return res.json();
}

Notice that planFor branches on intent rather than duration. That was a deliberate reversal.

I used to decide by length: under 30 seconds, analyze precisely; over three minutes, go coarse. But looking at a five-second swing coarsely tells you nothing, and studying a ten-minute lecture in high resolution adds nothing either. Length drives the bill; it has no bearing on the precision the task needs. Confuse the two and you overpay on short clips while missing the point on long ones.

Clipping earned its keep immediately. A swing analysis really only needs about two seconds. Let the user record thirty and cut the range server-side: pulling the relevant window at 4fps came out cheaper than scanning the whole thing at 1fps, and more accurate.

For the broader cost picture, How I Cut My Rork App's AI Costs from $350 to $35/Month covers the caching layer, and Rork × Multimodal AI: Design Patterns for Camera, Voice, and Text covers handling these inputs together in one app.

What was left after the rewiring

Same three-minute 720p clip, measured before and after.

The CPU time going from ~3,900ms to ~12ms is the number that settled it for me. Issuing a destination is not much work. Which means that for every video, the old design spent nearly four seconds carrying bytes without making a single decision about them.

There were losses too, and they belong in the record.

With video no longer crossing the Worker, I cannot inspect it server-side anymore. Things I could have rejected by reading the bytes now go through. For now I accept that. As a solo indie developer, detecting bad input from the analysis output is more realistic than running content inspection on a server I do not want to operate. For other subject matter, that trade would not hold.

The other loss: low media resolution stops reading fine print. On lecture slides it caught the body text and dropped the footnotes. Fine for summaries, disqualifying if you promise a transcript of the whiteboard. The 3x discount is charging you for something.

Ask whether the video needs to pass through at all

If I were adding video analysis to a Rork app today, the first thing I wrote would not be the relay handler. It would be a one-sentence answer to "what is this server protecting?"

For me the answer was the API key. That was the entire list. Routing video through a Worker to protect a key was like rerouting a delivery truck through my living room because that is where I keep the spare key.

If you have this running already, pull up your Worker logs and look at CPU time for a single upload. If it reads in the thousands of milliseconds, most of those milliseconds are probably spent making no decisions at all.

For handling footage straight from the camera, Rork × Vision Camera v4 covers that side, and Rork × LiveKit: Call Infrastructure for Production AI Voice Agents covers the realtime side of audio and video.

I resent that three-minute video more times than I want to admit before the path made sense. If it saves someone else the same detour, it was worth writing down.