Rork × Vision Camera v4: A Production-Grade Camera Stack with QR, OCR, and ML Inference

Permissions and AVAudioSession: kill the iOS landmines first

Before anything else, deal with iOS audio session behavior. The default audio category will shut off background music when you start recording with audio enabled, which is jarring for users.

// useCameraPermissions.ts
import { useEffect, useState } from "react";
import { Camera } from "react-native-vision-camera";
 
export function useCameraPermissions() {
  const [cameraStatus, setCameraStatus] = useState<string | null>(null);
  const [micStatus, setMicStatus] = useState<string | null>(null);
 
  useEffect(() => {
    (async () => {
      let camPerm = await Camera.getCameraPermissionStatus();
      if (camPerm !== "granted") {
        camPerm = await Camera.requestCameraPermission();
      }
      setCameraStatus(camPerm);
 
      // Only request the mic if the app actually records audio
      let micPerm = await Camera.getMicrophonePermissionStatus();
      if (micPerm !== "granted") {
        micPerm = await Camera.requestMicrophonePermission();
      }
      setMicStatus(micPerm);
    })();
  }, []);
 
  return { cameraStatus, micStatus };
}

That's enough to start. If you also need "don't kill the user's music", record video with audio: false and capture audio separately through AVAudioRecorder configured with mixWithOthers, then mux them later. I'd hold off on this complexity until users actually request it.

On Android, add MODIFY_AUDIO_SETTINGS to the permissions array in app.json next to RECORD_AUDIO. Without it, certain devices (I saw this on Pixel 7 series) record audio with the bass channels missing entirely.

Real-time QR and barcode detection with frame processors

Now the core of this article. Vision Camera v4's frame processor sends every frame through a Worklet, where you can drop in your own logic. For QR/barcodes, the built-in vision-camera-code-scanner is more than enough.

// QRScannerScreen.tsx
import { useState } from "react";
import { StyleSheet, View, Text } from "react-native";
import {
  Camera,
  useCameraDevice,
  useCodeScanner,
} from "react-native-vision-camera";
import { useCameraPermissions } from "./useCameraPermissions";
 
export default function QRScannerScreen() {
  const device = useCameraDevice("back");
  const { cameraStatus } = useCameraPermissions();
  const [lastValue, setLastValue] = useState<string | null>(null);
 
  const codeScanner = useCodeScanner({
    codeTypes: ["qr", "ean-13", "code-128", "data-matrix"],
    onCodeScanned: (codes) => {
      // The same code keeps firing every frame — debounce by value or by time
      const value = codes[0]?.value;
      if (value && value !== lastValue) {
        setLastValue(value);
        // Trigger your API call or navigation here
      }
    },
  });
 
  if (cameraStatus !== "granted") {
    return (
      <View style={styles.center}>
        <Text>Please grant camera access to continue.</Text>
      </View>
    );
  }
  if (!device) {
    return (
      <View style={styles.center}>
        <Text>No camera device available.</Text>
      </View>
    );
  }
 
  return (
    <View style={StyleSheet.absoluteFill}>
      <Camera
        style={StyleSheet.absoluteFill}
        device={device}
        isActive={true}
        codeScanner={codeScanner}
      />
      {lastValue && (
        <View style={styles.banner}>
          <Text style={styles.bannerText}>Detected: {lastValue}</Text>
        </View>
      )}
    </View>
  );
}
 
const styles = StyleSheet.create({
  center: { flex: 1, justifyContent: "center", alignItems: "center" },
  banner: {
    position: "absolute",
    bottom: 60,
    left: 20,
    right: 20,
    backgroundColor: "rgba(0,0,0,0.7)",
    padding: 16,
    borderRadius: 12,
  },
  bannerText: { color: "white", fontSize: 16 },
});

onCodeScanned keeps firing as long as the code is visible. If a user holds the camera over a code for two seconds, your callback runs maybe sixty times. Always compare against the previous value, or track a cooldown via useRef. Otherwise you'll spam your backend with the same lookup.

A common mistake is "more code types means better detection." It's the opposite. Narrowing codeTypes makes detection faster and reduces false positives. If your app only needs QR, use ["qr"] and nothing else.

ML inference inside frame processors: TFLite vs. MediaPipe

Once you go beyond QR — object detection, OCR, face landmarks — you need an actual ML runtime. The two real choices today are:

• react-native-fast-tflite — runs TensorFlow Lite models directly inside the Worklet. Best when you have your own model
• react-native-mlkit (v15+) — wraps Google MLKit. Text recognition, face detection, object tracking are all built in and very fast

For OCR, MLKit wins 99% of the time. Google handles the model download, updates, and per-device tuning, which removes the kind of bug surface you definitely don't want to own.

// OCRScreen.tsx — read paper documents in real time
import { useCameraDevice, useFrameProcessor, Camera } from "react-native-vision-camera";
import { runOnJS } from "react-native-worklets-core";
import { scanText } from "@react-native-ml-kit/text-recognition";
import { useState } from "react";
import { StyleSheet, Text, View } from "react-native";
 
export default function OCRScreen() {
  const device = useCameraDevice("back");
  const [recognized, setRecognized] = useState("");
 
  // Frame processing runs in a Worklet — separate from JS, doesn't block UI
  const frameProcessor = useFrameProcessor((frame) => {
    "worklet";
    // OCR is heavy. Throttle to one in five frames.
    if (frame.timestamp % 5 !== 0) return;
 
    const imageBuffer = frame.toArrayBuffer();
    runOnJS(processOCR)(imageBuffer);
  }, []);
 
  const processOCR = async (buffer: ArrayBuffer) => {
    try {
      const result = await scanText({ imageData: buffer });
      setRecognized(result.text);
    } catch (e) {
      // Fail silently — the next frame retries
      console.warn("OCR failed", e);
    }
  };
 
  if (!device) return <View />;
 
  return (
    <View style={StyleSheet.absoluteFill}>
      <Camera
        style={StyleSheet.absoluteFill}
        device={device}
        isActive={true}
        frameProcessor={frameProcessor}
      />
      <View style={styles.textBox}>
        <Text style={styles.text}>{recognized || "Point the camera at some text"}</Text>
      </View>
    </View>
  );
}
 
const styles = StyleSheet.create({
  textBox: {
    position: "absolute",
    bottom: 80,
    left: 20,
    right: 20,
    backgroundColor: "rgba(0,0,0,0.7)",
    padding: 16,
    borderRadius: 12,
  },
  text: { color: "white", fontSize: 14 },
});

The single most important habit here is don't run heavy work on every frame. OCR takes 100–400 ms per call. If you don't throttle, frames pile up and you'll get OOM crashes inside the Worklet. I learned this the hard way on a Pixel 6 that rebooted itself after about 90 seconds of unthrottled inference.

The reason for the Worklet structure is simple: calling MLKit on the JS thread will visibly stutter your UI. Vision Camera v4's frame processor runs on a separate Worklet thread, so throttling there and only handing the survivor frames to JS via runOnJS keeps the UI silky while the heavy work still happens.

Photo capture: the settings that lift you to production quality

Photo capture looks like a one-liner — camera.takePhoto({ ... }) — but the settings make a huge difference. The combination I keep coming back to:

const photo = await camera.current?.takePhoto({
  // Quality mode keeps exposure and white balance stable
  qualityPrioritization: "quality",
  flash: flashMode, // "off" | "on" | "auto"
  enableShutterSound: false,
  enableAutoStabilization: true,
});
 
// Read EXIF and rotate the image to display orientation before saving/uploading
const exifInfo = await ImageEditor.cropImage(photo.path, {
  offset: { x: 0, y: 0 },
  size: { width: photo.width, height: photo.height },
});

If you leave qualityPrioritization at "speed", the shutter feels instant but indoor and low-light photos come out noisy. For "I want to look at this later" apps, pick quality. For QR or document-scan flows where shutter speed matters more, speed is the better fit.

EXIF needs care. iPhones often store images in their native pixel orientation and rely on EXIF's Orientation tag to indicate the display rotation. If you upload that file straight to a server that ignores EXIF, photos appear sideways. Always read the orientation and rotate the pixels before uploading.

Video recording: how to choose codec, HDR, and resolution

Video recording is where format choices have the biggest impact on file size and quality. Three knobs decide most of it:

• Codec — H.264 (broad compatibility), HEVC (half the size of H.264 at the same quality, default on iOS), AV1 (very new, narrow device support). For SNS sharing, HEVC is great. For business apps where the file may be opened anywhere, stay on H.264
• HDR — Pays off in high-contrast outdoor scenes but HDR clips played on LDR displays often look washed out. If you go HDR, plan the playback grading too
• Resolution and fps — 4K@60fps eats battery and overheats devices in 5–10 minutes. 1080p@30fps is the universal sweet spot. Pick 1080p@60fps for sports or kids

// VideoRecorder.tsx
import { Camera, useCameraDevice, useCameraFormat } from "react-native-vision-camera";
 
const device = useCameraDevice("back");
 
// Pick the best available format from the device's list
const format = useCameraFormat(device, [
  { videoResolution: { width: 1920, height: 1080 } },
  { fps: 30 },
  { videoHdr: false }, // prioritize compatibility
]);
 
const startRecording = async () => {
  await camera.current?.startRecording({
    fileType: "mp4",
    videoCodec: "h265", // HEVC keeps file size down
    onRecordingFinished: (video) => {
      // Save video.path or upload it
    },
    onRecordingError: (error) => {
      console.error("recording error", error);
    },
  });
};

On Android, choosing h265 exposes a long-tail playback issue: some devices can't open the file in another app. Build a fallback path to H.264 on Android to avoid "I can't play my video in WhatsApp" support tickets.

Performance tuning: how I think about fps, preset, and format

Getting the most out of Vision Camera v4 comes down to using useCameraFormat well. The template I land on for production looks like this, and it's tuned to stay stable on low-end devices.

import { useCameraFormat, useCameraDevice } from "react-native-vision-camera";
import { Platform } from "react-native";
 
const device = useCameraDevice("back");
 
const format = useCameraFormat(device, [
  // Resolution comes first
  { videoResolution: { width: 1920, height: 1080 } },
  // 30 fps as the floor, 60 if available
  { fps: 30 },
  // HDR off — compatibility and perf win here
  { videoHdr: false },
  // Pixel format — YUV is the safest default
  { pixelFormat: Platform.OS === "ios" ? "yuv" : "yuv" },
]);

This list is an ordered priority queue. If no format matches the first criterion, it relaxes and tries the next. Pinning fps to 60 first will cause low-end devices to fall through to "no format, black screen", so keep 30 as the floor.

Common mistakes you'll hit before launch

Even with the setup above, every device test session uncovers a couple of issues. Roughly in the order I hit them:

• Frame processor silently does nothing — usually react-native-worklets-core isn't installed, or the Babel plugin (react-native-worklets-core/plugin) isn't in babel.config.js. Clearing Metro cache and rebuilding tends to surface the real error
• iOS device shows a black screen — NSCameraUsageDescription is missing or empty in Info.plist. The dialog never appears and the OS treats it as denied. If you set it via the Expo plugin, run expo prebuild and verify the value made it through
• Android recordings end up corrupted — backgrounding the app mid-record corrupts the file. Watch AppState and call stopRecording when the app goes to background
• Enabling HDR makes everything choppy — HDR-supporting formats are scarce on cheap devices. Auto-selection can drop fps to 12. Treat HDR as opt-in and only enable it when the user explicitly turns it on
• Japanese OCR accuracy is terrible — MLKit's text-recognition defaults to Latin. You have to opt into the Japanese (or any non-Latin) module separately. With @react-native-ml-kit/text-recognition, including the Japanese build module changes accuracy dramatically

These are barely covered in the docs, and I burned ten-plus hours of debugging working through them. Hopefully writing them down here saves you the same time.

Battery, memory, and thermal behavior in long sessions

The thing nobody warns you about: a camera app that works for sixty seconds in your office may not work for sixty minutes at a wedding. Vision Camera v4's frame pipeline is efficient, but anything you stack on top — encoding, ML inference, network uploads — compounds.

A few rules I now follow on every camera-heavy app:

• Stop the camera when it's not visible. isActive={false} should follow the screen's focus state. I learned this from an app that drained 12% battery in fifteen minutes because the camera kept running after the user navigated away
• Drop the resolution while previewing. If your app shows a preview before recording, render the preview at 1280×720 and bump up to 1920×1080 only when the user starts recording. Capture path resolution is independent from preview path
• Throttle the frame processor by time, not modulo. frame.timestamp % 5 works in dev but breaks if the device drops frames irregularly. A timestamp-based check (if (now - lastRun < 200) return;) handles thermal throttling more gracefully
• Free the buffer. When you call frame.toArrayBuffer(), the buffer is large. If you hold a reference to it on the JS side, garbage collection lags and memory usage climbs. Pass it to runOnJS, then let it go out of scope on both sides

Most of these are invisible until you instrument the app. Wire up Sentry or Firebase Performance to track session duration, max temperature, and OOM events. Without instrumentation, you only learn about thermal issues from one-star reviews.

Testing strategy: the matrix that catches production bugs

There's no substitute for real devices. Vision Camera v4 has parts that simply don't run in the iOS Simulator (no real camera) and parts that behave differently across Android OEMs (Samsung's camera HAL is uniquely picky). My minimum test matrix before any release:

• iPhone (current and previous generation): the mainline target. iPhone 14 still represents a huge slice of users
• iPhone SE / older: slow A-series chips reveal frame-rate floors that pricier phones hide
• Pixel (current generation): Google's reference Android. Most issues replicate here first
• Samsung Galaxy mid-range (A-series): the camera HAL most likely to expose codec mismatches
• Xiaomi or another low-end Android: low RAM and thermal limits reveal fps drops fast

Run each test scenario on every device:

• Open the camera, scan a QR, navigate away, return — verify camera resumes cleanly
• Start a 5-minute recording with audio — verify no crash, file plays back in another app
• Force-background the app mid-recording — verify no corrupted file
• Toggle airplane mode while ML inference is running — verify graceful degradation
• Fill the device storage to 90% — verify recording errors out cleanly (not silently)

Automating any of this is hard. Manual checking on physical devices is the only reliable approach I've found, and skipping it always burns me later in the form of bug reports that say "doesn't work on my Galaxy S22".

App Store and Play Store review: what reviewers actually look for

Camera-heavy apps face extra scrutiny in App Store and Play Store reviews. The most common rejection reasons I've seen:

• Permission text is generic. "We need camera access" is not enough. Spell out what the camera does for the user — "to scan QR codes for instant payment" or "to capture photos for product listings"
• Camera starts before consent. If your camera component mounts before the user grants permission, you may get a privacy rejection. Wrap the <Camera /> mount behind a permission check
• Background camera usage. If your app declares background camera capabilities and reviewers can't reproduce a legitimate use case, they reject the build. Don't enable background camera unless you absolutely need it
• Video calls without CallKit (iOS). If your camera feature integrates with VoIP, Apple expects you to use CallKit. Bare RTC implementations get flagged
• Image upload without privacy disclosure. If photos leave the device, your privacy policy and the App Privacy nutrition label both need to reflect that. Reviewers check the disclosed data types against what the app actually uploads

Plan a 30-minute review pass before each submission. Re-read your permission strings, check the App Privacy fields, and simulate "deny camera access" to make sure the app degrades gracefully. Most camera-related rejections I've seen would have been caught in that 30 minutes.

Production rollout: feature flags and gradual launch

Camera features have a wider blast radius than most React Native code. A bad release can disable camera functionality across thousands of devices, and "fix" tickets often need a native rebuild — meaning OTA updates won't save you.

I now ship every Vision Camera change behind a feature flag, controlled remotely. Concretely:

• Codec and resolution defaults are remote-configured. If a specific OS version starts crashing with HEVC, I roll Android over to H.264 in minutes rather than days
• The frame processor's heavy work is gated. The OCR feature can be turned off remotely without taking down the camera itself. This means I can ship the camera, then enable OCR for 5% of users, watch crash rates, and roll forward
• Migration logic respects rollback. If the new camera screen has an issue, the old route is still in the app and the feature flag points users back. Every Vision Camera v4 release I've shipped has kept the previous Expo Camera screen in the codebase for at least one minor version cycle

This pattern wouldn't be unusual for a web team, but in mobile it's still under-adopted. The savings on the first incident pay for the overhead many times over. Tools like GrowthBook, Firebase Remote Config, or LaunchDarkly all work — pick one and treat the configuration as part of the camera release.

Adjacent deep dives worth reading next

If you're building a camera-heavy app, the post-capture pipeline is where the bigger wins are. Two natural follow-ups:

• Saving Photos and Videos to Device Library Reliably with Rork
• Designing Resumable, Multipart Uploads for Large Media in Rork
• Normalizing EXIF Orientation and Stripping Metadata Before Upload

Why frame processors work the way they do (and why it matters)

Vision Camera v4's frame processor architecture isn't an arbitrary API choice. Understanding why it looks the way it does saves you from a category of bugs that otherwise look like magic.

When the camera captures a frame, the data lives in native memory — a CMSampleBuffer on iOS, a Image on Android. Bringing that data into JavaScript through a normal bridge call means copying every pixel across the bridge for every frame. At 30 fps with 1080p data, that's roughly 250 MB per second of copies. The bridge can't keep up, and you get the classic "frame processor blocks the UI" problem.

Worklets sidestep this by running JavaScript directly on the camera thread, sharing memory with the native frame buffer. There's no copy. The trade-off is that a worklet can't reach back into normal JS state — it's effectively a sandbox with its own scope. This is why patterns like runOnJS(callback)(args) exist: they're the controlled door between the worklet world and the rest of your app.

This shapes how you should design frame-handling code. Anything that needs to run every frame (drawing overlays with Skia, computing exposure compensation, doing fast OCR) belongs inside the worklet. Anything that needs network access, persistence, or React state updates belongs after a runOnJS call, and should be throttled. Mixing these layers — pretending you can call setState from inside a worklet — produces inconsistent crashes that are very hard to debug.

If you're going to spend a lot of time in frame processors, the react-native-vision-camera Frame Processor docs and the worklets-core README repay the time. The mental model that emerges — "worklets are tiny self-contained programs that share memory with the camera, and the JS thread is just a consumer" — clears up a surprising amount of Vision Camera behavior.

A practical decision tree for picking a camera library

There are now a half dozen serious camera libraries in the React Native ecosystem. The decision tree I use looks like this:

• Does the app only need a single QR scan or a one-shot photo? Use Expo Camera. The savings in build complexity dominate
• Does the app need video recording with audio, basic photo capture, and a few effects? Expo Camera with expo-camera/next covers this well in 2026
• Does the app need real-time analysis of frames (OCR, object detection, AR markers)? Vision Camera v4 is the only practical choice
• Does the app need GPU-accelerated effects or custom drawing on the preview? Vision Camera v4 + react-native-skia is the de facto stack
• Does the app need to integrate with custom hardware (depth sensors, infrared, Lidar)? Vision Camera with a custom native module is the way; alternatives mean rewriting the camera layer in Swift or Kotlin

The pattern that's burned me most is over-engineering. Picking Vision Camera because "we might need ML features later" added two days of build setup to a project that ultimately never used frame processors. If you're not sure, start with Expo Camera and migrate later — the migration cost is bounded, and you save the up-front complexity.

A short case study: shipping a receipt-scanning app

To make this concrete, here's how the patterns above came together for an app I shipped earlier this year. The product is a personal expense tracker that lets the user point the camera at a receipt and extract the total automatically.

The camera screen had three responsibilities: continuous OCR to find a number that looked like a total, a "tap to capture" gesture for manual confirmation, and a small live preview of the recognized text. The implementation looked like:

• Vision Camera v4 with useFrameProcessor, throttled to one in five frames
• MLKit's text-recognition (Latin + Japanese modules) running on the throttled frames via runOnJS
• A debounced React state holding the most recent recognized total, displayed in a banner
• A "capture" button that called takePhoto with qualityPrioritization: "quality" and saved the high-res photo plus the recognized total to Supabase

The first version of this used per-frame OCR and crashed within ninety seconds on any device that wasn't an iPhone 15. The second version added the throttle and ran for an hour without issues but heated the device noticeably. The third version added a "stop processing if the same total has been recognized for three seconds" check, which dropped average power consumption by about 40% and stopped the heating entirely.

None of these refinements were obvious from reading the docs. They came from watching real users use the app and instrumenting power and thermal events with Sentry. If I had a single piece of advice for someone starting their first Vision Camera project, it would be: ship a minimum version, add instrumentation, and let real-world traces tell you where to optimize.

What to do next

The setup in this guide covers QR scanning, OCR, and production-grade photo/video capture. The next step that pays off the most is wiring the camera into a full pipeline: capture → automatic upload → ML analysis → push notification. As a tangible action right now: install react-native-vision-camera and react-native-worklets-core in your project and get the QR scanner running on a real device. The moment it works, you'll feel "okay, this is a stack I can ship", and the rest of the work flows from there.