Long-Form On-Device Transcription with SpeechAnalyzer in Rork Max's Native Swift

Build the analysis session and feed it audio

Once the assets are ready, attach SpeechTranscriber to SpeechAnalyzer and feed audio as an AsyncStream. SpeechAnalyzer passes the audio it receives to the attached modules in turn.

import Speech
import AVFoundation
 
final class LiveTranscription {
    private let analyzer: SpeechAnalyzer
    private let transcriber: SpeechTranscriber
    private var inputBuilder: AsyncStream<AnalyzerInput>.Continuation?
    private let engine = AVAudioEngine()
 
    init(locale: Locale) {
        self.transcriber = SpeechTranscriber(locale: locale,
                                             preset: .progressiveLiveTranscription)
        self.analyzer = SpeechAnalyzer(modules: [transcriber])
    }
 
    /// Start feeding microphone input into the analysis session.
    func start() async throws {
        // 1. Prepare a stream to hand audio to, and connect it to the analyzer
        let (stream, continuation) = AsyncStream.makeStream(of: AnalyzerInput.self)
        self.inputBuilder = continuation
        try await analyzer.start(inputSequence: stream)
 
        // 2. Wrap the raw mic buffers in AnalyzerInput and send them to the stream
        let input = engine.inputNode
        let format = input.outputFormat(forBus: 0)
        input.installTap(onBus: 0, bufferSize: 4096, format: format) { [weak self] buffer, _ in
            self?.inputBuilder?.yield(AnalyzerInput(buffer: buffer))
        }
        engine.prepare()
        try engine.start()
    }
 
    func stop() async {
        engine.stop()
        engine.inputNode.removeTap(onBus: 0)
        inputBuilder?.finish()
        // After the supply stops, close out the remaining analysis
        try? await analyzer.finalizeAndFinish()
    }
}

The heart of this is wrapping the AVAudioPCMBuffer from the mic tap in AnalyzerInput and handing it to the stream with continuation.yield(_:). Because feeding audio and receiving results are two separate async flows, the UI stays responsive. To analyze a recorded file instead, read through the file and yield its buffers in place of the mic tap — the same skeleton works unchanged.

Draw volatile and final results differently

In live transcription, nothing shapes the experience more than how you handle volatile versus final results. While someone is speaking, the tentative result shifts around; a moment later it settles. Draw both the same way and the text swaps every time, jittering and becoming hard to read.

transcriber.results returns an AsyncSequence of results. Each result carries whether it's final, so you show volatile text in a faint gray placeholder and move it into the body once finalized.

func consumeResults() async {
    var confirmed = ""          // The confirmed body text
    for try? await result in transcriber.results {
        let text = String(result.text.characters)
        if result.isFinal {
            confirmed += text
            await MainActor.run {
                self.finalText = confirmed   // Reflect only the confirmed part
                self.volatileText = ""       // Clear the tentative placeholder
            }
        } else {
            await MainActor.run {
                self.volatileText = text      // Shifting tentative text, faint
            }
        }
    }
}

Adding just this distinction gives you the modern voice-app feel: text wells up as you speak and then settles a beat later. I first drew both together, and a tester told me "the text keeps flickering." Look at the final flag and change color and position — that alone changes the impression a great deal.

Decide the boundary between Rork Max native and Expo

So how do you wire this into Rork Max? Because Rork Max generates native Swift, it reaches cleanly for a modern framework like SpeechAnalyzer. Many existing apps, meanwhile, are built in Rork's own Expo / React Native. Drawing the line between the two up front makes the implementation much easier.

My design uses this split.

The key is to never hand raw audio buffers to the JS side. Round-tripping buffers across the bridge repeatedly is, by itself, a breeding ground for drops and latency. Keep the analysis fully on the native side and send only the confirmed text fragments to JS as events — a one-directional flow that stays stable.

As a native module, exposing just two methods, startTranscription / stopTranscription, plus one event that streams final text, is enough.

// Minimal surface with the Expo Modules API (sketch)
public class SpeechModule: Module {
    var live: LiveTranscription?
 
    public func definition() -> ModuleDefinition {
        Name("SpeechAnalyzer")
 
        // JS only touches these two methods and the onFinalText event
        AsyncFunction("startTranscription") { (localeId: String) in
            let live = LiveTranscription(locale: Locale(identifier: localeId))
            self.live = live
            try await live.start()
        }
 
        AsyncFunction("stopTranscription") {
            await self.live?.stop()
            self.live = nil
        }
 
        Events("onFinalText")   // Send only confirmed fragments to JS
    }
}

With this, the JS side is only ever "ask to start, receive the final text, save it." The heavy audio work stays sealed on the native side, so it never clogs the React Native bridge.

Pitfalls I found by actually porting

Finally, a few things I stumbled on while porting. If it saves someone walking the same path a little time, I'll be glad.

First, don't skip the asset-download check. As noted, first launch silently returns empty results. Always test once on a device in its clean, untouched state.

Second, AVAudioSession category configuration. If you record, enable .record or .playAndRecord and think about interference with other apps' audio. Neglect this and you get the confusing symptom of a live mic with no buffers arriving.

Third, no watchOS support. If your concept assumes a wearable, lean toward recording on the watch and passing analysis to the iPhone. Decide it during planning, or you'll be rebuilding later.

Fourth, if you handle long form, watch power and heat. On-device analysis is pleasant, but tens of minutes of continuous recording will warm the device. Pausing analysis when the app goes to the background helps cut battery complaints.

Start by building just one short recording, displayed with volatile and final results drawn differently. Once that feel clicks, extending to long form and splitting work between Rork Max and Expo both follow naturally. Thank you for reading.