◉ AI Models/2026-03-28Advanced

Rork × Core ML Custom Model Development × On-Device AI

Notes from training a model in Create ML, shipping it inside a Rork Max app with Core ML, and deciding per request whether inference runs on device or in the cloud — with measured routing, quantization, Vision and Natural Language integration, and OTA model swaps.

rork-max⁴⁰ core-ml² machine-learning² on-device-ai² vision-framework app-development¹²

✦ Premium Article

The Flight Where Half My AI Features Stopped Working

I tried the auto-tagging feature I'd just shipped while sitting on a plane. The spinner kept turning after I finished typing, and nothing came back before we landed. Obvious in hindsight — the request was going to a cloud API — but staring at that screen is when it hit me that the feature had only ever existed on top of a network connection.

That flight is why I started moving inference on device. Privacy and cost became reasons later. The first motivation was simply that it should work with no signal.

Rork Max works with Apple's Core ML directly: train in Create ML, quantize, ship the model inside the app, and keep inference local. What follows is that whole path, including the places where I got stuck.

Moving everything on device didn't solve everything, though. So we start with where the line actually sits.

On-Device AI Architecture and Core ML's Role

Understanding On-Device AI

On-device AI executes ML inference directly on the device's processors (CPU, GPU, Neural Engine) rather than sending data to cloud APIs. This approach delivers compelling advantages.

Key Benefits

Privacy by Default: Data never leaves the device
Sub-Second Latency: Millisecond-level inference without network roundtrips
Offline Operation: Works without internet connectivity
Cost Efficiency: No API costs or infrastructure overhead
Battery Optimization: Modern chips (Neural Engine) are power-efficient for inference

Core ML: The Unified Interface

Apple's Core ML is a unified framework that converts models trained in popular frameworks—TensorFlow, PyTorch, Scikit-learn, and others—into a single format (.mlmodel) that runs efficiently on iOS devices.

Core ML Strengths

Multi-Framework Support: TensorFlow, PyTorch, ONNX all convert to .mlmodel
Automatic Optimization: Intelligently dispatches to CPU, GPU, or Neural Engine
Tight Integration: Works seamlessly with Vision, Natural Language, and other frameworks
Memory Efficiency: Built-in quantization reduces model footprint significantly

Rork Max provides native support for Core ML, enabling you to integrate models without touching complex Swift code.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Custom model training in Create ML through Core ML quantization, step by step

✦End-to-end Vision and Natural Language integration inside a Rork Max app

✦An inference router that splits on-device and cloud work — measured at 67% local and 58% lower API spend

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Knowing When Not to Run On-Device: Routing Inference Decisions

Read this far and on-device inference starts to look like the answer to everything.

I thought so too. My plan was to push every inference in my app onto Core ML and drive the API bill to zero. That plan fell apart when a user wrote in to say the app "only knows old things."

A model baked into the device knows exactly the world it was trained on. You get low latency and privacy, and you hand back freshness and reasoning depth in exchange.

The practical answer isn't Core ML or cloud. It's choosing a target per request.

What Actually Decides the Target

The nature of the request settles it almost every time. Here are the axes I use in production.

Request characteristic	Target	Why
Carries personal data, location, or photos	Core ML (on-device)	The data never leaves the device — treat this as a promise, not a preference
Must work offline	Core ML (on-device)	Anything network-dependent breaks the moment signal drops
Short and high-frequency (completion, classification, tagging)	Core ML (on-device)	The round trip costs more than the inference itself
Needs knowledge newer than the training cutoff	Cloud LLM	An on-device model cannot answer this in principle
Needs multi-step reasoning or long-form generation	Cloud LLM	A quantized small model won't hold up

"When in doubt, stay local" is a fine default. Making the outbound decision carry the burden of proof — an asymmetric rule — is what keeps a privacy design from eroding over time.

Keep the Router in One Module

Scatter this decision across your screens and it will collapse within six months. Put it in one place.

// src/ai/inferenceTarget.ts
 
export type InferenceTarget = 'core-ml' | 'cloud';
 
export interface InferenceInput {
  text: string;
  /** Photos, location, contacts — anything that shouldn't leave the device */
  hasSensitivePayload: boolean;
  /** Asking for knowledge newer than the model's training cutoff */
  needsFreshKnowledge: boolean;
  /** Multi-step reasoning or long-form generation */
  needsDeepReasoning: boolean;
  isOffline: boolean;
}
 
export interface TargetDecision {
  target: InferenceTarget;
  /** For logs and debugging only — never surfaced in the UI */
  rationale: string;
}
 
const SHORT_INPUT_LIMIT = 50;
 
export function selectInferenceTarget(input: InferenceInput): TargetDecision {
  // Conditions that forbid leaving the device are evaluated before anything else
  if (input.isOffline) {
    return { target: 'core-ml', rationale: 'offline' };
  }
  if (input.hasSensitivePayload) {
    return { target: 'core-ml', rationale: 'sensitive-payload' };
  }
 
  // Only send out what the local model genuinely cannot answer
  if (input.needsFreshKnowledge) {
    return { target: 'cloud', rationale: 'knowledge-cutoff' };
  }
  if (input.needsDeepReasoning) {
    return { target: 'cloud', rationale: 'reasoning-depth' };
  }
 
  // Short inputs don't justify a round trip
  if (input.text.length <= SHORT_INPUT_LIMIT) {
    return { target: 'core-ml', rationale: 'short-input' };
  }
 
  return { target: 'core-ml', rationale: 'default-local-first' };
}

The character of this function lives in that last branch returning 'core-ml' rather than 'cloud'.

Anything ambiguous falls local. A request only leaves the device when it matches one of the two explicit reasons above. With the fallback pointed at the safe side, you can keep adding conditions over the years without quietly springing a leak.

Sending rationale to logs but never to the UI is deliberate too. A "processing in the cloud" indicator is handy while you're building and just offloads a decision onto your user once you ship.

Measure the Split

Judge the routing by ratios, not by feel. Three numbers are enough.

// src/ai/inferenceStats.ts
import AsyncStorage from '@react-native-async-storage/async-storage';
 
const KEY = 'inference_stats_v1';
 
type Bucket = { coreML: number; cloud: number; cloudSpendUsd: number };
 
export async function recordInference(
  target: 'core-ml' | 'cloud',
  spendUsd: number
): Promise<void> {
  const raw = await AsyncStorage.getItem(KEY);
  const bucket: Bucket = raw
    ? JSON.parse(raw)
    : { coreML: 0, cloud: 0, cloudSpendUsd: 0 };
 
  if (target === 'core-ml') {
    bucket.coreML += 1;
  } else {
    bucket.cloud += 1;
    bucket.cloudSpendUsd += spendUsd;
  }
 
  await AsyncStorage.setItem(KEY, JSON.stringify(bucket));
}
 
/** Share handled on-device. When this slips, suspect the routing rules. */
export async function localHitRate(): Promise<number> {
  const raw = await AsyncStorage.getItem(KEY);
  if (!raw) return 0;
  const b: Bucket = JSON.parse(raw);
  const total = b.coreML + b.cloud;
  return total === 0 ? 0 : b.coreML / total;
}

Here's a week of measurements from my own assistant-style app.

Metric	Measured
Handled entirely on-device	67%
Average latency (Core ML)	28ms
Average latency (cloud LLM)	1,240ms
Monthly API spend	$45 → $19 (down 58%)

When localHitRate() starts falling, look at the routing rules before you look at model accuracy. In my case the cause was always the same: I'd been too casual about adding cloud-bound conditions.

What Surfaces Once Routing Is In

Three things I only noticed after shipping this.

Model initialization is slow exactly once. A request arriving right after launch turns the MLModel load into felt wait time. Warm it in the background at startup, accept input while it loads, and queue internally until it's ready. In Rork Max, adding "initialize the Core ML model in the background at app launch" to the prompt is enough to get this.

Nothing releases the model on a memory warning. An on-device model stays resident. Without a release path at the didReceiveMemoryWarning equivalent — and a reload on the next inference — the app dies the moment someone uses it alongside a photo app.

On iOS 26 you can pull some cloud work back. Apple's FoundationModels runs an LLM on device, so a slice of what selectInferenceTarget() was sending to 'cloud' — summaries, rewrites — can move back inside. I wrote up that design separately in using Apple FoundationModels in a Rork app.

Core ML is iOS-only. To give Android the same experience, put TensorFlow Lite or MediaPipe's inference API behind the same seam, leave the return type of selectInferenceTarget() untouched, and swap only the implementation. This is where keeping the decision in one module pays for itself.

Training Custom Models with Create ML

Create ML Essentials

Create ML is Apple's macOS application that integrates with Xcode and lets you train ML models through an intuitive visual interface.

Supported Task Types

Image Classification: Distinguish between categories (dog breeds, plant species, defects)
Object Detection: Locate and label objects within images
Sound Classification: Recognize audio patterns (ambient sounds, machinery noise)
Text Classification: Categorize text (sentiment, spam detection, intent)
Tabular Data: Predict numerical values or categories from structured data

Practical Walkthrough: Training a Defect Detection Model

Let's train a real-world example: an automated quality control model for manufacturing.

Step 1: Prepare Your Dataset

Create ML uses folder structure for organization. Each category becomes a folder:

dataset/
├── OK/
│   ├── good_001.jpg
│   ├── good_002.jpg
│   └── ... (500+ images)
└── DEFECT/
    ├── bad_001.jpg
    ├── bad_002.jpg
    └── ... (500+ images)

Best practice: 500-5,000 images per class. Data quality directly influences accuracy.

Step 2: Train in Create ML

// In Xcode:
// 1. File > New > ML Model
// 2. Select "Image Classification"
// 3. Point to your dataset/ folder
// 4. Reserve 10-20% for validation
// 5. Click Train

Step 3: Evaluate and Iterate

Create ML displays accuracy and loss curves in real-time:

Accuracy ≥ 0.95: Production-ready
0.85-0.94: Add more data or improve preprocessing
< 0.85: Reconsider model architecture

Healthy training shows Train and Validation loss decreasing together. If Validation loss increases while Training loss drops, your model is overfitting—add more validation data.

Accelerate with Transfer Learning

Training from scratch takes hours. Apple provides pre-trained models (VGG, ResNet) that you can fine-tune in minutes.

In the Create ML interface, select a pre-trained architecture under "Architecture":

Benefits

90% faster training: Hours → seconds
5-10x less data needed: Maintains accuracy with smaller datasets
Better generalization: Retains useful image features

Core ML Model Optimization and Size Reduction

Understanding Model File Size

Create ML outputs .mlmodel files that can be surprisingly large. Given App Store distribution constraints (1GB app size limit), optimization is essential.

Typical Sizes

ResNet50 (image classification): ~100MB
YOLOv5 (object detection): 50-150MB (varies by precision)
BERT (text): ~500MB (full version)

Quantization: Shrink Without Losing Accuracy

Quantization converts floating-point numbers (Float32) to integer formats (Int8), reducing memory and computation.

Create ML's Quantization Options

After training, you'll see a Quantization section:

- Full Precision (Float32): Best accuracy, largest file
- Float16: ~50% smaller, minimal accuracy loss
- Int8 (Quantized): ~75% smaller, typically 1-3% accuracy drop

Example validation:

import CoreML
 
let modelURL = Bundle.main.url(forResource: "defect-detection", withExtension: "mlmodel")!
let model = try MLModel(contentsOf: modelURL)
print("Model size and details: \(model.modelDescription)")

Advanced Optimization with Python

For fine-grained control, use Apple's coremltools library:

pip install coremltools
 
import coremltools as ct
import tensorflow as tf
 
# Load your trained model
tf_model = tf.keras.models.load_model('defect_model.h5')
 
# Convert to Core ML
mlmodel = ct.convert(
    tf_model,
    inputs=[ct.ImageType(name="image", shape=(1, 224, 224, 3))],
    outputs=[ct.ClassifierOutputType(name="defects")],
    compute_units=ct.ComputeUnit.CPU_AND_NE
)
 
# Apply Int8 quantization
quantized = ct.quantization_utils.quantize_weights(mlmodel, nbits=8)
quantized.save('defect-detection.mlmodel')

Memory Optimization at Runtime

Minimize memory footprint during inference:

// Cache the model once
class ModelCache {
    static let shared = ModelCache()
    private let model: MLModel
 
    private init() {
        let url = Bundle.main.url(forResource: "defect-detection", withExtension: "mlmodel")!
        self.model = try! MLModel(contentsOf: url)
    }
}
 
// Use autoreleasepool to free temporary allocations
autoreleasepool {
    let prediction = try model.prediction(input: input)
    // Process prediction
}

Integrating Core ML Models into Rork Max

Model Management in Rork Max

Rork Max treats Core ML models as first-class assets with automatic optimization.

Integration Flow

Upload .mlmodel to Rork Max's Assets panel
Rork automatically optimizes (quantization, platform tuning)
Reference the model in your visual layout
Deploy to App Store—model is bundled automatically

No-Code Integration Example: Image Classification

Build this flow visually in Rork Max:

[Camera View] → [Core ML Model] → [Classification] → [Result Display]

The Generated Code

import CoreML
import Vision
 
struct QualityCheckView: View {
    @State private var cameraImage: UIImage?
    @State private var result: String = ""
 
    var body: some View {
        VStack {
            Image(uiImage: cameraImage ?? UIImage())
                .resizable()
                .scaledToFit()
 
            Button("Analyze") {
                analyzeImage()
            }
 
            Text(result)
                .font(.headline)
                .padding()
        }
    }
 
    private func analyzeImage() {
        guard let image = cameraImage else { return }
 
        let model = try? defect_detection(configuration: MLModelConfiguration())
        guard let buffer = image.pixelBuffer(size: CGSize(width: 224, height: 224)) else {
            return
        }
 
        let input = defect_detectionInput(imageAt: buffer)
        guard let output = try? model?.prediction(input: input) else {
            result = "Analysis failed"
            return
        }
 
        let confidence = output.classLabelProbs.max(by: { $0.value < $1.value })
        result = "Status: \(confidence?.key ?? "Unknown") (\(String(format: "%.1f%%", (confidence?.value ?? 0) * 100)))"
    }
}

Adding Custom Logic

Rork Max supports "Custom Swift" sections for advanced processing:

// Batch inference across multiple frames
func processMultipleFrames(_ frames: [UIImage]) -> [String] {
    let model = try! defect_detection(configuration: MLModelConfiguration())
    var results: [String] = []
 
    for frame in frames {
        guard let buffer = frame.pixelBuffer(size: CGSize(width: 224, height: 224)) else {
            continue
        }
 
        let input = defect_detectionInput(imageAt: buffer)
        let output = try! model.prediction(input: input)
        results.append(output.classLabel)
    }
 
    return results
}

Vision Framework Integration: Image Recognition and Object Detection

Vision Framework's Role

Vision works hand-in-hand with Core ML to preprocess images and post-process detections efficiently.

If you're heading toward camera, voice, and text living on the same screen, how you shape the input surface matters before any of this does — I collected those patterns in multimodal AI app design patterns for Rork.

Key Capabilities

Face Detection & Recognition: NSFW detection, emotion analysis
Object Detection: Fast, accurate region-based detection
Barcode & QR Recognition: Read codes in real-time
Text Recognition (OCR): Extract text from images
Homography Estimation: Perspective correction

Practical Example: Object Detection

Integrate a YOLO model converted to Core ML:

import Vision
import CoreML
 
class ObjectDetector {
    let model: defect_yolo
 
    init() throws {
        self.model = try defect_yolo(configuration: MLModelConfiguration())
    }
 
    func detect(in image: UIImage) -> [DetectionResult] {
        guard let ciImage = CIImage(image: image) else { return [] }
 
        let request = VNCoreMLRequest(model: try! model.model) { request, error in
            guard let observations = request.results as? [VNRecognizedObjectObservation] else {
                return
            }
 
            for observation in observations {
                let boundingBox = observation.boundingBox
                let confidence = observation.confidence
                let label = observation.labels.first?.identifier ?? "Unknown"
 
                print("Detected: \(label) at \(boundingBox) with \(confidence * 100)% confidence")
            }
        }
 
        let handler = VNImageRequestHandler(ciImage: ciImage, options: [:])
        try? handler.perform([request])
 
        return []
    }
}
 
struct DetectionResult {
    let label: String
    let boundingBox: CGRect
    let confidence: Float
}

Vision in Rork Max: Annotation Overlays

Rork Max provides visual components to annotate detection results:

func drawAnnotations(on image: CGImage, detections: [DetectionResult]) {
    let size = CGSize(width: image.width, height: image.height)
 
    UIGraphicsBeginImageContextWithOptions(size, false, 1.0)
    UIImage(cgImage: image).draw(at: .zero)
 
    let context = UIGraphicsGetCurrentContext()!
    context.setStrokeColor(UIColor.red.cgColor)
    context.setLineWidth(2.0)
 
    for detection in detections {
        let rect = CGRect(
            x: detection.boundingBox.origin.x * size.width,
            y: detection.boundingBox.origin.y * size.height,
            width: detection.boundingBox.width * size.width,
            height: detection.boundingBox.height * size.height
        )
        context.stroke(rect)
 
        let label = NSAttributedString(
            string: "\(detection.label) \(String(format: "%.0f%%", detection.confidence * 100))",
            attributes: [.foregroundColor: UIColor.red, .font: UIFont.systemFont(ofSize: 12)]
        )
        label.draw(at: CGPoint(x: rect.minX, y: rect.minY - 20))
    }
 
    let result = UIGraphicsGetImageFromCurrentImageContext()
    UIGraphicsEndImageContext()
}

Natural Language Framework: Text Classification and Sentiment Analysis

Natural Language Framework Overview

While Vision specializes in images, Natural Language optimizes text processing tasks.

Capabilities

Language Identification: Detect language across multilingual content
Tokenization & Tagging: Sentence/word segmentation and part-of-speech tagging
Named Entity Recognition: Extract persons, locations, organizations
Custom Text Classification: Sentiment, intent, topic categorization

Real-World Example: Sentiment Analysis

Train a sentiment classification model with Create ML:

training_data/
├── positive.txt
│   ├── This product is excellent
│   ├── Very satisfied with purchase
│   └── ...
└── negative.txt
    ├── Disappointed with quality
    ├── Not as expected
    └── ...

Convert to .mlmodel and integrate into Rork Max:

import NaturalLanguage
import CoreML
 
class SentimentAnalyzer {
    let model: sentiment_classifier
 
    init() throws {
        self.model = try sentiment_classifier(configuration: MLModelConfiguration())
    }
 
    func analyze(_ text: String) -> SentimentResult {
        // Preprocess text
        let tagger = NLTagger(tagSchemes: [.tokenType])
        tagger.string = text
 
        var tokens: [String] = []
        tagger.enumerateTags(in: text.startIndex..<text.endIndex, unit: .word, scheme: .tokenType) { tag, range in
            let token = String(text[range]).lowercased()
            tokens.append(token)
            return true
        }
 
        // Run prediction
        let input = sentiment_classifierInput(text: tokens.joined(separator: " "))
        guard let output = try? model.prediction(input: input) else {
            return SentimentResult(sentiment: "unknown", confidence: 0.0)
        }
 
        return SentimentResult(
            sentiment: output.classLabel,
            confidence: output.classLabelProbs[output.classLabel] ?? 0.0
        )
    }
}
 
struct SentimentResult {
    let sentiment: String // "positive" or "negative"
    let confidence: Double
}

Rork Max NLP Pipeline: Review Analysis

Automatically analyze customer reviews:

struct ReviewAnalysisView: View {
    @State private var reviewText: String = ""
    @State private var sentiment: SentimentResult?
 
    var body: some View {
        VStack {
            TextEditor(text: $reviewText)
                .border(Color.gray)
 
            Button("Analyze Sentiment") {
                analyzeSentiment()
            }
 
            if let result = sentiment {
                HStack {
                    Image(systemName: result.sentiment == "positive" ? "hand.thumbsup.fill" : "hand.thumbsdown.fill")
                        .foregroundColor(result.sentiment == "positive" ? .green : .red)
 
                    Text(result.sentiment == "positive" ? "Positive" : "Negative")
                    Text(String(format: "%.0f%%", result.confidence * 100))
                }
                .padding()
                .background(result.sentiment == "positive" ? Color.green.opacity(0.1) : Color.red.opacity(0.1))
                .cornerRadius(8)
            }
        }
    }
 
    private func analyzeSentiment() {
        let analyzer = try! SentimentAnalyzer()
        sentiment = analyzer.analyze(reviewText)
    }
}

Real-Time Inference Performance Tuning

Measuring Inference Latency

For real-time applications (AR, video processing), inference speed directly impacts user experience.

import os.log
 
class PerformanceMonitor {
    let logger = os.log.OSLog(subsystem: "com.rorklab.inference", category: "performance")
 
    func measureInference<T>(_ task: () throws -> T) rethrows -> (result: T, timeMs: Double) {
        let startTime = CACurrentMediaTime()
        let result = try task()
        let endTime = CACurrentMediaTime()
        let timeMs = (endTime - startTime) * 1000
 
        os_log(
            "Inference: %.1f ms",
            log: logger,
            type: .debug,
            timeMs
        )
 
        return (result, timeMs)
    }
}
 
// Usage
let monitor = PerformanceMonitor()
let (output, timeMs) = try monitor.measureInference {
    try model.prediction(input: input)
}
print("Completed in \(timeMs)ms")

Batch Processing for Throughput

Process multiple frames simultaneously to improve throughput:

// Single-frame: Low latency, low GPU utilization
for frame in frames {
    let output = try model.prediction(input: frameInput)
}
 
// Batch processing: Higher latency, higher throughput
let batchInput = MLMultiArray(shape: [frames.count, 224, 224, 3], dataType: .float32)
// ... populate batchInput with frames ...
let output = try model.prediction(input: batchInput)

CPU vs GPU vs Neural Engine

Core ML auto-selects, but you can tune explicitly:

let config = MLModelConfiguration()
 
// CPU only (highest accuracy, slowest)
config.computeUnits = .cpuOnly
 
// GPU + CPU (balanced)
config.computeUnits = .cpuAndGPU
 
// All (Neural Engine + GPU + CPU - fastest)
config.computeUnits = .all
 
let model = try defect_detection(configuration: config)

Typical Benchmarks (iPhone 15 Pro estimates)

CPU only: 50-100ms per inference
GPU + CPU: 20-30ms
All (Neural Engine): 5-10ms

Memory Efficiency and Threading

class InferenceQueue {
    private let queue = DispatchQueue(label: "com.rorklab.inference", qos: .userInitiated)
    private let model: defect_detection
 
    init() throws {
        self.model = try defect_detection(configuration: MLModelConfiguration())
    }
 
    func predict(
        _ input: defect_detectionInput,
        completion: @escaping (Result<defect_detectionOutput, Error>) -> Void
    ) {
        queue.async {
            do {
                let output = try self.model.prediction(input: input)
                DispatchQueue.main.async {
                    completion(.success(output))
                }
            } catch {
                DispatchQueue.main.async {
                    completion(.failure(error))
                }
            }
        }
    }
}

Model Versioning and Over-The-Air Updates

Progressive Model Updates

In production, you'll periodically release improved models. Updating all users immediately is impractical—App Store review takes days, and not all users update promptly.

Challenge: Update models without waiting for App Store approvals

One aside: if you also gate the app binary behind a minimum supported version, writing that comparison as a string check will break on releases like 1.10.0. I measured four implementations in why 1.10.0 devices get forced into an update.

Solution: Over-The-Air (OTA) Updates

Keep the app binary unchanged but download new models from your server:

struct ModelManager {
    let remoteURL = URL(string: "https://api.rorklab.com/models/defect-v2.mlmodel")!
    let cachePath = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)[0]
        .appendingPathComponent("ml-models")
 
    func fetchLatestModel() async throws -> MLModel {
        // Check local cache
        let versionFile = cachePath.appendingPathComponent("version.json")
        if FileManager.default.fileExists(atPath: versionFile.path) {
            let data = try Data(contentsOf: versionFile)
            let version = try JSONDecoder().decode(ModelVersion.self, from: data)
 
            if version.id == "v2" {
                let modelPath = cachePath.appendingPathComponent("defect.mlmodel")
                return try MLModel(contentsOf: modelPath)
            }
        }
 
        // Download latest model
        let (data, _) = try await URLSession.shared.data(from: remoteURL)
 
        // Cache it
        try FileManager.default.createDirectory(at: cachePath, withIntermediateDirectories: true)
        let modelPath = cachePath.appendingPathComponent("defect.mlmodel")
        try data.write(to: modelPath)
 
        // Record version
        let version = ModelVersion(id: "v2", timestamp: Date())
        let versionData = try JSONEncoder().encode(version)
        try versionData.write(to: versionFile)
 
        return try MLModel(contentsOf: modelPath)
    }
}
 
struct ModelVersion: Codable {
    let id: String
    let timestamp: Date
}

Version Management Best Practices

enum ModelVersionControl {
    case bundled    // Fallback included with app
    case remote(URL) // Latest from server
 
    static func load() async throws -> MLModel {
        do {
            // Try remote first
            return try await ModelManager().fetch(
                from: URL(string: "https://api.rorklab.com/models/latest.mlmodel")!
            )
        } catch {
            // Fallback to bundled
            print("Remote fetch failed, using bundled: \(error)")
            return try bundledModel()
        }
    }
 
    private static func bundledModel() throws -> MLModel {
        let url = Bundle.main.url(forResource: "defect-detection", withExtension: "mlmodel")!
        return try MLModel(contentsOf: url)
    }
}

Privacy-First AI Design Patterns

Privacy by Default

On-device AI's greatest advantage is privacy. Implement these practices rigorously:

Essential Safeguards

Never Persist Input Data: Don't save images or text used for inference to disk
Clear Memory Immediately: Overwrite buffers after use
Local-Only Processing: Never send user data to servers without explicit consent

Your own code sending nothing isn't the whole story — review asks what the SDKs you bundled collect. The audit path for tracing a Privacy Manifest through the SDK chain is in auditing Privacy Manifests in Rork-generated Expo apps. If on-device inference is your selling point, a mismatch here undoes it. 4. Transparent Permissions: Always ask before accessing camera/microphone

Secure Data Handling

class SecureMLProcessor {
    func processImageWithoutPersisting(_ image: UIImage) throws -> String {
        var processedBuffer: CVPixelBuffer?
 
        defer {
            // Clean up immediately after inference
            if var buffer = processedBuffer {
                CVPixelBufferUnlockBaseAddress(buffer, .readOnly)
                processedBuffer = nil
            }
        }
 
        guard let buffer = image.pixelBuffer(size: CGSize(width: 224, height: 224)) else {
            throw MLError.conversionFailed
        }
        processedBuffer = buffer
 
        // Run inference
        let input = defect_detectionInput(imageAt: buffer)
        let output = try model.prediction(input: input)
 
        // Return result only (never persist)
        return output.classLabel
    }
 
    func deleteAllLocalData() throws {
        let cachePath = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)[0]
        try FileManager.default.removeItem(at: cachePath)
    }
}
 
enum MLError: Error {
    case conversionFailed
}

User Permission Management

import AVFoundation
 
class PermissionManager {
    static func requestCameraAccess(completion: @escaping (Bool) -> Void) {
        AVCaptureDevice.requestAccess(for: .video) { granted in
            DispatchQueue.main.async {
                completion(granted)
            }
        }
    }
 
    static func requestMicrophoneAccess(completion: @escaping (Bool) -> Void) {
        AVCaptureDevice.requestAccess(for: .audio) { granted in
            DispatchQueue.main.async {
                completion(granted)
            }
        }
    }
}
 
// Always check before access
PermissionManager.requestCameraAccess { granted in
    if granted {
        startCamera()
    } else {
        showPermissionAlert()
    }
}

Compliance Framework

struct PrivacyCompliance {
    let allowDataSharing: Bool = false
    let allowAnalytics: Bool = false
    let dataRetentionDays: Int = 0
 
    func validateBeforePrediction() -> Bool {
        assert(!allowDataSharing, "Sharing must be disabled for privacy")
        assert(!allowAnalytics, "Analytics collection is disabled")
        return true
    }
}

Where to Start

If I had to pick the single change that mattered most here, it's selectInferenceTarget(). Before touching model accuracy, put the routing decision in one file. Every later change gets lighter, and because the fallback leans toward 'core-ml', the failure mode where data quietly leaks out as you bolt on conditions simply doesn't happen.

Quantization and OTA can wait until that foundation is settled. Mine did.

Start by wiring up localHitRate() alone and watching the local-processing share for a week. Once there's a number, your own data tells you which requests to pull back on device.

Core ML shifts underfoot with every Apple release, and I'm still keeping up with it — I'll add new measurements here as I get them.

Thank You for Reading

Rork Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.