Optimizing Machine Learning in Rork Max Apps — Core ML Integration, Per-Device Tuning, Quantization, Size Reduction

3. Quantization: the size-vs-quality tradeoff

On-device ML makes model size a product decision. App Store binaries have a 4GB uncompressed ceiling, and a 200MB model turns every update into a large download.

Three options in practice:

Float16: Half the size. Essentially no accuracy loss. Default.

Int8: Quarter the size. 1–3% accuracy loss in some tasks. Fine for image classification, ask questions for generative work.

Palettization: 1/8 or smaller, the current best option. Available iOS 17+. Weights are quantized to a limited set of "colors" (8–256). Remarkable compression with minimal accuracy drop.

import coremltools.optimize as cto
 
op_config = cto.coreml.OpPalettizerConfig(mode="kmeans", nbits=4)
config = cto.coreml.OptimizationConfig(global_config=op_config)
palettized_model = cto.coreml.palettize_weights(mlmodel, config=config)
palettized_model.save("MyModel_palettized.mlpackage")

Concrete measurement from my own project: a MobileNetV3-equivalent image classifier went from 22MB to 3.5MB with palettization, accuracy drop under 1%. It's my default in 2026.

4. Compute Units — making sure Neural Engine actually runs

Apple Silicon devices have three inference paths: CPU, GPU, Neural Engine (ANE). Core ML auto-selects, but doesn't always land on ANE, and ANE is the fastest path by a wide margin.

Explicitly:

import CoreML
 
let config = MLModelConfiguration()
config.computeUnits = .all              // All three (default)
// .cpuOnly
// .cpuAndGPU
// .cpuAndNeuralEngine                  // Prefer ANE

When a Rork Max-built app feels slow, the cause is almost always "ANE isn't actually handling the work." Verify with Xcode Instruments → Core ML template during development. Do this once per model.

5. Image preprocessing — push it into the model

Preprocessing is a hot path. If you resize, normalize, and channel-convert in Swift for every frame, the CPU burns cycles that should be idle.

Fold preprocessing into the model itself. Specify ct.ImageType during conversion with the right scale and reshape, and Core ML handles it internally. Swift code then passes CVPixelBuffer straight in.

// Good: pass CVPixelBuffer directly
let input = MyModelInput(input: pixelBuffer)
let output = try model.prediction(input: input)
 
// Bad: preprocess in Swift first
// (CPU-heavy per frame)

6. Battery management

On-device inference costs energy. Continuous inference drains batteries fast, and App Store review has flagged apps for this. Three controls:

Gate inference on intent. Don't run every frame; run when the user acts (shutter, tap). If continuous inference is genuinely needed, drop from 30fps to 10fps — often invisible to users, dramatic in power draw.

Respect Low Power Mode. Check ProcessInfo.processInfo.isLowPowerModeEnabled and widen intervals or swap to a smaller model when it's on.

Throttle on thermal state. ProcessInfo.processInfo.thermalState at .serious or .critical means stop inferring briefly. Preempts OS-level throttling and keeps sustained performance higher.

7. Per-device performance

Neural Engine improves roughly 2× per generation. Model that's slow on an iPhone 12 can be completely fine on iPhone 15.

Rough 2026 targets:

• iPhone 12 and earlier: lightweight models (<3MB, ~224×224 input)
• iPhone 13–15: mid-range (~20MB, ~384×384)
• iPhone 16+: larger (~100MB, 512×512+)

Rork Max projects targeting iOS 17+ on newer devices can tolerate large models. Still plan a fallback model for older hardware.

8. Multiple models and memory management

If your app uses more than one model, memory becomes the constraint. Loaded Core ML models easily consume hundreds of MB; stack a few and jetsam (iOS's low-memory kill) comes for you.

Load on demand, release explicitly.

class ModelManager {
    private var currentModel: MLModel?
 
    func loadModel(named name: String) throws {
        let url = Bundle.main.url(forResource: name, withExtension: "mlmodelc")!
        currentModel = try MLModel(contentsOf: url)
    }
 
    func unloadModel() {
        currentModel = nil  // ARC frees immediately
    }
}

This single pattern cut my peak memory to about a third and effectively eliminated crash-on-old-devices reports.

9. A/B testing and over-the-air model updates

Core ML models ship in the app binary by default, which means updating a model means a new release. Apple's compileModel API breaks that — compile a downloaded model at runtime.

let compiledUrl = try MLModel.compileModel(at: downloadedMlpackageUrl)
let model = try MLModel(contentsOf: compiledUrl)

With this, you can serve models from your backend and switch at runtime — A/B tests, per-user personalization, rapid iteration without App Store review. Surface a clean download-and-compile UX (progress indicator, background compile) and users won't notice it happening.

10. Debugging and profiling

Static analysis isn't reliable for Core ML performance. Measure on device.

Xcode Instruments has a Core ML template that visualizes per-inference latency, the actual Compute Unit used, and memory impact. Measurement rules: Release build, Low Power Mode off, app running alone (other foreground apps skew the numbers).

Priorities and principles

Distilling the above into a priority list for Rork Max + Core ML:

First: shrink the model. Binary size, download time, launch latency, memory — all downstream of model size. Palettization is the highest-leverage technique available in 2026; start there.

Second: verify ANE usage. Instruments will tell you whether your model is actually running on Neural Engine. If not, revisit conversion options — convert_to="mlprogram" and minimum_deployment_target usually fix it.

Third: manage battery and heat. This is what keeps you out of App Store rejection and makes your app feel native.

Rork Max generates beautiful SwiftUI. Pair it with a well-optimized Core ML model and you get an iOS app that holds its own against any hand-built native app — often better, because you got to the optimization work faster. Next time you're wiring up a model, walk the ten sections in order. Each one compounds.