◉ AI Models/2026-03-29Advanced

RAG with Rork — Building Knowledge-Powered AI Chat into Your Mobile App

Learn how to implement RAG (Retrieval-Augmented Generation) in a mobile app built with Rork. This guide covers the full pipeline using Supabase pgvector and Gemini API to build production-ready AI chat.

RAG knowledge base AI chat pgvector³ Supabase³³ Gemini API⁵ vector search Rork Max²³⁰

✦ Premium Article

What Is RAG and Why Does Your Mobile App Need It?

When you add AI chat to your app, you quickly run into a fundamental problem: LLMs don't know about your specific data. Whether it's your product FAQ, internal documentation, regional tourism guides, or proprietary business knowledge, the information your users need most is often absent from the model's training data.

RAG (Retrieval-Augmented Generation) is an architecture pattern that solves this problem elegantly. Instead of hoping the LLM "just knows" the answer, RAG first searches a vector database for relevant documents, then feeds those documents as context to the LLM. The result is answers grounded in your actual knowledge base rather than the model's general training.

The benefits for mobile apps are substantial.

Dramatically reduced hallucination: Instead of fabricating plausible-sounding answers, the LLM responds based on real documents from your knowledge base
Real-time information updates: Update your knowledge base and the AI's answers change immediately — no model retraining required
Cost efficiency: Compared to fine-tuning, RAG has lower setup costs and makes it trivial to add or modify knowledge
Domain-specific accuracy: Particularly powerful for specialized apps in healthcare, legal, education, and customer support

In this article, we'll walk through the complete process of integrating a RAG pipeline into a Rork-built mobile app. We'll use Supabase (with the pgvector extension) as our vector store and Google's Gemini API for both embeddings and answer generation.

RAG Architecture Overview

A RAG pipeline has two distinct phases: the Indexing Phase (preparation) and the Query Phase (answering user questions in real time).

Indexing Phase (Offline Processing)

Document collection: Gather source material — PDFs, Markdown files, web pages, or database records
Chunking: Split documents into appropriately sized fragments (chunks)
Embedding generation: Convert each chunk into a vector (array of numbers) using an embedding model
Vector storage: Store vectors and metadata in Supabase pgvector

Query Phase (Real-Time Processing)

Query embedding: Convert the user's question into a vector
Similarity search: Find the most relevant chunks using pgvector's cosine similarity
Context assembly: Format the retrieved chunks into a prompt
Answer generation: Send the context-enhanced prompt to Gemini API and return the response

Understanding this two-phase structure is essential for everything that follows.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Understand RAG architecture design principles and the optimal patterns for mobile environments

✦Master end-to-end implementation using Supabase pgvector and Gemini API

✦Build a complete pipeline — from chunking and embeddings to similarity search and answer generation — ready to integrate into your own app

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Setting Up the Foundation — Supabase and pgvector

Let's start with the vector database. Supabase is a PostgreSQL-based Backend as a Service that supports vector search through the pgvector extension.

Creating the Database Schema

After creating a new Supabase project, run the following in the SQL Editor:

-- Enable the pgvector extension
create extension if not exists vector;
 
-- Table for storing document chunks
create table documents (
  id bigserial primary key,
  content text not null,
  metadata jsonb default '{}',
  embedding vector(768),  -- Gemini text-embedding-004 dimensions
  created_at timestamptz default now()
);
 
-- Index for cosine similarity search
create index on documents
  using ivfflat (embedding vector_cosine_ops)
  with (lists = 100);
 
-- RPC function for similarity search
create or replace function match_documents(
  query_embedding vector(768),
  match_threshold float default 0.7,
  match_count int default 5
)
returns table (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
)
language plpgsql
as $$
begin
  return query
  select
    documents.id,
    documents.content,
    documents.metadata,
    1 - (documents.embedding <=> query_embedding) as similarity
  from documents
  where 1 - (documents.embedding <=> query_embedding) > match_threshold
  order by documents.embedding <=> query_embedding
  limit match_count;
end;
$$;

The key detail here is vector(768) — this must match the output dimensions of your embedding model. Gemini's text-embedding-004 outputs 768-dimensional vectors.

Why IVFFlat Over HNSW?

pgvector offers two index types, each with different trade-offs:

IVFFlat: Faster to build with lower memory usage. Well-suited for datasets under 100,000 records. The lists parameter controls the number of clusters
HNSW: Higher search accuracy with better support for real-time updates. Preferable for larger datasets, but consumes more memory

For most mobile app knowledge bases — typically ranging from a few thousand to tens of thousands of chunks — IVFFlat delivers excellent performance.

Chunking Strategy — The Most Important Design Decision

The quality of your RAG pipeline depends more on chunking strategy than almost any other factor. Chunks that are too large dilute relevance scores; chunks that are too small lose important context.

Basic Overlap-Based Chunking

// utils/chunker.ts — Split text into overlapping chunks
interface ChunkOptions {
  chunkSize: number;      // Maximum characters per chunk
  chunkOverlap: number;   // Overlapping characters between chunks
  separator: string;      // Delimiter for splitting
}
 
const DEFAULT_OPTIONS: ChunkOptions = {
  chunkSize: 500,
  chunkOverlap: 50,
  separator: '\n\n',
};
 
export function splitIntoChunks(
  text: string,
  options: ChunkOptions = DEFAULT_OPTIONS
): string[] {
  const { chunkSize, chunkOverlap, separator } = options;
  const paragraphs = text.split(separator);
  const chunks: string[] = [];
  let currentChunk = '';
 
  for (const paragraph of paragraphs) {
    // Add to current chunk if within size limit
    if ((currentChunk + separator + paragraph).length <= chunkSize) {
      currentChunk = currentChunk
        ? currentChunk + separator + paragraph
        : paragraph;
    } else {
      // Finalize current chunk and start a new one
      if (currentChunk) {
        chunks.push(currentChunk.trim());
        // Overlap: carry the tail end into the next chunk
        const overlap = currentChunk.slice(-chunkOverlap);
        currentChunk = overlap + separator + paragraph;
      } else {
        // Force-split paragraphs that exceed chunkSize
        currentChunk = paragraph;
      }
    }
  }
 
  if (currentChunk.trim()) {
    chunks.push(currentChunk.trim());
  }
 
  return chunks;
}

Choosing the Right Chunk Size

Optimal chunk sizes vary by document type:

FAQ / Q&A pairs: 300–500 characters. Each question-answer pair should fit in a single chunk
Technical documentation: 500–800 characters. Split by section, preserving code blocks intact
Long-form articles and manuals: 800–1,200 characters. Maintain paragraph coherence while providing sufficient context
Conversation logs: 200–400 characters. Split by utterance for higher search precision

For the overlap parameter, aim for 10–15% of your chunk size. Overlap prevents meaning from being severed at chunk boundaries.

Semantic Chunking — A More Sophisticated Approach

Rather than splitting by character count alone, semantic chunking respects the logical structure of your documents:

// utils/semantic-chunker.ts — Structure-aware splitting
export function semanticChunk(markdown: string): string[] {
  const sections: string[] = [];
  const lines = markdown.split('\n');
  let currentSection = '';
  let currentHeading = '';
 
  for (const line of lines) {
    // Start a new section at H2 or H3 headings
    if (line.match(/^#{2,3}\s/)) {
      if (currentSection.trim()) {
        sections.push(
          `${currentHeading}\n${currentSection.trim()}`
        );
      }
      currentHeading = line;
      currentSection = '';
    } else {
      currentSection += line + '\n';
    }
  }
 
  if (currentSection.trim()) {
    sections.push(
      `${currentHeading}\n${currentSection.trim()}`
    );
  }
 
  return sections;
}

Semantic chunking is especially effective for well-structured documents like technical manuals and FAQs. By including the heading at the start of each chunk, you provide the search system with clear context about which section the information belongs to.

Generating Embeddings with Gemini text-embedding-004

With chunks prepared, the next step is converting them into vectors using Gemini's embedding model.

Supabase Edge Function Implementation

// supabase/functions/generate-embeddings/index.ts
import { serve } from 'https://deno.land/std@0.168.0/http/server.ts';
import { createClient } from 'https://esm.sh/@supabase/supabase-js@2';
 
const GEMINI_API_URL =
  'https://generativelanguage.googleapis.com/v1beta/models/text-embedding-004:embedContent';
 
interface EmbeddingRequest {
  chunks: { content: string; metadata?: Record<string, unknown> }[];
}
 
serve(async (req) => {
  const { chunks } = (await req.json()) as EmbeddingRequest;
 
  const supabase = createClient(
    Deno.env.get('SUPABASE_URL')!,
    Deno.env.get('SUPABASE_SERVICE_ROLE_KEY')!
  );
 
  const results = [];
 
  // Batch processing: 5 at a time for rate limit compliance
  for (let i = 0; i < chunks.length; i += 5) {
    const batch = chunks.slice(i, i + 5);
 
    const embeddings = await Promise.all(
      batch.map(async (chunk) => {
        const res = await fetch(
          `${GEMINI_API_URL}?key=${Deno.env.get('GEMINI_API_KEY')}`,
          {
            method: 'POST',
            headers: { 'Content-Type': 'application/json' },
            body: JSON.stringify({
              model: 'models/text-embedding-004',
              content: { parts: [{ text: chunk.content }] },
              taskType: 'RETRIEVAL_DOCUMENT',
            }),
          }
        );
 
        const data = await res.json();
        return {
          content: chunk.content,
          metadata: chunk.metadata || {},
          embedding: data.embedding.values,
        };
      })
    );
 
    // Store in Supabase
    const { error } = await supabase
      .from('documents')
      .insert(embeddings);
 
    if (error) throw error;
    results.push(...embeddings.map((e) => ({ success: true })));
 
    // Rate limit buffer: 100ms between batches
    if (i + 5 < chunks.length) {
      await new Promise((r) => setTimeout(r, 100));
    }
  }
 
  return new Response(
    JSON.stringify({ processed: results.length }),
    { headers: { 'Content-Type': 'application/json' } }
  );
});

The Importance of taskType

Gemini's embedding API includes a taskType parameter that optimizes vector generation for specific use cases:

RETRIEVAL_DOCUMENT: Use when embedding document chunks during the indexing phase
RETRIEVAL_QUERY: Use when embedding user queries during the search phase
SEMANTIC_SIMILARITY: Use for general text-to-text similarity comparisons

Using different task types for documents versus queries significantly improves search accuracy. This is due to Gemini's asymmetric retrieval optimization, which adjusts the embedding space for the different roles of queries and documents.

Implementing Similarity Search

Now let's build the search function that takes a user's question and finds the most relevant chunks.

// supabase/functions/search-documents/index.ts
import { serve } from 'https://deno.land/std@0.168.0/http/server.ts';
import { createClient } from 'https://esm.sh/@supabase/supabase-js@2';
 
const GEMINI_API_URL =
  'https://generativelanguage.googleapis.com/v1beta/models/text-embedding-004:embedContent';
 
serve(async (req) => {
  const { query, matchCount = 5, threshold = 0.7 } = await req.json();
 
  // 1. Generate query embedding (taskType: RETRIEVAL_QUERY)
  const embeddingRes = await fetch(
    `${GEMINI_API_URL}?key=${Deno.env.get('GEMINI_API_KEY')}`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        model: 'models/text-embedding-004',
        content: { parts: [{ text: query }] },
        taskType: 'RETRIEVAL_QUERY',
      }),
    }
  );
 
  const embeddingData = await embeddingRes.json();
  const queryEmbedding = embeddingData.embedding.values;
 
  // 2. Run similarity search via pgvector
  const supabase = createClient(
    Deno.env.get('SUPABASE_URL')!,
    Deno.env.get('SUPABASE_SERVICE_ROLE_KEY')!
  );
 
  const { data: documents, error } = await supabase.rpc(
    'match_documents',
    {
      query_embedding: queryEmbedding,
      match_threshold: threshold,
      match_count: matchCount,
    }
  );
 
  if (error) throw error;
 
  return new Response(
    JSON.stringify({ documents }),
    { headers: { 'Content-Type': 'application/json' } }
  );
});

Techniques for Better Search Quality

Hybrid Search

Combining vector search with PostgreSQL's full-text search (tsvector) lets you cover both semantic similarity and exact keyword matches:

-- Hybrid search function combining vector and text search
create or replace function hybrid_search(
  query_text text,
  query_embedding vector(768),
  match_count int default 5,
  vector_weight float default 0.7,
  text_weight float default 0.3
)
returns table (
  id bigint,
  content text,
  metadata jsonb,
  combined_score float
)
language plpgsql
as $$
begin
  return query
  with vector_results as (
    select
      d.id,
      d.content,
      d.metadata,
      1 - (d.embedding <=> query_embedding) as vector_score
    from documents d
    order by d.embedding <=> query_embedding
    limit match_count * 2
  ),
  text_results as (
    select
      d.id,
      d.content,
      d.metadata,
      ts_rank(
        to_tsvector('english', d.content),
        plainto_tsquery('english', query_text)
      ) as text_score
    from documents d
    where to_tsvector('english', d.content)
      @@ plainto_tsquery('english', query_text)
    limit match_count * 2
  )
  select
    coalesce(v.id, t.id) as id,
    coalesce(v.content, t.content) as content,
    coalesce(v.metadata, t.metadata) as metadata,
    (coalesce(v.vector_score, 0) * vector_weight +
     coalesce(t.text_score, 0) * text_weight) as combined_score
  from vector_results v
  full outer join text_results t on v.id = t.id
  order by combined_score desc
  limit match_count;
end;
$$;

Re-ranking for Precision

After the initial retrieval returns the top candidates, a re-ranking step can reorder them using a more precise scoring method:

// utils/reranker.ts — LLM-based re-ranking
export async function rerankDocuments(
  query: string,
  documents: { content: string; similarity: number }[],
  apiKey: string
): Promise<{ content: string; score: number }[]> {
  const prompt = `Rate the relevance of each document to the question below on a scale of 0.0 to 1.0.
Return a JSON array.
 
Question: ${query}
 
Documents:
${documents.map((d, i) => `[${i}] ${d.content.slice(0, 200)}`).join('\n')}
 
Response format: [{"index": 0, "score": 0.9}, ...]`;
 
  const res = await fetch(
    `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=${apiKey}`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        contents: [{ parts: [{ text: prompt }] }],
        generationConfig: { temperature: 0 },
      }),
    }
  );
 
  const data = await res.json();
  const text = data.candidates[0].content.parts[0].text;
  const scores = JSON.parse(
    text.replace(/```json\n?|\n?```/g, '')
  );
 
  return scores
    .map((s: { index: number; score: number }) => ({
      content: documents[s.index].content,
      score: s.score,
    }))
    .sort(
      (a: { score: number }, b: { score: number }) =>
        b.score - a.score
    );
}

Generating Answers — Prompt Design for RAG

The final step in the query pipeline is passing the retrieved chunks as context to the LLM and generating an answer. The prompt design here has an outsized impact on answer quality.

// utils/rag-chain.ts — RAG answer generation chain
interface RAGContext {
  query: string;
  documents: { content: string; metadata: Record<string, unknown> }[];
  systemPrompt?: string;
}
 
export function buildRAGPrompt(context: RAGContext): string {
  const { query, documents, systemPrompt } = context;
 
  const contextText = documents
    .map(
      (doc, i) =>
        `[Source ${i + 1}] ${doc.metadata?.source || 'Unknown'}\n${doc.content}`
    )
    .join('\n\n---\n\n');
 
  const defaultSystemPrompt = `You are a helpful assistant.
Follow these rules when answering:
 
1. Base your answer ONLY on the provided context
2. If the context doesn't contain the needed information, say "This information is not available in the knowledge base"
3. Include source references [Source N] in your answer
4. Do not supplement with guesses or general knowledge
5. Be concise and accurate`;
 
  return `${systemPrompt || defaultSystemPrompt}
 
## Context
 
${contextText}
 
## User Question
 
${query}
 
## Answer`;
}
 
export async function generateRAGResponse(
  context: RAGContext,
  apiKey: string
): Promise<string> {
  const prompt = buildRAGPrompt(context);
 
  const res = await fetch(
    `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=${apiKey}`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        contents: [{ parts: [{ text: prompt }] }],
        generationConfig: {
          temperature: 0.3,  // Low for factual answers
          maxOutputTokens: 1024,
          topP: 0.8,
        },
      }),
    }
  );
 
  const data = await res.json();
  return data.candidates[0].content.parts[0].text;
}

Prompt Design Best Practices

Effective RAG prompts share several characteristics:

Explicit constraints: Stating "only answer from the provided context" significantly reduces hallucination
Source attribution: Requiring source references lets users verify the basis for each answer
Low temperature: Use 0.1–0.3 for factual responses. Raise to 0.5–0.7 when creative or conversational answers are appropriate
Format guidance: Specify whether you want Markdown, bullet points, numbered lists, or plain prose

Integrating RAG into Your Rork App

With the backend pipeline in place, let's build the mobile frontend using React Native components in Rork.

Chat Screen Component

// components/RAGChat.tsx — Main RAG chat component
import React, { useState, useRef, useCallback } from 'react';
import {
  View,
  TextInput,
  FlatList,
  Text,
  TouchableOpacity,
  ActivityIndicator,
  StyleSheet,
  KeyboardAvoidingView,
  Platform,
} from 'react-native';
 
interface Message {
  id: string;
  role: 'user' | 'assistant';
  content: string;
  sources?: { title: string; excerpt: string }[];
  timestamp: Date;
}
 
const SUPABASE_URL = 'YOUR_SUPABASE_URL';
const SUPABASE_ANON_KEY = 'YOUR_SUPABASE_ANON_KEY';
 
export default function RAGChat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState('');
  const [loading, setLoading] = useState(false);
  const flatListRef = useRef<FlatList>(null);
 
  const sendMessage = useCallback(async () => {
    if (!input.trim() || loading) return;
 
    const userMessage: Message = {
      id: Date.now().toString(),
      role: 'user',
      content: input.trim(),
      timestamp: new Date(),
    };
 
    setMessages((prev) => [...prev, userMessage]);
    setInput('');
    setLoading(true);
 
    try {
      // Call Supabase Edge Function for RAG processing
      const response = await fetch(
        `${SUPABASE_URL}/functions/v1/rag-chat`,
        {
          method: 'POST',
          headers: {
            'Content-Type': 'application/json',
            Authorization: `Bearer ${SUPABASE_ANON_KEY}`,
          },
          body: JSON.stringify({ query: userMessage.content }),
        }
      );
 
      const data = await response.json();
 
      const assistantMessage: Message = {
        id: (Date.now() + 1).toString(),
        role: 'assistant',
        content: data.answer,
        sources: data.sources,
        timestamp: new Date(),
      };
 
      setMessages((prev) => [...prev, assistantMessage]);
    } catch (error) {
      setMessages((prev) => [
        ...prev,
        {
          id: (Date.now() + 1).toString(),
          role: 'assistant',
          content:
            'Sorry, an error occurred while generating the answer. Please try again.',
          timestamp: new Date(),
        },
      ]);
    } finally {
      setLoading(false);
    }
  }, [input, loading]);
 
  const renderMessage = ({ item }: { item: Message }) => (
    <View
      style={[
        styles.messageContainer,
        item.role === 'user'
          ? styles.userMessage
          : styles.assistantMessage,
      ]}
    >
      <Text style={styles.messageText}>{item.content}</Text>
      {item.sources && item.sources.length > 0 && (
        <View style={styles.sourcesContainer}>
          <Text style={styles.sourcesTitle}>Sources:</Text>
          {item.sources.map((source, index) => (
            <Text key={index} style={styles.sourceItem}>
              [{index + 1}] {source.title}
            </Text>
          ))}
        </View>
      )}
    </View>
  );
 
  return (
    <KeyboardAvoidingView
      style={styles.container}
      behavior={Platform.OS === 'ios' ? 'padding' : 'height'}
    >
      <FlatList
        ref={flatListRef}
        data={messages}
        renderItem={renderMessage}
        keyExtractor={(item) => item.id}
        onContentSizeChange={() =>
          flatListRef.current?.scrollToEnd()
        }
        contentContainerStyle={styles.messagesList}
      />
      <View style={styles.inputContainer}>
        <TextInput
          style={styles.input}
          value={input}
          onChangeText={setInput}
          placeholder="Ask a question..."
          multiline
          editable={!loading}
        />
        <TouchableOpacity
          style={[
            styles.sendButton,
            loading && styles.sendButtonDisabled,
          ]}
          onPress={sendMessage}
          disabled={loading}
        >
          {loading ? (
            <ActivityIndicator color="#fff" size="small" />
          ) : (
            <Text style={styles.sendButtonText}>Send</Text>
          )}
        </TouchableOpacity>
      </View>
    </KeyboardAvoidingView>
  );
}
 
// Expected behavior:
// 1. User types a question and taps Send
// 2. Loading indicator appears
// 3. RAG backend searches relevant documents and generates an answer
// 4. Answer with source citations appears in the chat
 
const styles = StyleSheet.create({
  container: { flex: 1, backgroundColor: '#f5f5f5' },
  messagesList: { padding: 16 },
  messageContainer: {
    maxWidth: '80%',
    padding: 12,
    borderRadius: 16,
    marginBottom: 8,
  },
  userMessage: {
    alignSelf: 'flex-end',
    backgroundColor: '#007AFF',
  },
  assistantMessage: {
    alignSelf: 'flex-start',
    backgroundColor: '#fff',
    borderWidth: 1,
    borderColor: '#e0e0e0',
  },
  messageText: { fontSize: 15, lineHeight: 22 },
  sourcesContainer: {
    marginTop: 8,
    paddingTop: 8,
    borderTopWidth: 1,
    borderTopColor: '#e0e0e0',
  },
  sourcesTitle: {
    fontSize: 12,
    fontWeight: '600',
    color: '#666',
    marginBottom: 4,
  },
  sourceItem: { fontSize: 12, color: '#888', marginBottom: 2 },
  inputContainer: {
    flexDirection: 'row',
    padding: 12,
    backgroundColor: '#fff',
    borderTopWidth: 1,
    borderTopColor: '#e0e0e0',
  },
  input: {
    flex: 1,
    backgroundColor: '#f0f0f0',
    borderRadius: 20,
    paddingHorizontal: 16,
    paddingVertical: 10,
    fontSize: 15,
    maxHeight: 100,
  },
  sendButton: {
    marginLeft: 8,
    backgroundColor: '#007AFF',
    borderRadius: 20,
    paddingHorizontal: 16,
    justifyContent: 'center',
  },
  sendButtonDisabled: { opacity: 0.6 },
  sendButtonText: {
    color: '#fff',
    fontSize: 15,
    fontWeight: '600',
  },
});

Performance Optimization for Mobile

Mobile apps face unique constraints — variable network conditions, limited processing power, and battery considerations. Here are the key optimizations for a production RAG pipeline.

Caching Responses

Running a full vector search and LLM call for every question is wasteful when users ask similar or identical questions:

// utils/rag-cache.ts — Query-answer caching layer
import AsyncStorage from '@react-native-async-storage/async-storage';
 
const CACHE_PREFIX = 'rag_cache_';
const CACHE_TTL = 24 * 60 * 60 * 1000; // 24 hours
 
interface CacheEntry {
  answer: string;
  sources: { title: string; excerpt: string }[];
  timestamp: number;
}
 
export async function getCachedAnswer(
  query: string
): Promise<CacheEntry | null> {
  const key = CACHE_PREFIX + hashQuery(query);
  const cached = await AsyncStorage.getItem(key);
 
  if (!cached) return null;
 
  const entry: CacheEntry = JSON.parse(cached);
 
  // TTL check
  if (Date.now() - entry.timestamp > CACHE_TTL) {
    await AsyncStorage.removeItem(key);
    return null;
  }
 
  return entry;
}
 
export async function setCachedAnswer(
  query: string,
  answer: string,
  sources: { title: string; excerpt: string }[]
): Promise<void> {
  const key = CACHE_PREFIX + hashQuery(query);
  const entry: CacheEntry = {
    answer,
    sources,
    timestamp: Date.now(),
  };
  await AsyncStorage.setItem(key, JSON.stringify(entry));
}
 
// Simple hash function (normalize then hash)
function hashQuery(query: string): string {
  const normalized = query
    .toLowerCase()
    .trim()
    .replace(/\s+/g, ' ');
  let hash = 0;
  for (let i = 0; i < normalized.length; i++) {
    const char = normalized.charCodeAt(i);
    hash = (hash << 5) - hash + char;
    hash |= 0;
  }
  return Math.abs(hash).toString(36);
}

Streaming Responses

Displaying the answer incrementally as it's generated dramatically improves perceived performance:

// hooks/useStreamingRAG.ts — Streaming answer hook
import { useState, useCallback } from 'react';
 
export function useStreamingRAG(supabaseUrl: string, apiKey: string) {
  const [streamedText, setStreamedText] = useState('');
  const [isStreaming, setIsStreaming] = useState(false);
 
  const streamAnswer = useCallback(
    async (query: string) => {
      setStreamedText('');
      setIsStreaming(true);
 
      try {
        const response = await fetch(
          `${supabaseUrl}/functions/v1/rag-chat-stream`,
          {
            method: 'POST',
            headers: {
              'Content-Type': 'application/json',
              Authorization: `Bearer ${apiKey}`,
            },
            body: JSON.stringify({ query }),
          }
        );
 
        const reader = response.body?.getReader();
        const decoder = new TextDecoder();
 
        if (!reader) throw new Error('Stream not available');
 
        while (true) {
          const { done, value } = await reader.read();
          if (done) break;
 
          const chunk = decoder.decode(value, { stream: true });
          setStreamedText((prev) => prev + chunk);
        }
      } finally {
        setIsStreaming(false);
      }
    },
    [supabaseUrl, apiKey]
  );
 
  return { streamedText, isStreaming, streamAnswer };
}

Rate Limiting and Error Handling

Robust retry logic with exponential backoff is essential for production use:

// utils/retry.ts — Retry with exponential backoff
export async function withRetry<T>(
  fn: () => Promise<T>,
  maxRetries: number = 3,
  baseDelay: number = 1000
): Promise<T> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error: unknown) {
      if (attempt === maxRetries) throw error;
 
      const status =
        error instanceof Response ? error.status : 0;
 
      // Only retry on 429 (Rate Limit) or 5xx errors
      if (status !== 429 && status < 500 && status !== 0) {
        throw error;
      }
 
      const delay =
        baseDelay * Math.pow(2, attempt) +
        Math.random() * 1000;
      await new Promise((r) => setTimeout(r, delay));
    }
  }
  throw new Error('Max retries exceeded');
}

Security and Privacy Considerations

When your knowledge base contains sensitive information, security isn't optional — it's foundational.

Row Level Security (RLS)

Use Supabase RLS to restrict document access on a per-user basis:

-- Per-user document access control
alter table documents enable row level security;
 
create policy "Users can only access their own documents"
  on documents for select
  using (
    metadata->>'user_id' = auth.uid()::text
    or metadata->>'visibility' = 'public'
  );

PII Filtering

Before ingesting documents into your knowledge base, implement a filtering step that detects and strips personally identifiable information. Similarly, consider whether user queries might inadvertently contain sensitive data before sending them to the LLM.

Secure API Key Management

Never hardcode API keys in your mobile app's source code. Use Supabase Edge Functions as a proxy layer, keeping all API keys on the server side. Throughout this article, Gemini API keys are retrieved from server-side environment variables via Deno.env.get('GEMINI_API_KEY') — the mobile client never sees them.

Looking back

RAG (Retrieval-Augmented Generation) is a transformative architecture pattern for AI-powered mobile apps. In this guide, we covered the full pipeline — from chunking strategies and embedding generation to similarity search, prompt design, and integration into a Rork-built React Native app.

The core advantage of RAG is its ability to deliver accurate, real-time answers grounded in your proprietary data, without the cost and complexity of model fine-tuning. Whether you're building a customer support chatbot, an internal knowledge search tool, or an educational Q&A platform, RAG provides the foundation for trustworthy AI interactions.

Start small — perhaps with a few dozen FAQ entries — to understand how the pipeline behaves. Once you're confident in the results, scale gradually by adding more documents, implementing hybrid search, and fine-tuning your chunking strategy.

Thank You for Reading

Rork Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.