Back to blog
#turso #vector-search #rag #cloudflare #drizzle #sqlite

Building RAG with Turso: From Native Vector Search to a Split Architecture

When I started building a RAG (Retrieval-Augmented Generation) system as a hobby project on Cloudflare Workers, one of the first decisions was where to store vectors. Cloudflare Vectorize is the obvious choice on that platform, but Turso caught my attention because it lets you keep vectors in the same SQLite database as your regular relational data. One schema, one migration system, one connection. That simplicity is worth exploring.

Here is what I learned after building it, and why I eventually split the responsibilities between Turso and Vectorize anyway.

What Turso Vector Support Looks Like

Turso is a distributed SQLite database built on libSQL. The libSQL layer adds a native column type called F32_BLOB(N) for storing fixed-width float arrays, along with SQL functions for searching them.

A vector table looks like this:

CREATE TABLE chunk_vectors_1024 (
  id TEXT PRIMARY KEY,
  chunk_hash TEXT,
  agent_id TEXT,
  vector F32_BLOB(1024),
  created_at TEXT DEFAULT (CURRENT_TIMESTAMP)
);

The F32_BLOB(1024) column stores a 1024-dimensional embedding as a compact binary blob of 32-bit floats. There is no JSON, no base64, no string encoding. The database treats it as a first-class vector type.

For search, two approaches exist.

Brute-force cosine similarity:

SELECT id, vector_distance_cos(vector, vector32(?)) AS distance
FROM chunk_vectors_1024
WHERE agent_id = ?
ORDER BY distance
LIMIT ?

vector32(?) converts the query vector bytes into the native type. vector_distance_cos computes cosine distance between two vectors. The result is a full table scan sorted by distance, which works fine for small datasets.

HNSW index for approximate nearest neighbor:

SELECT chunk_vectors_1024.id
FROM vector_top_k('chunk_vectors_1024_idx', vector32(?), ?)
JOIN chunk_vectors_1024 ON chunk_vectors_1024.rowid = vector_top_k.id
WHERE agent_id = ?

vector_top_k queries an HNSW (Hierarchical Navigable Small World) index by name. The index is created as a separate DDL step outside of your regular table schema. The join retrieves the actual row data for whatever row IDs the index returns.

Representing F32_BLOB in Drizzle ORM

The interesting part on the code side is that Drizzle ORM does not know about F32_BLOB out of the box. You need to define a custom column type that maps between JavaScript’s number[] and the binary format libSQL expects.

const f32Blob = (dim: number) =>
  customType<{
    data: number[];
    driverData: Buffer;
  }>({
    dataType() {
      return `F32_BLOB(${dim})`;
    },
    fromDriver(value) {
      const floatArray = new Float32Array(new Uint8Array(value).buffer);
      return Array.from(floatArray);
    },
    toDriver(value: number[]) {
      return Buffer.from(new Float32Array(value).buffer);
    },
  });

The dataType() return value is injected directly into the CREATE TABLE statement, so Drizzle passes it straight to libSQL without interpreting it. The toDriver and fromDriver functions handle the conversion: JavaScript arrays become binary via Float32Array, and the stored binary comes back as a plain JavaScript array.

Using it in a table definition:

export const chunkVectors1024Table = sqliteTable('chunk_vectors_1024', {
  id: text('id').primaryKey().$defaultFn(() => crypto.randomUUID()),
  chunkHash: text('chunk_hash'),
  agentId: text('agent_id'),
  vector: f32Blob(1024)('vector'),
  createdAt: text('created_at').default(sql`(CURRENT_TIMESTAMP)`),
});

This works cleanly. Drizzle generates the right DDL, and the custom type handles the binary conversion transparently.

The Multi-Tenant Filtering Problem

Here is where the single-database approach ran into trouble.

My project had multiple agents, and each agent should only search its own chunks. The brute-force approach handles this cleanly because the WHERE agent_id = ? filter runs before computing distances. You scan only the rows belonging to that agent.

The HNSW index does not work that way. vector_top_k searches the entire index and returns the globally closest vectors. The WHERE agent_id = ? in the query above filters the results after the ANN search, which means:

  1. The index might return topK results globally, but fewer than topK belong to your agent.
  2. If you ask for topK = 5, you might get 2 results after filtering, even when the agent has hundreds of matching chunks.
  3. To guarantee you get enough results, you need to fetch far more than topK from the index and discard, which negates the speed advantage.

This is a well-known problem in multi-tenant vector search: per-tenant namespace filtering and ANN indices do not compose cleanly unless the index itself understands namespaces. Turso’s vector_top_k does not offer namespace-level filtering built in.

What I Landed On

I split the storage responsibility. Turso handles the relational data: agents, sources, chunk text, and chunk hashes. Cloudflare Vectorize handles the actual vector index, with each agent mapped to a Vectorize namespace.

The search path became:

User query
  -> embed the query text
  -> query Vectorize index with (vector, topK, namespace=agentId)
  -> Vectorize returns vector IDs
  -> fetch chunk text from Turso using those IDs
  -> return ranked results

Turso’s vector tables still exist in the schema, but they serve as the persistent storage for raw embeddings. When an agent is indexed, embeddings are written to both Turso and Vectorize. The migration function to move existing Turso vectors into Vectorize was a one-time batch job that pushed vectors in pages of 4000:

const batchSize = 4000;
const batchCount = Math.ceil(vectorCount / batchSize);

for (let i = 0; i < batchCount; i++) {
  const batch = await db
    .select()
    .from(chunkVectorTables[agent.dimension])
    .offset(i * batchSize)
    .limit(batchSize);

  await insertVectors(context, agentId, batch.map(vector => ({
    id: vector.id,
    values: vector.vector as number[],
    namespace: agentId,
  })));
}

Vectorize uses namespaces natively, so the per-agent filtering problem disappears. The topK results returned are already scoped to the agent’s namespace.

What Turso Vectors Are Good For

The native vector support in Turso is genuinely useful in the right context:

  • Single-tenant use: If you have one user’s data in a database and do not need namespace filtering, brute-force cosine search is simple and correct.
  • Small datasets: For a few thousand vectors, the full table scan is fast enough that you do not need ANN indexing at all.
  • Prototyping: The single-database model is faster to get running. You add a F32_BLOB column, write the custom Drizzle type, and you have vector search without any external service.

The multi-tenant filtering limitation is what pushed me toward a dedicated vector index. For a single-user RAG or a small dataset, I would keep everything in Turso and not bother splitting.

On AWS S3 Vectors

While debugging the architecture, I also tried AWS S3 Vectors (a relatively new service for storing and querying embeddings directly in S3). It is interesting as a concept because the storage cost of S3 is effectively zero compared to hosted vector databases. The Python migration script in the project ended up being a test harness for it rather than anything production-ready. The API is similar to other vector stores: put vectors with a key and metadata, query by similarity. The tooling is less mature than Cloudflare Vectorize at this point for Workers-based projects.

The final setup is Turso for data and Vectorize for search. Not the cleanest architecture, but an honest reflection of what each service is good at.