One directory lookup.
One contiguous scan.
Filtered vector search is usually a scatter-gather across an index that knows nothing about your filter. Telys owns the physical layout instead: on the partition key, nearest-neighbors-where-key-equals-x is an O(1) directory lookup plus one sequential scan of a contiguous block. This page walks the runtime that ships today, then the architecture it is converging on — labeled as spec.
The engine owns where every vector lives.
A collection declares partition_by at creation, and the layout follows the key. The base segment is clustered by partition — every key maps to one contiguous slice — and recent writes sit beside it in a mutable, Arrow-shaped delta.
Partition-clustered
Vectors stored sorted by partition key, so each partition is one contiguous block. A scoped query reads one sequential run of memory, not rows scattered across an index.
O(1) key → slice
A directory maps each partition value to its (offset, length) in the base segment. Resolving the filter is a dictionary lookup, not an index traversal.
Arrow-shaped writes
add and upsert append here, each row stamped with a monotonic write LSN. The delta is queryable immediately and unions with the base slice through one scan path.
What one scoped query executes.
col.search(qvec, where={"tenant_id": "acme"}, top_k=10)
The filter names the partition key the collection was created with.
O(1) lookup: the partition value resolves to (offset, length) in the base segment. No candidate generation, no graph entry point.
SIMD exact scoring over one sequential block. Recall on this path is 1.0 — it is a scan, not an approximation. Measured p50/p95 for this step publish on the benchmarks page, each bound to its recall gate and rig.
Rows for the same key in the Arrow-shaped delta are scored through the same path. A write is visible to the very next query.
Hits filter at the snapshot LSN: tombstoned and superseded versions never surface. Top-k returns with the explain payload attached.
Oversized partitions take a declared fork: build_ivf places a per-partition IVF over any partition above the row threshold, and queries route through it with an exact rerank — calibrated against a recall floor, and named in the plan.
Every result names its plan.
explain=True returns the physical plan the query actually took. On the partition key you get the contiguous-slice scan. Oversized partitions declare the IVF fork. Off-key filters fall back to scatter-gather — and the payload says why.
hits = col.search(qvec, where={"tenant_id": "acme"}, top_k=10, explain=True)
hits["explain"]["plan"]
# on the partition key → "PartitionSliceExactF32" # one contiguous slice, exact, recall 1.0
# oversized partition → "PartitionIVFRerankF32" # per-partition IVF + exact rerank
# off-key filter → "ScatterGatherExact" # fallback — and it says why:
hits["explain"]["fallback_reason"]
# "path is not the physical partition key"Nothing is overwritten. Versions are superseded.
The write model is append-only MVCC. upsert writes a new version of a logical id and marks the prior one superseded; delete places a tombstone at a delete LSN; snapshot() pins a consistent read view; compact() folds the delta into the base and drops what no snapshot can see.
col.upsert(vectors, ids=ids, metadata=metadata) # a new version; the prior one is superseded col.delete(["doc-41"]) # tombstone at a delete LSN — no in-place erase lsn = col.snapshot() # pin a consistent read view at an LSN col.compact() # fold delta into base; drop superseded rows col.build_ivf(min_rows=20000, target_recall=0.98)
| Call | What it does |
|---|---|
| add / add_texts | Insert new rows. Duplicate ids are rejected — a second physical row for one logical id requires upsert, on purpose. |
| upsert / upsert_texts | Write a new version of a logical id; the prior version is superseded, never overwritten in place. |
| search / search_text | Filtered top-k. explain=True attaches the physical plan to the result. |
| delete | Tombstone logical rows at a delete LSN. |
| compact | Fold the delta into the base segment; drop tombstoned and superseded versions. |
| build_ivf | Build per-partition IVF over partitions above the row threshold, calibrated to a recall floor. |
| snapshot | Return an LSN that pins a consistent read view. |
| save / stats | Persist the collection to disk; report row counts, partitions, and layout facts. |
Everything below is specification, not shipped API reference. It is the substrate the runtime is converging on. The SDK-facing seam is frozen, so the swap underneath is invisible to callers.
One planner. One flat IR. One executor.
Three front ends lower into one planner, which emits one flat physical intermediate representation, executed by one Mojo vectorized executor directly over Arrow buffers.
The memory model (remember · recall · as_of), the query API (point_get · scan · search · hybrid_search), and the fabric. One entry contract; no per-API engine.
Logical and physical planning in one place. Selectivity is estimated from segment statistics; the plan is explicit and returned with the result.
A flat array of operators bound by integer slots. The inter-operator register is a row set — a bitmap, sorted row-ids, or a selection vector.
Vectorized execution over Arrow buffers and FAISS candidate arrays. The SIMD hot path stays out of Python; compressed Parquet pages decode into scratch buffers first.
The mutation layer
A write-ahead log for durability and ordering, and a mutable Arrow-shaped delta for immediate visibility — the table layer Parquet lacks on its own.
Sealed, immutable
Durable compressed columnar segments. Footer statistics drive segment and row-group pruning; sealed files are never mutated.
FAISS ANN
Per-segment FAISS indexes, bound to the segment's checksum and version, mapped read-only with lifetime tied to the read snapshot.
Pruning and lexical
Zone maps, bloom filters, and bitmaps for pruning; a BM25 sidecar for sparse retrieval and score fusion.
SCAN FILTER PROJECT POINT_GET RANGE_GET AGGREGATE TOP_K HASH_JOIN HASH_GROUP_BY SORT ANN_SEARCH SPARSE_SEARCH FUSE RERANK MATERIALIZE
WAL first. Visible immediately. Sealed in the background.
Append-only frames with checksums and monotonic LSNs. Recovery replays records past the last sealed LSN and truncates at the first bad checksum.
The write lands in the mutable delta and is queryable immediately — sealed segments and delta union through one scan path.
Past a size, row, or age threshold, the delta is sorted and written as a Parquet segment with its .vidx and sidecars, then published by an atomic manifest swap.
Background merges fold small segments together, drop tombstoned rows, and rebuild sidecars — budget-controlled, so on-device deployments stay quiet.
Sealed segments are never mutated. A read pins (manifest snapshot, LSN); writers seal new segments and swap the manifest without disturbing in-flight readers. Garbage collection waits for the oldest live snapshot.
The filter chooses the plan, not the other way around.
Filtered ANN is adaptive planning by selectivity: the planner estimates how many rows survive the filter from segment statistics, then picks the cheapest strategy that keeps the recall contract.
Prefilter → exact
Scalar or bitmap prefilter first, then exact SIMD scoring over the survivors. Below a row threshold ANN is skipped entirely — exact scan, recall 1.0.
Allow-bitmap traversal
ANN traversal carries an allow-bitmap, so the index only surfaces rows the filter admits.
ANN → post-filter
ANN first with adaptive over-fetch, then post-filter. The filter removes little, so candidate generation leads.
Correctness gates precede any speed number.
The shipped runtime is verified by parity suites run against both engine implementations, and every query can name the physical plan it took. That is the order of operations here: correctness gates first, measurement second.
Benchmarks publish under a fairness contract — matched recall, matched hardware, transport-separated results, win / tie / lose reported. The first measured results are now live on the benchmarks page, every figure carried with its full conditions: hardware, dataset, dimension, selectivity, recall, and transport. Every multiplier remains a hypothesis outside the conditions stated beside it.