Building vek: SIMD Vector Kernels from Scratch
Vector search is the backbone of RAG pipelines, embeddings, and recommendation systems. We built vek to solve this from scratch — zero dependencies, hand-tuned SIMD, and a stable C ABI.
Why vek?
Most vector libraries are heavy. We wanted something that:
- Compiles anywhere (Linux, macOS, Windows)
- Has no runtime dependencies
- Picks the best SIMD backend at startup
- Stays out of the hot path
The Approach
Runtime CPU dispatch. One binary detects SSE2, AVX2, AVX-512, or NEON at startup and routes to the optimal kernel.
Multiple precisions. f32, f16, bf16, int8, uint8, and 1-bit binary vectors, each with dot product, L2 distance, and cosine similarity.
Numerical stability. Neumaier compensated summation prevents catastrophic cancellation in f32 dot products.
The Hot Path
Every function follows the same pattern: CPU feature detection at load time, then indirect through a function pointer table. No branches in the inner loop.
The masked load approach for AVX-512 means all elements use the same arithmetic path — no scalar tails, no cleanup code.
Benchmarks
Coming soon. For now, the kernels are being integrated into our own RAG pipelines.
Get It
git clone https://github.com/wraithen0/vek
cd vek
makeLicensed MIT OR Apache-2.0.