Your vector database is bleeding money.
You started with Pinecone's free tier. Built a RAG prototype. It worked great.
Then you went to production.
Suddenly you're paying $3,000/month. For what? Storing embeddings and running similarity searches. The same operation that costs $64/month on TurboPuffer.
Or maybe you went the "smart" route and self-hosted Milvus. Zero licensing costs! Except now you're paying an engineer 20-40 hours/month to manage Kubernetes clusters, tune indexes, and debug performance issues. That's $3,000-6,000/month in hidden DevOps costs.
There's a better way.
TurboPuffer is a serverless vector database built from first principles on object storage. It's 10-100x cheaper than traditional vector databases while maintaining competitive performance for most use cases.
Real numbers from production:
- Cursor: 95% cost reduction, 100B+ vectors
- Notion: 80% cost reduction, 10B+ vectors, "millions saved annually"
- At 100M vectors: $380/month (TurboPuffer) vs $1,200-1,800/month (Pinecone)
This isn't about choosing the "best" vector database. It's about choosing one you can afford to scale without hiring a dedicated team.
The Vector Database Problem: Hidden Costs Everywhere
Let's talk about what actually happens when you build RAG systems at scale.
The Storage Amplification Problem
You have 100GB of documents. Should be cheap to store, right?
Wrong.
When you vectorize text for RAG:
- Chunk your documents (500-1000 tokens each)
- Generate embeddings (1536 dimensions for OpenAI ada-002)
- Store vectors + metadata
Result: 100GB of text becomes 1.6TB of vector data. 16x storage amplification.
That 1.6TB in Pinecone costs ~$6,000/month in Standard tier.
The same 1.6TB in TurboPuffer costs ~$528/month.
Difference: $5,472/month = $65,664/year.
The Pinecone Tax
Pinecone's pricing looks reasonable at first:
Free tier: 1 index, 100K vectors Starter: $70/month, 20GB storage Standard: Starts at $70/month, scales with usage
Then you hit production and discover the hidden costs:
Index limits:
- Standard plan: Max 20 indexes
- Need more? Move to Enterprise (call for pricing, probably $5,000+/month)
Write costs:
- $4-6 per million writes
- Updating 10M vectors/day = $120-180/day = $3,600-5,400/month
Feature add-ons:
- Built-in embeddings: Extra cost
- Reranking: Extra cost
- Enterprise features (HIPAA, SLA): Call for pricing
Real story: Start at $70/month for experiments. Scale to $3,000/month in production. Wonder where the money went.
The Self-Hosting Hell
"I'll just self-host Weaviate/Milvus/Qdrant. It's open source!"
Great plan. Now you need:
Infrastructure:
- Kubernetes cluster (or equivalent)
- Load balancers
- Storage volumes
- Monitoring stack
- Backup system
Cost: $500-1,000/month for modest scale
Operational burden:
- Initial setup: 40-80 hours
- Ongoing maintenance: 20-40 hours/month
- Performance tuning: Constant
- Scaling decisions: Weekly
- Security patching: Monthly
Real cost: $3,000-6,000/month in DevOps time + infrastructure
For a 5-15 person team, you can't afford a dedicated vector DB specialist. And without one, you're constantly fighting infrastructure instead of building product.
The pgvector Trap
"Why not just add pgvector to Postgres?"
Because pgvector is excellent until it isn't.
Works great for:
- <1M vectors
- Simple queries
- Low query volume
Falls apart at scale:
- Memory contention: HNSW indexes need massive RAM, starving your transactional queries
- Performance cliff: Beyond 1M vectors, latency becomes unpredictable
- Index size explosion: 10M vectors × 1536 dimensions = 50GB+ in memory
- Backup nightmares: Your 20GB database becomes 200GB with vectors
One portfolio company tried pgvector for their RAG system. Worked fine in development (10K documents). Production (500K documents) brought their Postgres instance to its knees. Query latency jumped from 50ms to 2+ seconds. Transactional queries started timing out.
The solution: Move vectors to a dedicated database. That's what they're built for.
What You Actually Need from a Vector Database
Let's be specific:
Core Functionality:
- Vector similarity search (cosine, L2, dot product)
- Metadata filtering
- Fast queries (<100ms p99 for user-facing apps)
- Scales to 10M-100M+ vectors
Operational:
- Zero maintenance overhead
- Predictable pricing
- No surprise bills
- Production-grade reliability (99.9%+ uptime)
Nice to Have:
- Hybrid search (vector + BM25)
- Multi-tenancy support
- SOC2/HIPAA compliance without enterprise tax
Don't Need:
- Built-in embeddings (5 lines of code to call OpenAI)
- Built-in reranking (Cohere/Voyage APIs are simple)
- Complex query DSLs (just return candidates, rerank in app code)
- 47 different indexing strategies
Most vector databases over-engineer this. TurboPuffer focuses on the essentials.
How TurboPuffer Actually Works
TurboPuffer is built from first principles on object storage (S3/GCS/Azure).
The "Pufferfish" Architecture
Traditional vector databases store everything in RAM or SSDs. Fast but expensive.
TurboPuffer uses a three-tier storage hierarchy:
┌─────────────────────────────────────────────┐
│ RAM (Hot Data) │
│ • <10ms queries │
│ • $5/GB cost │
│ • Frequently accessed vectors │
└─────────────────┬───────────────────────────┘
↓
┌─────────────────────────────────────────────┐
│ NVMe SSD (Warm Data) │
│ • 10-50ms queries │
│ • $0.60/GB cost │
│ • Recently accessed vectors │
└─────────────────┬───────────────────────────┘
↓
┌─────────────────────────────────────────────┐
│ S3/Object Storage (Cold Data) │
│ • 200-500ms queries (first access) │
│ • $0.033/GB cost │
│ • All data, always │
└─────────────────────────────────────────────┘
How it works:
- All writes go to object storage as the source of truth
- Data "inflates" based on access patterns
- First query: Fetches from S3 (200-500ms)
- Subsequent queries: Cached in NVMe (10-50ms) or RAM (<10ms)
- Unused data "deflates" back to object storage
Result: You pay S3 prices ($0.033/GB) for storage but get SSD/RAM performance for frequently accessed data.
Why Object Storage Enables 10-100x Cost Reduction
Storage cost comparison:
| Storage Tier | Cost/GB/Month | Example: 1TB | | ---------------------------- | ------------- | ------------ | | RAM (Pinecone, Qdrant) | $5.00 | $5,000 | | SSD (Weaviate, Milvus) | $0.60 | $600 | | Object Storage (TurboPuffer) | $0.033 | $33 |
For 10TB of vectors:
- Traditional vector DB: $50,000-60,000/month
- TurboPuffer: $330/month (storage) + usage
Why this works:
Most vector data follows power-law access patterns:
- 10% of data is accessed 90% of the time (hot)
- 90% of data is rarely accessed (cold)
Traditional vector databases pay RAM prices for ALL data. TurboPuffer pays RAM prices only for hot data and object storage prices for cold data.
The Trade-Off: Cold Query Latency
Here's the honest truth about TurboPuffer's performance:
Warm queries (cached): p50 = 8ms Cold queries (first access): p90 = 444ms
That 444ms cold query is the trade-off for 10-100x cost savings.
When this is fine:
- Multi-tenant apps where each tenant's data is accessed infrequently
- RAG systems where most documents are rarely retrieved
- Applications that can pre-warm caches (load data when user opens a dialog)
When this is NOT fine:
- Real-time recommendation engines requiring <50ms p99 for every query
- High-frequency trading systems
- Applications where all data is accessed uniformly
For most RAG use cases, you can pre-warm the cache. Notion does this: when a user opens Q&A, they send a hint query to warm the cache during the dialog animation. By the time the user types a question, data is already warm.
Result: Sub-10ms queries for user-facing interactions.
The Numbers That Matter
Let's compare real costs at different scales.
Small Scale: 1M Vectors, 10K Queries/Day
TurboPuffer:
- Storage: 1M × 1536 dims × 4 bytes / 1GB × $0.33 = $2
- Queries: 10K/day × 30 days × $5/petabyte ≈ negligible
- Writes: Minimal
- Total: $64/month (minimum plan)
Pinecone Standard:
- Base: $70/month
- Storage included for small scale
- Total: $70/month
pgvector (Postgres):
- Postgres instance: $50-100/month
- Works fine at this scale
- Total: $50-100/month
Winner at small scale: pgvector (if you already run Postgres)
Medium Scale: 10M Vectors, 100K Queries/Day
TurboPuffer:
- Storage: 10M vectors × 6KB = 60GB × $0.33 = $20
- Queries: 100K/day × 30 days = minimal cost
- Writes: Modest
- Total: $85/month
Pinecone Standard:
- ~$180/month at this scale
- Total: $180/month
Qdrant Cloud:
- Instance-based pricing
- Total: $102/month (optimized)
Self-hosted Milvus:
- Infrastructure: $500/month
- DevOps: 20 hours/month × $150/hour = $3,000/month
- Total: $3,500/month
pgvector:
- Starts showing performance issues
- Requires significant RAM ($200-500/month instance)
- Total: $200-500/month + increasing maintenance
Winner: TurboPuffer ($85 vs $180-3,500)
Large Scale: 100M Vectors, 1M Queries/Day
TurboPuffer:
- Storage: 600GB × $0.33 = $198
- Queries: 1M/day × 30 days = minimal
- Writes: 100K/day × 30 days × $2/million = $6
- Total: $380/month
Pinecone:
- $1,200-1,800/month
Qdrant Cloud:
- $450-800/month
Weaviate Cloud:
- $800-1,200/month
Self-hosted:
- Infrastructure: $2,000-5,000/month
- DevOps: 40 hours/month × $150 = $6,000/month
- Total: $8,000-11,000/month
Winner: TurboPuffer ($380 vs $1,200-11,000)
Enterprise Scale: 10B Vectors (Notion's Scale)
TurboPuffer:
- Exact pricing not public
- Estimated: $10,000-15,000/month
- Notion reports "millions saved annually"
Pinecone Enterprise:
- Call for pricing
- Estimated: $30,000-50,000/month
Notion's result: 80% cost reduction by switching to TurboPuffer
Real Production Use Cases
Cursor: 100B+ Vectors, 10M+ Namespaces
The problem:
- Each (user, codebase) needs isolated vector storage
- Peak writes: 1M+ vectors/second during codebase indexing
- Need to reuse embeddings between similar codebases
The solution:
- One TurboPuffer namespace per (user, codebase)
copy_from_namespaceAPI reuses embeddings- Cache warming when codebase opens
Results:
- 95% cost reduction vs previous vector DB
- 20x cheaper than alternatives
- "One of the few pieces of infrastructure we haven't had to worry about as we've scaled"
Cost impact:
- Previous solution: ~$50,000/month (estimated)
- TurboPuffer: ~$2,500/month
- Annual savings: $570,000
Notion: 10B+ Vectors, AI Q&A for Workspaces
The problem:
- Every workspace needs searchable knowledge base
- Millions of workspaces = millions of namespaces
- Can't charge per-user for AI features if infrastructure is expensive
The solution:
- One namespace per workspace
- Hybrid search (vector + BM25)
- Cache pre-warming when Q&A dialog opens
Results:
- 80% cost reduction
- Removed per-user AI charges (competitive advantage)
- Millions saved annually
- 99.99%+ uptime
Business impact: Free AI features for all users because infrastructure costs dropped 80%.
Linear: Migrating from Elasticsearch + pgvector
The problem:
- Elasticsearch: Operational overhead, expensive
- pgvector: Performance degradation beyond 1M vectors
The solution:
- Migrate to TurboPuffer
- One namespace per team/org
- Hybrid search for issues, comments, documents
Results:
- Eliminated vector DB infrastructure management
- "Hands-free experience"
- 100M+ full-text documents without machine type concerns
Code: How to Actually Use It
Basic RAG Pipeline
import turbopuffer as tpuf
import openai
# Initialize
tpuf_client = tpuf.Turbopuffer(api_key="your_key")
ns = tpuf_client.namespace("customer_docs")
# Chunk and embed documents
def ingest_document(doc_id: str, text: str):
chunks = chunk_text(text, size=500)
# Batch embed
embeddings = openai.embeddings.create(
model="text-embedding-3-small",
input=chunks
)
# Upsert to TurboPuffer
ns.write(
upsert_rows=[
{
'id': f"{doc_id}_chunk_{i}",
'vector': emb.embedding,
'text': chunk,
'doc_id': doc_id,
'category': 'support',
}
for i, (chunk, emb) in enumerate(zip(chunks, embeddings.data))
],
distance_metric='cosine_distance',
schema={
'text': {'type': 'string', 'full_text_search': True},
'doc_id': {'type': 'string'},
'category': {'type': 'string'},
}
)
# Query with hybrid search
def search(question: str, category: str = None):
# Embed question
query_emb = openai.embeddings.create(
model="text-embedding-3-small",
input=question
).data[0].embedding
# Build filters
filters = None
if category:
filters = ('category', 'Eq', category)
# Search
results = ns.query(
rank_by=('vector', 'ANN', query_emb),
top_k=10,
filters=filters,
include_attributes=['text', 'doc_id'],
)
return results
# Use it
ingest_document("policy_123", "Return policy: 30 days...")
results = search("How do I return a product?", category="support")
8 lines to ingest. 8 lines to query. No complex configuration. No index tuning.
Multi-Tenant Architecture
class MultiTenantRAG:
def __init__(self):
self.tpuf = tpuf.Turbopuffer()
def get_namespace(self, tenant_id: str):
return self.tpuf.namespace(f"tenant_{tenant_id}")
def prewarm_cache(self, tenant_id: str):
"""Warm cache when tenant opens app."""
ns = self.get_namespace(tenant_id)
ns.query(
rank_by=('vector', 'ANN', [0.0] * 1536),
top_k=1,
) # Free if already warm
def query(self, tenant_id: str, question: str):
ns = self.get_namespace(tenant_id)
# ... query logic
Unlimited namespaces. One per customer. Perfect isolation. Predictable costs.
Hybrid Search with Reranking
# Stage 1: TurboPuffer retrieval
vector_results = ns.query(
rank_by=('vector', 'ANN', query_vector),
top_k=100,
)
bm25_results = ns.query(
rank_by=('text', 'BM25', question),
top_k=100,
)
# Stage 2: Reciprocal rank fusion (in your code)
fused = reciprocal_rank_fusion([vector_results, bm25_results])
# Stage 3: Rerank top candidates
reranked = cohere_client.rerank(
model='rerank-english-v3.0',
query=question,
documents=[doc['text'] for doc in fused[:50]],
top_n=10,
)
TurboPuffer philosophy: Do simple retrieval fast and cheap. Complex ranking in app code where you control it.
When You Should NOT Use TurboPuffer
Let's be honest about limitations.
1. You Need Consistent <10ms P99 for ALL Queries
If your application requires:
- Every query <10ms, no exceptions
- Uniform access patterns (all data equally hot)
- Real-time recommendation engines
- High-frequency trading systems
Use Qdrant or in-memory solution. Pay RAM prices for ALL data. Accept the cost.
TurboPuffer's cold queries (200-500ms) will break your SLA.
2. You're Pre-Revenue with <100K Vectors
If you're:
- Still in prototype phase
- Have <100K vectors
- Zero budget
Start with pgvector or Pinecone free tier. They work fine at small scale.
Migrate to TurboPuffer when you hit 1M+ vectors or $200/month on alternatives.
3. You Need Open Source
TurboPuffer is proprietary. If you:
- Require open-source for policy compliance
- Want to inspect source code
- Need to self-host everything
Use Qdrant, Weaviate, or Milvus. Accept the operational overhead.
BYOC (Bring Your Own Cloud) available for TurboPuffer Enterprise, but that's not truly open source.
4. You Need Built-In Embeddings/Reranking
If you want:
- Vector DB to generate embeddings for you
- Built-in reranking without external APIs
- All-in-one solution
Pinecone offers these as add-ons (at extra cost).
With TurboPuffer, you call OpenAI/Cohere APIs yourself. It's 5 lines of code, but it's not "built-in."
5. You Love Operational Complexity
Some teams want:
- Control over every index parameter
- Custom Kubernetes configurations
- Deep database tuning
Self-host Milvus. You get complete control. And complete operational burden.
TurboPuffer abstracts this away. If you love tweaking databases, you'll be bored.
How It Fits Our Unified Stack
TurboPuffer is the final piece of Grey Haven's unified tech stack:
Frontend & API: TanStack Start Infrastructure: Control Plane Database: PlanetScale LLM Framework: Mirascope Vector Search: TurboPuffer
The Philosophy:
Pick tools once. Get world-class at them. Choose cost efficiency and operational simplicity over feature bloat.
Why TurboPuffer Fits:
- 10-100x cheaper than alternatives at portfolio scale
- Zero DevOps overhead across all companies
- Unlimited namespaces for multi-tenant architecture
- Predictable pricing (storage + writes + queries)
- Production compliance (SOC2, HIPAA) without enterprise tax
Real Impact for Portfolio Companies:
Portfolio of 10 companies, average 10M vectors each:
With Pinecone:
- 10 companies × $500/month = $5,000/month
- Annual: $60,000
With TurboPuffer:
- 10 companies × $64 minimum = $640/month
- Usage: ~$1,200/month
- Annual: $22,080
- Savings: $37,920/year
With self-hosted Milvus:
- Infrastructure: $3,000/month
- DevOps (1 FTE): $12,000/month
- Annual: $180,000
- TurboPuffer saves: $157,920/year
One person manages vector search for the entire portfolio. No dedicated specialist needed. No operational burden.
That's the compounding value of consistency.
Stop Paying the Enterprise Tax
Here's what matters when building RAG systems at scale:
Traditional vector databases are expensive:
- Pinecone: $70/month → $3,000/month as you scale
- Self-hosted: $3,000-11,000/month in infrastructure + DevOps
- Hidden costs everywhere (write fees, index limits, enterprise features)
TurboPuffer is 10-100x cheaper:
- $64/month minimum
- $380/month for 100M vectors (vs $1,200-11,000)
- No hidden costs
- No DevOps burden
The trade-off is real:
- Cold queries: 200-500ms (vs <10ms in-memory)
- But warm queries: <10ms (same as alternatives)
- Pre-warm caches for user-facing apps (Notion pattern)
- Perfect for power-law access patterns (90% cold, 10% hot)
Real production results:
- Cursor: 95% cost reduction, 100B+ vectors
- Notion: 80% cost reduction, millions saved annually
- Linear: Eliminated infrastructure management
When TurboPuffer wins:
- Multi-tenant applications (unlimited namespaces)
- RAG systems 10M-100B vectors
- Small teams (5-15 engineers)
- Cost-sensitive applications
- Power-law access patterns
When to choose alternatives:
- Need <10ms p99 for ALL queries (Qdrant)
- Uniform access patterns (in-memory DB)
- Early prototypes <100K vectors (pgvector, Pinecone free tier)
- Open source requirement (Qdrant, Weaviate, Milvus)
For small teams building production RAG systems, TurboPuffer is the obvious choice. It delivers enterprise-grade performance at startup prices with zero operational overhead.
We're not chasing features. We're choosing cost efficiency and simplicity that lets us scale without drowning in infrastructure costs.
That's the unified stack philosophy in action.
At Grey Haven, we build RAG systems across multiple portfolio companies. We've evaluated every major vector database for production use. TurboPuffer's cost efficiency (10-100x cheaper), operational simplicity (zero DevOps), and unlimited multi-tenancy make it the clear choice for small teams building at scale.
Want to learn more about our unified tech stack? Read Part 1: Why We Ditched Next.js for TanStack Start, Part 2: Why We Chose Control Plane for Multicloud Infrastructure, Part 3: Why We Chose Mirascope Over LangChain, and Part 5: PlanetScale PostgreSQL vs Supabase.
Building RAG systems and bleeding money on vector databases? Let's talk.
