Why We Stopped Using Vector Databases. The $40k/year Pinecone Bill That Postgres Could Handle.

We were paying Pinecone $3,200 per month. $38,400 per year. For what?

Semantic search across 500,000 documents. The demo was beautiful — "Find documents similar to this query!" Our AI product felt cutting-edge.

Then I asked our engineer a simple question: "How often do users actually need semantic search?"

He pulled the data. Answer: 12% of queries.

The other 88% were exact matches or keyword searches. Traditional database queries. The vector database wasn't even involved.

We were running a $40k/year dedicated vector database for 12% of queries.

We migrated to pgvector (a free extension for our existing Postgres database). Latency went up 40ms. Nobody noticed. Nobody complained. The product worked identically.

Here's when you actually need a dedicated vector database — and when you're just burning money on hype.

Section 1: The Vector Database Hype Cycle

Vector databases had their moment. And it was loud.

The 2023-2024 Gold Rush:

When ChatGPT exploded, everyone wanted to build AI applications. And "every AI app needs a vector database" became gospel.

The logic sounded reasonable: LLMs work with embeddings. Embeddings are vectors. You need to search vectors similarity. Therefore: vector database.

Pinecone, Weaviate, Qdrant, Milvus, Chroma — venture capital poured in. Pinecone raised at a $750M valuation. Vector databases were the picks and shovels of the AI gold rush.

The Consulting Industrial Complex:

Every AI tutorial, every consulting engagement, every "how to build RAG" blog post defaulted to: "First, set up your vector database."

It became the standard architecture. Questioning it marked you as unsophisticated. "You're not using a vector database? How do you do semantic search?"

The answer — that most apps don't need sophisticated semantic search — was heresy.

The Reality:

Most AI applications don't have the scale or use case to justify dedicated vector infrastructure.

• They have thousands or tens of thousands of documents, not billions.
• They need good-enough search, not sub-10ms latency.
• They're built by small teams who can't afford another infrastructure dependency.

The vector database hype was a hammer looking for nails. Vendors sold solutions to problems most teams didn't have.

Section 2: When Vector Databases Actually Make Sense

I'm not saying vector databases are useless. There are legitimate use cases. But they're narrower than the marketing suggests.

Billions of Vectors with Sub-10ms Latency:

If you're Spotify (recommending from 100 million songs) or Pinterest (searching billions of images), you need specialized vector infrastructure.

At that scale, every millisecond matters. The optimization work done by Pinecone, Weaviate, et al. is genuinely valuable.

Most companies are not Spotify. Most have 100k-1M documents. That's trivially handleable by simpler solutions.

Complex Vector Operations:

Some applications need sophisticated vector queries:

• Hybrid search (combining vector similarity with keyword filters)
• Complex metadata filtering at scale
• Real-time index updates with consistency guarantees

If your queries are "find the 10 most similar documents," you don't need this complexity.

Dedicated ML Infrastructure Team:

Running a vector database well requires expertise. Index tuning. Query optimization. Monitoring. Cost management.

If you have an ML platform team, they can handle this. If you're a 10-person startup, adding a vector database means adding operational burden to an already-stretched team.

The 1% Rule:

Fewer than 1% of AI applications actually need a dedicated vector database. The other 99% are paying for infrastructure they don't need because they followed tutorials written by vector database vendors.

Section 3: The Postgres/pgvector Alternative

For most use cases, there's a simpler answer: pgvector.

What is pgvector?

It's a Postgres extension that adds vector similarity search to your existing database.

• Store embeddings as a vector column type
• Create indexes for fast similarity search (IVFFlat, HNSW)
• Query with standard SQL: ORDER BY embedding <-> query_embedding LIMIT 10

That's it. No new database. No new vendor. No new operational burden.

Performance:

pgvector handles millions of vectors with good performance:

• 100k vectors: Single-digit millisecond queries
• 1M vectors: 10-50ms queries (depending on index configuration)
• 10M+ vectors: This is where dedicated vector databases start making sense

For 90% of applications, that's more than fast enough. Your users aren't timing your semantic search with a stopwatch.

Operational Simplicity:

You're already running Postgres. (Everyone is running Postgres.) pgvector requires:

• One extension installation: CREATE EXTENSION vector;
• Adding a vector column to your table
• Creating an index

Compare to Pinecone: New vendor relationship. New billing. New API. New monitoring. New failure modes. New on-call rotation.

The simplest infrastructure is the infrastructure you already have.

"Good Enough" Is Actually Good Enough:

There's a mindset in engineering that more specialized = better. "Pinecone is optimized for vectors, so it must be better than a general-purpose database."

True. Pinecone is faster. On benchmarks.

But does your application need that speed? If pgvector returns results in 50ms instead of 15ms, will users notice? Will it affect business metrics?

Usually, no. The 35ms difference is invisible. The $40k/year difference is not.

Section 4: Our Migration Story

Here's exactly how we migrated from Pinecone to pgvector.

The Audit:

First, we analyzed our query patterns:

• 88% of searches were keyword-based or exact match. Vector search not involved.
• 12% used semantic similarity. Of these, 95% worked fine with top-10 results (no complex ranking needed).
• Latency requirements: 100ms was acceptable. Users searched, waited, got results. Nobody complained.

Conclusion: We were massively overengineered.

The Migration:

Weekend project:

1. Added pgvector extension to our Postgres RDS instance.
2. Created a vector column on our documents table.
3. Batch-copied 500k embeddings from Pinecone to Postgres.
4. Created an HNSW index for fast approximate search.
5. Updated our API to query Postgres instead of Pinecone.

Total engineering time: ~16 hours.

Results:

• Latency: 15ms (Pinecone) → 55ms (pgvector). 40ms increase. Users noticed? No.
• Cost: $3,200/month → $0 incremental. We were already paying for the Postgres instance.
• Ops burden: Eliminated. No more monitoring a separate service. No more Pinecone-specific on-call.
• Development velocity: Improved. One fewer system to understand and maintain.

Annualized savings: $38,400. Plus the intangible benefit of simpler architecture.

Conclusion

Vector databases are a solution looking for a problem. For most AI applications, the problem is smaller than it appears.

Before you sign up for Pinecone (or Weaviate, or any dedicated vector database):

• How many vectors do you actually have? If less than 10M, pgvector is probably fine.
• What latency do you actually need? If 50-100ms is acceptable, pgvector is definitely fine.
• Do you have the team to operate another database? If not, don't add one.

Start with Postgres. Upgrade when you hit its limits — not before.

The best infrastructure is the infrastructure you already have.