WeeklyTalk #159

Why AI databases are becoming so huge – a look at vector stores

Name: Why AI databases are becoming so huge – a look at vector stores
Uploaded: 2026-02-26T16:00:00+01:00
Duration: 21 min 38 s
Description: In this episode, we take a look at what's inside a vector store and how it works.

In this episode, we take a look at what's inside a vector store and how it works.

What actually happens behind the scenes when an AI searches thousands of documents for the right answer? Whether it's ChatGPT, RAG applications, or semantic search, so-called vector stores are working in the background. You rarely see them, but nothing works without them: they play a decisive role in determining the quality and performance of AI systems. The problem is that these databases can become really large. And really expensive.

In this episode, we take a look at what's inside a vector store and how it works.

00:00:00 - What happens behind the scenes of an AI?
00:01:04 - The Vector Store as the memory of the chat application
00:01:38 - Why embeddings are needed for semantic search
00:03:14 - Functionality: Mathematics and statistics instead of actual understanding
00:04:16 - Data processing: Breaking documents into chunks
00:05:41 - Trial and error in choosing the right embedding model
00:06:53 - Importance of index structure for performance
00:08:17 - A look into practice: Elastic Search and data visualization
00:09:54 - How a user query becomes the right answer
00:12:36 - Storage requirements: When databases grow to terabyte size
00:14:14 - Operating costs: Up to €200 per day for large systems
00:15:24 - Evolution of systems: From hype to normal progression
00:17:54 - Self-hosting vs. Cloud: Hardware requirements and data security
00:20:45 - Outlook: The massive energy demand of future data centers

February 26, 2026