Vector Databases
A Vector Database (VDB) is a specialized database designed to store, index, and query high-dimensional vectors efficiently. These vectors typically represent data points in a multi-dimensional space, such as embeddings generated by machine learning models.
Remember the embeddings? Where we discussed how LLMs convert text into numerical vectors? These vectors capture the semantic meaning of the text, allowing for more effective comparisons and searches. Vector databases are optimized to handle these high-dimensional vectors, enabling fast similarity searches and nearest neighbor queries.
Common vector databases include:
LLMs can be run locally for instance in the case of LLAMA from meta, or via API calls to services like OpenAI, Anthropic, Cohere, or Hugging Face.
The way you interact with an LLM via API is passing text (prompts) via the context window. The context window is the amount of text the LLM can consider at one time. You don’t have to pass the entire document to the LLM, just the relevant parts. This is where vector databases come in. By storing document embeddings in a vector database, you can quickly retrieve the most relevant sections of a document. That happens before passing the text to the LLM for processing. This approach is often referred to as “retrieval-augmented generation” (RAG).