Pgvector Document Store for Retrieval Augmented Generation (RAG)

When implementing Retrieval Augmented Generation (RAG), a capable document store is necessary. This guide will explain how to leverage a pgvector database as the document store.

Leveraging the pgvector Document Store

To utilize the PostgreSQL pgVector document store, you’ll need to include the following dependency:

<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-pgvector</artifactId>
    <version>0.21.0</version>
</dependency>

This extension will check for a default datasource, ensure you have defined at least one datasource. For detailed guidance, refer to the CONFIGURE DATA SOURCES IN QUARKUS.

The pgvector store requires the dimension of the vector to be set. Add the quarkus.langchain4j.pgvector.dimension property to your application.properties file and set it to the dimension of the vector. The dimension depends on the embedding model you use. For example, AllMiniLmL6V2QuantizedEmbeddingModel produces vectors of dimension 384. OpenAI’s text-embedding-ada-002 produces vectors of dimension 1536.

Upon installing the extension, you can utilize the pgvector store using the following code:

package io.quarkiverse.langchain4j.samples;

import static dev.langchain4j.data.document.splitter.DocumentSplitters.recursive;

import java.util.List;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;

import dev.langchain4j.data.document.Document;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.store.embedding.EmbeddingStoreIngestor;
import io.quarkiverse.langchain4j.pgvector.PgVectorEmbeddingStore;

@ApplicationScoped
public class IngestorExampleWithPgvector {

    /**
     * The embedding store (the database).
     * The bean is provided by the quarkus-langchain4j-pgvector extension.
     */
    @Inject
    PgVectorEmbeddingStore store;

    /**
     * The embedding model (how is computed the vector of a document).
     * The bean is provided by the LLM (like openai) extension.
     */
    @Inject
    EmbeddingModel embeddingModel;

    public void ingest(List<Document> documents) {
        EmbeddingStoreIngestor ingestor = EmbeddingStoreIngestor.builder()
                .embeddingStore(store)
                .embeddingModel(embeddingModel)
                .documentSplitter(recursive(500, 0))
                .build();
        // Warning - this can take a long time...
        ingestor.ingest(documents);
    }
}

Configuration Settings

Customize the behavior of the extension by exploring various configuration options:

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property

Type

Default

quarkus.langchain4j.pgvector.datasource

The name of the configured Postgres datasource to use for this store. If not set, the default datasource from the Agroal extension will be used.

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_DATASOURCE

string

quarkus.langchain4j.pgvector.table

The table name for storing embeddings

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_TABLE

string

embeddings

quarkus.langchain4j.pgvector.dimension

The dimension of the embedding vectors. This has to be the same as the dimension of vectors produced by the embedding model that you use. For example, AllMiniLmL6V2QuantizedEmbeddingModel produces vectors of dimension 384. OpenAI’s text-embedding-ada-002 produces vectors of dimension 1536.

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_DIMENSION

int

required

quarkus.langchain4j.pgvector.use-index

Use index or not

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_USE_INDEX

boolean

false

quarkus.langchain4j.pgvector.index-list-size

index size

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_INDEX_LIST_SIZE

int

0

quarkus.langchain4j.pgvector.create-table

Whether the table should be created if not already existing.

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_CREATE_TABLE

boolean

true

quarkus.langchain4j.pgvector.drop-table-first

Whether the table should be dropped prior to being created.

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_DROP_TABLE_FIRST

boolean

false

quarkus.langchain4j.pgvector.metadata.storage-mode

Metadata type:

  • COLUMN_PER_KEY: for static metadata, when you know in advance the list of metadata fields. In this case, you should also override the quarkus.langchain4j.pgvector.metadata.column-definitions property to define the right columns.

  • COMBINED_JSON: For dynamic metadata, when you don’t know the list of metadata fields that will be used.

  • COMBINED_JSONB: Same as JSON, but stored in a binary way. Optimized for query on large dataset. In this case, you should also override the quarkus.langchain4j.pgvector.metadata.column-definitions property to change the type of the metadata column to COMBINED_JSONB.

Default value: COMBINED_JSON

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_METADATA_STORAGE_MODE

column-per-key, combined-json, combined-jsonb

combined-json

quarkus.langchain4j.pgvector.metadata.column-definitions

Metadata Definition: SQL definition of metadata field(s). By default, "metadata JSON NULL" configured. This is only suitable if using the JSON metadata type.

If using JSONB metadata type, this should in most cases be set to metadata JSONB NULL.

If using COLUMNS metadata type, this should be a list of columns, one column for each desired metadata field. Example: condominium_id uuid null, user uuid null

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_METADATA_COLUMN_DEFINITIONS

list of string

metadata JSON NULL

quarkus.langchain4j.pgvector.metadata.indexes

Metadata Indexes, list of fields to use as index.

For instance:

  • JSON: with JSON metadata, indexes are not allowed, so this property must be empty. To use indexes, switch to JSONB metadata.

  • JSONB: (metadata→'key'), (metadata→'name'), (metadata→'age')

  • COLUMNS: key, name, age

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_METADATA_INDEXES

list of string

quarkus.langchain4j.pgvector.metadata.index-type

Index Type:

Environment variable: QUARKUS_LANGCHAIN4J_PGVECTOR_METADATA_INDEX_TYPE

string

BTREE

Under the Hood

Each ingested document is saved as a row in a Postgres table, containing the embedding column stored as a vector.