In-Process Embedding Models

When ingesting document or implementing the RAG pattern, you need an embedding model. This is a model that takes a document and returns a vector representation of that document. The vector representation is stored in a vector database, and is used to find similar documents.

When using LLMs like OpenAI or HuggingFace, it provides remote embedding models. To compute the embedding of a document, you need to send the document to the remote model.

In-process models avoids this overhead by running the model in the same process as the application. This is generally faster, but requires more memory.

Supported in-process models

The Quarkus LangChain4j extension provides supports for a set of in-process embedding models. They are not included by default, and you need to add the explicit dependency to your project.

The following table lists the supported models, and the corresponding dependency to add to your project.

Model Name Dependency Vector Dimension Injected type

all-minlm-l6-v2-q

dev.langchain4j:langchain4j-embeddings-all-minilm-l6-v2-q:0.27.1

384

dev.langchain4j.model.embedding.AllMiniLmL6V2QuantizedEmbeddingModel

all-minlm-l6-v2

dev.langchain4j:langchain4j-embeddings-all-minilm-l6-v2:0.27.1

384

dev.langchain4j.model.embedding.AllMiniLmL6V2EmbeddingModel

bge-small-en-q

dev.langchain4j:langchain4j-embeddings-bge-small-en-q:0.27.1

384

dev.langchain4j.model.embedding.BgeSmallEnQuantizedEmbeddingModel

bge-small-en

dev.langchain4j:langchain4j-embeddings-bge-small-en:0.27.1

384

dev.langchain4j.model.embedding.BgeSmallEnEmbeddingModel

bge-small-zh-q

dev.langchain4j:langchain4j-embeddings-bge-small-zh-q:0.27.1

384

dev.langchain4j.model.embedding.BgeSmallZhQuantizedEmbeddingModel

bge-small-zh

dev.langchain4j:langchain4j-embeddings-bge-small-zh:0.27.1

384

dev.langchain4j.model.embedding.BgeSmallZhEmbeddingModel

e5-small-v2-q

dev.langchain4j:langchain4j-embeddings-e5-small-v2-q:0.27.1

384

dev.langchain4j.model.embedding.E5SmallV2QuantizedEmbeddingModel

e5-small-v2-q

dev.langchain4j:langchain4j-embeddings-e5-small-v2-q:0.27.1

384

dev.langchain4j.model.embedding.E5SmallV2EmbeddingModel

Injecting an embedding model

You can inject the model in your application using:

@Inject E5SmallV2QuantizedEmbeddingModel model;

Use the corresponding model type for the model you want to use, and make sure you have added the corresponding dependency to your project.

Note that if you do not have any other embedding model in your project, you can inject the EmbeddingModel interface, and it will be automatically injected with the available model:

@Inject EmbeddingModel model;