Embedding Providers

react-ai-chat supports multiple embedding providers for RAG.

The server package exports:

import {
  googleEmbedding,
  openAIEmbedding,
  voyageEmbedding,
  cohereEmbedding,
  jinaEmbedding,
  huggingFaceEmbedding,
} from "react-ai-chat/server";

The provider you use to query the index must be compatible with the embedding configuration used when the index was generated.

Google

Install the Google GenAI SDK:

npm install @google/genai

Create a Google client:

import { GoogleGenAI } from "@google/genai";
import { googleEmbedding } from "react-ai-chat/server";

const client = new GoogleGenAI({
  apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY,
});

const provider = googleEmbedding(client, {
  model: "gemini-embedding-001",
});

When using a generated embedding index, use the model stored in the index:

const provider = googleEmbedding(client, {
  model: embeddings.model,
});

Batch size

The default maxBatchSize is 250.

The RAG indexer uses this value to determine how many chunks can be sent in a single embedding request.

OpenAI

Install the OpenAI SDK:

npm install openai

Create an OpenAI client:

import OpenAI from "openai";
import { openAIEmbedding } from "react-ai-chat/server";

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

const provider = openAIEmbedding(client, {
  model: "text-embedding-3-small",
});

Use the model recorded in your generated index when configuring RAG:

const provider = openAIEmbedding(client, {
  model: embeddings.model,
});

Batch size

The default maxBatchSize is 2,048.

The RAG indexer uses this value to determine how many chunks can be sent in a single embedding request.

Voyage AI

Install the Voyage AI SDK:

npm install voyageai

Create the client and embedding provider:

import VoyageAI from "voyageai";
import { voyageEmbedding } from "react-ai-chat/server";

const client = new VoyageAI({
  apiKey: process.env.VOYAGE_API_KEY,
});

const provider = voyageEmbedding(client, {
  model: "voyage-3-large",
});

For an existing generated index:

const provider = voyageEmbedding(client, {
  model: embeddings.model,
});

Batch size

The default maxBatchSize is 1,000.

The RAG indexer uses this value to determine how many chunks can be sent in a single embedding request.

Cohere

Install the Cohere SDK:

npm install cohere-ai

Create the client:

import { CohereClientV2 } from "cohere-ai";
import { cohereEmbedding } from "react-ai-chat/server";

const client = new CohereClientV2({
  token: process.env.COHERE_API_KEY,
});

const provider = cohereEmbedding(client, {
  model: "embed-v4.0",
});

For an existing generated index:

const provider = cohereEmbedding(client, {
  model: embeddings.model,
});

Batch size

The default maxBatchSize is 96.

The RAG indexer uses this value to determine how many chunks can be sent in a single embedding request.

Jina AI

Jina uses its own API client helper.

Create the client with createJinaClient:

import { createJinaClient, jinaEmbedding } from "react-ai-chat/server";

const client = createJinaClient({
  apiKey: process.env.JINA_API_KEY,
});

const provider = jinaEmbedding(client, {
  model: "jina-embeddings-v3",
});

For an existing index:

const provider = jinaEmbedding(client, {
  model: embeddings.model,
});

Batch size

The default maxBatchSize is 2,048.

The RAG indexer uses this value to determine how many chunks can be sent in a single embedding request.

Hugging Face

Install the Hugging Face inference SDK:

npm install @huggingface/inference

Create the client:

import { HfInference } from "@huggingface/inference";
import { huggingFaceEmbedding } from "react-ai-chat/server";

const client = new HfInference(process.env.HUGGINGFACE_API_KEY);

const provider = huggingFaceEmbedding(client, {
  model: "sentence-transformers/all-MiniLM-L6-v2",
});

For an existing generated index:

const provider = huggingFaceEmbedding(client, {
  model: embeddings.model,
});

Batch size

The default maxBatchSize is 32.

The RAG indexer uses this value to determine how many chunks can be sent in a single embedding request.

Batch embedding

All supported embedding providers expose a maximum batch size used during RAG index generation.

Instead of sending every document chunk as an individual request, the indexer groups chunks into batches.

For example:

70 document chunks
        ↓
Provider maxBatchSize: 32
        ↓
Batch 1: 32 chunks
Batch 2: 32 chunks
Batch 3: 6 chunks

The batches are processed sequentially.

This reduces the number of API requests and helps reduce rate-limit pressure during large indexing jobs.

`embedMany()`

Embedding providers support batched embedding through embedMany().

The indexer uses this operation when generating embeddings for multiple document chunks.

Conceptually:

const embeddings = await provider.embedMany(chunks);

The provider handles the request according to its API requirements.

The indexer then stores the resulting vectors alongside their document chunks.

Batch processing is performed sequentially so the indexer can control request volume and handle provider errors at the batch level.

Provider and index compatibility

An embedding index is tied to the embedding model that generated its vectors.

For example, if your index was generated with:

google / gemini-embedding-001

your RAG route should use the same embedding configuration:

const provider = googleEmbedding(client, {
  model: embeddings.model,
});

Changing the model can produce vectors with different dimensions or a different embedding space.

If you change the embedding provider or model, regenerate the index.

npx react-ai-chat embed

API keys

Keep provider API keys on the server.

For example:

GOOGLE_GENERATIVE_AI_API_KEY=...
OPENAI_API_KEY=...
VOYAGE_API_KEY=...
COHERE_API_KEY=...
JINA_API_KEY=...
HUGGINGFACE_API_KEY=...

Do not expose these values in client-side code.

Using a provider in `createChatRoute`

Once your provider is configured, pass it to the rag option:

import { google } from "@ai-sdk/google";
import { GoogleGenAI } from "@google/genai";
import { createChatRoute, googleEmbedding } from "react-ai-chat/server";
import embeddings from "@/chatbot/embeddings.json";

const client = new GoogleGenAI({
  apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY,
});

const provider = googleEmbedding(client, {
  model: embeddings.model,
});

export const POST = createChatRoute({
  model: google("gemini-3.5-flash"),
  rag: {
    index: embeddings,
    provider,
    topK: 3,
  },
});

The embedding provider handles vector generation for retrieval.

The model passed to createChatRoute handles the final response.

These can use different model providers.

Rate limits and fallback models

The embedding CLI supports a fallback model when generating an index.

Embedding requests are processed in batches. If the primary provider encounters an error while processing a batch, the configured fallback provider can be used for that batch.

The fallback configuration must be compatible with the selected embedding dimensions.

For example:

Primary provider
      ↓
Embedding batch
      ↓
Provider error
      ↓
Fallback provider
      ↓
Continue indexing

The generated index records the embedding configuration used during generation.

If embeddings were generated using a fallback model, configure your retrieval provider with the model recorded in the generated index:

const provider = googleEmbedding(client, {
  model: embeddings.model,
});

Embedding Providers

RAG

CLI

createChatRoute

On this page