Connect LlamaIndex to TrustedRails

LlamaIndex is a framework for building RAG pipelines and agents over your own data. Its OpenAI-compatible LLM class accepts a custom base URL, so you can point it at TrustedRails and run retrieval and agents on open-source models: the only change from a standard OpenAI setup is the base URL, the key, and the model id.

The integration is the same in Python and JavaScript/TypeScript; only the package name and the constructor differ. Pick your language below.

Prerequisites

• A TrustedRails API key (starts with tr-prx-). See Create a TrustedRails API Key.
• Python 3.9+ or Node.js 18+, depending on your stack.

Install and configure

Python

Install the OpenAI-compatible LLM integration:

pip install llama-index-llms-openai-like

Point OpenAILike at TrustedRails:

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="zai-org/GLM-5.3-Flash",
    api_base="https://proxy.trustedrails.com/v1",
    api_key="tr-prx-your-api-key",
    is_chat_model=True,
    context_window=128000,
)

response = llm.complete("Say hello from TrustedRails.")
print(response)

Use OpenAILike (not the plain OpenAI class); the standard class only accepts OpenAI's own model names. Keep is_chat_model=True so calls go to the chat endpoint, and set context_window to your model's window.

JavaScript / TypeScript

Install the OpenAI integration package:

npm install @llamaindex/openai

Point OpenAI at TrustedRails:

import { OpenAI } from "@llamaindex/openai";

const llm = new OpenAI({
  model: "zai-org/GLM-5.3-Flash",
  apiKey: "tr-prx-your-api-key",
  baseURL: "https://proxy.trustedrails.com/v1",
});

const response = await llm.complete({ prompt: "Say hello from TrustedRails." });
console.log(response.text);

The model must match a TrustedRails-supported id exactly, for example zai-org/GLM-5.3-Flash or MiniMaxAI/MiniMax-M2.7 (see Supported Models). Once the LLM is configured, pass it to a Settings.llm / index / query engine and the rest of LlamaIndex works unchanged.

Embeddings for RAG

For the retrieval side of a RAG pipeline, use BAAI/bge-m3 through the same base URL and key (see Embeddings & RAG).

Python

Install the OpenAI-like embedding integration:

pip install llama-index-embeddings-openai-like

Point OpenAILikeEmbedding at TrustedRails and pass it to your index:

from llama_index.core import VectorStoreIndex, Document
from llama_index.embeddings.openai_like import OpenAILikeEmbedding

embed_model = OpenAILikeEmbedding(
    model_name="BAAI/bge-m3",
    api_base="https://proxy.trustedrails.com/v1",
    api_key="tr-prx-your-api-key",
)

index = VectorStoreIndex.from_documents(
    [Document(text="TrustedRails bills in USD per token.")],
    embed_model=embed_model,
)
print(index.as_retriever().retrieve("How am I charged?")[0].text)

As with the LLM, use OpenAILikeEmbedding (not the plain OpenAIEmbedding class, which only accepts OpenAI's own embedding-model names), and note the parameter is model_name, not model.

JavaScript / TypeScript

The OpenAIEmbedding class from the same @llamaindex/openai package accepts custom model ids directly:

import { OpenAIEmbedding } from "@llamaindex/openai";

const embedModel = new OpenAIEmbedding({
  model: "BAAI/bge-m3",
  apiKey: "tr-prx-your-api-key",
  baseURL: "https://proxy.trustedrails.com/v1",
});

const vector = await embedModel.getTextEmbedding("How am I charged?");
console.log(vector.length); // 1024

Pass it as embedModel wherever LlamaIndex expects an embedding model: indexes, retrievers, and query engines work unchanged.

Verify

Run the snippet above. A printed reply confirms LlamaIndex is reaching TrustedRails through your key.

Troubleshooting

• 401 / invalid API key: wrong or paused key. Create a fresh one from Create a TrustedRails API Key.
• Model not found / unsupported: the model must match a model TrustedRails serves exactly (see Supported Models).
• Connection errors: confirm the base URL is exactly https://proxy.trustedrails.com/v1.
• Output is truncated: raise context_window (Python) to match your model; the default is small.
• "'BAAI/bge-m3' is not a valid OpenAIEmbeddingModelType": the plain OpenAIEmbedding class validates model names against OpenAI's list. Use OpenAILikeEmbedding from llama-index-embeddings-openai-like (see the embeddings section above).
• Looking for the model's thinking: reasoning models return their thinking in a separate reasoning response field, never inside the response text — LlamaIndex output is clean answer text. To skip thinking entirely (shorter, cheaper responses), see reasoning control.