Connect LlamaIndex to TrustedRails

LlamaIndex is a framework for building RAG pipelines and agents over your own data. Its OpenAI-compatible LLM class accepts a custom base URL, so you can point it at TrustedRails and run retrieval and agents on open-source models — the only change from a standard OpenAI setup is the base URL, the key, and the model id.

The integration is the same in Python and JavaScript/TypeScript; only the package name and the constructor differ. Pick your language below.

Prerequisites

A TrustedRails API key (starts with tr-prx-). See Create a TrustedRails API Key.
Python 3.9+ or Node.js 18+, depending on your stack.

Install the OpenAI-compatible LLM integration:

pip install llama-index-llms-openai-like

Point OpenAILike at TrustedRails:

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="moonshotai/Kimi-K2.6",
    api_base="https://proxy.trustedrails.com/v1",
    api_key="tr-prx-your-api-key",
    is_chat_model=True,
    context_window=128000,
)

response = llm.complete("Say hello from TrustedRails.")
print(response)

Use OpenAILike (not the plain OpenAI class) — the standard class only accepts OpenAI's own model names. Keep is_chat_model=True so calls go to the chat endpoint, and set context_window to your model's window.

Install the OpenAI integration package:

npm install @llamaindex/openai

Point OpenAI at TrustedRails:

import { OpenAI } from "@llamaindex/openai";

const llm = new OpenAI({
  model: "moonshotai/Kimi-K2.6",
  apiKey: "tr-prx-your-api-key",
  baseURL: "https://proxy.trustedrails.com/v1",
});

const response = await llm.complete({ prompt: "Say hello from TrustedRails." });
console.log(response.text);

The model must match a TrustedRails-supported id exactly — for example moonshotai/Kimi-K2.6 or MiniMaxAI/MiniMax-M2.7 (see Supported Models). Once the LLM is configured, pass it to a Settings.llm / index / query engine and the rest of LlamaIndex works unchanged.

Verify

Run the snippet above. A printed reply confirms LlamaIndex is reaching TrustedRails through your key.

Troubleshooting

401 / invalid API key — wrong or paused key. Create a fresh one from Create a TrustedRails API Key.
Model not found / unsupported — the model must match a model TrustedRails serves exactly (see Supported Models).
Connection errors — confirm the base URL is exactly https://proxy.trustedrails.com/v1.
Output is truncated — raise context_window (Python) to match your model; the default is small.
Visible <think> reasoning in the output — reasoning models such as MiniMaxAI/MiniMax-M2.7 emit their thinking inline, which LlamaIndex returns verbatim in the response text. The example uses moonshotai/Kimi-K2.6, which returns clean text; strip the <think> block yourself if you prefer a reasoning model.