Connect LibreChat to TrustedRails

LibreChat is an open-source, self-hosted chat UI: a ChatGPT-style interface that talks to many AI providers. It reaches TrustedRails through its custom endpoint mechanism, so you add TrustedRails once in librechat.yaml and its models show up in the model picker for everyone on your instance.

Prerequisites

• A TrustedRails API key (starts with tr-prx-). See Create a TrustedRails API Key.
• A running, self-hosted LibreChat instance. See the LibreChat docs for installation.
• Your instance must actually load librechat.yaml. The default Docker setup does not mount it; enable it by configuring docker-compose.override.yml exactly as described in LibreChat's custom endpoints guide. Without this, the endpoint you add below will not appear.

Add TrustedRails as a custom endpoint

Open your librechat.yaml and add an entry under endpoints.custom:

librechat.yaml

version: 1.3.13
endpoints:
  custom:
    - name: "TrustedRails"
      apiKey: "${TrustedRails_API_KEY}"
      baseURL: "https://proxy.trustedrails.com/v1"
      models:
        default: ["MiniMaxAI/MiniMax-M2.7"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      modelDisplayLabel: "TrustedRails"
      iconURL: "https://trustedrails.com/img/trustedrails-icon.svg"

With fetch: true, LibreChat loads the live model list from TrustedRails's public /v1/models endpoint, so new models appear automatically; the default array is only the initial selection.

Provide your API key

The config above reads the key from an environment variable. Add it to your .env file:

.env

TrustedRails_API_KEY=tr-prx-your-key-here

To let each user enter their own key from the LibreChat UI instead of using a shared one, set apiKey: "user_provided"; users then see a key field when they select the endpoint.

Restart and select TrustedRails

Restart LibreChat so it reloads librechat.yaml. For Docker, recreate the containers with docker compose down && docker compose up -d; a plain docker compose restart does not apply a newly added librechat.yaml mount. Then open the app: TrustedRails appears in the endpoint selector. Pick it, choose a model, and start chatting.

RAG file chat with TrustedRails embeddings

LibreChat's "chat with your files" feature is handled by its companion RAG API service (rag_api, included in the standard Docker compose). By default it needs an OpenAI key for embeddings; point it at TrustedRails instead (see Embeddings & RAG). Add to your .env:

# .env (read by the rag_api service)
EMBEDDINGS_PROVIDER=openai
EMBEDDINGS_MODEL=BAAI/bge-m3
RAG_OPENAI_BASEURL=https://proxy.trustedrails.com/v1
RAG_OPENAI_API_KEY=tr-prx-your-key-here
RAG_CHECK_EMBEDDING_CTX_LENGTH=false

RAG_CHECK_EMBEDDING_CTX_LENGTH=false is required. With the default (true), the RAG API pre-tokenizes your documents with OpenAI's tokenizer and sends token arrays instead of text, and every file upload then fails with an "`input` must be a string or a non-empty array of strings" error. Also leave EMBEDDINGS_DIMENSIONS unset; BGE-M3 vectors are always 1024-dimensional.

Recreate the containers (docker compose down && docker compose up -d) and file uploads in chat are embedded and retrieved through TrustedRails.

Verify

Send any message with TrustedRails selected. A streamed reply confirms the connection. If you run LibreChat with logs visible, the request shows the model id that served it (for example MiniMaxAI/MiniMax-M2.7).

To check the RAG path, attach a small text file to a chat and ask about its contents; an answer grounded in the file confirms embeddings run through TrustedRails.

Troubleshooting

• TrustedRails not in the selector: librechat.yaml wasn't reloaded (restart LibreChat) or the endpoints.custom indentation is off.
• No models listed: the key env var is unset or misnamed, so fetch: true can't read the model list. Confirm TrustedRails_API_KEY is set and the value starts with tr-prx-.
• 401 / invalid API key: wrong or paused key. Create a fresh one from Create a TrustedRails API Key.
• File uploads fail with "input must be a string or a non-empty array of strings": the RAG API is pre-tokenizing text (its default). Set RAG_CHECK_EMBEDDING_CTX_LENGTH=false and recreate the containers (see the RAG section above).