Importing custom LLMs into RealTimeX

RealTimeX allows you to easily load any valid GGUF file and select that as your LLM with zero-setup. Please use text-based LLMs only for this process. Embedder models will not function as chat models.

Import a model into RealTimeX

Use Ollama to Import and Run Models

Ollama makes it extremely easy to run LLMs locally, and RealTimeX can connect to any Ollama-served model.

Step-by-step:

1. Install Ollama

   • Go to ollama.com/download (opens in a new tab) and download/install for your OS.
   • Launch Ollama (just run the app or the ollama serve command).

2. Pull an LLM with Ollama

   • Open your terminal and run:

     ollama pull llama3
   • Or replace llama3 with any supported model name (see Ollama's model library (opens in a new tab)).

3. Connect RealTimeX to Ollama

   • In RealTimeX Desktop, go to the LLM setup or model selection screen.
   • Choose Ollama as the model provider.
   • RealTimeX will auto-detect running Ollama models. Select the one you want to use.

4. Start using the model!

   • The model will now be available in RealTimeX just like any built-in option.

Troubleshooting

• If RealTimeX doesn’t detect Ollama, make sure the Ollama server is running.
• You can find more models for Ollama at ollama.com/library (opens in a new tab).