Setting Up Ollama — Free Local AI by node

Ollama lets you run AI models on your own computer. No API keys, no costs, no data leaving your machine. Here's how to set it up with CivNode. ## What is Ollama? Ollama is a tool that runs large language models locally. You download it, pull a model, and it runs a local server that CivNode can talk to. The AI processing happens on your hardware. Nothing goes to the cloud. ## Installation **Mac:** Download from ollama.com. Open the app. It installs itself and runs in the menu bar. Done. **Linux:** ``` curl -fsSL https://ollama.com/install.sh | sh ``` The installer sets up Ollama as a system service. It starts automatically. **Windows:** Download from ollama.com. Run the installer. Ollama runs in the system tray. ## Pulling a model Open a terminal and run: ``` ollama pull llama3.1:8b ``` This downloads the Llama 3.1 8B model. It's about 4.7 GB. Wait for it to finish. ## Which models work well with CivNode? For writing assistance, these models balance quality and speed: - **llama3.1:8b** — Good all-rounder. Fast on most hardware. Recommended starting point. - **mistral:7b** — Slightly different style, some prefer it for creative writing. - **llama3.1:70b** — Significantly better quality, but needs a powerful GPU (24GB+ VRAM) or lots of RAM. - **qwen2.5:14b** — Good middle ground between 8b and 70b. For most people, the 8b model is the right choice. It's fast enough to feel responsive and smart enough to be useful. ## Connecting to CivNode 1. Go to **Settings → AI Providers** 2. Under Local AI, enter the Ollama URL: `http://localhost:11434` 3. Select your model from the dropdown (CivNode will detect what you have installed) 4. Save That's it. CivNode will use Ollama for AI features when it's running. ## Why use Ollama? - **Free.** No API costs. Run it as much as you want. - **Private.** Your writing never leaves your machine. - **No internet required.** Works offline, on planes, in cabins, wherever. - **No rate limits.** Use AI features as heavily as you like. The tradeoff is speed and quality. Cloud providers like Claude and GPT-4 are faster and more capable than local models. But for many writing tasks — brainstorming, getting unstuck, generating character ideas — a local model is more than enough. ## Troubleshooting **"Connection refused" error:** Ollama isn't running. On Mac, open the Ollama app. On Linux, run `ollama serve`. On Windows, check the system tray. **Slow responses:** Your hardware matters. If responses take more than 30 seconds for a short prompt, your machine may be too slow for the model. Try a smaller model: `ollama pull phi3:mini` gives you a lightweight option. **Out of memory:** Larger models need more RAM. The 8b models need roughly 8GB of available RAM. The 70b models need 48GB+. If your system is swapping, use a smaller model. **Model not showing in CivNode:** Make sure you've pulled the model (`ollama list` shows what's installed) and that the Ollama URL is correct in settings.