Ollama backend for Local AI
The runtime that hosts your local Llama and other models, exposes a simple HTTP API, and connects the Mac Mini to your apps and tools.
Local model server
Ollama runs your Llama and other models directly on the Mac Mini—no calls to external APIs.
Simple HTTP API
Exposes a standard API you can reach from Open WebUI, Obsidian plugins, AI Dev Suite, and custom tools.
Model catalogue
Pull, update, and switch between models (reasoning, chat, code) while keeping everything on-prem.
Role in the Local AI architecture
- Acts as the single model backend for the Mac Mini and any other Local AI services.
- Accepts prompts from Open WebUI, Obsidian Copilot, and custom apps over HTTP.
- Handles token streaming so frontends can show live responses.
Model management
- Pull models (e.g. Llama 3) once; reuse them across all Local AI tools.
- Keep a small set of “blessed” models for production use and pin their versions.
- Experiment with new models on a separate port without disturbing stable workloads.
Performance, monitoring, and safety
Performance
- Optimised for Apple Silicon; take advantage of GPU and unified memory.
- Run different model sizes for different workloads (drafting vs. reasoning).
- Use the Debugger / Observer to see request latency and errors in real time.
Safety & isolation
- Requests never leave your network; you control who can reach the Ollama port.
- Combine Ollama with gateway rules or reverse proxies for stricter access control.
- Log prompts and responses locally for auditing when needed.
Next steps
If you want help sizing hardware, choosing models, or wiring Ollama into your existing tools, book a call and we’ll design the backend together.
Talk about Ollama backend