Ollama Api

Every model I use runs locally, and I've used the Ollama API since its first public release: for embeddings, for OCR and for text generation. I've followed it as it evolved ever since.

Where I use it

I've used it on many projects: RAG systems, chatbots and OCR tools inside existing applications. Two of the RAG systems are told on this site: the one I built at STCC, on the Python page, and Local RAG Example, open source and written in Go. At HSPI most of my Go microservices use models through Ollama or LM Studio.

An OCR app moved to LM Studio

Around the turn of 2026, an app I was working on ran an OCR model on a server with an AMD GPU. Ollama's Vulkan support was still experimental at the time. I moved the model to LM Studio, which runs models through Vulkan, and performance improved. Ollama supports Vulkan now.

Autocompletion on code that can't leave

Every day I use a local model for code autocompletion. It's for confidential codebases (those of parliamentary institutions, enterprise systems) that can't be sent to a cloud AI provider: running the models on my own infrastructure, I get AI assistance without a single line of code leaving the development environment.

Trying models at home

In my home server room an RTX 3090 runs models such as Qwen Coder and Gemma through Ollama. I try them there, in Docker, before proposing them at work.