Schedule a Free AI Consultation →
SRJ Consulting &Services The Operating Discipline for AI Library™

Ollama

Ollama

The simplest way to run open-weight models entirely on your own machine.

Open Source & Self-Hosted Models Active #OpenSource#LocalDeployment#Privacy#DeveloperTools

In short

Turns self-hosting from an infrastructure project into a single install. Pulls and runs open-weight models locally with an OpenAI-compatible endpoint, which means existing code usually points at it unchanged.

What it is best at

  1. Running an assistant on confidential material with no network path at all
  2. Local development against a model without per-token cost
  3. Air-gapped or high-sensitivity environments where no external API is permitted

Built for: Dev Teams · Compliance/Audit Professionals · Solopreneurs

Technical foundation

Base model
Not a model. A runtime and model manager that serves open-weight models locally.
Context and file handling
Depends on the model served. Text and, for multimodal models, images.
Latency
Bounded entirely by local hardware. Small models are fast on a modern laptop; large ones need serious GPU capacity.
Output quality and limits
Exactly the quality of whichever open-weight model is loaded. Ollama adds no capability and takes none away.

Pricing and access tiers

TierModelKey inclusionsLimits
Open source$0Full local runtime and model libraryBounded by your hardware

Pricing, version numbers, context-window sizes, and compliance certifications change frequently. Where stated they are accurate as of the as_of date and should be confirmed with the vendor before any procurement or compliance decision. Where they could not be stated confidently they are omitted rather than guessed.

Security, privacy and governance

Training data opt-out
Not applicable. Nothing leaves the machine, which is the entire reason to use it.

The governance question this raises

This is the strongest available answer to 'we cannot send this data anywhere'. It converts an AI vendor risk assessment into an endpoint security question, which most organizations already know how to run. The residual risks move to the model licence and to unmanaged installs on staff laptops, so pair adoption with a licence review and an endpoint policy.

No compliance certifications are listed here. Certification status is vendor-specific and time-specific, so it is stated only where verified rather than assumed. Check the vendor’s trust centre and confirm it covers the specific tier you are buying.

Integrations and ecosystem

  • OpenAI-compatible API surface
  • Most orchestration frameworks
  • Editor and desktop clients

API and SDKs: Local REST API that is largely OpenAI-compatible, so most existing code can be repointed with a base-URL change.

The verdict

Strengths

  • Data never leaves the machine
  • Trivial to install compared with raw self-hosting
  • No per-token cost

Drawbacks

  • Capability limited by local hardware
  • Model licences still apply and still need review
  • Easy for staff to install without oversight

Consider instead: Hugging Face, llama.cpp, Llama (Meta)

Frequently asked questions

What is Ollama used for?

Turns self-hosting from an infrastructure project into a single install. Pulls and runs open-weight models locally with an OpenAI-compatible endpoint, which means existing code usually points at it unchanged.

What model does Ollama run on?

Not a model. A runtime and model manager that serves open-weight models locally.

Does Ollama train on your data?

Not applicable. Nothing leaves the machine, which is the entire reason to use it.

What are the alternatives to Ollama?

The closest comparable tools are Hugging Face, llama.cpp, Llama (Meta). Which fits depends on where the work already lives and what the organization's data terms require.

Listing a tool is not governing it

The AI Business Enablement Audit™ builds the inventory, measures your organization against every framework in the AI Governance Reference Library, and delivers a defensible governance dossier.

Start or finish your AI Audit →