Every line of code you send to a cloud AI provider leaves your machine. For many developers — especially those working on proprietary software, regulated industries, or air-gapped environments — that's a non-starter. OpenCode + Ollama solves this: a fully private AI coding agent that runs entirely on your hardware with zero API costs and zero data leaks.

With models like Qwen 3.6-35B-A3B (73.4% SWE-bench with only 3B active parameters), local AI coding has reached a quality threshold where it's genuinely useful for production work — not just a novelty.

This guide covers model selection, hardware requirements, configuration, performance tuning, and hybrid local/cloud setups for the best of both worlds.

1Why Run AI Coding Locally?

2Installing Ollama

Ollama is the easiest way to run LLMs locally. It handles model downloading, quantization, and serving behind an OpenAI-compatible API:

Ollama runs a local server on http://localhost:11434 that exposes an OpenAI-compatible API. OpenCode connects to this endpoint directly.

3Best Models for Coding (April 2026)

4Hardware Requirements

5Connecting OpenCode to Ollama

Add Ollama as a provider in your opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "api_url": "http://localhost:11434/v1"
    }
  },
  "model": {
    "default": "ollama/qwen3.6:35b-a3b",
    "fast": "ollama/qwen3.6:8b"
  }
}

Start Ollama, then launch OpenCode:

6Performance Tuning

Get the most out of local models with these optimizations:

• Context window: Set num_ctx to match your available VRAM. Larger context = more memory. Start with 8192 and increase if needed.
• GPU layers: Ensure all layers are offloaded to GPU with num_gpu: -1 (auto). CPU inference is 10-50x slower.
• Quantization: Use Q4_K_M for the best quality/speed balance. Q8_0 for maximum quality if VRAM allows.
• Keep alive: Set keep_alive: "30m" to keep the model loaded between requests, avoiding reload delays.
• Batch size: Increase num_batch to 512 or 1024 for faster prompt processing on high-VRAM systems.

# Create a Modelfile for optimized settings
FROM qwen3.6:35b-a3b
PARAMETER num_ctx 16384
PARAMETER num_batch 1024
PARAMETER temperature 0.1
PARAMETER top_p 0.9

7Hybrid Local + Cloud Setup

The most practical approach combines local and cloud models. Use local models for routine tasks and cloud models for complex reasoning:

{
  "provider": {
    "ollama": {
      "api_url": "http://localhost:11434/v1"
    },
    "anthropic": {
      "api_key": "env:ANTHROPIC_API_KEY"
    }
  },
  "model": {
    "default": "ollama/qwen3.6:35b-a3b",
    "fast": "ollama/qwen3.6:8b",
    "reasoning": "anthropic/claude-opus-4"
  }
}

Use /model in the TUI to switch between local and cloud models on the fly. Routine file edits and simple questions go to Ollama (free). Complex architecture decisions and multi-file refactors go to Claude (paid but worth it).

8Air-Gapped & Enterprise Deployment

For fully air-gapped environments:

• Pre-download models on a connected machine, then transfer via USB/network share
• Install OpenCode and Ollama from local binaries (no internet required at runtime)
• Use AGENTS.md to enforce coding standards without cloud connectivity
• Deploy Ollama on a shared GPU server for team-wide access
• All session logs stay on your infrastructure — full audit trail

9Limitations & When to Use Cloud

Local models have real limitations:

• Complex reasoning: Claude Opus 4.7 and GPT-5.4 still outperform local models on multi-step reasoning and large refactors
• Context window: Local models typically max out at 32K-128K tokens vs 200K+ for cloud models
• Speed on large prompts: Processing a 50K-token codebase is slow on consumer hardware
• Hardware cost: A capable GPU (RTX 4090) costs $1,500-2,000 upfront

10Lushbinary Private AI Services

Lushbinary helps teams deploy private AI coding infrastructure — from single-developer Ollama setups to team-wide GPU servers with model routing and access controls. We also build custom AI tools that work entirely within your network.

❓ Frequently Asked Questions

📚 Sources

• Ollama Official Site
• OpenCode Official Site
• Qwen 3.6 Documentation

Content was rephrased for compliance with licensing restrictions. Benchmark data sourced from official model documentation as of April 2026. Performance varies by hardware — always benchmark on your own system.