# Local AI Coding Environment — Reference & Lessons Learned > Consolidates `local_ai_setup.md` and `local_llm.md`. Those files are superseded by this one. --- ## Hardware | Component | Spec | |-----------|------| | GPU | NVIDIA RTX 5070 Ti, 12 GB VRAM | | RAM | 32 GB system RAM | | OS | Windows 11 | | WSL | WSL2 Ubuntu | --- ## Software Stack | Tool | Purpose | Notes | |------|---------|-------| | Ollama | Local model server | Serves OpenAI-compatible API at `localhost:11434/v1` | | OpenCode | Primary AI coding agent | Installed via npm (`opencode-ai`) | | Aider | Alternative AI coding agent | pip: `aider-chat` | | Claude Code | Cloud AI coding agent | Optional; can point at Ollama | | VS Code | Editor | With Continue extension for inline chat | --- ## Architecture ``` VS Code ├── Continue Extension ──────────────┐ ├── Claude Code CLI ────────────────┤ └── OpenCode / Aider ───────────────┤ ▼ Ollama │ ▼ Qwen3-Coder 30B (primary) ``` All code stays local. Cloud models are optional fallback only. --- ## Ollama — Available Models | Model | Size | Use | |-------|------|-----| | `qwen3-coder:30b-fixed` | 18 GB | **Primary** — OpenCode agent (fixed config, see below) | | `qwen3-coder:30b` | 18 GB | Original pull (unmodified) | | `gemma4:latest` | 9.6 GB | English-reliable alternative | | `gemma4:12b` | 7.6 GB | Lighter English-reliable option | | `qwen2.5-coder:14b` | 9.0 GB | Mid-weight coding model | | `qwen2.5-coder:7b` | 4.7 GB | Fast, lightweight coding | | `qwen2.5-coder:1.5b` | 986 MB | Minimal footprint | | `qwen3:latest` | 5.2 GB | General chat | | `qwen3:1.7b` | 1.4 GB | Fast general chat | | `bge-m3:latest` | 1.2 GB | Embeddings | --- ## OpenCode Configuration **Config file:** `~/.config/opencode/opencode.jsonc` ```jsonc { "$schema": "https://opencode.ai/config.json", "provider": { "ollama": { "options": { "baseURL": "http://localhost:11434/v1" }, "models": { "qwen3-coder:30b-fixed": { "name": "Qwen3 Coder 30B" }, "qwen2.5-coder:14b": { "name": "Qwen 2.5 Coder 14B" }, "qwen2.5-coder:7b": { "name": "Qwen 2.5 Coder 7B" }, "gemma4:latest": { "name": "Gemma 4 8B (latest)" }, "qwen3:latest": { "name": "Qwen3 8B" } } } }, "model": "ollama/qwen3-coder:30b-fixed" } ``` --- ## Active Model — qwen3-coder:30b-fixed Modelfile ``` FROM TEMPLATE {{ .Prompt }} PARAMETER repeat_penalty 1.05 PARAMETER stop <|im_start|> PARAMETER stop <|im_end|> PARAMETER stop <|endoftext|> PARAMETER temperature 0.7 PARAMETER top_k 20 PARAMETER top_p 0.8 PARAMETER num_ctx 32768 ← fixed (was missing) PARAMETER num_predict 8192 ← fixed (was missing) ``` Model specs: 30.5B params, MoE architecture (qwen3moe), Q4_K_M quantization, supports tools + completion. --- ## Issue & Fix — Tool Call JSON Truncation ### Error ``` llama-server returned invalid tool call arguments for "edit": unexpected end of ... [retrying in 51s attempt N] ``` ### Root Cause Two Ollama parameters were missing from the model's Modelfile: | Parameter | Default when missing | Required for tool calls | |-----------|---------------------|------------------------| | `num_ctx` | **2048 tokens** | Minimum ~8192; 32768 recommended | | `num_predict` | **128 tokens** | Minimum ~1024; 8192 recommended | With only 128 output tokens allowed, the model began generating valid JSON for the `edit` tool call but ran out of budget before closing all braces, producing malformed JSON. The 2048-token context also filled up quickly during long sessions, compounding the problem. ### Fix Applied Rebuilt `qwen3-coder:30b-fixed` with the two parameters added: ```bash # Export current Modelfile, append fixes, rebuild ollama show qwen3-coder:30b-fixed --modelfile > Modelfile_fixed # (add num_ctx 32768 and num_predict 8192) ollama create qwen3-coder:30b-fixed -f Modelfile_fixed ``` No changes needed to the OpenCode config — same model name, same config file. ### Verification ```bash ollama show qwen3-coder:30b-fixed # Should show: num_ctx 32768 and num_predict 8192 in Parameters section ``` --- ## Aider Configuration **Config file:** `.aider.conf.yml` in project root ```yaml model: ollama/qwen3-coder:30b openai-api-base: http://localhost:11434/v1 openai-api-key: ollama edit-format: whole map-tokens: 2048 no-auto-commits: true system-prompt: "Always respond in English only. Never use Chinese or any other language." ``` | Option | Why | |--------|-----| | `edit-format: whole` | More reliable file writes for local models | | `map-tokens: 2048` | Limits repo map size for smaller context windows | | `no-auto-commits: true` | Review changes before they hit git | | `system-prompt` | Qwen3 defaults to Chinese; forces English | ### Aider Quick Reference | Command | What it does | |---------|-------------| | `/add src/foo.py` | Add file to context | | `/drop src/foo.py` | Remove file from context | | `/diff` | See pending changes | | `/undo` | Revert last change | | `/run pytest` | Run command, share output with model | | `/clear` | Clear chat history (keep files) | | `/settings` | Show loaded config | | `/model ollama/gemma4:12b` | Switch model on the fly | --- ## Known Issues & Fixes ### Tool call JSON truncated (OpenCode / llama-server) **Error:** `llama-server returned invalid tool call arguments for "edit": unexpected end of...` **Cause:** `num_predict` too low (default 128) cuts off JSON mid-generation. **Fix:** See "Issue & Fix" section above — add `num_ctx` and `num_predict` to Modelfile. ### Model responds in Chinese **Cause:** Qwen3 is a Chinese model and defaults to Chinese. **Fix (Aider):** `system-prompt` in `.aider.conf.yml`. **Fix (OpenCode):** Add a system prompt in OpenCode's project config if supported, or use `gemma4` models which default to English. ### Aider writes code to chat but not to disk **Cause:** Model didn't follow Aider's edit format. **Fix 1:** `/add target_file.py` before the request — hints the target. **Fix 2:** `edit-format: whole` in config (already included above). ### Config changes not taking effect (Aider) **Fix:** Exit (`/exit`) and restart. Config is read only at startup. ### "File not found" / repo-map warnings for deleted files **Cause:** File deleted from disk but still tracked in git index. **Fix:** ```bash git rm --cached filename.py # or from inside Aider: /run git rm --cached filename.py ``` ### Model loops with filler text, never executes tool calls **Symptom:** Model repeatedly says "I'll continue..." but never reads/edits files. Context counter in OpenCode bottom-left shows a number near your `num_ctx` limit. **Cause:** Context window is full. No tokens left for tool call JSON, so the model outputs short filler and stalls. **Fix (immediate):** Start a new OpenCode session. Break large tasks (e.g. "translate all files") into one file per session. **Fix (permanent):** Increase `num_ctx` in the Modelfile (e.g. 65536), rebuild, restart Ollama. Watch RAM usage — each doubling of `num_ctx` roughly doubles KV cache memory. ### Context fills up, model performance degrades **Cause:** Long session accumulates tokens beyond `num_ctx`. **Fix (Aider):** `/clear` to reset chat history without losing file context. **Fix (Ollama model):** Increase `num_ctx` in Modelfile (current: 32768). --- ## Performance Notes - qwen3-coder:30b is a MoE model — fits in VRAM+RAM split on RTX 5070 Ti (12 GB VRAM). - `num_gpu 99` in Modelfile maximizes layers on VRAM; remainder spills to RAM. - Safe `num_ctx` range for this hardware: **8192–32768**. Above 32768 risks RAM pressure. - Q4_K_M quantization gives best quality/speed/memory balance for this size class. --- ## Environment Verification Checklist ```bash ollama list # models appear ollama show qwen3-coder:30b-fixed # num_ctx and num_predict present curl http://localhost:11434/api/tags # Ollama serving aider --version # Aider installed opencode --version # OpenCode installed nvidia-smi # GPU visible ```