gpt_from_scratch/local_ai_setup.md

Local AI Coding Setup — Windows 11 (Aider + Ollama)

Hardware

• GPU: RTX 5070 Ti, 12GB VRAM
• RAM: 32GB
• OS: Windows 11

---

Prerequisites

1. Install Ollama

Download from https://ollama.com and install.

2. Pull a model

ollama pull qwen3-coder:30b   # 18GB, strong coding model (MoE, VRAM+RAM split)
ollama pull gemma4:12b        # 12GB, responds in English more reliably

Confirm models are available:

ollama list

Confirm Ollama is serving:

curl http://localhost:11434/api/tags

---

Install Aider

pip install aider-chat
aider --version   # should print version number

---

Project Config

Create .aider.conf.yml in your project root:

model: ollama/gemma4:12b
openai-api-base: http://localhost:11434/v1
openai-api-key: ollama
edit-format: whole
map-tokens: 2048
no-auto-commits: true
system-prompt: "Always respond in English only. Never use Chinese or any other language."

Config notes:

• edit-format: whole — more reliable file writes for local models (avoids code showing in chat only)
• map-tokens: 2048 — limits repo map size, important for smaller context windows
• no-auto-commits: true — review changes before they hit git
• system-prompt — forces English responses (important for Qwen3 which defaults to Chinese)

Switch models without editing the file:

/model ollama/qwen3-coder:30b

---

Optional: Performance Tuning (if inference is slow)

Create a Modelfile in the project root:

FROM qwen3-coder:30b
PARAMETER num_ctx 8192
PARAMETER num_gpu 99

Register it:

ollama create qwen3-coder-tuned -f Modelfile

Update .aider.conf.yml:

model: ollama/qwen3-coder-tuned

Why: num_gpu 99 maximizes VRAM usage. num_ctx 8192 prevents the KV cache from spilling too much into RAM. Recommended context range: 4096–16384.

---

Daily Aider Workflow

Start Aider (reads config automatically):

aider

Command	What it does
/add src/foo.py	Add file to context
/drop src/foo.py	Remove file from context
/diff	See pending changes
/undo	Revert last change
/run pytest	Run command, share output with model
/run git rm --cached file.py	Run git commands from inside Aider
/clear	Clear chat history (keep files)
/settings	Show loaded config (verify auto_commits, system_prompt)
/model ollama/gemma4:12b	Switch model on the fly
/exit	Quit

---

Known Issues & Fixes

Model responds in Chinese

Cause: Qwen3 is a Chinese model and defaults to Chinese. Fix: system-prompt in .aider.conf.yml (already included above). Restart Aider after config change.

Aider shows code in chat but doesn't write files

Cause: Model didn't follow Aider's edit format. Fix 1: /add target_file.py before asking — hints the target file to the model. Fix 2: edit-format: whole in config (already included above).

"File not found" / repo-map warnings for deleted files

Cause: File deleted from disk but still tracked in git index. Fix:

git rm --cached filename.py

Or from inside Aider:

/run git rm --cached filename.py

Config changes not taking effect

Fix: Exit Aider (/exit) and restart. Config is only read at startup.

---

Verify Everything Works

1. ollama list — model appears
2. aider --version — prints version
3. Start Aider, run /settings — confirm auto_commits: False and system_prompt present
4. Ask Aider to create a simple file — it should write to disk, not just chat
5. Response is in English