CodeK
Does Karpathy-Style Code Clarity Improve SWE-Bench Repair?
What It Is
A research dataset and training pipeline testing whether Karpathy-style code clarity (descriptive names, short functions, minimum abstractions) improves automated software-repair performance on SWE-bench. The seed batch is 201 hand-rolled Karpathy-style examples; the pipeline runs three augmentation passes to reach ~10K rows across splits before LoRA fine-tuning on Qwen2.5-Coder.
Most code-LLM training treats all working code as equally good supervision. CodeK tests an old intuition formally: if the supervision is more readable and minimally abstracted, does the resulting model write code that's easier to repair? The answer matters for code-LLM evaluation methodology, not just one model.
What I Built
- Phase 1: extracted and codified Karpathy-style guidance into a reusable style guide (`karpathy_style_codex.md`) from primary sources
- Phase 2.1: 201/201 synthetic Karpathy-style training rows in `data/seeds/synthetic_karpathy.jsonl` — tracked and reproducible
- Phase 2 plan: three augmentation passes (style-rewrite, function-level decomposition, naming-clarity transforms) to scale to ~10K rows
- Reproducible dataset packaging targeting public HuggingFace release
- Explicit style rules and primary-source attribution baked into the augmentation prompts
Tech Stack
Phase 1 complete, Phase 2.1 complete (201/201 rows). Phase 2.2 augmentation passes are next. Final dataset and model variants slated for HuggingFace.