CODE Magazine / Articles

Local LLMs

Articles filed under Local LLMs

Articles

Showing 1–2 of 2

  1. CODE Magazine: 2026 - November/December cover

    Distilling Models

    Sahil demonstrates that model distillation is the practical path to making AI useful, affordable, and private by compressing the knowledge of large “teacher” models into small, fast “student” models that run locally. Using a Git error–explainer as an example, he shows how to generate synthetic training data with Ollama, fine-tune a tiny model in Google Colab with Unsloth and LoRA, and export it for local use.

  2. CODE Magazine: 2026 - November/December cover

    Fine Tuning Large Language Models, Part 2

    Wei-Meng expands the fine-tuning toolkit beyond basic LoRA by explaining how quantization enables QLoRA on modest hardware, how prompt tuning can steer a frozen model with only a few learned virtual tokens, and how to judge whether training truly improved performance through baselines, standard metrics, and LLM-as-a-judge methods. This article also introduces knowledge distillation as a way to compress a strong teacher into a smaller student, then shows practical deployment paths with direct Transformers use or ONNX export.