HomeBooks

A Sivaro book · 17 chapters

Fine-tuning LLMs

When fine-tuning a language model pays off, when retrieval or a better prompt is enough, and what fine-tuning takes.

Chapters

  1. 01 Is Fine Tuning an LLM Worth It for Production in 2026?Is fine-tuning an LLM worth it in production? Hosting costs against hosted APIs, RAG versus fine-tuning, the steps, risks, tools, and a matrix to decide.
  2. 02 Fine Tuning vs Post Training for LLMs: A 2026 GuideThe difference between supervised fine-tuning and alignment post-training for LLMs, when each applies, how costs compare, and a framework for choosing.
  3. 03 Fine Tuning LLM on Mac Studio M4 Performance: A 2026 Field GuideFine-tuning an LLM on a Mac Studio M4: memory limits, software stack, LoRA steps for classification, base model choice, and when to pick RAG instead.
  4. 04 Self-Improving Open-Source Models: The Agentic Coding PlaybookHow self-improving coding agents on open models work: the feedback loop, failure modes, how to build one, metrics, and when a static model is better.
  5. 05 Fine Tuning LLMs for Domain Specific Tasks: 2026 GuideHow to fine-tune LLMs for domain tasks: how much data you need, Unsloth, Axolotl and managed tools, LoRA steps, RAG versus fine-tuning, and when to skip.
  6. 06 fine tuning qwen3.5 bug fixes and workaroundsCommon Qwen3.5 fine-tuning bugs and fixes: tokenizer mismatch, misleading loss curves, memory, DeepSpeed with LoRA, chat templates and the pad token.
  7. 07 best datasets for llm fine-tuning: A Practitioner’s Guide (2026)Which datasets suit LLM fine-tuning: instruction, preference, domain and synthetic data, plus how to evaluate and build a dataset before training.
  8. 08 Can You Fine Tune an LLM on a Mac Studio? (2026 Guide)Can a Mac Studio fine-tune an LLM? Which models fit in memory, how PyTorch MPS, MLX and Unsloth compare, where it breaks, and when to use the cloud.
  9. 09 The Real Guide to Best Open Source Models for Fine Tuning in 2026Choosing an open source model to fine-tune: Llama, Mistral, Qwen and DeepSeek compared, when to skip fine-tuning, the workflow and hosting costs.
  10. 10 LLM Fine-Tuning Dataset Size: Best Practices 2026How much data LLM fine-tuning needs: small datasets, quality over quantity, a minimum viable set, synthetic data, when to stop and hardware demands.
  11. 11 How much does it cost to fine tune an LLM in 2026?What LLM fine-tuning costs: compute by model size and method, data prep, iteration, inference after training, when it does not pay off, estimating.
  12. 12 How Long Does It Take to Fine-Tune an LLM?How long LLM fine-tuning takes across data prep, training, and evaluation, which factors stretch the timeline, and when fine-tuning is the wrong choice.
  13. 13 Can You Fine-Tune GPT-4 on Your Own Data? A 2026 GuideCan GPT-4 be fine-tuned on your own data? What fine-tuning changes, how it compares with RAG and prompt engineering, and a decision tree for choosing.
  14. 14 Fine Tuned LLM vs Prompt Engineering: Which Is Better?Fine-tuning or prompt engineering? A decision framework based on volume, task stability, failure tolerance and data, with costs, evaluation and hybrids.
  15. 15 LLM Fine Tuning vs Prompt Engineering: A Practitioner's GuideWhen to prompt and when to fine-tune an LLM: a decision framework, LoRA example, hardware needs, tools, RAG as a third option and common mistakes to avoid.
  16. 16 Fine Tune Open Source LLM vs GPT API: 2026 GuideFine-tune an open source LLM or call the GPT API? Compares cost, hidden data and labeling work, tooling and RAG, with a decision checklist for each.
  17. 17 How Much Does LLM Training Cost?What LLM training costs, from fine-tuning to frontier models. Covers compute, data, engineering and evaluation costs, and why training got cheaper.