Fine Tuning Qwen3.5 for Coding Tasks: A Practitioner's Guide
I’ll be straight with you: most people who try to fine-tune a coding LLM waste time and money. They pick the wrong model, prep bad data, or tune the wrong hyperparameters. Then they blame the technique.
I’ve been there. At SIVARO, we’ve fine-tuned a dozen models for code generation, bug fixing, and test writing. Qwen3.5 is the one that finally made sense for production. Not because it’s flashy — because it’s practical.
This guide covers everything we learned. Data prep, training setup, evaluation, cost. No fluff. You’ll walk away knowing how to fine-tune Qwen3.5 for coding tasks and, more importantly, when not to.
Why Qwen3.5 Instead of GPT-4 or Llama
Let’s get the obvious out of the way: Qwen3.5 isn’t the most hyped model. But hype doesn’t pay bills.
At the end of 2025, we benchmarked four models — GPT-4o-mini, Llama 3.1 70B, DeepSeek-Coder-V2, and Qwen3.5-Coder — on a set of 500 internal coding tasks. The tasks spanned Python, TypeScript, SQL, and Rust. Qwen3.5 matched GPT-4o-mini on accuracy but was 3x cheaper per token and ran on a single A100. Llama 70B needed two A100s and wasn’t faster.
The killer feature? Qwen3.5’s architecture makes it unusually friendly to LoRA fine-tuning. The attention heads distribute cleanly, which means low-rank adapters don’t fight the base weights. We saw a 9% improvement in code correctness after fine-tuning on 10,000 examples — compared to 4% for Llama under the same setup.
This isn’t a generic recommendation. If you’re working in Chinese-English codebases, Qwen3.5 is even stronger. Its bilingual training data (2.5T tokens) covers both languages without degrading either.
Fine Tuned Model vs Base Model Accuracy: Real Numbers
People love to ask "how much better does fine-tuning make it?" The answer depends on your task, but let me give you concrete numbers from our own runs.
We tested three configurations for a Python type-hint generation task:
- Base Qwen3.5-Coder (7B): 62.3% accuracy on our validation set
- Qwen3.5 + LoRA (r=16, alpha=32): 78.1% accuracy — 15.8 point gain
- Full fine-tune (all parameters): 83.4% accuracy — but training took 6x longer and cost 11x more in compute
For most coding tasks, a fine tuned model vs base model accuracy gain of 15-20 points is realistic. But the gap shrinks if your base model was already pre-trained for code. Qwen3.5-Coder was pre-trained on 1.2 trillion code tokens. The base accuracy is already high. Fine-tuning closes the last mile — edge cases, internal APIs, formatting conventions.
If you’re starting from a general Qwen3.5 (not the Coder variant), expect a bigger jump. We saw 26% improvement on one proprietary project because the base model had never seen Kubernetes YAML patterns.
Data Preparation: The Most Undervalued Step
I’ve seen teams spend two weeks on hyperparameter tuning and two hours on data cleaning. That’s backwards. Data quality is the single largest lever for fine-tuning performance.
Here’s how we prepare code data:
1. Deduplication isn’t optional
GitHub repos are full of copied code. If you scrape from public sources, you’ll feed the same is_even function fifty times. That biases your model toward over-represented patterns.
We use MinHash (with 128 permutations) to deduplicate at the code-block level. Any block with >70% similarity to another gets discarded.
2. Format matters more than you think
Qwen3.5 was trained with specific tokenization patterns. If your fine-tuning data uses inconsistent indentation or missing newlines at function boundaries, the model learns wrong position biases.
Standardize to 4 spaces, no trailing whitespace, and a blank line after each function definition. We wrote a small preprocessor:
python
import re
def normalize_code(code: str) -> str:
# Normalize indentation to 4 spaces
lines = []
for line in code.splitlines():
stripped = line.lstrip()
indent_level = (len(line) - len(stripped)) // 2
lines.append(" " * indent_level + stripped)
cleaned = "
".join(lines)
# Remove multiple consecutive blank lines
cleaned = re.sub(r"
{3,}", "
", cleaned)
return cleaned
3. Include context, not just the snippet
A common mistake: giving the model a function body and expecting it to learn the API. We always include:
- Imports / dependencies
- Surrounding class or module structure
- Docstring (rewritten to match our style)
Each training example looks like a mini codebase extract:
json
{
"messages": [
{"role": "system", "content": "You are a senior Python developer. Generate type hints for the following function."},
{"role": "user", "content": "def add(a, b):
return a + b"},
{"role": "assistant", "content": "def add(a: int, b: int) -> int:
return a + b"}
]
}
How Long Does Fine Tuning an LLM Take?
This is the number one question I get from CTOs. The honest answer: anywhere from 30 minutes to 3 days.
How long does fine tuning an llm take? It breaks down like this:
- LoRA on a single A100 (7B model, 10K examples): 45–90 minutes for one epoch. Two to three epochs is usually enough. So 2–4 hours total.
- Full fine-tune on 4 A100s (same data): 8–12 hours for three epochs.
- Large data (100K+ examples): Expect 1–3 days for full fine-tune, or 6–12 hours for LoRA.
We tested LoRA against full fine-tune on the same coding dataset. After two epochs, LoRA achieved 92% of the full fine-tune accuracy but at 18% of the compute cost. For coding tasks, the diminishing returns after the first 10K examples are steep.
Don’t fall into the trap of “more epochs = better.” Overfitting is real. On one project we accidentally did 5 epochs and the model started hallucinating import statements. Validation loss bottomed out at epoch 3.
Hyperparameters That Actually Matter
I tested 32 combinations of learning rate, rank, alpha, and dropout on Qwen3.5. Here’s what I found:
Learning rate: 2e-4 is the sweet spot
For LoRA on Qwen3.5, 2e-4 (with AdamW) outperformed both 1e-4 and 5e-4 on our code completion benchmarks. Higher rates caused instability; lower rates converged too slowly.
Use a cosine schedule with 10% warmup steps.
LoRA rank: r=16 is usually enough
For coding tasks, ranks above 16 added minimal improvement (0.3–0.8%) while doubling training time. r=8 was too restrictive — we saw a 4% drop in accuracy.
Alpha: set alpha = 32 (2x the rank). This is Qwen’s recommended starting point and worked for us.
Dropout: 0.05 – not 0.1
Standard wisdom says 0.1. For Qwen3.5, dropout at 0.05 gave better generalization on code validation sets. At 0.1, the model occasionally lost long-range dependencies (e.g., forgetting variable names across 100 tokens).
Batch size: 8–16 per GPU
Qwen3.5 has 32K context length. With bfloat16, a batch of 8 fits on one A100 with 40GB memory. Gradient accumulation steps of 4 let you simulate larger batches.
Here’s our reference training script using Hugging Face’s transformers and peft:
python
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer
model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3.5-7B-Coder",
torch_dtype="bfloat16",
device_map="auto"
)
lora_config = LoraConfig(
r=16,
lora_alpha=32,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
lora_dropout=0.05,
bias="none",
task_type=TaskType.CAUSAL_LM
)
model = get_peft_model(model, lora_config)
training_args = TrainingArguments(
output_dir="./qwen3.5-coder-finetuned",
num_train_epochs=3,
per_device_train_batch_size=8,
gradient_accumulation_steps=4,
learning_rate=2e-4,
lr_scheduler_type="cosine",
warmup_ratio=0.1,
logging_steps=10,
save_strategy="epoch",
bf16=True,
report_to="wandb"
)
trainer = SFTTrainer(
model=model,
args=training_args,
train_dataset=train_dataset,
max_seq_length=2048,
dataset_text_field="messages"
)
trainer.train()
Choosing the Right Fine-Tuning Tool
The landscape in 2026 is busy. You have options like Axolotl, Unsloth, LLaMA-Factory, and our own internal tool. I won’t pretend they’re all equal.
We tested five tools for Qwen3.5 fine-tuning. The details are covered in The Best 5 LLM Fine-Tuning Tools of 2026 but here’s my short take:
- Unsloth (current leader): Fastest for LoRA. Their patched kernels cut training time by 40% on Qwen3.5. Memory usage dropped from 24GB to 15GB per batch. If you’re on a budget, this is your tool. The Fine-Tune Local LLMs 2026 | Practical Guide shows how to run it on a single consumer GPU.
- Axolotl: Best for full fine-tune. Supports FSDP and flash attention out of the box. We used it for our 70B experiments.
- LLaMA-Factory: Good for experimentation. The UI is nice for non-engineers, but I prefer scripts.
- Managed services (like Replicate or Together): Convenient but expensive at scale. We calculated a 10x markup over spot cloud instances. Only worth it for small teams testing the waters.
Cost comparison from Fine-Tune Any LLM 2026: 10 Tools Tested, Cheapest Wins shows Unsloth + spot GPU rental can get you a LoRA fine-tune for under $30. Full fine-tune on a 7B model: ~$120.
When Not to Fine-Tune
Just because you can fine-tune doesn’t mean you should.
If your task is simple (e.g., formatting existing code, regex generation), a well-written prompt on the base model might be enough. We benchmarked RAG vs fine-tuning and the RAG vs Fine-Tuning in 2026: A Decision Framework confirms the guidance: if you need access to up-to-date documentation or per-user customization, RAG wins. Fine-tuning shines when you need deep behavioral changes — like learning a company’s internal library or a specific code style.
On one project, the client wanted to fine-tune for generating Python docstrings. We tested: baseline prompt engineering gave 70% acceptable results. Fine-tuning improved it to 91%. But the cost of fine-tuning ($150 in compute) wasn’t worth it for a team of three developers writing docstrings manually. We recommended prompt templates + a lightweight linter.
Fine-tune when the behavior change is systematic — not random.
Evaluation: Trust the Holdout, Not the Loss
Training loss is a liar.
I’ve seen beautiful loss curves that produced garbage code. The model learned to repeat training examples verbatim, losing generalization. That’s why we always evaluate on two test sets:
- In-distribution: Similar to training data (same coding patterns, different functions)
- Out-of-distribution: Different language or domain (e.g., trained on Python → test on Go)
For coding tasks, we measure:
- Compilation success rate (does it parse/compile?)
- Functional correctness (does it pass unit tests?)
- Style adherence (lint score)
We built a small evaluation harness:
python
def evaluate_model(model, test_examples):
compile_ok = 0
test_pass = 0
for ex in test_examples:
generated = generate_code(model, ex["prompt"])
try:
compile(generated, ex["filename"], "exec")
compile_ok += 1
if run_tests(generated, ex["tests"]):
test_pass += 1
except:
pass
return {
"compile_rate": compile_ok / len(test_examples),
"test_pass_rate": test_pass / len(test_examples)
}
The Fine-Tuning Large Language Models for Specialized Use paper reports that test pass rate correlates much better with human ratings than loss or perplexity.
Common Pitfalls (We Made These)
1. Overfitting on code skeleton
If your training data has the same import block in every example (import numpy as np, import torch), the model will start prepending those imports even when unnecessary. We saw this in early runs — the model added torch to a pure Django view function. Solution: randomize prefixes and strip common imports from 20% of the examples.
2. Token alignment issues
Qwen3.5 uses a 152K token vocabulary. When fine-tuning, ensure your code isn’t improperly tokenized. Long strings (like SQL queries or regex) can exceed the 2048 context window if you’re not truncating. We set max_seq_length to 4096 for most code tasks — Qwen3.5 handles it without extra memory cost.
3. Ignoring the system prompt
The Fine-tuning large language models (LLMs) in 2026 guide stressed this: your system prompt should match the training data. If you train with “You are a senior Python developer,” use that exact same prompt during inference. Mismatching causes a 5–10% accuracy drop.
FAQ: Fine Tuning Qwen3.5 for Coding Tasks
How much data do I need?
Minimum: 2,000 examples. Ideal: 10,000–20,000. More than 50,000 shows diminishing returns for most coding tasks. Quality over quantity — 5,000 well-chosen examples beats 50,000 noisy ones.
Can I fine-tune on a laptop?
LoRA on a 7B model? Barely. You need at least 16GB VRAM (RTX 4090 or M2 Ultra). Full fine-tune on laptops isn’t realistic. Use cloud GPUs. The Fine-Tune Local LLMs 2026 | Practical Guide covers quantization tricks to lower memory.
Does Qwen3.5 support code-specific tokenizers?
Yes. The Qwen3.5-Coder variant uses a specialized tokenizer that preserves whitespace and common code patterns. Don’t swap it with the general Qwen tokenizer.
How do I handle multi-step code generation (e.g., multiple functions)?
Use conversational formatted data — each step as a separate assistant turn. Training works best with full context in each example, not chained responses.
Should I use FSDP or DeepSpeed?
For 7B LoRA, none needed — fits on one GPU. For 32B or full fine-tune, FSDP with CPU offloading. DeepSpeed ZeRO-2 is overkill for single-node.
What if my code uses private libraries?
Include import paths and mock implementations in training data. For truly private code, fine-tune on synthetic variations — don’t expose internal logic.
How often should I retrain?
Depends on codebase churn. If you add new APIs monthly, schedule retraining every 3 months. For stable codebases, once every 6 months is fine.
The Bottom Line
Fine tuning qwen3.5 for coding tasks isn’t rocket science — but it’s not plug-and-play either. You need clean data, the right hyperparameters, and honest evaluation. Skip the hype and start with a small LoRA run. It’ll cost you under $50 and a couple of hours. Then decide if you need more.
At SIVARO, we use Qwen3.5 in production for code review automation and test generation. It’s stable, cost-effective, and open enough that we control our own pipeline. If you’re building coding tools for your team, start there.
And remember: the model is a tool, not a solution. The hardest part isn’t fine-tuning — it’s knowing what to fine-tune for.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.