dpo

Hotel-domain conversational assistant using Qwen2.5-7B-Instruct, focusing on practical, reproducible fine-tuning. It supports SFT + LoRA for a stable baseline, SFT + QLoRA for 7B model training with 4-bit quantization under limited GPU memory, and DPO to enhance responses via human preference pairs

Advanced LLM fine-tuning techniques: SFT (LoRA, QLoRA, DoRA, P-/Prefix-Tuning), GRPO, DPO, ORPO, KTO & PPO; composable correctness/format rewards + LLM-as-a-Judge evals (DeepEval, Evidently AI) across math, multi-hop, medical & general QA on Llama 3, Mistral, Phi-4, Gemma & Qwen3. Built on TRL, PEFT & Unsloth.

Different post-training techniques for LLMs, including: SFT, DPO and Online RL

Companion code for the Manning book 'LLM Customization and Fine-Tuning.' Adapt open-weights LLMs end to end: prompting and RAG, LoRA/QLoRA, full SFT, distillation, and DPO/RLHF alignment.