Full fine-tuning recipe: DPO on Llama 3.3 70B via FSDP, targeting AWS p4d.24xlarge, with data mix and eval plan.
Full fine-tuning recipe: DPO on Llama 3.1 70B via LLaMA-Factory, targeting AWS p4d.24xlarge, with data mix and eval plan.
Full fine-tuning recipe: DPO on Mistral Small 3 via Axolotl, targeting Lambda Labs 8xH100, with data mix and eval plan.
Full fine-tuning recipe: DPO on Qwen 2.5 7B via LitGPT, targeting Lambda Labs 8xH100, with data mix and eval plan.
Full fine-tuning recipe: DPO on Qwen 2.5 32B via torchtune, targeting Lambda Labs 8xH100, with data mix and eval plan.
Full fine-tuning recipe: DPO on Gemma 2 9B via Unsloth, targeting single A100 80GB, with data mix and eval plan.
Full fine-tuning recipe: DPO on Gemma 2 27B via LitGPT, targeting single A100 80GB, with data mix and eval plan.
Full fine-tuning recipe: DPO on Phi-3.5-mini via torchtune, targeting single H100 80GB, with data mix and eval plan.
Full fine-tuning recipe: DPO on DeepSeek-V3 base via Unsloth, targeting single H100 80GB, with data mix and eval plan.
Full fine-tuning recipe: DPO on Mixtral 8x7B via OpenRLHF, targeting single H100 80GB, with data mix and eval plan.
Full fine-tuning recipe: DPO on Yi 1.5 34B via DeepSpeed, targeting 2x A100 80GB, with data mix and eval plan.
Full fine-tuning recipe: DPO on Llama 3.3 70B via Hugging Face TRL, targeting 2x A100 80GB, with data mix and eval plan.