Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

10.7k
Stars
+2.4k
Gained
29.4%
Growth
Python
Language

💡 Why It Matters

Fine-tuning large language models (LLMs) remains resource-intensive and complicated, especially on limited hardware. This project tackles that by enabling fine-tuning of 8 billion parameter models on a 4 GB laptop GPU through a technique called layer streaming, which feeds one decoder layer at a time to the GPU, significantly reducing VRAM usage. It automates many aspects like batch size and quantization via a single YAML config, removing the need for SSH into GPU servers and simplifying local training workflows. With over 10,000 stars and active development as of late 2026, it shows solid community engagement but lacks significant growth momentum. Teams aiming for accessible local fine-tuning with minimal setup will benefit most, particularly ML engineers and platform teams. It is not suitable for those requiring large-scale distributed training or cloud-based infrastructure, where frameworks like Hugging Face’s Accelerate or DeepSpeed provide more scalable solutions.

🎯 When to Use

Choose this project when you need to fine-tune LLMs locally on constrained GPUs, such as laptops with 4 GB VRAM, and want an automated, minimal-configuration workflow. For large-scale or multi-GPU training, consider Hugging Face Accelerate or DeepSpeed instead, as they offer more mature distributed training capabilities and broader hardware support.

👥 Team Fit & Use Cases

It is typically used in teams developing smaller-scale custom LLMs for research or product prototypes where local hardware is a constraint, such as personalised chatbots or preference-tuned models. It fits well in environments where engineers want to iterate quickly on fine-tuning without cloud dependency, like internal tools or experimental AI features on edge devices.

🎭 Best For

🏷️ Topics & Ecosystem

cli consumer-gpu dpo fine-tuning gguf huggingface llm llmops local-ai local-llm lora low-vram machine-learning ollama peft python pytorch qlora sft transformers

📊 Activity

Latest commit: 2026-11-20. Over the past 42 days, this repository gained 2.4k stars (+29.4% growth). Activity data is based on daily RepoPi snapshots of the GitHub repository.