LoRA Speedrun: A Leaderboard for Fine-Tuning Techniques
A leaderboard measuring and comparing the speed of techniques.
1What is LoRA Speedrun?
LoRA Speedrun is a leaderboard that measures the fine-tuning times of models like Qwen2.5-1.5B on tasks like GSM8K. Techniques are compared fairly by keeping hardware and tasks constant.
2How Does It Work?
Participants test their techniques on a Modal L40S sandbox for a specific task. Each record is independently run three times on the same hardware and verified. This process is free and open to everyone.
3Current Records
The current record for the GSM8K task is 6 minutes 5 seconds by @Saivineeth147. This technique uses sequence packing and loss masking to achieve higher accuracy and faster results.
4Second Track: SQuAD v1.1
The second track targets the SQuAD v1.1 task with the SmolLM2-1.7B model. This track does not yet have a record and is waiting for its first entry. Both tracks include different model families and task types.
5Why It Matters
LoRA/QLoRA is widely used for fine-tuning in the real world. However, techniques are tested on different models, data, and hardware, making comparisons difficult. This leaderboard provides a fair competitive environment.
6Future and Contribution
LoRA Speedrun can be expanded with more techniques and tracks. Participants can develop new techniques to secure a place on the leaderboard and contribute to the advancement of AI fine-tuning techniques.
Original article: Hacker News — LoRA Hız Yarışı: İnce Ayar Teknikleri için Lider Tablosu