Выпуск

Why Models Improve by Re-Reading the Same Data

My Weird Prompts · 17 ноября 2026 г. · 19 мин

Repeating the same training data works because each epoch reshuffles mini-batches and takes new gradient descent steps from a different starting point. Exploring multi-epoch training, this discussion covers Muennighoff's four-epoch scaling threshold, grokking's internal reorganization at zero loss, and why under-training is a common fine-tuning trap. It also examines voice LoRA runs on Qwen3-TTS, where falling loss can hide rising room artifacts and inference adapter strength matters as much as epoch count. Key sources: • Muennighoff et al., Scaling Data-Constrained Language Models, v5 revised 2025-06-28. Retrieved 2026-11-17. — https://arxiv.org/abs/2305.16264v5 • When Fine-Tuning Fails and when it Generalises: Role of Data Diversity and Mixed Training in LLM-based TTS. Retrieved 2026-11-17. — https://arxiv.org/pdf/2603.10904 • Wei Jie Chee, Can I adapt Qwen3-TTS without losing control of the voice?, published 2026-02-07, claim TTS-QWEN3-LORA-2026-08 / v1 checked 2026-08-07, with March 2026 update. Retrieved 2026-11-17. — https://instavar.com/research/tts/qwen3-tts-lora-fine-tuning Episode 6658 — myweirdprompts.com/n/6658 (permanent short link: myweirdprompts.com/022083) Listen on Android: MWP FM Player — myweirdprompts.com/apk

Страница выпуска