https://111qqz.com/en/tags/sft/Recent content in SFT on 111qqz's blogHugo -- gohugo.ioenhust.111qqz@gmail.com (111qqz)hust.111qqz@gmail.com (111qqz)© 2011-2026 111qqzMon, 19 Oct 2026 20:40:00 +0800https://111qqz.com/en/post/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/2026-10-19-pretraining-to-post-training/Mon, 19 Oct 2026 20:40:00 +0800hust.111qqz@gmail.com (111qqz)https://111qqz.com/en/post/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/2026-10-19-pretraining-to-post-training/<h2 class="relative group">起因 <div id="起因" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e8%b5%b7%e5%9b%a0" aria-label="Anchor">#</a> </span> </h2> <p>最近继续跟 MIT 6.S191，加上工作中或多或少接触到一些 LLM 训练相关的讨论，我开始试着把大模型的训练流程从头到尾理一遍。</p> <p>之前写过几篇笔记：<a href="https://111qqz.com/en/post/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/2026-09-08-next-token-prediction/" >NTP 那篇</a>推导了 next-token prediction 的条件概率和 cross entropy loss，<a href="https://111qqz.com/en/post/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/2026-09-26-training-signal-learning-paradigms/" >训练信号来源那篇</a>整理了 self-supervised pretraining → representation → fine-tuning 的基本链路。</p>