al-folio

a simple whitespace theme for academics


Advancing LLM Capabilities: GFlowNets for Amortized Inference and Enhanced Diversity

Large Language Models (LLMs) have fundamentally transformed artificial intelligence, showcasing unprecedented capabilities in generating human-like text and solving complex problems. However, their practical utility and responsible deployment in real-world scenarios are critically dependent on two key aspects: effective alignment with human preferences and the ability to perform robust, multi-step reasoning. This blogpost explores how Generative Flow Networks (GFlowNets) offer a novel and principled framework to address these challenges, aligning with the broader pursuit of efficient ML by reframing LLM fine-tuning as a distribution-matching problem rather than a reward-maximization one.

31 min read   ·   2025

Learning to Search: Amortized Reasoning in LLMs with GFlowNets

Many tasks we care about—chain‑of‑thought (CoT) reasoning, story infilling, tool‑augmented arithmetic—are instances of intractable posterior inference inside a pretrained LLM. Common fine‑tuning strategies such as supervised learning, PPO‑style RLHF, or DPO chase one high‑reward trajectory and ignore the rest, forfeiting diversity and reliability. This post explains how Generative Flow Networks (GFlowNets) turn LLM fine‑tuning into train‑time search: the model is taught to sample complete reasoning paths with probability proportional to their joint likelihood, thereby amortizing Bayesian inference. We weave together intuition, a toy demo, and results from Hu et al. (ICLR 2024) to show why GFlowNets can be a drop‑in alternative that is (i) more data‑efficient, (ii) more robust to reward miss‑specification, and (iii) naturally enables model‑averaged predictions.

3 min read   ·   2025

a distill-style blog post

an example of a distill-style blog post and main elements

23 min read   ·   2021

a post with code

an example of a blog post with some code

4 min read   ·   2015