[Long Review] Finetuned Language Models Are Zero-Shot Learners
Before InstructGPT made instruction tuning a household phrase, Google's FLAN paper made the case that a modest amount of multi-task fine-tuning on natural-language instructions could turn a plain LM into a surprisingly c
Before InstructGPT made instruction tuning a household phrase, Google's FLAN paper made the case that a modest amount of multi-task fine-tuning on natural-language instructions could turn a plain LM into a surprisingly capable zero-shot learner. Highlighted in Jeff Dean's 2021 research recap, FLAN reformulates dozens of NLP tasks as instructions, fine-tunes a 137B-parameter model on them, and then evaluates on held-out task clusters it never saw during training.
This long review works through the task taxonomy, the instruction templates, and the results that show FLAN beating raw GPT-3 zero-shot on a majority of benchmarks. For voice AI engineers designing prompt-driven dialogue systems or building custom LLM back-ends for conversational products, FLAN is essential background on why instruction tuning works, when it hurts smaller models, and what task diversity really buys you. Well worth the time if you want to understand the intellectual lineage running from FLAN through modern chat-tuned assistants.
