Which item is listed as missing from n-gram models to resemble large language models?

Which item is listed as missing from n-gram models to resemble large language models?

• A. Transformer architecture
• B. Scale
• C. Post-training
• D. Data augmentation

Answer: (C)

Post-training is the step that truly separates basic language models from large language models. N-gram models are simple probability estimators built from counting word sequences; they have no learning phase beyond those counts and no subsequent refinement stage. Large language models, on the other hand, undergo a two-stage process: a broad pretraining on massive text to learn general language patterns, followed by a post-training phase that tunes the model for specific behavior—such as instruction following, alignment with human preferences, and safety constraints—through fine-tuning and sometimes reinforcement learning from human feedback. This post-training stage is what endows LLMs with the ability to understand and follow complex prompts, adapt to different tasks, and produce coherent, context-aware responses.

Post-training is the step that truly separates basic language models from large language models. N-gram models are simple probability estimators built from counting word sequences; they have no learning phase beyond those counts and no subsequent refinement stage. Large language models, on the other hand, undergo a two-stage process: a broad pretraining on massive text to learn general language patterns, followed by a post-training phase that tunes the model for specific behavior—such as instruction following, alignment with human preferences, and safety constraints—through fine-tuning and sometimes reinforcement learning from human feedback. This post-training stage is what endows LLMs with the ability to understand and follow complex prompts, adapt to different tasks, and produce coherent, context-aware responses.