What is a transformer architecture notable for in NLP?

What is a transformer architecture notable for in NLP?

• A. A model that processes sequences strictly left-to-right.
• B. A neural network architecture that uses attention to process sequences in parallel.
• C. A CNN variant for image processing only.
• D. A decision tree for text classification.

Answer: (B)

Transformers in NLP are defined by their use of attention to process sequences in parallel. This means the model looks at all positions in the input simultaneously and weighs how much each token should influence others when forming representations. That attention mechanism lets the model capture dependencies across long spans of text without having to read the sequence strictly from start to finish, which enables much faster training and better handling of context than sequential models. Why this matters: because every token can attend to every other token, the model can learn relationships, themes, and long-range dependencies more effectively. This parallelizable attention is a core reason transformers scaled up so well and became the foundation for many large language models. The other options don’t fit because they describe approaches that don’t match how transformers operate. Processing strictly left-to-right characterizes many older sequence models and oversimplifies transformers (in many uses, especially in encoding, attention across the whole sequence is key). A CNN variant for image processing is a different architecture geared toward spatial data, not the language-focused attention mechanism. A decision tree is a completely different, non-neural approach for text classification.

Transformers in NLP are defined by their use of attention to process sequences in parallel. This means the model looks at all positions in the input simultaneously and weighs how much each token should influence others when forming representations. That attention mechanism lets the model capture dependencies across long spans of text without having to read the sequence strictly from start to finish, which enables much faster training and better handling of context than sequential models.

Why this matters: because every token can attend to every other token, the model can learn relationships, themes, and long-range dependencies more effectively. This parallelizable attention is a core reason transformers scaled up so well and became the foundation for many large language models.

The other options don’t fit because they describe approaches that don’t match how transformers operate. Processing strictly left-to-right characterizes many older sequence models and oversimplifies transformers (in many uses, especially in encoding, attention across the whole sequence is key). A CNN variant for image processing is a different architecture geared toward spatial data, not the language-focused attention mechanism. A decision tree is a completely different, non-neural approach for text classification.