Data
Long Syntactic Dependencies
Category
Data
Abstraction level
Primitive
Operation level
DataInference
Use cases
Assessing models' syntactic understandingSyntactic parsingDiagnosing difficult linguistic constructions
How it works
Parsing must retain information about a distant element across a long stretch of text; attention mechanisms in models allow directly linking distant tokens without the signal decaying over distance.
Problem solved
How to correctly link sentence elements (e.g. subject and verb) when many words separate them?
Strengths & limitations
Strengths
✓A good test of syntactic representation quality
✓Justifies the advantage of attention mechanisms
Limitations
✗Burdens working memory and lengthens context
✗A source of agreement and interpretation errors
✗Harder in languages with free word order