Syntax-sensitive compositional distributional semantics

What is it about?

This paper outlines a theory for composing distributional representations of words which is syntax-sensitive. In our approach, co-occurrence features are typed using higher-order dependency paths. For example, in "The small boy kicks the ball", the word 'boy' co-occurs with 'kicks' in the first order dependency of 'subject of' and with the word 'small' in the first order dependency of 'modified-by'. However, there is also a second-order dependency path between 'small' and 'kicks' which is 'modifies something which is a subject of'. By aligning distributional representations according to the syntax of the phrase being composed, higher order dependency relationships can be used to predict the co-occurrences of other compositional phrases.

Featured Image

Why is it important?

Most current work on compositional distributional semantics relies on operations which are commutative and insensitive to word order or structure. Therefore the representation of "window glass" is the same as "glass window" and the representation of "The dog bites the man." is the same as "The man bites the dog." This is not true in our approach due to the different dependency relations in each case. Therefore it is possible to make more subtle semantic distinctions, leading to better natural language understanding.