University of Calgary
UofC Navigation

Word Embedding Models from a Cognitive and Computational Linguistics perspective

Date & Time:
January 18, 2019 | 3:00 pm
Location:
CHE 212
Speaker:
Fatemeh Torabi Asr, Post-doctoral fellow at the Discourse Processing Lab, Department of Linguistics (Simon Fraser University)

 
Abstract:

Word embeddings obtained from neural networks trained on big text corpora have become popular representations of word meaning in computational linguistics. The most popular models such as Word2Vec and GloVe generate two sets of embeddings, i.e., word and context embeddings during training. However, context vectors are usually discarded after training and not used in many applications. We demonstrate how these two layers of distributional representation should be interpreted and used in predicting taxonomic similarity vs. asymmetric association between words.  Our study is composed of a set of artificial language experiments as well as evaluations based on word similarity and relatedness datasets collected through crowdsourcing and psycholinguistic experiments. In particular, we use two datasets obtained from experiments on human subjects: SimLex-999 (Hills et al. 2016) including explicitly instructed ratings for word similarity, and explicitly instructed production norms (Jouravlev & McRae, 2016) for word relatedness. We find that people respond with words closer to the cue within the context embedding space (rather than the word embedding space) when they are explicitly asked to generate thematically related words. Taxonomic similarity ratings are however better predicted by word embeddings alone. This suggests that the distributional information encoded in different layers of the neural network reflect different aspects of word meaning. Our experiments also elaborate on word embeddings as a model of human lexical memory by showing that both types of semantic relations among words are encoded within a unified network through reinforcement learning.
 

Fatemeh Torabi Asr

I am currently a post-doctoral fellow at the Discourse Processing Lab within the Department of Linguistics at Simon Fraser University. With a background in software engineering and research experience in the area of cognitive computing and computational linguistics (NLP), I aim at understanding semantic processing in human brain and developing language-intelligent systems that benefit human society. My current research is focused on social media and news text analysis for the purposes of misinformation detection and gender gap tracking in media.

See an overview of my professional journey here!