The course belongs to the MA Programme Linguistic Diversity in the Digital Age.
- study track: language technology
- modules: Studies in Language Technology (LDA-T3100), Essentials in Language Technology (LDA-TA500), Comprehensive specialization in Language Technology (LDA-TB500)
This is an optional course.
The course is available to students from other study tracks and degree programmes.
After successfully completing the course, students will be able to:
- explain the common computational vector space models for words applied in language technology.
- understand how words representations are used to generate sentence-based vector models.
- describe the challenges related to word and sentence based vector models.
- know the main neural language models and apply them for different applications.
- read, understand and present scientific papers on current challenges in representation learning for natural language processing.
Students can take this course in year 1 or 2. The course is offered during the spring term in period 4.
- Introduction to neural networks and semantic vector representations
- Common model architectures for word-type models: CBOW, skip-gram, GloVe, fastText
- Cross-lingual word embeddings
- Common models for sentence embeddings and their application in NLP
- Current approaches in language modeling
- Recent topics in representation learning in natural language processing: multimodal learning for distributional representations, compositionality with distributed representations, multilingual representations, analyzing and interpreting neural networks for NLP, etc.
- Weekly lectures and hands-on assignments
- Final assignment with report
- Seminar presentation on current topic
- Dan Jurafsky and James H. Martin (2019, 3rd ed. draft). Speech and Language Processing.
- Yoav Goldberg (2015). A Primer on Neural Network Models for Natural Language Processing.
- Additional material distributed in the course.
Grading follows the standard scale 0 – 5. The following aspects are taken into account in grading:
- Hands-on assignments (⅓)
- Final assignment with report (⅓)
- Seminar presentation on current topic (⅓)
- Programming for linguists or equivalent (BA level)
Recommended optional studies:
- Machine learning for linguists or equivalent (BA level)
- Mathematics for linguists or equivalent (BA level)
- Introduction to deep learning or equivalent (MA level)
- Command-line course or equivalent