The course belongs to the MA Programme Linguistic Diversity in the Digital Age.

• study track: language technology
• modules: Studies in Language Technology (LDA-T3100), Essentials in Language Technology (LDA-TA500), Comprehensive specialization in Language Technology (LDA-TB500)

This is an optional course.

The course is available to students from other study tracks and degree programmes.

After successfully completing the course, students will be able to:

• explain the common computational vector space models for words applied in language technology.
• understand how words representations are used to generate sentence-based vector models.
• describe the challenges related to word and sentence based vector models.
• know the main neural language models and apply them for different applications.
• read, understand and present scientific papers on current challenges in representation learning for natural language processing.

Students can take this course in year 1 or 2. The course is offered during the spring term in period 4.

• Introduction to neural networks and semantic vector representations
• Common model architectures for word-type models: CBOW, skip-gram, GloVe, fastText
• Cross-lingual word embeddings
• Common models for sentence embeddings and their application in NLP
• Current approaches in language modeling
• Recent topics in representation learning in natural language processing: multimodal learning for distributional representations, compositionality with distributed representations, multilingual representations, analyzing and interpreting neural networks for NLP, etc.

• Weekly lectures and hands-on assignments
• Final assignment with report
• Seminar presentation on current topic

• Dan Jurafsky and James H. Martin (2019, 3rd ed. draft). Speech and Language Processing.
• Yoav Goldberg (2015). A Primer on Neural Network Models for Natural Language Processing.
• Additional material distributed in the course.

Grading follows the standard scale 0 – 5. The following aspects are taken into account in grading:

• Hands-on assignments (⅓)
• Final assignment with report (⅓)
• Seminar presentation on current topic (⅓)

Prerequisites:

• Programming for linguists or equivalent (BA level)

Recommended optional studies:

• Machine learning for linguists or equivalent (BA level)
• Mathematics for linguists or equivalent (BA level)
• Introduction to deep learning or equivalent (MA level)
• Command-line course or equivalent