Natural Language Generation

Practice Exam Review

Final Exam Review

In this module, we'll continue our exploration of linguistic analysis of sentences rather than focusing on the structure of sentences like we did on the parsing module. To do so, we'll cover logical representations of sentence meaning, semantic role labeling and information extraction.

In this module, we'll delve more into the linguistics side of natural language processing. We'll take a look at different approaches to parsing and learn about the structure of sentences from a linguistics perspective.

In this module, we will delve into two related NLP topics: dialogue systems and question answering.

In this module, we will go over machine translation, one of the most important NLP applications, the challenges it involves, and how to evaluate machine translation models.

In this module, we will delve into some of the capabilities of cutting edge pre-trained language models. We will explore the vital concepts of prompt engineering and instruction following. We'll first discuss the pre-train, prompt and predict paradigm, a new approach which questions whether fine tuning is even necessary. After. we'll cover several advanced prompting techniques such as few shot learning, instruction following and chain of thought prompting. These advanced techniques will provide us with a more efficient and more creative ways to harness the full potential of pre-trained language models.

In this module, we will cover encoder-decoder models, BERT, fine-tuning and masked language models. Understanding them will give you a good understanding of state-of-the-art NLP models, and why pre-trained large language models have become so important.

In this module, we will cover transformers and pre-trained language models, and text generation. For the latter section, we will be joined by guest lecturer and Penn PhD graduate, Dr. Daphne Ippolito.

In this module, we're going to cover part of speech tagging. This is a fundamental task in natural language processing and has traditionally been used in a variety of applications. We'll cover some of the models that are used for part of speech tagging, going into some depth into hidden Markov models and dynamic programing, which is a technique widely useful to avoid exponential complexity and discrete optimization problems.

In this module, we'll take a look at neural network based language models, which, unlike the previous N-gram based language models that we looked at earlier, use word embedding based representations for their contexts. This allows them to make much better probabilistic prediction about the next word in a sequence, and they have become the foundation for large pre-trained language models like Chat GPT that have led to exciting innovations in the field of NLP.

This week, we will continue our exploration of vector space semantics and embeddings. We'll begin the module by wrapping up word embeddings and discussing bias in vector space models. Then, we'll discuss a variety of goals that any representation of word meaning should aim to achieve. These six goals will help us understand different aspects of word meaning and the relationships of words with other words. Then, we'll pivot to a coding demo that will provide you with a hands on experience working with vector space models and see how word embeddings can be used to retrieve words with similar meaning and to solve word analogy tasks.

In this module, we'll begin to explore vector space semantics in natural language processing. (This will continue into next week.) Vector space semantics are powerful because they allow us to represent words in a way that allows us to measure similarity between words and capture several other kinds of meaning. We'll start this module by exploring important concepts that underpin this topic, like the distributional hypothesis and term-by-document matrices, and then switch to cover a recent approach to vector space models called word embeddings

In this module, we are going to cover essential topics that will allow us to move into important tasks in NLP: a review of probability and defining a probabilistic model. We will then delve into one of the simpler language models, the N-gram language model.

In this module, we'll begin with delving into text preprocessing. We'll go through tasks that transform an unstructured text into a structured format that we can analyze via machine learning. Once we've preprocessed our text data, we can then move on to building machine learning models for text classification tasks. To do this, we'll introduce logistic regression, which is a popular algorithm for text classification, and explain the concept of gradient descent.

In this module, we’ll get started by looking at a classic natural language processing problem: text classification. Using the example of sentiment analysis, where we can determine the emotions and attitudes that an author expresses towards the subject of their writing, we will delve into classifying text. Also this week, we will introduce regular expressions, a powerful tool for natural language processing.