Module 3: ML monitoring for unstructured data

This module covers evaluating and monitoring the production performance for models that use unstructured data, including NLP, LLMs, and embeddings.

This module covers evaluating and monitoring the production performance for models that use unstructured data, including LLM-based systems.

We will cover:

• Why monitoring unstructured data is difficult;

• How to measure text data quality;

• What are text descriptors and how to use them;

• How to deal with embeddings;

• How to deal with multimodal data.

This module includes both a theoretical part and code practice. At the end of this module, you will understand the possible approaches to monitoring ML models that work with texts and other unstructured data.