> For the complete documentation index, see [llms.txt](https://learn.evidentlyai.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://learn.evidentlyai.com/ml-observability-course/module-4-designing-effective-ml-monitoring.md).

# Module 4: Designing effective ML monitoring

In previous modules, we reviewed possible metrics and approaches to tracking the performance of ML models in production.

In this module, we will put it all together and review specific questions you might have when setting up an ML monitoring system for a particular model. We’ll do a deeper dive and cover:

* How to select and prioritize ML monitoring metrics.
* How and when to retrain ML models.
* How to choose a reference dataset.
* How to implement custom metrics in ML monitoring.
* How to choose an appropriate ML monitoring architecture.

At the end of this module, you will understand how to design an optimal approach to ML monitoring considering the model risks, criticality, and deployment scenario.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://learn.evidentlyai.com/ml-observability-course/module-4-designing-effective-ml-monitoring.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.