# Module 4: Designing effective ML monitoring

In previous modules, we reviewed possible metrics and approaches to tracking the performance of ML models in production.

In this module, we will put it all together and review specific questions you might have when setting up an ML monitoring system for a particular model. We’ll do a deeper dive and cover:

* How to select and prioritize ML monitoring metrics.
* How and when to retrain ML models.
* How to choose a reference dataset.
* How to implement custom metrics in ML monitoring.
* How to choose an appropriate ML monitoring architecture.

At the end of this module, you will understand how to design an optimal approach to ML monitoring considering the model risks, criticality, and deployment scenario.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://learn.evidentlyai.com/ml-observability-course/module-4-designing-effective-ml-monitoring.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.