Focus metric configurable alert

It would be great to be able to configure an alert for a selected metric, a selected alert threshold and a selected number of failed turns. E.g. switch automatically to monitor mode for edge similarity only when score gets below 60% for 3 turns in a row. That would allow you focus on something else and still be able to detect asap regression on the alert metric.