PerformanceReport#
- class PerformanceReport[source]#
Summarize given scores on a dataset and model.
- Parameters
- alternative_scorersDict[str, Callable], default: None
An optional dictionary of scorer name to scorer functions. If none given, using default scorers
Notes
Scorers are a convention of sklearn to evaluate a model. See scorers documentation A scorer is a function which accepts (model, X, y_true) and returns a float result which is the score. For every scorer higher scores are better than lower scores.
You can create a scorer out of existing sklearn metrics:
from sklearn.metrics import roc_auc_score, make_scorer training_labels = [1, 2, 3] auc_scorer = make_scorer(roc_auc_score, labels=training_labels, multi_class='ovr') # Note that the labels parameter is required for multi-class classification in metrics like roc_auc_score or # log_loss that use the predict_proba function of the model, in case that not all labels are present in the test # set.
Or you can implement your own:
from sklearn.metrics import make_scorer def my_mse(y_true, y_pred): return (y_true - y_pred) ** 2 # Mark greater_is_better=False, since scorers always suppose to return # value to maximize. my_mse_scorer = make_scorer(my_mse, greater_is_better=False)
- __new__(*args, **kwargs)#
Methods
|
Add new condition function to the check. |
|
Add condition. |
|
Add condition - metric scores are not less than given score. |
|
Add condition that will check that test performance is not degraded by more than given percentage in train. |
Remove all conditions from this check instance. |
|
Run conditions on given result. |
|
Finalize the check result by adding the check instance and processing the conditions. |
|
|
Return check metadata. |
Name of class in split camel case. |
|
|
Return parameters to show when printing the check. |
Remove given condition by index. |
|
|
Run check. |
|
Run check. |