RocReport

class RocReport

Calculate the ROC curve for each class.

For each class, plots the ROC curve for a one vs all classification of that class based on the model’s predicted probabilities. In addition, for each class, it marks the optimal probability threshold cut-off point based on Youden’s index defined as sensitivity + specificity - 1. See https://en.wikipedia.org/wiki/Youden%27s_J_statistic for additional details.

Parameters

excluded_classesList , default: None

List of classes to exclude from the calculation. If None, calculate one vs all ROC curve for all classes for multiclass and only the positive class for binary classification. If an empty list was provided, calculate for all classes.

n_samplesint , default: 1_000_000

number of samples to use for this check.

random_stateint, default: 42

random seed for all check internals.

References

Ewald, B. (2006). Post hoc choice of cut points introduced bias to diagnostic research. Journal of clinical epidemiology, 59(8), 798-801.

Steyerberg, E.W., Van Calster, B., & Pencina, M.J. (2011). Performance measures for prediction models and markers: evaluation of predictions and classifications. Revista Espanola de Cardiologia (English Edition), 64(9), 788-794.

Jiménez-Valverde, A., & Lobo, J.M. (2007). Threshold criteria for conversion of probability of species presence to either–or presence–absence. Acta oecologica, 31(3), 361-369.

__init__(excluded_classes: Optional[List] = None, n_samples: int = 1000000, random_state: int = 42, **kwargs)

__new__(*args, **kwargs)

Methods

RocReport.add_condition(name, ...)	Add new condition function to the check.
RocReport.add_condition_auc_greater_than([...])	Add condition - require min allowed AUC score per class.
RocReport.clean_conditions()	Remove all conditions from this check instance.
RocReport.conditions_decision(result)	Run conditions on given result.
RocReport.config([include_version, ...])	Return check configuration (conditions' configuration not yet supported).
RocReport.from_config(conf[, version_unmatch])	Return check object from a CheckConfig object.
RocReport.from_json(conf[, version_unmatch])	Deserialize check instance from JSON string.
RocReport.metadata([with_doc_link])	Return check metadata.
RocReport.name()	Name of class in split camel case.
RocReport.params([show_defaults])	Return parameters to show when printing the check.
RocReport.remove_condition(index)	Remove given condition by index.
RocReport.run(dataset[, model, ...])	Run check.
RocReport.run_logic(context, dataset_kind)	Run check.
RocReport.to_json([indent, include_version, ...])	Serialize check instance to JSON string.

Examples

ROC Report

previous

RegressionSystematicError.to_json

next

RocReport.add_condition