DeepcheckScorer#

class DeepcheckScorer[source]#

Encapsulate scorer function with extra methods.

Scorer functions are functions used to compute various performance metrics, using the model and data as inputs, rather than the labels and predictions. Scorers are callables with the signature scorer(model, features, y_true). Additional data on scorer functions can be found at https://scikit-learn.org/stable/modules/model_evaluation.html.

Parameters
scorert.Union[str, t.Callable]

sklearn scorer name or callable

model_classes: t.Optional[t.List]

possible classes output for model. None for regression tasks.

namestr, default = None

scorer name

__init__(scorer: Union[str, Callable], model_classes: Optional[List], observed_classes: Optional[List], name: Optional[str] = None)[source]#
__new__(*args, **kwargs)#

Methods

DeepcheckScorer.filter_nulls(dataset)

Return data of dataset without null labels.

DeepcheckScorer.run_on_data_and_label(model, ...)

Run scorer with model, data and labels without null filtering.

DeepcheckScorer.run_on_pred(y_true[, ...])

Run sklearn scorer on the labels and the pred/proba according to scorer type.

DeepcheckScorer.score_perfect(dataset)

Calculate the perfect score of the current scorer for given dataset.

DeepcheckScorer.validate_fitting(model, dataset)

Validate given scorer for the model and dataset.

DeepcheckScorer.validate_scorer_multilabel_output(scores)

Validate output and return scores for the observed classes as well as for the model classes.

previous

MULTICLASS_SCORERS_NON_AVERAGE

next

DeepcheckScorer.filter_nulls