model_evaluation#

model_evaluation(alternative_scorers: Optional[Dict[str, Callable]] = None, columns: Optional[Union[Hashable, List[Hashable]]] = None, ignore_columns: Optional[Union[Hashable, List[Hashable]]] = None, n_top_columns: Optional[int] = None, n_samples: Optional[int] = None, random_state: int = 42, n_to_show: int = 5, **kwargs) → Suite[source]#

Suite for evaluating the model’s performance over different metrics, segments, error analysis, examining overfitting, comparing to baseline, and more.

List of Checks:

List of Checks#
Check Example	API Reference
ROC Report	`RocReport`
Confusion Matrix Report	`ConfusionMatrixReport`
Weak Segments Performance	`WeakSegmentPerformance`
Prediction Drift	`PredictionDrift`
Simple Model Comparison	`SimpleModelComparison`
Calibration Score	`CalibrationScore`
Regression Systematic Error	`RegressionSystematicError`
Regression Error Distribution	`RegressionErrorDistribution`
Unused Features	`UnusedFeatures`
Boosting Overfit	`BoostingOverfit`
Model Inference Time	`ModelInferenceTime`
Prediction Drift	`PredictionDrift`

Parameters

alternative_scorersDict[str, Callable], default: None: An optional dictionary of scorer name to scorer functions. If none given, use default scorers
columnsUnion[Hashable, List[Hashable]] , default: None: The columns to be checked. If None, all columns will be checked except the ones in ignore_columns.
ignore_columnsUnion[Hashable, List[Hashable]] , default: None: The columns to be ignored. If None, no columns will be ignored.
n_top_columnsint , optional: number of columns to show ordered by feature importance (date, index, label are first) (check dependent)
n_samplesint , default: 1_000_000: number of samples to use for checks that sample data. If none, use the default n_samples per check.
random_stateint, default: 42: random seed for all checks.
n_to_showint , default: 5: number of top results to show (check dependent)
**kwargsdict: additional arguments to pass to the checks.

Returns

Suite: A suite for evaluating the model’s performance.

See also

Full Suite Quickstart

Examples

>>> from deepchecks.tabular.suites import model_evaluation
>>> suite = model_evaluation(columns=['a', 'b', 'c'], n_samples=1_000_000)
>>> result = suite.run()
>>> result.show()

run(self, train_dataset: Optional[Union[Dataset, DataFrame]] = None, test_dataset: Optional[Union[Dataset, DataFrame]] = None, model: Optional[BasicModel] = None, feature_importance: Optional[Series] = None, feature_importance_force_permutation: bool = False, feature_importance_timeout: int = 120, with_display: bool = True, y_pred_train: Optional[ndarray] = None, y_pred_test: Optional[ndarray] = None, y_proba_train: Optional[ndarray] = None, y_proba_test: Optional[ndarray] = None, run_single_dataset: Optional[str] = None, model_classes: Optional[List] = None) → SuiteResult#

Run all checks.

Parameters

train_dataset: Optional[Union[Dataset, pd.DataFrame]] , default None: object, representing data an estimator was fitted on
test_datasetOptional[Union[Dataset, pd.DataFrame]] , default None: object, representing data an estimator predicts on
modelOptional[BasicModel] , default None: A scikit-learn-compatible fitted estimator instance
run_single_dataset: Optional[str], default None: ‘Train’, ‘Test’ , or None to run on both train and test.
feature_importance: pd.Series , default: None: pass manual features importance
feature_importance_force_permutationbool , default: False: force calculation of permutation features importance
feature_importance_timeoutint , default: 120: timeout in second for the permutation features importance calculation
y_pred_train: Optional[np.ndarray] , default: None: Array of the model prediction over the train dataset.
y_pred_test: Optional[np.ndarray] , default: None: Array of the model prediction over the test dataset.
y_proba_train: Optional[np.ndarray] , default: None: Array of the model prediction probabilities over the train dataset.
y_proba_test: Optional[np.ndarray] , default: None: Array of the model prediction probabilities over the test dataset.
model_classes: Optional[List] , default: None: For classification: list of classes known to the model

Returns

SuiteResult: All results by all initialized checks

train_test_validation

full_suite