Calibration Score#

import warnings

import pandas as pd
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

from deepchecks.tabular.checks import CalibrationScore
from deepchecks.tabular.datasets.classification import adult

def custom_formatwarning(msg, *args, **kwargs):
    # ignore everything except the message
    return str(msg) + '\n'

warnings.formatwarning = custom_formatwarning

Binary Classification#

Load data#

The dataset is the adult dataset which can be downloaded from the UCI machine learning repository.

Dua, D. and Graff, C. (2019). UCI Machine Learning Repository []. Irvine, CA: University of California, School of Information and Computer Science.

from urllib.request import urlopen

from sklearn.preprocessing import LabelEncoder

label_name = 'income'
from deepchecks.tabular import Dataset

train_ds, test_ds = adult.load_data()
Calibration Metric

Multi-class classification#

iris = load_iris(as_frame=True)
clf = LogisticRegression()
frame = iris.frame
X =
y =
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=55), y_train)
ds = Dataset(pd.concat([X_test, y_test], axis=1),
check = CalibrationScore(), clf)
Calibration Metric

Total running time of the script: ( 0 minutes 3.708 seconds)

Gallery generated by Sphinx-Gallery