Calibration Metric

.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "checks_gallery/tabular/model_evaluation/plot_calibration_score.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_checks_gallery_tabular_model_evaluation_plot_calibration_score.py: .. _plot_tabular_calibration_score: Calibration Score ***************** .. GENERATED FROM PYTHON SOURCE LINES 10-28 .. code-block:: default import warnings import pandas as pd from sklearn.datasets import load_iris from sklearn.linear_model import LogisticRegression from sklearn.model_selection import train_test_split from deepchecks.tabular.checks import CalibrationScore from deepchecks.tabular.datasets.classification import adult def custom_formatwarning(msg, *args, **kwargs): # ignore everything except the message return str(msg) + '\n' warnings.formatwarning = custom_formatwarning .. GENERATED FROM PYTHON SOURCE LINES 29-38 Binary Classification ===================== Load data --------- The dataset is the adult dataset which can be downloaded from the UCI machine learning repository. Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science. .. GENERATED FROM PYTHON SOURCE LINES 38-45 .. code-block:: default from urllib.request import urlopen from sklearn.preprocessing import LabelEncoder label_name = 'income' .. GENERATED FROM PYTHON SOURCE LINES 46-51 .. code-block:: default from deepchecks.tabular import Dataset train_ds, test_ds = adult.load_data() .. GENERATED FROM PYTHON SOURCE LINES 52-54 .. code-block:: default model = adult.load_fitted_model() .. GENERATED FROM PYTHON SOURCE LINES 55-59 .. code-block:: default check = CalibrationScore() check.run(test_ds, model) .. raw:: html

Calibration Metric

.. GENERATED FROM PYTHON SOURCE LINES 60-62 Multi-class classification ========================== .. GENERATED FROM PYTHON SOURCE LINES 62-74 .. code-block:: default iris = load_iris(as_frame=True) clf = LogisticRegression() frame = iris.frame X = iris.data y = iris.target X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=55) clf.fit(X_train, y_train) ds = Dataset(pd.concat([X_test, y_test], axis=1), features=iris.feature_names, label='target') .. GENERATED FROM PYTHON SOURCE LINES 75-78 .. code-block:: default check = CalibrationScore() check.run(ds, clf) .. raw:: html

Calibration Metric

.. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 5.975 seconds) .. _sphx_glr_download_checks_gallery_tabular_model_evaluation_plot_calibration_score.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_calibration_score.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_calibration_score.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_