.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "checks_gallery/tabular/train_test_validation/plot_train_test_label_drift.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_checks_gallery_tabular_train_test_validation_plot_train_test_label_drift.py: Train Test Label Drift ********************** This notebooks provides an overview for using and understanding label drift check. **Structure:** * `What Is Label Drift? <#what-is-label-drift>`__ * `Run Check on a Classification Label <#run-check-on-a-classification-label>`__ * `Run Check on a Regression Label <#run-check-on-a-regression-label>`__ * `Add a Condition <#run-check>`__ What Is Label Drift? ======================== Drift is simply a change in the distribution of data over time, and it is also one of the top reasons why machine learning model's performance degrades over time. Label drift is when drift occurs in the label itself. For more information on drift, please visit our :doc:`drift guide `. How Deepchecks Detects Label Drift ------------------------------------ This check detects label drift by using :ref:`univariate measures ` on the label column. .. GENERATED FROM PYTHON SOURCE LINES 34-43 .. code-block:: default import pprint import numpy as np import pandas as pd from deepchecks.tabular import Dataset from deepchecks.tabular.checks import TrainTestLabelDrift .. GENERATED FROM PYTHON SOURCE LINES 44-46 Run Check on a Classification Label ==================================== .. GENERATED FROM PYTHON SOURCE LINES 46-62 .. code-block:: default # Generate data: # -------------- np.random.seed(42) train_data = np.concatenate([np.random.randn(1000,2), np.random.choice(a=[1,0], p=[0.5, 0.5], size=(1000, 1))], axis=1) #Create test_data with drift in label: test_data = np.concatenate([np.random.randn(1000,2), np.random.choice(a=[1,0], p=[0.35, 0.65], size=(1000, 1))], axis=1) df_train = pd.DataFrame(train_data, columns=['col1', 'col2', 'target']) df_test = pd.DataFrame(test_data, columns=['col1', 'col2', 'target']) train_dataset = Dataset(df_train, label='target') test_dataset = Dataset(df_test, label='target') .. rst-class:: sphx-glr-script-out Out: .. code-block:: none It is recommended to initialize Dataset with categorical features by doing "Dataset(df, cat_features=categorical_list)". No categorical features were passed, therefore heuristically inferring categorical features in the data. 0 categorical features were inferred .. GENERATED FROM PYTHON SOURCE LINES 63-66 .. code-block:: default df_train.head() .. raw:: html
col1 col2 target
0 0.496714 -0.138264 1.0
1 0.647689 1.523030 1.0
2 -0.234153 -0.234137 1.0
3 1.579213 0.767435 1.0
4 -0.469474 0.542560 0.0


.. GENERATED FROM PYTHON SOURCE LINES 67-69 Run Check =============================== .. GENERATED FROM PYTHON SOURCE LINES 69-74 .. code-block:: default check = TrainTestLabelDrift() result = check.run(train_dataset=train_dataset, test_dataset=test_dataset) result .. raw:: html
Train Test Label Drift


.. GENERATED FROM PYTHON SOURCE LINES 75-77 Run Check on a Regression Label ================================ .. GENERATED FROM PYTHON SOURCE LINES 77-92 .. code-block:: default # Generate data: # -------------- train_data = np.concatenate([np.random.randn(1000,2), np.random.randn(1000, 1)], axis=1) test_data = np.concatenate([np.random.randn(1000,2), np.random.randn(1000, 1)], axis=1) df_train = pd.DataFrame(train_data, columns=['col1', 'col2', 'target']) df_test = pd.DataFrame(test_data, columns=['col1', 'col2', 'target']) #Create drift in test: df_test['target'] = df_test['target'].astype('float') + abs(np.random.randn(1000)) + np.arange(0, 1, 0.001) * 4 train_dataset = Dataset(df_train, label='target') test_dataset = Dataset(df_test, label='target') .. rst-class:: sphx-glr-script-out Out: .. code-block:: none It is recommended to initialize Dataset with categorical features by doing "Dataset(df, cat_features=categorical_list)". No categorical features were passed, therefore heuristically inferring categorical features in the data. 0 categorical features were inferred .. GENERATED FROM PYTHON SOURCE LINES 93-95 Run check --------- .. GENERATED FROM PYTHON SOURCE LINES 95-100 .. code-block:: default check = TrainTestLabelDrift() result = check.run(train_dataset=train_dataset, test_dataset=test_dataset) result .. raw:: html
Train Test Label Drift


.. GENERATED FROM PYTHON SOURCE LINES 101-103 Add a Condition =============== .. GENERATED FROM PYTHON SOURCE LINES 103-106 .. code-block:: default check_cond = TrainTestLabelDrift().add_condition_drift_score_not_greater_than() check_cond.run(train_dataset=train_dataset, test_dataset=test_dataset) .. raw:: html
Train Test Label Drift


.. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 0.360 seconds) .. _sphx_glr_download_checks_gallery_tabular_train_test_validation_plot_train_test_label_drift.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_train_test_label_drift.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_train_test_label_drift.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_