Multivariate Drift

.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "user-guide/general/exporting_results/examples/plot_exports_output_to_wandb.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_user-guide_general_exporting_results_examples_plot_exports_output_to_wandb.py: Weights & Biases (wandb) ************************ In this guide, we will demonstrate how to export the Check's and the Suite's output to wandb. This enables to view the exported results in wandb and compare the results you receive with different parameters or model/data. **Structure:** * `Load Data <#load-data>`__ * `Run a Check <#run-a-check>`__ * `Export Check to wandb <#export-a-check-s-output-checkresult-to-wandb>`__ * `Export Suite to wandb <#exporting-a-suite-s-output-suiteresult-to-wandb>`__ .. GENERATED FROM PYTHON SOURCE LINES 20-22 Load Data --------- .. GENERATED FROM PYTHON SOURCE LINES 22-28 .. code-block:: default from deepchecks.tabular.datasets.classification import iris train_dataset, test_dataset = iris.load_data() model = iris.load_fitted_model() .. rst-class:: sphx-glr-script-out .. code-block:: none classification_label value for label type is deprecated, allowed task types are multiclass, binary and regression. .. GENERATED FROM PYTHON SOURCE LINES 29-31 Run a Check ----------- .. GENERATED FROM PYTHON SOURCE LINES 31-37 .. code-block:: default from deepchecks.tabular.checks import MultivariateDrift result = MultivariateDrift().add_condition_overall_drift_value_less_than( ).run(train_dataset, test_dataset, model) .. GENERATED FROM PYTHON SOURCE LINES 38-40 Observe CheckResult Display and Value ------------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 40-43 .. code-block:: default result .. raw:: html

Multivariate Drift

.. GENERATED FROM PYTHON SOURCE LINES 44-45 the value can be observed .. GENERATED FROM PYTHON SOURCE LINES 45-47 .. code-block:: default result.value .. rst-class:: sphx-glr-script-out .. code-block:: none {'domain_classifier_auc': 0.4545454545454546, 'domain_classifier_drift_score': 0, 'domain_classifier_feature_importance': {'petal length (cm)': 1.0, 'sepal length (cm)': 0.0, 'sepal width (cm)': 0.0, 'petal width (cm)': 0.0}} .. GENERATED FROM PYTHON SOURCE LINES 48-62 Export a Check's Output (CheckResult) to wandb ============================================== Exporting the output to wandb is possible using the ``to_wandb`` function. This function exports the check outputs to a wandb project. The output display that will be export will be a bit different from what you usually see. Only the tables and the plots are being exported. On default if you export a single check to wandb without a wandb run active it will create a project with the name deepchecks and the and the check's metadata in the config and export the results there. See Check to_wandb options -------------------------- .. GENERATED FROM PYTHON SOURCE LINES 62-67 .. code-block:: default from deepchecks.core import CheckResult help(CheckResult.to_wandb) .. rst-class:: sphx-glr-script-out .. code-block:: none Help on function to_wandb in module deepchecks.core.check_result: to_wandb(self, **kwargs) Send result to wandb. Parameters ---------- kwargs: Keyword arguments to pass to wandb.init. Default project name is deepchecks. Default config is the check metadata (params, train/test/ name etc.). .. GENERATED FROM PYTHON SOURCE LINES 68-69 To use this function you first need to login to your wandb account .. GENERATED FROM PYTHON SOURCE LINES 69-74 .. code-block:: default import wandb wandb.login() .. rst-class:: sphx-glr-script-out .. code-block:: none False .. GENERATED FROM PYTHON SOURCE LINES 75-78 .. code-block:: default result.to_wandb() .. rst-class:: sphx-glr-script-out .. code-block:: none wandb: Tracking run with wandb version 0.12.21 wandb: W&B syncing is set to `offline` in this directory. wandb: Run `wandb online` or set WANDB_MODE=online to enable cloud syncing. wandb: Waiting for W&B process to finish... (success). wandb: - 0.000 MB of 0.000 MB uploaded (0.000 MB deduped) wandb: \ 0.000 MB of 0.000 MB uploaded (0.000 MB deduped) wandb: wandb: You can sync this run to the cloud by running: wandb: wandb sync /home/runner/work/deepchecks/deepchecks/docs/source/user-guide/general/exporting_results/wandb/offline-run-20221031_160319-vnlcmvbc wandb: Find logs at: ./wandb/offline-run-20221031_160319-vnlcmvbc/logs .. GENERATED FROM PYTHON SOURCE LINES 79-89 Exporting a Suite's Output (SuiteResult) to wandb ================================================= Run Suite and export to wandb ----------------------------- Exporting a suite to wandb is the same as exporting a check. The display in wand will have a section per check. On default if you export a suite to wandb without a wandb run active it will create a project with the name deepchecks and the suite's name in the config and send the results there. .. GENERATED FROM PYTHON SOURCE LINES 89-94 .. code-block:: default from deepchecks.tabular.suites import full_suite suite = full_suite() .. GENERATED FROM PYTHON SOURCE LINES 95-99 .. code-block:: default suite_result = suite.run(train_dataset=train_dataset, test_dataset=test_dataset, model=model) suite_result.to_wandb() .. rst-class:: sphx-glr-script-out .. code-block:: none Full Suite: | | 0/35 [Time: 00:00] Full Suite: |# | 1/35 [Time: 00:00, Check=Train Test Performance] Full Suite: |#### | 4/35 [Time: 00:00, Check=Train Test Prediction Drift] Full Suite: |###### | 6/35 [Time: 00:02, Check=Weak Segments Performance] Full Suite: |########### | 11/35 [Time: 00:02, Check=Model Inference Time] Full Suite: |#################### | 20/35 [Time: 00:02, Check=Feature Label Correlation Change] Full Suite: |######################### | 25/35 [Time: 00:02, Check=Special Characters] Full Suite: |################################# | 33/35 [Time: 00:02, Check=Feature Label Correlation] wandb: Tracking run with wandb version 0.12.21 wandb: W&B syncing is set to `offline` in this directory. wandb: Run `wandb online` or set WANDB_MODE=online to enable cloud syncing. wandb: Waiting for W&B process to finish... (success). wandb: - 0.000 MB of 0.000 MB uploaded (0.000 MB deduped) wandb: \ 0.000 MB of 0.000 MB uploaded (0.000 MB deduped) wandb: wandb: You can sync this run to the cloud by running: wandb: wandb sync /home/runner/work/deepchecks/deepchecks/docs/source/user-guide/general/exporting_results/wandb/offline-run-20221031_160327-30n37z4j wandb: Find logs at: ./wandb/offline-run-20221031_160327-30n37z4j/logs .. GENERATED FROM PYTHON SOURCE LINES 100-101 You can also set all the kwargs the wandb.init will get: .. GENERATED FROM PYTHON SOURCE LINES 101-103 .. code-block:: default suite_result.to_wandb(project='my-suite-project', config={'suite-name': 'full-suite'}) .. rst-class:: sphx-glr-script-out .. code-block:: none wandb: Tracking run with wandb version 0.12.21 wandb: W&B syncing is set to `offline` in this directory. wandb: Run `wandb online` or set WANDB_MODE=online to enable cloud syncing. wandb: Waiting for W&B process to finish... (success). wandb: - 0.000 MB of 0.000 MB uploaded (0.000 MB deduped) wandb: \ 0.000 MB of 0.000 MB uploaded (0.000 MB deduped) wandb: wandb: You can sync this run to the cloud by running: wandb: wandb sync /home/runner/work/deepchecks/deepchecks/docs/source/user-guide/general/exporting_results/wandb/offline-run-20221031_160333-1d5kj07b wandb: Find logs at: ./wandb/offline-run-20221031_160333-1d5kj07b/logs .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 22.538 seconds) .. _sphx_glr_download_user-guide_general_exporting_results_examples_plot_exports_output_to_wandb.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_exports_output_to_wandb.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_exports_output_to_wandb.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_