.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "checks_gallery/vision/model_evaluation/plot_single_dataset_performance.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_checks_gallery_vision_model_evaluation_plot_single_dataset_performance.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_checks_gallery_vision_model_evaluation_plot_single_dataset_performance.py:


.. _plot_vision_single_dataset_performance:

Single Dataset Performance
*********************************

This notebooks provides an overview for using and understanding single dataset performance check.

**Structure:**

* `What Is the Purpose of the Check? <#what-is-the-purpose-of-the-check>`__
* `Generate data an model <#generate-data-and-model>`__
* `Run the check <#run-the-check>`__
* `Define a condition <#define-a-condition>`__

What Is the Purpose of the Check?
=================================
This check returns the results from a dict of metrics, in the format metric name: scorer, calculated for the
given model dataset. The scorer can be an ignite.Metric or Sklearn scorer.
Use this check to evaluate the performance on a single vision dataset such as a test set.

.. GENERATED FROM PYTHON SOURCE LINES 25-27

Generate data and model
-----------------------

.. GENERATED FROM PYTHON SOURCE LINES 27-31

.. code-block:: default


    from deepchecks.vision.checks import SingleDatasetPerformance
    from deepchecks.vision.datasets.classification import mnist








.. GENERATED FROM PYTHON SOURCE LINES 32-37

.. code-block:: default



    mnist_model = mnist.load_model()
    train_ds = mnist.load_dataset(train=True, object_type='VisionData')








.. GENERATED FROM PYTHON SOURCE LINES 38-45

Run the check
-------------
We will run the check with the model defined above.

The check will use the default classification metrics - `Precision
<https://pytorch.org/ignite/generated/ignite.metrics.precision.Precision.html#ignite.metrics.precision.Precision>`__.
and `Recall <https://pytorch.org/ignite/generated/ignite.metrics.recall.Recall.html#ignite.metrics.recall.Recall>`__.

.. GENERATED FROM PYTHON SOURCE LINES 45-51

.. code-block:: default



    check = SingleDatasetPerformance()
    result = check.run(train_ds, mnist_model)
    result





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Validating Input:
    |     | 0/1 [Time: 00:00]
    Validating Input:
    |#####| 1/1 [Time: 00:00]

    Ingesting Batches:
    |                                                                                                                                                             | 0/157 [Time: 00:00]

    Ingesting Batches:
    |###############                                                                                                                                              | 15/157 [Time: 00:00]

    Ingesting Batches:
    |##############################                                                                                                                               | 30/157 [Time: 00:00]

    Ingesting Batches:
    |##############################################                                                                                                               | 46/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################                                                                                                | 61/157 [Time: 00:00]

    Ingesting Batches:
    |############################################################################                                                                                 | 76/157 [Time: 00:00]

    Ingesting Batches:
    |###########################################################################################                                                                  | 91/157 [Time: 00:00]

    Ingesting Batches:
    |##########################################################################################################                                                   | 106/157 [Time: 00:00]

    Ingesting Batches:
    |##########################################################################################################################                                   | 122/157 [Time: 00:00]

    Ingesting Batches:
    |##########################################################################################################################################                   | 138/157 [Time: 00:00]

    Ingesting Batches:
    |##########################################################################################################################################################   | 154/157 [Time: 00:01]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]


    Computing Check:
    |     | 0/1 [Time: 00:00]


    Computing Check:
    |#####| 1/1 [Time: 00:00]
                                           

                                                                                                                                                                                                            


                                              

.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Single Dataset Performance</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"0d1f752902234c5ca77904531ef32ffd": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_ff546ac92d504eb58af5440f0ae7074b", "IPY_MODEL_fa6e9e03583d447fb973ce402a262e96", "IPY_MODEL_dda8af89c4ec45199bddf703849c4fb2", "IPY_MODEL_61fc08f9ee4542e8ab2494d48502af4b"], "layout": "IPY_MODEL_05507d9b697e47d1b62d55ee7d51f9a2"}}, "ff546ac92d504eb58af5440f0ae7074b": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_34e9512a02af4a6d8677dd342099f5c0", "style": "IPY_MODEL_1ae3eee16f14430585f670eec4452175", "value": "<h4><b>Single Dataset Performance</b></h4>"}}, "34e9512a02af4a6d8677dd342099f5c0": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "1ae3eee16f14430585f670eec4452175": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "fa6e9e03583d447fb973ce402a262e96": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_48e4309d984e46e68dde623253868cce", "style": "IPY_MODEL_495865c6a5b24f4983cd1f84e01d6488", "value": "<p>Calculate performance metrics of a given model on a given dataset. <a href=\"https://docs.deepchecks.com/0.9/checks_gallery/vision/model_evaluation/plot_single_dataset_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "48e4309d984e46e68dde623253868cce": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "495865c6a5b24f4983cd1f84e01d6488": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "dda8af89c4ec45199bddf703849c4fb2": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_ff92bc0f8f3d4d7da530a87b6f63775b", "style": "IPY_MODEL_bd568883c7e543e2ba49f57f0d04891e"}}, "ff92bc0f8f3d4d7da530a87b6f63775b": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "bd568883c7e543e2ba49f57f0d04891e": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "61fc08f9ee4542e8ab2494d48502af4b": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_027029d12d9740198188535d3aad1da4", "IPY_MODEL_a532341b02564ea49250d4ee0f08771b", "IPY_MODEL_3fa1824ad8c84eac8570365cc1f71b80"], "layout": "IPY_MODEL_6ff144401e8e4a6eb3ff52a8391ccd3c"}}, "027029d12d9740198188535d3aad1da4": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_4bae242b6b3e4a558d8ad47d8e74b08b", "style": "IPY_MODEL_815bf674ea8c456ab3d445a3c7d71180", "value": "<h5><b>Additional Outputs</b></h5>"}}, "4bae242b6b3e4a558d8ad47d8e74b08b": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "815bf674ea8c456ab3d445a3c7d71180": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "a532341b02564ea49250d4ee0f08771b": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_11d2c7944af44fa69be10ee4b6c83544", "style": "IPY_MODEL_fe0b15ea7d23467b89c1c521c893e858", "value": "<style type=\"text/css\">\n#T_59080_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_59080_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_59080_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_59080_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_59080_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_59080_\">\n  <thead>\n    <tr>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >Metric</th>\n      <th class=\"col_heading level0 col1\" >Class</th>\n      <th class=\"col_heading level0 col2\" >Class Name</th>\n      <th class=\"col_heading level0 col3\" >Value</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_59080_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n      <td id=\"T_59080_row0_col0\" class=\"data row0 col0\" >Precision</td>\n      <td id=\"T_59080_row0_col1\" class=\"data row0 col1\" >0</td>\n      <td id=\"T_59080_row0_col2\" class=\"data row0 col2\" >0</td>\n      <td id=\"T_59080_row0_col3\" class=\"data row0 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n      <td id=\"T_59080_row1_col0\" class=\"data row1 col0\" >Precision</td>\n      <td id=\"T_59080_row1_col1\" class=\"data row1 col1\" >1</td>\n      <td id=\"T_59080_row1_col2\" class=\"data row1 col2\" >1</td>\n      <td id=\"T_59080_row1_col3\" class=\"data row1 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n      <td id=\"T_59080_row2_col0\" class=\"data row2 col0\" >Precision</td>\n      <td id=\"T_59080_row2_col1\" class=\"data row2 col1\" >2</td>\n      <td id=\"T_59080_row2_col2\" class=\"data row2 col2\" >2</td>\n      <td id=\"T_59080_row2_col3\" class=\"data row2 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n      <td id=\"T_59080_row3_col0\" class=\"data row3 col0\" >Precision</td>\n      <td id=\"T_59080_row3_col1\" class=\"data row3 col1\" >3</td>\n      <td id=\"T_59080_row3_col2\" class=\"data row3 col2\" >3</td>\n      <td id=\"T_59080_row3_col3\" class=\"data row3 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n      <td id=\"T_59080_row4_col0\" class=\"data row4 col0\" >Precision</td>\n      <td id=\"T_59080_row4_col1\" class=\"data row4 col1\" >4</td>\n      <td id=\"T_59080_row4_col2\" class=\"data row4 col2\" >4</td>\n      <td id=\"T_59080_row4_col3\" class=\"data row4 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n      <td id=\"T_59080_row5_col0\" class=\"data row5 col0\" >Precision</td>\n      <td id=\"T_59080_row5_col1\" class=\"data row5 col1\" >5</td>\n      <td id=\"T_59080_row5_col2\" class=\"data row5 col2\" >5</td>\n      <td id=\"T_59080_row5_col3\" class=\"data row5 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n      <td id=\"T_59080_row6_col0\" class=\"data row6 col0\" >Precision</td>\n      <td id=\"T_59080_row6_col1\" class=\"data row6 col1\" >6</td>\n      <td id=\"T_59080_row6_col2\" class=\"data row6 col2\" >6</td>\n      <td id=\"T_59080_row6_col3\" class=\"data row6 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n      <td id=\"T_59080_row7_col0\" class=\"data row7 col0\" >Precision</td>\n      <td id=\"T_59080_row7_col1\" class=\"data row7 col1\" >7</td>\n      <td id=\"T_59080_row7_col2\" class=\"data row7 col2\" >7</td>\n      <td id=\"T_59080_row7_col3\" class=\"data row7 col3\" >0.96</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n      <td id=\"T_59080_row8_col0\" class=\"data row8 col0\" >Precision</td>\n      <td id=\"T_59080_row8_col1\" class=\"data row8 col1\" >8</td>\n      <td id=\"T_59080_row8_col2\" class=\"data row8 col2\" >8</td>\n      <td id=\"T_59080_row8_col3\" class=\"data row8 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n      <td id=\"T_59080_row9_col0\" class=\"data row9 col0\" >Precision</td>\n      <td id=\"T_59080_row9_col1\" class=\"data row9 col1\" >9</td>\n      <td id=\"T_59080_row9_col2\" class=\"data row9 col2\" >9</td>\n      <td id=\"T_59080_row9_col3\" class=\"data row9 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n      <td id=\"T_59080_row10_col0\" class=\"data row10 col0\" >Recall</td>\n      <td id=\"T_59080_row10_col1\" class=\"data row10 col1\" >0</td>\n      <td id=\"T_59080_row10_col2\" class=\"data row10 col2\" >0</td>\n      <td id=\"T_59080_row10_col3\" class=\"data row10 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n      <td id=\"T_59080_row11_col0\" class=\"data row11 col0\" >Recall</td>\n      <td id=\"T_59080_row11_col1\" class=\"data row11 col1\" >1</td>\n      <td id=\"T_59080_row11_col2\" class=\"data row11 col2\" >1</td>\n      <td id=\"T_59080_row11_col3\" class=\"data row11 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n      <td id=\"T_59080_row12_col0\" class=\"data row12 col0\" >Recall</td>\n      <td id=\"T_59080_row12_col1\" class=\"data row12 col1\" >2</td>\n      <td id=\"T_59080_row12_col2\" class=\"data row12 col2\" >2</td>\n      <td id=\"T_59080_row12_col3\" class=\"data row12 col3\" >0.97</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row13\" class=\"row_heading level0 row13\" >13</th>\n      <td id=\"T_59080_row13_col0\" class=\"data row13 col0\" >Recall</td>\n      <td id=\"T_59080_row13_col1\" class=\"data row13 col1\" >3</td>\n      <td id=\"T_59080_row13_col2\" class=\"data row13 col2\" >3</td>\n      <td id=\"T_59080_row13_col3\" class=\"data row13 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row14\" class=\"row_heading level0 row14\" >14</th>\n      <td id=\"T_59080_row14_col0\" class=\"data row14 col0\" >Recall</td>\n      <td id=\"T_59080_row14_col1\" class=\"data row14 col1\" >4</td>\n      <td id=\"T_59080_row14_col2\" class=\"data row14 col2\" >4</td>\n      <td id=\"T_59080_row14_col3\" class=\"data row14 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row15\" class=\"row_heading level0 row15\" >15</th>\n      <td id=\"T_59080_row15_col0\" class=\"data row15 col0\" >Recall</td>\n      <td id=\"T_59080_row15_col1\" class=\"data row15 col1\" >5</td>\n      <td id=\"T_59080_row15_col2\" class=\"data row15 col2\" >5</td>\n      <td id=\"T_59080_row15_col3\" class=\"data row15 col3\" >0.97</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row16\" class=\"row_heading level0 row16\" >16</th>\n      <td id=\"T_59080_row16_col0\" class=\"data row16 col0\" >Recall</td>\n      <td id=\"T_59080_row16_col1\" class=\"data row16 col1\" >6</td>\n      <td id=\"T_59080_row16_col2\" class=\"data row16 col2\" >6</td>\n      <td id=\"T_59080_row16_col3\" class=\"data row16 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row17\" class=\"row_heading level0 row17\" >17</th>\n      <td id=\"T_59080_row17_col0\" class=\"data row17 col0\" >Recall</td>\n      <td id=\"T_59080_row17_col1\" class=\"data row17 col1\" >7</td>\n      <td id=\"T_59080_row17_col2\" class=\"data row17 col2\" >7</td>\n      <td id=\"T_59080_row17_col3\" class=\"data row17 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row18\" class=\"row_heading level0 row18\" >18</th>\n      <td id=\"T_59080_row18_col0\" class=\"data row18 col0\" >Recall</td>\n      <td id=\"T_59080_row18_col1\" class=\"data row18 col1\" >8</td>\n      <td id=\"T_59080_row18_col2\" class=\"data row18 col2\" >8</td>\n      <td id=\"T_59080_row18_col3\" class=\"data row18 col3\" >0.97</td>\n    </tr>\n    <tr>\n      <th id=\"T_59080_level0_row19\" class=\"row_heading level0 row19\" >19</th>\n      <td id=\"T_59080_row19_col0\" class=\"data row19 col0\" >Recall</td>\n      <td id=\"T_59080_row19_col1\" class=\"data row19 col1\" >9</td>\n      <td id=\"T_59080_row19_col2\" class=\"data row19 col2\" >9</td>\n      <td id=\"T_59080_row19_col3\" class=\"data row19 col3\" >0.96</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "11d2c7944af44fa69be10ee4b6c83544": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "fe0b15ea7d23467b89c1c521c893e858": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "3fa1824ad8c84eac8570365cc1f71b80": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_20e04d697b1341c8aa2a41f00fd5c9fb", "style": "IPY_MODEL_e7af702fc2cc424a9d03f8063e6bd0bc", "value": "<div><p style=\"font-size:0.9em;line-height:1;\"><i>Note - data sampling: Data is sampled from the original dataset, running on 10000 samples out of 60000. Sample size can be controlled with the \"n_samples\" parameter.</i></p></div>"}}, "20e04d697b1341c8aa2a41f00fd5c9fb": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "e7af702fc2cc424a9d03f8063e6bd0bc": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "6ff144401e8e4a6eb3ff52a8391ccd3c": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "05507d9b697e47d1b62d55ee7d51f9a2": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "0d1f752902234c5ca77904531ef32ffd"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 52-55

If you have a GPU, you can speed up this check by passing it as an argument to .run() as device=<your GPU>

To display the results in an IDE like PyCharm, you can use the following code:

.. GENERATED FROM PYTHON SOURCE LINES 55-57

.. code-block:: default


    #  result.show_in_window()







.. GENERATED FROM PYTHON SOURCE LINES 58-59

The result will be displayed in a new window.

.. GENERATED FROM PYTHON SOURCE LINES 61-62

Now we will run a check with a metric different from the defaults- F-1.

.. GENERATED FROM PYTHON SOURCE LINES 62-68

.. code-block:: default

    from ignite.metrics import Fbeta

    check = SingleDatasetPerformance(scorers={'f1': Fbeta(1)})
    result = check.run(train_ds, mnist_model)
    result





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Validating Input:
    |     | 0/1 [Time: 00:00]
    Validating Input:
    |#####| 1/1 [Time: 00:00]

    Ingesting Batches:
    |                                                                                                                                                             | 0/157 [Time: 00:00]

    Ingesting Batches:
    |###############                                                                                                                                              | 15/157 [Time: 00:00]

    Ingesting Batches:
    |##############################                                                                                                                               | 30/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################                                                                                                                | 45/157 [Time: 00:00]

    Ingesting Batches:
    |############################################################                                                                                                 | 60/157 [Time: 00:00]

    Ingesting Batches:
    |###########################################################################                                                                                  | 75/157 [Time: 00:00]

    Ingesting Batches:
    |##########################################################################################                                                                   | 90/157 [Time: 00:00]

    Ingesting Batches:
    |#########################################################################################################                                                    | 105/157 [Time: 00:00]

    Ingesting Batches:
    |########################################################################################################################                                     | 120/157 [Time: 00:00]

    Ingesting Batches:
    |#######################################################################################################################################                      | 135/157 [Time: 00:00]

    Ingesting Batches:
    |######################################################################################################################################################       | 150/157 [Time: 00:01]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]


    Computing Check:
    |     | 0/1 [Time: 00:00]


    Computing Check:
    |#####| 1/1 [Time: 00:00]
                                           

                                                                                                                                                                                                            


                                              

.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Single Dataset Performance</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"65976b94c5f94a6699d41a98dc96e2a9": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_f807e13a85df4012bcafa5738fbae528", "IPY_MODEL_e2b54b128a7647ed96052cbcf175cdd1", "IPY_MODEL_e47ad539dcbc400b86ec4186a8324fea", "IPY_MODEL_f787916031d54dd49bfc67656a0e95c7"], "layout": "IPY_MODEL_1f2850e95e75479581be66a18629bf15"}}, "f807e13a85df4012bcafa5738fbae528": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_8080c65f34984c2ebae93db43f461c55", "style": "IPY_MODEL_dc1d69464bce432aa6160ea02da29f57", "value": "<h4><b>Single Dataset Performance</b></h4>"}}, "8080c65f34984c2ebae93db43f461c55": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "dc1d69464bce432aa6160ea02da29f57": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "e2b54b128a7647ed96052cbcf175cdd1": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_91b5608fb5da4be4ba14b10402f437f1", "style": "IPY_MODEL_f307611b7b154560b0404f7d7b5475b3", "value": "<p>Calculate performance metrics of a given model on a given dataset. <a href=\"https://docs.deepchecks.com/0.9/checks_gallery/vision/model_evaluation/plot_single_dataset_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "91b5608fb5da4be4ba14b10402f437f1": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "f307611b7b154560b0404f7d7b5475b3": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "e47ad539dcbc400b86ec4186a8324fea": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_1bd964c962e2457c9257aeb3ec255663", "style": "IPY_MODEL_e881c6c3db2846bb89fa5ac4bfcb62c1"}}, "1bd964c962e2457c9257aeb3ec255663": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "e881c6c3db2846bb89fa5ac4bfcb62c1": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "f787916031d54dd49bfc67656a0e95c7": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_db7ed0d1f1c8414eae7e6c6a950b64ac", "IPY_MODEL_555e5a312d65424797fa53bc81970a27", "IPY_MODEL_bec5b1e0aeb844baa3e2c9e0d2b69f7f"], "layout": "IPY_MODEL_41aa1999015a41c0a155ddb1cf65272d"}}, "db7ed0d1f1c8414eae7e6c6a950b64ac": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_6217d08d2daa482d93dd717666244a66", "style": "IPY_MODEL_edf9a032b6c74bb692054a553f9655ca", "value": "<h5><b>Additional Outputs</b></h5>"}}, "6217d08d2daa482d93dd717666244a66": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "edf9a032b6c74bb692054a553f9655ca": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "555e5a312d65424797fa53bc81970a27": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_6b8776fe0756464db29b53caaa26c3bd", "style": "IPY_MODEL_106ad8a4ca594e3eb4ccb396c70d8658", "value": "<style type=\"text/css\">\n#T_02f18_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_02f18_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_02f18_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_02f18_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_02f18_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_02f18_\">\n  <thead>\n    <tr>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >Metric</th>\n      <th class=\"col_heading level0 col1\" >Class</th>\n      <th class=\"col_heading level0 col2\" >Class Name</th>\n      <th class=\"col_heading level0 col3\" >Value</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_02f18_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n      <td id=\"T_02f18_row0_col0\" class=\"data row0 col0\" >f1</td>\n      <td id=\"T_02f18_row0_col1\" class=\"data row0 col1\" ><NA></td>\n      <td id=\"T_02f18_row0_col2\" class=\"data row0 col2\" ><NA></td>\n      <td id=\"T_02f18_row0_col3\" class=\"data row0 col3\" >0.98</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "6b8776fe0756464db29b53caaa26c3bd": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "106ad8a4ca594e3eb4ccb396c70d8658": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "bec5b1e0aeb844baa3e2c9e0d2b69f7f": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_7298cdee6f504a23a535d15b3a74427e", "style": "IPY_MODEL_ded627bafa0b4c73a16dc015582db13e", "value": "<div><p style=\"font-size:0.9em;line-height:1;\"><i>Note - data sampling: Data is sampled from the original dataset, running on 10000 samples out of 60000. Sample size can be controlled with the \"n_samples\" parameter.</i></p></div>"}}, "7298cdee6f504a23a535d15b3a74427e": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "ded627bafa0b4c73a16dc015582db13e": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "41aa1999015a41c0a155ddb1cf65272d": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "1f2850e95e75479581be66a18629bf15": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "65976b94c5f94a6699d41a98dc96e2a9"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 69-74

Define a Condition
==================
We can define a condition to validate that our model performance score is above or below a certain threshold.
The condition is defined as a function that takes the results of the check as input and
returns a ConditionResult object.

.. GENERATED FROM PYTHON SOURCE LINES 74-80

.. code-block:: default


    check = SingleDatasetPerformance()
    check.add_condition_greater_than(0.5)
    result = check.run(train_ds, mnist_model)
    result.show(show_additional_outputs=False)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Validating Input:
    |     | 0/1 [Time: 00:00]
    Validating Input:
    |#####| 1/1 [Time: 00:00]

    Ingesting Batches:
    |                                                                                                                                                             | 0/157 [Time: 00:00]

    Ingesting Batches:
    |###############                                                                                                                                              | 15/157 [Time: 00:00]

    Ingesting Batches:
    |###############################                                                                                                                              | 31/157 [Time: 00:00]

    Ingesting Batches:
    |##############################################                                                                                                               | 46/157 [Time: 00:00]

    Ingesting Batches:
    |##############################################################                                                                                               | 62/157 [Time: 00:00]

    Ingesting Batches:
    |##############################################################################                                                                               | 78/157 [Time: 00:00]

    Ingesting Batches:
    |##############################################################################################                                                               | 94/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################################                                                | 109/157 [Time: 00:00]

    Ingesting Batches:
    |############################################################################################################################                                 | 124/157 [Time: 00:00]

    Ingesting Batches:
    |############################################################################################################################################                 | 140/157 [Time: 00:00]

    Ingesting Batches:
    |############################################################################################################################################################ | 156/157 [Time: 00:01]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]


    Computing Check:
    |     | 0/1 [Time: 00:00]


    Computing Check:
    |#####| 1/1 [Time: 00:00]
                                           

                                                                                                                                                                                                            


                                              

.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Single Dataset Performance</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"bc5079c383c54b29b9cb4fa799c2c885": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_7701070288f8423681f575c8c68a480e", "IPY_MODEL_8182e2c706844532b7477f2ba4110da0", "IPY_MODEL_cec3919e94eb494eb5ce710760710c98"], "layout": "IPY_MODEL_1a2bea261faf4fcd81528362e33bf773"}}, "7701070288f8423681f575c8c68a480e": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_6fb336d98476492686c297b0072d4b04", "style": "IPY_MODEL_48d862866f2943918581e6100f098461", "value": "<h4><b>Single Dataset Performance</b></h4>"}}, "6fb336d98476492686c297b0072d4b04": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "48d862866f2943918581e6100f098461": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "8182e2c706844532b7477f2ba4110da0": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_9a810ba25e194796b01917a2272537de", "style": "IPY_MODEL_f36308bdf794405dabff2106ca38921a", "value": "<p>Calculate performance metrics of a given model on a given dataset. <a href=\"https://docs.deepchecks.com/0.9/checks_gallery/vision/model_evaluation/plot_single_dataset_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "9a810ba25e194796b01917a2272537de": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "f36308bdf794405dabff2106ca38921a": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "cec3919e94eb494eb5ce710760710c98": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_05acaf8370d14860ae96e7d59413730a", "style": "IPY_MODEL_f7dc7cf5902b43428b2b13ee9804b77d", "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_370f7_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_370f7_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_370f7_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_370f7_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_370f7_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_370f7_\">\n  <thead>\n    <tr>\n      <th class=\"col_heading level0 col0\" >Status</th>\n      <th class=\"col_heading level0 col1\" >Condition</th>\n      <th class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_370f7_row0_col0\" class=\"data row0 col0\" ><div style=\"color: green;text-align: center\">\u2713</div></td>\n      <td id=\"T_370f7_row0_col1\" class=\"data row0 col1\" >Score is greater than 0.5 for classes: all</td>\n      <td id=\"T_370f7_row0_col2\" class=\"data row0 col2\" >Passed for all of the metrics.</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "05acaf8370d14860ae96e7d59413730a": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "f7dc7cf5902b43428b2b13ee9804b77d": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "1a2bea261faf4fcd81528362e33bf773": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "bc5079c383c54b29b9cb4fa799c2c885"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 81-83

We can also define a condition on a specific metric (or a subset of the metrics) that was passed to the check and a
specific class, instead of testing all the metrics and all the classes which is the default mode.

.. GENERATED FROM PYTHON SOURCE LINES 83-88

.. code-block:: default


    check = SingleDatasetPerformance()
    check.add_condition_greater_than(0.8, metrics=['Precision'], class_mode='3')
    result = check.run(train_ds, mnist_model)
    result.show(show_additional_outputs=False)




.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Validating Input:
    |     | 0/1 [Time: 00:00]
    Validating Input:
    |#####| 1/1 [Time: 00:00]

    Ingesting Batches:
    |                                                                                                                                                             | 0/157 [Time: 00:00]

    Ingesting Batches:
    |################                                                                                                                                             | 16/157 [Time: 00:00]

    Ingesting Batches:
    |################################                                                                                                                             | 32/157 [Time: 00:00]

    Ingesting Batches:
    |################################################                                                                                                             | 48/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################                                                                                             | 64/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################################                                                                             | 80/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################################################                                                             | 96/157 [Time: 00:00]

    Ingesting Batches:
    |###############################################################################################################9                                             | 112/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################################################################################                             | 128/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################################################################################################             | 144/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]


    Computing Check:
    |     | 0/1 [Time: 00:00]


    Computing Check:
    |#####| 1/1 [Time: 00:00]
                                           

                                                                                                                                                                                                            


                                              

.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Single Dataset Performance</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"057abca0c9e94b109974ccdcd9bd63b8": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_9697d50d670b4541836cbcf803fabb54", "IPY_MODEL_e056dd30c213447cbad023a7debfc4ba", "IPY_MODEL_964896c546af43cfa4c50f6313fdd1bd"], "layout": "IPY_MODEL_7b320a39612849dc8d421b7a5188ed86"}}, "9697d50d670b4541836cbcf803fabb54": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_0d7cc46d4c444a26b572956b2c6f4fcc", "style": "IPY_MODEL_05360907b7244533b3c9bff0d3b59647", "value": "<h4><b>Single Dataset Performance</b></h4>"}}, "0d7cc46d4c444a26b572956b2c6f4fcc": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "05360907b7244533b3c9bff0d3b59647": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "e056dd30c213447cbad023a7debfc4ba": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_32ca875d3057463a8e6e7480dab2af77", "style": "IPY_MODEL_52481f74240a4d0fbea0abfa029b11cf", "value": "<p>Calculate performance metrics of a given model on a given dataset. <a href=\"https://docs.deepchecks.com/0.9/checks_gallery/vision/model_evaluation/plot_single_dataset_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "32ca875d3057463a8e6e7480dab2af77": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "52481f74240a4d0fbea0abfa029b11cf": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "964896c546af43cfa4c50f6313fdd1bd": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_e5e9e499165c4d43b5f56ce39acf06b5", "style": "IPY_MODEL_16d447fa256d459ea5be2baaa7d76726", "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_81a5f_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_81a5f_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_81a5f_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_81a5f_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_81a5f_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_81a5f_\">\n  <thead>\n    <tr>\n      <th class=\"col_heading level0 col0\" >Status</th>\n      <th class=\"col_heading level0 col1\" >Condition</th>\n      <th class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_81a5f_row0_col0\" class=\"data row0 col0\" ><div style=\"color: firebrick;text-align: center;font-weight:bold\">\u2048</div></td>\n      <td id=\"T_81a5f_row0_col1\" class=\"data row0 col1\" >Score is greater than 0.8 for classes: 3</td>\n      <td id=\"T_81a5f_row0_col2\" class=\"data row0 col2\" >Exception in condition: KeyError: '3'</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "e5e9e499165c4d43b5f56ce39acf06b5": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "16d447fa256d459ea5be2baaa7d76726": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "7b320a39612849dc8d421b7a5188ed86": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "057abca0c9e94b109974ccdcd9bd63b8"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  4.490 seconds)


.. _sphx_glr_download_checks_gallery_vision_model_evaluation_plot_single_dataset_performance.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_single_dataset_performance.py <plot_single_dataset_performance.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_single_dataset_performance.ipynb <plot_single_dataset_performance.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_