.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "checks_gallery/vision/model_evaluation/plot_single_dataset_performance.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_checks_gallery_vision_model_evaluation_plot_single_dataset_performance.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_checks_gallery_vision_model_evaluation_plot_single_dataset_performance.py:


.. _plot_vision_single_dataset_performance:

Single Dataset Performance
*********************************

This notebooks provides an overview for using and understanding single dataset performance check.

**Structure:**

* `What Is the Purpose of the Check? <#what-is-the-purpose-of-the-check>`__
* `Generate data an model <#generate-data-and-model>`__
* `Run the check <#run-the-check>`__
* `Define a condition <#define-a-condition>`__

What Is the Purpose of the Check?
=================================
This check returns the results from a dict of metrics, in the format metric name: scorer, calculated for the
given model dataset. The scorer can be an ignite.Metric or Sklearn scorer.
Use this check to evaluate the performance on a single vision dataset such as a test set.

.. GENERATED FROM PYTHON SOURCE LINES 25-27

Generate data and model
-----------------------

.. GENERATED FROM PYTHON SOURCE LINES 27-31

.. code-block:: default


    from deepchecks.vision.checks import SingleDatasetPerformance
    from deepchecks.vision.datasets.classification import mnist








.. GENERATED FROM PYTHON SOURCE LINES 32-37

.. code-block:: default



    mnist_model = mnist.load_model()
    train_ds = mnist.load_dataset(train=True, object_type='VisionData')








.. GENERATED FROM PYTHON SOURCE LINES 38-45

Run the check
-------------
We will run the check with the model defined above.

The check will use the default classification metrics - `Precision
<https://pytorch.org/ignite/generated/ignite.metrics.precision.Precision.html#ignite.metrics.precision.Precision>`__.
and `Recall <https://pytorch.org/ignite/generated/ignite.metrics.recall.Recall.html#ignite.metrics.recall.Recall>`__.

.. GENERATED FROM PYTHON SOURCE LINES 45-51

.. code-block:: default



    check = SingleDatasetPerformance()
    result = check.run(train_ds, mnist_model)
    result





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Validating Input:
    |     | 0/1 [Time: 00:00]
    Validating Input:
    |#####| 1/1 [Time: 00:00]

    Ingesting Batches:
    |                                                                                                                                                             | 0/157 [Time: 00:00]

    Ingesting Batches:
    |###############                                                                                                                                              | 15/157 [Time: 00:00]

    Ingesting Batches:
    |##############################                                                                                                                               | 30/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################                                                                                                                | 45/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################                                                                                                | 61/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################                                                                                | 77/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################                                                                | 93/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################################                                                | 109/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################################################                                | 125/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################################################################                | 141/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]


    Computing Check:
    |     | 0/1 [Time: 00:00]


    Computing Check:
    |#####| 1/1 [Time: 00:00]
                                           

                                                                                                                                                                                                            


                                              

.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Single Dataset Performance</title>
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"e658c4ac1cd3467d8861893d59b8e43b": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_be37d575c98443c7ad92ece4d07f6d84", "IPY_MODEL_cd8ceb21fefc45c8844121f8942b035b", "IPY_MODEL_7eeda4b927874742bc1ba9f1d950c354", "IPY_MODEL_73b9b3ac373048bf93329a76c7a62d89"], "layout": "IPY_MODEL_14176a6269b7408cb4ae1a6f356158df"}}, "be37d575c98443c7ad92ece4d07f6d84": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_6f86469ba168452d98b7d9c1b5cec1be", "style": "IPY_MODEL_b924a7f096c24d8c97b8877d27be3cfb", "value": "<h4><b>Single Dataset Performance</b></h4>"}}, "6f86469ba168452d98b7d9c1b5cec1be": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "b924a7f096c24d8c97b8877d27be3cfb": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "cd8ceb21fefc45c8844121f8942b035b": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_86fb7801d3574ee5bac77fd8cf704366", "style": "IPY_MODEL_340f039097504922aac584515b584660", "value": "<p>Calculate performance metrics of a given model on a given dataset. <a href=\"https://docs.deepchecks.com/0.8/checks_gallery/vision/model_evaluation/plot_single_dataset_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "86fb7801d3574ee5bac77fd8cf704366": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "340f039097504922aac584515b584660": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "7eeda4b927874742bc1ba9f1d950c354": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_673e3aefa42246f4af9c384aaa0fda71", "style": "IPY_MODEL_788301fbf57e47e798948d66a8e19a10"}}, "673e3aefa42246f4af9c384aaa0fda71": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "788301fbf57e47e798948d66a8e19a10": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "73b9b3ac373048bf93329a76c7a62d89": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_df51280b50a9473dbff855d992673b74", "IPY_MODEL_c7511d9259ca41059d21e7b4285291e8", "IPY_MODEL_9e6979d7c69041ecbc198ebf865220a4"], "layout": "IPY_MODEL_ede6def8cd544bceac0be44e0e60c33f"}}, "df51280b50a9473dbff855d992673b74": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_52d8745c33a04c008ff95c99a19c6119", "style": "IPY_MODEL_9b372d37a8b3486fbc3fe2a2e9d5ba42", "value": "<h5><b>Additional Outputs</b></h5>"}}, "52d8745c33a04c008ff95c99a19c6119": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "9b372d37a8b3486fbc3fe2a2e9d5ba42": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "c7511d9259ca41059d21e7b4285291e8": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_c2efecd252da4356b00d2681ef8268b0", "style": "IPY_MODEL_0f90ac656a8249c599a4a73420d22e57", "value": "<style type=\"text/css\">\n#T_c64af_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_c64af_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_c64af_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_c64af_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_c64af_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_c64af_\">\n  <thead>\n    <tr>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >Metric</th>\n      <th class=\"col_heading level0 col1\" >Class</th>\n      <th class=\"col_heading level0 col2\" >Class Name</th>\n      <th class=\"col_heading level0 col3\" >Value</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_c64af_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n      <td id=\"T_c64af_row0_col0\" class=\"data row0 col0\" >Precision</td>\n      <td id=\"T_c64af_row0_col1\" class=\"data row0 col1\" >0</td>\n      <td id=\"T_c64af_row0_col2\" class=\"data row0 col2\" >0</td>\n      <td id=\"T_c64af_row0_col3\" class=\"data row0 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n      <td id=\"T_c64af_row1_col0\" class=\"data row1 col0\" >Precision</td>\n      <td id=\"T_c64af_row1_col1\" class=\"data row1 col1\" >1</td>\n      <td id=\"T_c64af_row1_col2\" class=\"data row1 col2\" >1</td>\n      <td id=\"T_c64af_row1_col3\" class=\"data row1 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n      <td id=\"T_c64af_row2_col0\" class=\"data row2 col0\" >Precision</td>\n      <td id=\"T_c64af_row2_col1\" class=\"data row2 col1\" >2</td>\n      <td id=\"T_c64af_row2_col2\" class=\"data row2 col2\" >2</td>\n      <td id=\"T_c64af_row2_col3\" class=\"data row2 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n      <td id=\"T_c64af_row3_col0\" class=\"data row3 col0\" >Precision</td>\n      <td id=\"T_c64af_row3_col1\" class=\"data row3 col1\" >3</td>\n      <td id=\"T_c64af_row3_col2\" class=\"data row3 col2\" >3</td>\n      <td id=\"T_c64af_row3_col3\" class=\"data row3 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n      <td id=\"T_c64af_row4_col0\" class=\"data row4 col0\" >Precision</td>\n      <td id=\"T_c64af_row4_col1\" class=\"data row4 col1\" >4</td>\n      <td id=\"T_c64af_row4_col2\" class=\"data row4 col2\" >4</td>\n      <td id=\"T_c64af_row4_col3\" class=\"data row4 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n      <td id=\"T_c64af_row5_col0\" class=\"data row5 col0\" >Precision</td>\n      <td id=\"T_c64af_row5_col1\" class=\"data row5 col1\" >5</td>\n      <td id=\"T_c64af_row5_col2\" class=\"data row5 col2\" >5</td>\n      <td id=\"T_c64af_row5_col3\" class=\"data row5 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n      <td id=\"T_c64af_row6_col0\" class=\"data row6 col0\" >Precision</td>\n      <td id=\"T_c64af_row6_col1\" class=\"data row6 col1\" >6</td>\n      <td id=\"T_c64af_row6_col2\" class=\"data row6 col2\" >6</td>\n      <td id=\"T_c64af_row6_col3\" class=\"data row6 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n      <td id=\"T_c64af_row7_col0\" class=\"data row7 col0\" >Precision</td>\n      <td id=\"T_c64af_row7_col1\" class=\"data row7 col1\" >7</td>\n      <td id=\"T_c64af_row7_col2\" class=\"data row7 col2\" >7</td>\n      <td id=\"T_c64af_row7_col3\" class=\"data row7 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n      <td id=\"T_c64af_row8_col0\" class=\"data row8 col0\" >Precision</td>\n      <td id=\"T_c64af_row8_col1\" class=\"data row8 col1\" >8</td>\n      <td id=\"T_c64af_row8_col2\" class=\"data row8 col2\" >8</td>\n      <td id=\"T_c64af_row8_col3\" class=\"data row8 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n      <td id=\"T_c64af_row9_col0\" class=\"data row9 col0\" >Precision</td>\n      <td id=\"T_c64af_row9_col1\" class=\"data row9 col1\" >9</td>\n      <td id=\"T_c64af_row9_col2\" class=\"data row9 col2\" >9</td>\n      <td id=\"T_c64af_row9_col3\" class=\"data row9 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n      <td id=\"T_c64af_row10_col0\" class=\"data row10 col0\" >Recall</td>\n      <td id=\"T_c64af_row10_col1\" class=\"data row10 col1\" >0</td>\n      <td id=\"T_c64af_row10_col2\" class=\"data row10 col2\" >0</td>\n      <td id=\"T_c64af_row10_col3\" class=\"data row10 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n      <td id=\"T_c64af_row11_col0\" class=\"data row11 col0\" >Recall</td>\n      <td id=\"T_c64af_row11_col1\" class=\"data row11 col1\" >1</td>\n      <td id=\"T_c64af_row11_col2\" class=\"data row11 col2\" >1</td>\n      <td id=\"T_c64af_row11_col3\" class=\"data row11 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n      <td id=\"T_c64af_row12_col0\" class=\"data row12 col0\" >Recall</td>\n      <td id=\"T_c64af_row12_col1\" class=\"data row12 col1\" >2</td>\n      <td id=\"T_c64af_row12_col2\" class=\"data row12 col2\" >2</td>\n      <td id=\"T_c64af_row12_col3\" class=\"data row12 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row13\" class=\"row_heading level0 row13\" >13</th>\n      <td id=\"T_c64af_row13_col0\" class=\"data row13 col0\" >Recall</td>\n      <td id=\"T_c64af_row13_col1\" class=\"data row13 col1\" >3</td>\n      <td id=\"T_c64af_row13_col2\" class=\"data row13 col2\" >3</td>\n      <td id=\"T_c64af_row13_col3\" class=\"data row13 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row14\" class=\"row_heading level0 row14\" >14</th>\n      <td id=\"T_c64af_row14_col0\" class=\"data row14 col0\" >Recall</td>\n      <td id=\"T_c64af_row14_col1\" class=\"data row14 col1\" >4</td>\n      <td id=\"T_c64af_row14_col2\" class=\"data row14 col2\" >4</td>\n      <td id=\"T_c64af_row14_col3\" class=\"data row14 col3\" >0.99</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row15\" class=\"row_heading level0 row15\" >15</th>\n      <td id=\"T_c64af_row15_col0\" class=\"data row15 col0\" >Recall</td>\n      <td id=\"T_c64af_row15_col1\" class=\"data row15 col1\" >5</td>\n      <td id=\"T_c64af_row15_col2\" class=\"data row15 col2\" >5</td>\n      <td id=\"T_c64af_row15_col3\" class=\"data row15 col3\" >0.97</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row16\" class=\"row_heading level0 row16\" >16</th>\n      <td id=\"T_c64af_row16_col0\" class=\"data row16 col0\" >Recall</td>\n      <td id=\"T_c64af_row16_col1\" class=\"data row16 col1\" >6</td>\n      <td id=\"T_c64af_row16_col2\" class=\"data row16 col2\" >6</td>\n      <td id=\"T_c64af_row16_col3\" class=\"data row16 col3\" >1.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row17\" class=\"row_heading level0 row17\" >17</th>\n      <td id=\"T_c64af_row17_col0\" class=\"data row17 col0\" >Recall</td>\n      <td id=\"T_c64af_row17_col1\" class=\"data row17 col1\" >7</td>\n      <td id=\"T_c64af_row17_col2\" class=\"data row17 col2\" >7</td>\n      <td id=\"T_c64af_row17_col3\" class=\"data row17 col3\" >0.98</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row18\" class=\"row_heading level0 row18\" >18</th>\n      <td id=\"T_c64af_row18_col0\" class=\"data row18 col0\" >Recall</td>\n      <td id=\"T_c64af_row18_col1\" class=\"data row18 col1\" >8</td>\n      <td id=\"T_c64af_row18_col2\" class=\"data row18 col2\" >8</td>\n      <td id=\"T_c64af_row18_col3\" class=\"data row18 col3\" >0.97</td>\n    </tr>\n    <tr>\n      <th id=\"T_c64af_level0_row19\" class=\"row_heading level0 row19\" >19</th>\n      <td id=\"T_c64af_row19_col0\" class=\"data row19 col0\" >Recall</td>\n      <td id=\"T_c64af_row19_col1\" class=\"data row19 col1\" >9</td>\n      <td id=\"T_c64af_row19_col2\" class=\"data row19 col2\" >9</td>\n      <td id=\"T_c64af_row19_col3\" class=\"data row19 col3\" >0.97</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "c2efecd252da4356b00d2681ef8268b0": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "0f90ac656a8249c599a4a73420d22e57": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "9e6979d7c69041ecbc198ebf865220a4": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_fc68ec0977d54a4abcfcca2c912b7961", "style": "IPY_MODEL_8149b0d550cf499f870a3a13c33a4048", "value": "<div><p style=\"font-size:0.9em;line-height:1;\"><i>Note - data sampling: Data is sampled from the original dataset, running on 10000 samples out of 60000. Sample size can be controlled with the \"n_samples\" parameter.</i></p></div>"}}, "fc68ec0977d54a4abcfcca2c912b7961": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "8149b0d550cf499f870a3a13c33a4048": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "ede6def8cd544bceac0be44e0e60c33f": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "14176a6269b7408cb4ae1a6f356158df": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "e658c4ac1cd3467d8861893d59b8e43b"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 52-55

If you have a GPU, you can speed up this check by passing it as an argument to .run() as device=<your GPU>

To display the results in an IDE like PyCharm, you can use the following code:

.. GENERATED FROM PYTHON SOURCE LINES 55-57

.. code-block:: default


    #  result.show_in_window()







.. GENERATED FROM PYTHON SOURCE LINES 58-59

The result will be displayed in a new window.

.. GENERATED FROM PYTHON SOURCE LINES 61-62

Now we will run a check with a metric different from the defaults- F-1.

.. GENERATED FROM PYTHON SOURCE LINES 62-68

.. code-block:: default

    from ignite.metrics import Fbeta

    check = SingleDatasetPerformance(scorers={'f1': Fbeta(1)})
    result = check.run(train_ds, mnist_model)
    result





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Validating Input:
    |     | 0/1 [Time: 00:00]
    Validating Input:
    |#####| 1/1 [Time: 00:00]

    Ingesting Batches:
    |                                                                                                                                                             | 0/157 [Time: 00:00]

    Ingesting Batches:
    |###############                                                                                                                                              | 15/157 [Time: 00:00]

    Ingesting Batches:
    |##############################                                                                                                                               | 30/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################                                                                                                                | 45/157 [Time: 00:00]

    Ingesting Batches:
    |############################################################                                                                                                 | 60/157 [Time: 00:00]

    Ingesting Batches:
    |###########################################################################                                                                                  | 75/157 [Time: 00:00]

    Ingesting Batches:
    |##########################################################################################                                                                   | 90/157 [Time: 00:00]

    Ingesting Batches:
    |#########################################################################################################                                                    | 105/157 [Time: 00:00]

    Ingesting Batches:
    |########################################################################################################################                                     | 120/157 [Time: 00:00]

    Ingesting Batches:
    |#######################################################################################################################################                      | 135/157 [Time: 00:00]

    Ingesting Batches:
    |######################################################################################################################################################       | 150/157 [Time: 00:01]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]


    Computing Check:
    |     | 0/1 [Time: 00:00]


    Computing Check:
    |#####| 1/1 [Time: 00:00]
                                           

                                                                                                                                                                                                            


                                              

.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Single Dataset Performance</title>
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"58a7d1ef5c22447d8c104dc9c2f4715e": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_20a1d80290974488ae6ffc5c1bb488ef", "IPY_MODEL_a43c93c8762549b88daacbc8dabe30c0", "IPY_MODEL_4aea5b9baf6c4bee99d321df4d97f0b5", "IPY_MODEL_35fdb5ec2ab142869090513cb84f3e80"], "layout": "IPY_MODEL_b1c3de4232b74ed99c1e6ad002090cdc"}}, "20a1d80290974488ae6ffc5c1bb488ef": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_3037841f48ba478d87475b5e5ba84fc4", "style": "IPY_MODEL_3bc5068d6a494a4789c52895ea3177bb", "value": "<h4><b>Single Dataset Performance</b></h4>"}}, "3037841f48ba478d87475b5e5ba84fc4": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "3bc5068d6a494a4789c52895ea3177bb": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "a43c93c8762549b88daacbc8dabe30c0": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_2dc2961b9fe24f5aa3d94c34622723ab", "style": "IPY_MODEL_f44b28884e5b4c7da4b61872b55ed2a5", "value": "<p>Calculate performance metrics of a given model on a given dataset. <a href=\"https://docs.deepchecks.com/0.8/checks_gallery/vision/model_evaluation/plot_single_dataset_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "2dc2961b9fe24f5aa3d94c34622723ab": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "f44b28884e5b4c7da4b61872b55ed2a5": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "4aea5b9baf6c4bee99d321df4d97f0b5": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_e2439f27017240719571ceaea237598c", "style": "IPY_MODEL_bc9afd0b60124114ad32bad25e188bec"}}, "e2439f27017240719571ceaea237598c": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "bc9afd0b60124114ad32bad25e188bec": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "35fdb5ec2ab142869090513cb84f3e80": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_a787fed666104a22b29e4ae5b8cd333d", "IPY_MODEL_fa5c4a5f142e467ea3762e06cb960578", "IPY_MODEL_23965d4855ef430bb81e90ce5b85a214"], "layout": "IPY_MODEL_5dac8ccf94c54af79c0911e99e936826"}}, "a787fed666104a22b29e4ae5b8cd333d": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_cef1115318424246bdb11604e681cdf8", "style": "IPY_MODEL_2d6a337843cf49cfa4f9d05479e59d33", "value": "<h5><b>Additional Outputs</b></h5>"}}, "cef1115318424246bdb11604e681cdf8": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "2d6a337843cf49cfa4f9d05479e59d33": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "fa5c4a5f142e467ea3762e06cb960578": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_c0a4ebe4381944f1898a09dea9d0243f", "style": "IPY_MODEL_1516c6baa68d443db55e5d89a71d6035", "value": "<style type=\"text/css\">\n#T_54483_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_54483_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_54483_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_54483_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_54483_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_54483_\">\n  <thead>\n    <tr>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >Metric</th>\n      <th class=\"col_heading level0 col1\" >Class</th>\n      <th class=\"col_heading level0 col2\" >Class Name</th>\n      <th class=\"col_heading level0 col3\" >Value</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_54483_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n      <td id=\"T_54483_row0_col0\" class=\"data row0 col0\" >f1</td>\n      <td id=\"T_54483_row0_col1\" class=\"data row0 col1\" ><NA></td>\n      <td id=\"T_54483_row0_col2\" class=\"data row0 col2\" ><NA></td>\n      <td id=\"T_54483_row0_col3\" class=\"data row0 col3\" >0.98</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "c0a4ebe4381944f1898a09dea9d0243f": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "1516c6baa68d443db55e5d89a71d6035": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "23965d4855ef430bb81e90ce5b85a214": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_dbff401488b543e69c496a115528d410", "style": "IPY_MODEL_03e33b6349c54a0dbf5deb8d6a59d90d", "value": "<div><p style=\"font-size:0.9em;line-height:1;\"><i>Note - data sampling: Data is sampled from the original dataset, running on 10000 samples out of 60000. Sample size can be controlled with the \"n_samples\" parameter.</i></p></div>"}}, "dbff401488b543e69c496a115528d410": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "03e33b6349c54a0dbf5deb8d6a59d90d": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "5dac8ccf94c54af79c0911e99e936826": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "b1c3de4232b74ed99c1e6ad002090cdc": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "58a7d1ef5c22447d8c104dc9c2f4715e"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 69-74

Define a Condition
==================
We can define a condition to validate that our model performance score is above or below a certain threshold.
The condition is defined as a function that takes the results of the check as input and
returns a ConditionResult object.

.. GENERATED FROM PYTHON SOURCE LINES 74-80

.. code-block:: default


    check = SingleDatasetPerformance()
    check.add_condition_greater_than(0.5)
    result = check.run(train_ds, mnist_model)
    result.show(show_additional_outputs=False)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Validating Input:
    |     | 0/1 [Time: 00:00]
    Validating Input:
    |#####| 1/1 [Time: 00:00]

    Ingesting Batches:
    |                                                                                                                                                             | 0/157 [Time: 00:00]

    Ingesting Batches:
    |################                                                                                                                                             | 16/157 [Time: 00:00]

    Ingesting Batches:
    |################################                                                                                                                             | 32/157 [Time: 00:00]

    Ingesting Batches:
    |################################################                                                                                                             | 48/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################                                                                                             | 64/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################################                                                                             | 80/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################################################                                                             | 96/157 [Time: 00:00]

    Ingesting Batches:
    |###############################################################################################################9                                             | 112/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################################################################################                             | 128/157 [Time: 00:00]

    Ingesting Batches:
    |################################################################################################################################################             | 144/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]


    Computing Check:
    |     | 0/1 [Time: 00:00]


    Computing Check:
    |#####| 1/1 [Time: 00:00]
                                           

                                                                                                                                                                                                            


                                              

.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Single Dataset Performance</title>
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"5d13e06384224d2ba698ba09d918323c": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_749ff58f2e5a41f5be052e7afd7948ce", "IPY_MODEL_5c888544b493433e847a8298d2fd59c9", "IPY_MODEL_8f678bf1cfb44348bc491c2d0cd6edde"], "layout": "IPY_MODEL_143658efdae1447ab7b84adae92946fa"}}, "749ff58f2e5a41f5be052e7afd7948ce": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_e5fb1b46fa8b403f93610b353ccc3b5f", "style": "IPY_MODEL_7c2deb9adced466f972b83474b92a5a9", "value": "<h4><b>Single Dataset Performance</b></h4>"}}, "e5fb1b46fa8b403f93610b353ccc3b5f": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "7c2deb9adced466f972b83474b92a5a9": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "5c888544b493433e847a8298d2fd59c9": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_2a6bb7fe12ee4373bc1ddb771c72d0a7", "style": "IPY_MODEL_3738f5eb6bce4a8dacc80a39b97e685a", "value": "<p>Calculate performance metrics of a given model on a given dataset. <a href=\"https://docs.deepchecks.com/0.8/checks_gallery/vision/model_evaluation/plot_single_dataset_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "2a6bb7fe12ee4373bc1ddb771c72d0a7": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "3738f5eb6bce4a8dacc80a39b97e685a": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "8f678bf1cfb44348bc491c2d0cd6edde": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_f0b0118d28b34db88b351b1830398e49", "style": "IPY_MODEL_f301c68b989b41329cff8a05d3e6ae36", "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_83f27_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_83f27_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_83f27_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_83f27_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_83f27_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_83f27_\">\n  <thead>\n    <tr>\n      <th class=\"col_heading level0 col0\" >Status</th>\n      <th class=\"col_heading level0 col1\" >Condition</th>\n      <th class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_83f27_row0_col0\" class=\"data row0 col0\" ><div style=\"color: green;text-align: center\">\u2713</div></td>\n      <td id=\"T_83f27_row0_col1\" class=\"data row0 col1\" >Score is greater than 0.5 for classes: all</td>\n      <td id=\"T_83f27_row0_col2\" class=\"data row0 col2\" >Passed for all of the mertics.</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "f0b0118d28b34db88b351b1830398e49": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "f301c68b989b41329cff8a05d3e6ae36": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "143658efdae1447ab7b84adae92946fa": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "5d13e06384224d2ba698ba09d918323c"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 81-83

We can also define a condition on a specific metric (or a subset of the metrics) that was passed to the check and a
specific class, instead of testing all the metrics and all the classes which is the default mode.

.. GENERATED FROM PYTHON SOURCE LINES 83-88

.. code-block:: default


    check = SingleDatasetPerformance()
    check.add_condition_greater_than(0.8, metrics=['Precision'], class_mode='3')
    result = check.run(train_ds, mnist_model)
    result.show(show_additional_outputs=False)




.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Validating Input:
    |     | 0/1 [Time: 00:00]
    Validating Input:
    |#####| 1/1 [Time: 00:00]

    Ingesting Batches:
    |                                                                                                                                                             | 0/157 [Time: 00:00]

    Ingesting Batches:
    |###############                                                                                                                                              | 15/157 [Time: 00:00]

    Ingesting Batches:
    |###############################                                                                                                                              | 31/157 [Time: 00:00]

    Ingesting Batches:
    |###############################################                                                                                                              | 47/157 [Time: 00:00]

    Ingesting Batches:
    |###############################################################                                                                                              | 63/157 [Time: 00:00]

    Ingesting Batches:
    |###############################################################################                                                                              | 79/157 [Time: 00:00]

    Ingesting Batches:
    |###############################################################################################                                                              | 95/157 [Time: 00:00]

    Ingesting Batches:
    |##############################################################################################################9                                              | 111/157 [Time: 00:00]

    Ingesting Batches:
    |###############################################################################################################################                              | 127/157 [Time: 00:00]

    Ingesting Batches:
    |###############################################################################################################################################              | 143/157 [Time: 00:00]

    Ingesting Batches:
    |#############################################################################################################################################################| 157/157 [Time: 00:01]


    Computing Check:
    |     | 0/1 [Time: 00:00]


    Computing Check:
    |#####| 1/1 [Time: 00:00]
                                           

                                                                                                                                                                                                            


                                              

.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Single Dataset Performance</title>
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"a579bfe9e22a4095ae05d844137493eb": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_e0dfa538d51246bea64012a6990f3485", "IPY_MODEL_6c37eb2fe3b7441495ba06cc372ddef8", "IPY_MODEL_921f6f7d14374715b5586c52698f675f"], "layout": "IPY_MODEL_8d1dfbb9502344dca065572eb974f068"}}, "e0dfa538d51246bea64012a6990f3485": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_a685279c39bc4564b9b6c8e838fb674e", "style": "IPY_MODEL_1257c49b249c4962a9e71456a55b3c3a", "value": "<h4><b>Single Dataset Performance</b></h4>"}}, "a685279c39bc4564b9b6c8e838fb674e": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "1257c49b249c4962a9e71456a55b3c3a": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "6c37eb2fe3b7441495ba06cc372ddef8": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_ed6108a66dd444588fd798177d802c59", "style": "IPY_MODEL_5e903daa5b9e49768aa373a3d623e89d", "value": "<p>Calculate performance metrics of a given model on a given dataset. <a href=\"https://docs.deepchecks.com/0.8/checks_gallery/vision/model_evaluation/plot_single_dataset_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "ed6108a66dd444588fd798177d802c59": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "5e903daa5b9e49768aa373a3d623e89d": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "921f6f7d14374715b5586c52698f675f": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_23a32e8fdf144cae95787bd11eb3ab91", "style": "IPY_MODEL_b11befacabd94609aaa6ed02823da923", "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_74c83_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_74c83_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_74c83_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_74c83_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_74c83_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_74c83_\">\n  <thead>\n    <tr>\n      <th class=\"col_heading level0 col0\" >Status</th>\n      <th class=\"col_heading level0 col1\" >Condition</th>\n      <th class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_74c83_row0_col0\" class=\"data row0 col0\" ><div style=\"color: green;text-align: center\">\u2713</div></td>\n      <td id=\"T_74c83_row0_col1\" class=\"data row0 col1\" >Score is greater than 0.8 for classes: 3</td>\n      <td id=\"T_74c83_row0_col2\" class=\"data row0 col2\" >Passed for all of the mertics.</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "23a32e8fdf144cae95787bd11eb3ab91": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "b11befacabd94609aaa6ed02823da923": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "8d1dfbb9502344dca065572eb974f068": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "a579bfe9e22a4095ae05d844137493eb"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  4.508 seconds)


.. _sphx_glr_download_checks_gallery_vision_model_evaluation_plot_single_dataset_performance.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_single_dataset_performance.py <plot_single_dataset_performance.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_single_dataset_performance.ipynb <plot_single_dataset_performance.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_