.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "checks_gallery/tabular/performance/plot_performance_report.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_checks_gallery_tabular_performance_plot_performance_report.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_checks_gallery_tabular_performance_plot_performance_report.py:


Performance Report
******************
This notebooks provides an overview for using and understanding performance report check.

**Structure:**

* `What is the purpose of the check? <#what-is-the-purpose-of-the-check>`__
* `Generate data & model <#generate-data-model>`__
* `Run the check <#run-the-check>`__
* `Define a condition <#define-a-condition>`__
* `Using alternative scorers <#using-alternative-scorers>`__

What is the purpose of the check?
=================================
This check helps you compare your model's performance between two datasets.
The default metric that are used are F1, Percision, and Recall for Classification
and Negative Root Mean Square Error, Negative Mean Absolute Error, and R2 for Regression. RMSE and MAE Scorers are
negative because we subscribe to the sklearn convention of defining scoring functions.
`See scorers documentation <https://scikit-learn.org/stable/modules/model_evaluation.html#scoring>`_

.. GENERATED FROM PYTHON SOURCE LINES 26-28

Generate data & model
=====================

.. GENERATED FROM PYTHON SOURCE LINES 28-35

.. code-block:: default


    from deepchecks.tabular.datasets.classification.phishing import (
        load_data, load_fitted_model)

    train_dataset, test_dataset = load_data()
    model = load_fitted_model()








.. GENERATED FROM PYTHON SOURCE LINES 36-38

Run the check
=============

.. GENERATED FROM PYTHON SOURCE LINES 38-44

.. code-block:: default


    from deepchecks.tabular.checks.performance import PerformanceReport

    check = PerformanceReport()
    check.run(train_dataset, test_dataset, model)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">

    <script
        src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js"
        integrity="sha512-c3Nl8+7g4LMSTdrm621y7kf9v3SDPnhxLNhcjFJbKECVnmZHTdo+IRO05sNLTH/D3vA6u1X32ehoLC7WFVdheg=="
        crossorigin="anonymous"
        referrerpolicy="no-referrer">
    </script>

    <script type="text/javascript">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>
    <script type="text/javascript">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: "STIX-Web"}});}</script>
    <script type="text/javascript">
        if (typeof require !== 'undefined') {
            require.undef("plotly");
            requirejs.config({
                paths: {'plotly': ['https://cdn.plot.ly/plotly-2.8.3.min']}
            });
            require(
                ['plotly'],
                function(Plotly) {
                    window._Plotly = Plotly;
                    window.Plotly = Plotly;
                    console.log('Loaded plotly successfully');
                },
                function() {console.log('Failed to load plotly')}
            );
        } else {
            console.log('requirejs is not present');
        }
    </script>
    <h4><b>Performance Report</b></h4><p>Summarize given scores on a dataset and model.</p><h5><b>Additional Outputs</b></h5><div>                            <div id="89da79ef-5946-4cf5-9429-01a43cada23c" class="plotly-graph-div" style="height:525px; width:100%;"></div>            <script type="text/javascript">                require(["plotly"], function(Plotly) {                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById("89da79ef-5946-4cf5-9429-01a43cada23c")) {                    Plotly.newPlot(                        "89da79ef-5946-4cf5-9429-01a43cada23c",                        [{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Train<br>Metric=F1<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Train","marker":{"color":"#636efa","pattern":{"shape":""}},"name":"Train","offsetgroup":"Train","orientation":"v","showlegend":true,"x":[0,1],"xaxis":"x","y":[0.9998737533139755,0.9960159362549801],"yaxis":"y","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Train<br>Metric=Precision<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Train","marker":{"color":"#636efa","pattern":{"shape":""}},"name":"Train","offsetgroup":"Train","orientation":"v","showlegend":false,"x":[0,1],"xaxis":"x2","y":[0.9997475385003787,1.0],"yaxis":"y2","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Train<br>Metric=Recall<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Train","marker":{"color":"#636efa","pattern":{"shape":""}},"name":"Train","offsetgroup":"Train","orientation":"v","showlegend":false,"x":[0,1],"xaxis":"x3","y":[1.0,0.9920634920634921],"yaxis":"y3","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Test<br>Metric=F1<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Test","marker":{"color":"#EF553B","pattern":{"shape":""}},"name":"Test","offsetgroup":"Test","orientation":"v","showlegend":true,"x":[0,1],"xaxis":"x","y":[0.9969210824825797,0.8972972972972973],"yaxis":"y","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Test<br>Metric=Precision<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Test","marker":{"color":"#EF553B","pattern":{"shape":""}},"name":"Test","offsetgroup":"Test","orientation":"v","showlegend":false,"x":[0,1],"xaxis":"x2","y":[0.9951472015528955,0.9540229885057471],"yaxis":"y2","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Test<br>Metric=Recall<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Test","marker":{"color":"#EF553B","pattern":{"shape":""}},"name":"Test","offsetgroup":"Test","orientation":"v","showlegend":false,"x":[0,1],"xaxis":"x3","y":[0.9987012987012988,0.8469387755102041],"yaxis":"y3","type":"histogram"}],                        {"template":{"data":{"bar":[{"error_x":{"color":"#2a3f5f"},"error_y":{"color":"#2a3f5f"},"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"bar"}],"barpolar":[{"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"barpolar"}],"carpet":[{"aaxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"baxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"type":"carpet"}],"choropleth":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"choropleth"}],"contour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"contour"}],"contourcarpet":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"contourcarpet"}],"heatmap":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmap"}],"heatmapgl":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmapgl"}],"histogram":[{"marker":{"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"histogram"}],"histogram2d":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2d"}],"histogram2dcontour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2dcontour"}],"mesh3d":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"mesh3d"}],"parcoords":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"parcoords"}],"pie":[{"automargin":true,"type":"pie"}],"scatter":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter"}],"scatter3d":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter3d"}],"scattercarpet":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattercarpet"}],"scattergeo":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergeo"}],"scattergl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergl"}],"scattermapbox":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattermapbox"}],"scatterpolar":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolar"}],"scatterpolargl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolargl"}],"scatterternary":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterternary"}],"surface":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"surface"}],"table":[{"cells":{"fill":{"color":"#EBF0F8"},"line":{"color":"white"}},"header":{"fill":{"color":"#C8D4E3"},"line":{"color":"white"}},"type":"table"}]},"layout":{"annotationdefaults":{"arrowcolor":"#2a3f5f","arrowhead":0,"arrowwidth":1},"autotypenumbers":"strict","coloraxis":{"colorbar":{"outlinewidth":0,"ticks":""}},"colorscale":{"diverging":[[0,"#8e0152"],[0.1,"#c51b7d"],[0.2,"#de77ae"],[0.3,"#f1b6da"],[0.4,"#fde0ef"],[0.5,"#f7f7f7"],[0.6,"#e6f5d0"],[0.7,"#b8e186"],[0.8,"#7fbc41"],[0.9,"#4d9221"],[1,"#276419"]],"sequential":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"sequentialminus":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]},"colorway":["#636efa","#EF553B","#00cc96","#ab63fa","#FFA15A","#19d3f3","#FF6692","#B6E880","#FF97FF","#FECB52"],"font":{"color":"#2a3f5f"},"geo":{"bgcolor":"white","lakecolor":"white","landcolor":"#E5ECF6","showlakes":true,"showland":true,"subunitcolor":"white"},"hoverlabel":{"align":"left"},"hovermode":"closest","mapbox":{"style":"light"},"paper_bgcolor":"white","plot_bgcolor":"#E5ECF6","polar":{"angularaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","radialaxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"scene":{"xaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"yaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"zaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"}},"shapedefaults":{"line":{"color":"#2a3f5f"}},"ternary":{"aaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"baxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","caxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"title":{"x":0.05},"xaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2},"yaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2}}},"xaxis":{"anchor":"y","domain":[0.0,0.3],"title":{},"tickprefix":"Class ","tickangle":60,"type":"category"},"yaxis":{"anchor":"x","domain":[0.0,1.0],"title":{},"showticklabels":true},"xaxis2":{"anchor":"y2","domain":[0.35,0.6499999999999999],"matches":"x","title":{},"tickprefix":"Class ","tickangle":60,"type":"category"},"yaxis2":{"anchor":"x2","domain":[0.0,1.0],"showticklabels":true,"title":{}},"xaxis3":{"anchor":"y3","domain":[0.7,1.0],"matches":"x","title":{},"tickprefix":"Class ","tickangle":60,"type":"category"},"yaxis3":{"anchor":"x3","domain":[0.0,1.0],"showticklabels":true,"title":{}},"annotations":[{"font":{},"showarrow":false,"text":"F1","x":0.15,"xanchor":"center","xref":"paper","y":1.0,"yanchor":"bottom","yref":"paper"},{"font":{},"showarrow":false,"text":"Precision","x":0.49999999999999994,"xanchor":"center","xref":"paper","y":1.0,"yanchor":"bottom","yref":"paper"},{"font":{},"showarrow":false,"text":"Recall","x":0.85,"xanchor":"center","xref":"paper","y":1.0,"yanchor":"bottom","yref":"paper"}],"legend":{"title":{"text":"Dataset"},"tracegroupgap":0},"margin":{"t":60},"barmode":"group"},                        {"responsive": true}                    ).then(function(){
                            
    var gd = document.getElementById('89da79ef-5946-4cf5-9429-01a43cada23c');
    var x = new MutationObserver(function (mutations, observer) {{
            var display = window.getComputedStyle(gd).display;
            if (!display || display === 'none') {{
                console.log([gd, 'removed!']);
                Plotly.purge(gd);
                observer.disconnect();
            }}
    }});

    // Listen for the removal of the full notebook cells
    var notebookContainer = gd.closest('#notebook-container');
    if (notebookContainer) {{
        x.observe(notebookContainer, {childList: true});
    }}

    // Listen for the clearing of the current output cell
    var outputEl = gd.closest('.output');
    if (outputEl) {{
        x.observe(outputEl, {childList: true});
    }}

                            })                };                });            </script>        </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 45-51

Define a condition
==================
We can define on our check a condition that will validate that our model doesn't degrade
on new data.

Let's add a condition to the check and see what happens when it fails:

.. GENERATED FROM PYTHON SOURCE LINES 51-57

.. code-block:: default


    check = PerformanceReport()
    check.add_condition_train_test_relative_degradation_not_greater_than(0.05)
    result = check.run(train_dataset, test_dataset, model)
    result.show(show_additional_outputs=False)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Performance Report</title>
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
        <script>
            (!window.navigator.onLine ? alert('needs internet connection') : '');
        </script>
    </head>
    <body>
    

    <!-- Load require.js. Delete this if your page already loads require.js -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" crossorigin="anonymous"></script>
    <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed-amd.js" crossorigin="anonymous"></script>

    <script type="application/vnd.jupyter.widget-state+json">
    {
      "version_major": 2,
      "version_minor": 0,
      "state": {
        "5c12e41bd6254771977966a815b0e0c1": {
          "model_name": "VBoxModel",
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [
              "rendered_html",
              "jp-RenderedHTMLCommon",
              "jp-RenderedHTML",
              "jp-OutputArea-output"
            ],
            "children": [
              "IPY_MODEL_2c926c98c978490ab41bf5cc09507b3e",
              "IPY_MODEL_ac675be139574d009ebcbadd314adaec",
              "IPY_MODEL_8c2fc2eaeb4f4e39a9702b942f30293d"
            ],
            "layout": "IPY_MODEL_05744712f0c4481593e888e921fc545b"
          }
        },
        "2c926c98c978490ab41bf5cc09507b3e": {
          "model_name": "HTMLModel",
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "layout": "IPY_MODEL_59d7d5791a64407a81b491a0a88f0519",
            "style": "IPY_MODEL_c68978f66ec34a3aa6ee1b0f4b8790fd",
            "value": "<h4><b>Performance Report</b></h4>"
          }
        },
        "59d7d5791a64407a81b491a0a88f0519": {
          "model_name": "LayoutModel",
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "state": {}
        },
        "c68978f66ec34a3aa6ee1b0f4b8790fd": {
          "model_name": "DescriptionStyleModel",
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "state": {}
        },
        "ac675be139574d009ebcbadd314adaec": {
          "model_name": "HTMLModel",
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "layout": "IPY_MODEL_8e3528e6daff47c5b591a82f465831d3",
            "style": "IPY_MODEL_7cf159d8d52d4506a4677554460d8ed5",
            "value": "<p>Summarize given scores on a dataset and model.</p>"
          }
        },
        "8e3528e6daff47c5b591a82f465831d3": {
          "model_name": "LayoutModel",
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "state": {}
        },
        "7cf159d8d52d4506a4677554460d8ed5": {
          "model_name": "DescriptionStyleModel",
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "state": {}
        },
        "8c2fc2eaeb4f4e39a9702b942f30293d": {
          "model_name": "HTMLModel",
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "layout": "IPY_MODEL_220b73a4d29d4cecb05c1163d5b28b39",
            "style": "IPY_MODEL_0bb03550ee774bef9a9626a4394372b2",
            "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_69e05 table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_69e05 thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_69e05 tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_69e05 th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_69e05 td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_69e05\">\n  <thead>\n    <tr>\n      <th id=\"T_69e05_level0_col0\" class=\"col_heading level0 col0\" >Status</th>\n      <th id=\"T_69e05_level0_col1\" class=\"col_heading level0 col1\" >Condition</th>\n      <th id=\"T_69e05_level0_col2\" class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_69e05_row0_col0\" class=\"data row0 col0\" ><div style=\"color: red;text-align: center\">\u2716</div></td>\n      <td id=\"T_69e05_row0_col1\" class=\"data row0 col1\" >Train-Test scores relative degradation is not greater than 0.05</td>\n      <td id=\"T_69e05_row0_col2\" class=\"data row0 col2\" >F1 for class 1 (train=1 test=0.9)\nRecall for class 1 (train=0.99 test=0.85)</td>\n    </tr>\n  </tbody>\n</table>\n"
          }
        },
        "220b73a4d29d4cecb05c1163d5b28b39": {
          "model_name": "LayoutModel",
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "state": {}
        },
        "0bb03550ee774bef9a9626a4394372b2": {
          "model_name": "DescriptionStyleModel",
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "state": {}
        },
        "05744712f0c4481593e888e921fc545b": {
          "model_name": "LayoutModel",
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "state": {}
        }
      }
    }
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "5c12e41bd6254771977966a815b0e0c1"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 58-59

We detected that for class "0" our the Precision result is degraded by more than 5%

.. GENERATED FROM PYTHON SOURCE LINES 61-64

Using alternative scorers
=========================
We can define alternative scorers that are not run by default:

.. GENERATED FROM PYTHON SOURCE LINES 64-71

.. code-block:: default


    from sklearn.metrics import fbeta_score, make_scorer

    fbeta_scorer = make_scorer(fbeta_score, labels=[0, 1], average=None, beta=0.2)

    check = PerformanceReport(alternative_scorers={'my scorer': fbeta_scorer})
    check.run(train_dataset, test_dataset, model)





.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">

    <script
        src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js"
        integrity="sha512-c3Nl8+7g4LMSTdrm621y7kf9v3SDPnhxLNhcjFJbKECVnmZHTdo+IRO05sNLTH/D3vA6u1X32ehoLC7WFVdheg=="
        crossorigin="anonymous"
        referrerpolicy="no-referrer">
    </script>

    <script type="text/javascript">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>
    <script type="text/javascript">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: "STIX-Web"}});}</script>
    <script type="text/javascript">
        if (typeof require !== 'undefined') {
            require.undef("plotly");
            requirejs.config({
                paths: {'plotly': ['https://cdn.plot.ly/plotly-2.8.3.min']}
            });
            require(
                ['plotly'],
                function(Plotly) {
                    window._Plotly = Plotly;
                    window.Plotly = Plotly;
                    console.log('Loaded plotly successfully');
                },
                function() {console.log('Failed to load plotly')}
            );
        } else {
            console.log('requirejs is not present');
        }
    </script>
    <h4><b>Performance Report</b></h4><p>Summarize given scores on a dataset and model.</p><h5><b>Additional Outputs</b></h5><div>                            <div id="90a6305e-6526-4fee-ba61-0fe62dcf2d07" class="plotly-graph-div" style="height:525px; width:100%;"></div>            <script type="text/javascript">                require(["plotly"], function(Plotly) {                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById("90a6305e-6526-4fee-ba61-0fe62dcf2d07")) {                    Plotly.newPlot(                        "90a6305e-6526-4fee-ba61-0fe62dcf2d07",                        [{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Train<br>Metric=my scorer<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Train","marker":{"color":"#636efa","pattern":{"shape":""}},"name":"Train","offsetgroup":"Train","orientation":"v","showlegend":true,"x":[0,1],"xaxis":"x","y":[0.999757246200903,0.9996924023377423],"yaxis":"y","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Test<br>Metric=my scorer<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Test","marker":{"color":"#EF553B","pattern":{"shape":""}},"name":"Test","offsetgroup":"Test","orientation":"v","showlegend":true,"x":[0,1],"xaxis":"x","y":[0.9952834297803498,0.9494060712714474],"yaxis":"y","type":"histogram"}],                        {"template":{"data":{"bar":[{"error_x":{"color":"#2a3f5f"},"error_y":{"color":"#2a3f5f"},"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"bar"}],"barpolar":[{"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"barpolar"}],"carpet":[{"aaxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"baxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"type":"carpet"}],"choropleth":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"choropleth"}],"contour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"contour"}],"contourcarpet":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"contourcarpet"}],"heatmap":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmap"}],"heatmapgl":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmapgl"}],"histogram":[{"marker":{"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"histogram"}],"histogram2d":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2d"}],"histogram2dcontour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2dcontour"}],"mesh3d":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"mesh3d"}],"parcoords":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"parcoords"}],"pie":[{"automargin":true,"type":"pie"}],"scatter":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter"}],"scatter3d":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter3d"}],"scattercarpet":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattercarpet"}],"scattergeo":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergeo"}],"scattergl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergl"}],"scattermapbox":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattermapbox"}],"scatterpolar":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolar"}],"scatterpolargl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolargl"}],"scatterternary":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterternary"}],"surface":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"surface"}],"table":[{"cells":{"fill":{"color":"#EBF0F8"},"line":{"color":"white"}},"header":{"fill":{"color":"#C8D4E3"},"line":{"color":"white"}},"type":"table"}]},"layout":{"annotationdefaults":{"arrowcolor":"#2a3f5f","arrowhead":0,"arrowwidth":1},"autotypenumbers":"strict","coloraxis":{"colorbar":{"outlinewidth":0,"ticks":""}},"colorscale":{"diverging":[[0,"#8e0152"],[0.1,"#c51b7d"],[0.2,"#de77ae"],[0.3,"#f1b6da"],[0.4,"#fde0ef"],[0.5,"#f7f7f7"],[0.6,"#e6f5d0"],[0.7,"#b8e186"],[0.8,"#7fbc41"],[0.9,"#4d9221"],[1,"#276419"]],"sequential":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"sequentialminus":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]},"colorway":["#636efa","#EF553B","#00cc96","#ab63fa","#FFA15A","#19d3f3","#FF6692","#B6E880","#FF97FF","#FECB52"],"font":{"color":"#2a3f5f"},"geo":{"bgcolor":"white","lakecolor":"white","landcolor":"#E5ECF6","showlakes":true,"showland":true,"subunitcolor":"white"},"hoverlabel":{"align":"left"},"hovermode":"closest","mapbox":{"style":"light"},"paper_bgcolor":"white","plot_bgcolor":"#E5ECF6","polar":{"angularaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","radialaxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"scene":{"xaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"yaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"zaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"}},"shapedefaults":{"line":{"color":"#2a3f5f"}},"ternary":{"aaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"baxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","caxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"title":{"x":0.05},"xaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2},"yaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2}}},"xaxis":{"anchor":"y","domain":[0.0,1.0],"title":{},"tickprefix":"Class ","tickangle":60,"type":"category"},"yaxis":{"anchor":"x","domain":[0.0,1.0],"title":{},"showticklabels":true},"annotations":[{"font":{},"showarrow":false,"text":"my scorer","x":0.5,"xanchor":"center","xref":"paper","y":1.0,"yanchor":"bottom","yref":"paper"}],"legend":{"title":{"text":"Dataset"},"tracegroupgap":0},"margin":{"t":60},"barmode":"group"},                        {"responsive": true}                    ).then(function(){
                            
    var gd = document.getElementById('90a6305e-6526-4fee-ba61-0fe62dcf2d07');
    var x = new MutationObserver(function (mutations, observer) {{
            var display = window.getComputedStyle(gd).display;
            if (!display || display === 'none') {{
                console.log([gd, 'removed!']);
                Plotly.purge(gd);
                observer.disconnect();
            }}
    }});

    // Listen for the removal of the full notebook cells
    var notebookContainer = gd.closest('#notebook-container');
    if (notebookContainer) {{
        x.observe(notebookContainer, {childList: true});
    }}

    // Listen for the clearing of the current output cell
    var outputEl = gd.closest('.output');
    if (outputEl) {{
        x.observe(outputEl, {childList: true});
    }}

                            })                };                });            </script>        </div>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  7.770 seconds)


.. _sphx_glr_download_checks_gallery_tabular_performance_plot_performance_report.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: plot_performance_report.py <plot_performance_report.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: plot_performance_report.ipynb <plot_performance_report.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_