.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tabular/auto_checks/data_integrity/plot_data_duplicates.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_tabular_auto_checks_data_integrity_plot_data_duplicates.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_tabular_auto_checks_data_integrity_plot_data_duplicates.py:


.. _tabular__data_duplicates:

Data Duplicates
***************

This notebook provides an overview for using and understanding the data duplicates check:

**Structure:**

* `Why data duplicates? <#why-data-duplicates>`__
* `Load Data <#load-data>`__
* `Run the Check <#run-the-check>`__
* `Define a Condition <#define-a-condition>`__

.. GENERATED FROM PYTHON SOURCE LINES 19-26

.. code-block:: default


    from datetime import datetime

    import pandas as pd

    from deepchecks.tabular.datasets.classification.phishing import load_data








.. GENERATED FROM PYTHON SOURCE LINES 27-38

Why data duplicates?
====================
The ``DataDuplicates`` check finds multiple instances of identical samples in the
Dataset. Duplicate samples increase the weight the model gives to those samples.
If these duplicates are there intentionally (e.g. as a result of intentional
oversampling, or due to the dataset's nature it has identical-looking samples)
this may be valid, however if this is an hidden issue we're not expecting to occur,
it may be an indicator for a problem in the data pipeline that requires attention.

Load Data
=========

.. GENERATED FROM PYTHON SOURCE LINES 38-43

.. code-block:: default



    phishing_dataset = load_data(as_train_test=False, data_format='DataFrame')
    phishing_dataset






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>target</th>
          <th>month</th>
          <th>scrape_date</th>
          <th>ext</th>
          <th>urlLength</th>
          <th>numDigits</th>
          <th>numParams</th>
          <th>num_%20</th>
          <th>num_@</th>
          <th>entropy</th>
          <th>has_ip</th>
          <th>hasHttp</th>
          <th>hasHttps</th>
          <th>urlIsLive</th>
          <th>dsr</th>
          <th>dse</th>
          <th>bodyLength</th>
          <th>numTitles</th>
          <th>numImages</th>
          <th>numLinks</th>
          <th>specialChars</th>
          <th>scriptLength</th>
          <th>sbr</th>
          <th>bscr</th>
          <th>sscr</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>net</td>
          <td>102</td>
          <td>8</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.384032</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>4921</td>
          <td>191</td>
          <td>32486</td>
          <td>3</td>
          <td>5</td>
          <td>330</td>
          <td>9419</td>
          <td>23919</td>
          <td>0.736286</td>
          <td>0.289940</td>
          <td>2.539442</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>country</td>
          <td>154</td>
          <td>60</td>
          <td>0</td>
          <td>2</td>
          <td>0</td>
          <td>-3.566515</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>0</td>
          <td>0</td>
          <td>16199</td>
          <td>0</td>
          <td>4</td>
          <td>39</td>
          <td>2735</td>
          <td>794</td>
          <td>0.049015</td>
          <td>0.168838</td>
          <td>0.290311</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>net</td>
          <td>171</td>
          <td>5</td>
          <td>11</td>
          <td>0</td>
          <td>0</td>
          <td>-4.608755</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>5374</td>
          <td>104</td>
          <td>103344</td>
          <td>18</td>
          <td>9</td>
          <td>302</td>
          <td>27798</td>
          <td>83817</td>
          <td>0.811049</td>
          <td>0.268985</td>
          <td>2.412174</td>
        </tr>
        <tr>
          <th>3</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>com</td>
          <td>94</td>
          <td>10</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.548921</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>6107</td>
          <td>466</td>
          <td>34093</td>
          <td>11</td>
          <td>43</td>
          <td>199</td>
          <td>9087</td>
          <td>19427</td>
          <td>0.569824</td>
          <td>0.266536</td>
          <td>2.137889</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>other</td>
          <td>95</td>
          <td>11</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.717188</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>3819</td>
          <td>928</td>
          <td>202</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>39</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.193069</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>11345</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>country</td>
          <td>89</td>
          <td>7</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.254491</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>0</td>
          <td>0</td>
          <td>4117</td>
          <td>5</td>
          <td>0</td>
          <td>1</td>
          <td>971</td>
          <td>1866</td>
          <td>0.625302</td>
          <td>0.213266</td>
          <td>2.932029</td>
        </tr>
        <tr>
          <th>11346</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>other</td>
          <td>107</td>
          <td>13</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.758879</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>9073</td>
          <td>1882</td>
          <td>17788</td>
          <td>47</td>
          <td>58</td>
          <td>645</td>
          <td>3185</td>
          <td>4228</td>
          <td>0.291069</td>
          <td>0.214348</td>
          <td>1.357928</td>
        </tr>
        <tr>
          <th>11347</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>com</td>
          <td>112</td>
          <td>10</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.723014</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>2640</td>
          <td>1011</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>11348</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>html</td>
          <td>111</td>
          <td>3</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.289384</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>2291</td>
          <td>265</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>11349</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>html</td>
          <td>97</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.304523</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>6273</td>
          <td>298</td>
          <td>149</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>25</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.167785</td>
          <td>0.000000</td>
        </tr>
      </tbody>
    </table>
    <p>11350 rows × 25 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 44-46

Run the Check
=============

.. GENERATED FROM PYTHON SOURCE LINES 46-58

.. code-block:: default


    from deepchecks.tabular.checks import DataDuplicates

    DataDuplicates().run(phishing_dataset)

    # With Check Parameters
    # ---------------------
    # ``DataDuplicates`` check can also use a specific subset of columns (or alternatively
    # use all columns except specific ignore_columns to check duplication):

    DataDuplicates(columns=["entropy", "numParams"]).run(phishing_dataset)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2023 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"2320dd9aa1e14c788d007771ad2f22e8": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_1c3b4f8815bf47d78457cf5eb994363f", "IPY_MODEL_70c9a5310d2c4ad7b8a017a59dbc79b0", "IPY_MODEL_072489a49d884881b695abbec41f0574", "IPY_MODEL_b9cb6b5f2dca46c6bde3d636921967b4"], "layout": "IPY_MODEL_46e0c63f3e5146c6b45011ee8efc9a9e"}}, "1c3b4f8815bf47d78457cf5eb994363f": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_abde1b94d0d0480f802bade41b510a4d", "style": "IPY_MODEL_40123d0ccb5841bbab588d86c9bdd5bb", "value": "<h4><b>Data Duplicates</b></h4>"}}, "abde1b94d0d0480f802bade41b510a4d": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "40123d0ccb5841bbab588d86c9bdd5bb": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "70c9a5310d2c4ad7b8a017a59dbc79b0": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_be6e20e53c5f4c5f80179185da5e62a1", "style": "IPY_MODEL_8693d25d903244edba08be0a301db26d", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.18/tabular/auto_checks/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "be6e20e53c5f4c5f80179185da5e62a1": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "8693d25d903244edba08be0a301db26d": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "072489a49d884881b695abbec41f0574": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_d25d487ae67343e69d7708cfbf9a3481", "style": "IPY_MODEL_9ba81b3a32d24942a8104a8353aae22c"}}, "d25d487ae67343e69d7708cfbf9a3481": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "9ba81b3a32d24942a8104a8353aae22c": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "b9cb6b5f2dca46c6bde3d636921967b4": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_954b8b0976f048df817f40db9890bc58", "IPY_MODEL_c1f5a6b1e1c840d5a8047da1056cee27", "IPY_MODEL_1de5719f128e471f9845c2d65292f15d", "IPY_MODEL_08aabd879fe0417b97c295c222c62480"], "layout": "IPY_MODEL_50cf18eec88e4d5ebb867dfbc53675ad"}}, "954b8b0976f048df817f40db9890bc58": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_c4aa14a91c4b46f3b3f75d0558ebc028", "style": "IPY_MODEL_c4cd56e7aed6489a80a1065e7e577006", "value": "<h5><b>Additional Outputs</b></h5>"}}, "c4aa14a91c4b46f3b3f75d0558ebc028": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "c4cd56e7aed6489a80a1065e7e577006": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "c1f5a6b1e1c840d5a8047da1056cee27": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_aea8b5125dba48b294affd1471fe9027", "style": "IPY_MODEL_5515efd16b3141ab930da8cfb9ed16f5", "value": "<div>4.11% of data samples are duplicates. </div>"}}, "aea8b5125dba48b294affd1471fe9027": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "5515efd16b3141ab930da8cfb9ed16f5": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "1de5719f128e471f9845c2d65292f15d": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_bf83713f6a8d488ab0988f6073e0e3f8", "style": "IPY_MODEL_ce7899e5bcc54fc79238066013e2f460", "value": "<div>Each row in the table shows an example of duplicate data and the number of times it appears.</div>"}}, "bf83713f6a8d488ab0988f6073e0e3f8": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "ce7899e5bcc54fc79238066013e2f460": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "08aabd879fe0417b97c295c222c62480": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_a3c359d52e724cdf9b65f63a8cada681", "style": "IPY_MODEL_a4e8119f48b147d2ab55a57b7b5d7d78", "value": "<style type=\"text/css\">\n#T_571e7_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_571e7_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_571e7_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_571e7_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_571e7_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_571e7_\">\n  <thead>\n    <tr>\n      <th class=\"blank\" >&nbsp;</th>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >entropy</th>\n      <th class=\"col_heading level0 col1\" >numParams</th>\n    </tr>\n    <tr>\n      <th class=\"index_name level0\" >Instances</th>\n      <th class=\"index_name level1\" >Number of Duplicates</th>\n      <th class=\"blank col0\" >&nbsp;</th>\n      <th class=\"blank col1\" >&nbsp;</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_571e7_level0_row0\" class=\"row_heading level0 row0\" >2360, 6560, 11342, 2150, 3528, 1557, 810...</th>\n      <th id=\"T_571e7_level1_row0\" class=\"row_heading level1 row0\" >13</th>\n      <td id=\"T_571e7_row0_col0\" class=\"data row0 col0\" >-4.31</td>\n      <td id=\"T_571e7_row0_col1\" class=\"data row0 col1\" >0</td>\n    </tr>\n    <tr>\n      <th id=\"T_571e7_level0_row1\" class=\"row_heading level0 row1\" >4638, 4412, 1729, 2213, 1641, 2234, 6328...</th>\n      <th id=\"T_571e7_level1_row1\" class=\"row_heading level1 row1\" >8</th>\n      <td id=\"T_571e7_row1_col0\" class=\"data row1 col0\" >-4.57</td>\n      <td id=\"T_571e7_row1_col1\" class=\"data row1 col1\" >4</td>\n    </tr>\n    <tr>\n      <th id=\"T_571e7_level0_row2\" class=\"row_heading level0 row2\" >6774, 7528, 9592, 4634, 10046, 6504, 271...</th>\n      <th id=\"T_571e7_level1_row2\" class=\"row_heading level1 row2\" >8</th>\n      <td id=\"T_571e7_row2_col0\" class=\"data row2 col0\" >-4.49</td>\n      <td id=\"T_571e7_row2_col1\" class=\"data row2 col1\" >8</td>\n    </tr>\n    <tr>\n      <th id=\"T_571e7_level0_row3\" class=\"row_heading level0 row3\" >4047, 8391, 2499, 9932, 929, 11345, 9348...</th>\n      <th id=\"T_571e7_level1_row3\" class=\"row_heading level1 row3\" >8</th>\n      <td id=\"T_571e7_row3_col0\" class=\"data row3 col0\" >-4.25</td>\n      <td id=\"T_571e7_row3_col1\" class=\"data row3 col1\" >0</td>\n    </tr>\n    <tr>\n      <th id=\"T_571e7_level0_row4\" class=\"row_heading level0 row4\" >2984, 1670, 1020, 6666, 10923, 9138, 180...</th>\n      <th id=\"T_571e7_level1_row4\" class=\"row_heading level1 row4\" >7</th>\n      <td id=\"T_571e7_row4_col0\" class=\"data row4 col0\" >-4.65</td>\n      <td id=\"T_571e7_row4_col1\" class=\"data row4 col1\" >5</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "a3c359d52e724cdf9b65f63a8cada681": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "a4e8119f48b147d2ab55a57b7b5d7d78": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "50cf18eec88e4d5ebb867dfbc53675ad": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "46e0c63f3e5146c6b45011ee8efc9a9e": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "2320dd9aa1e14c788d007771ad2f22e8"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 59-62

.. code-block:: default


    DataDuplicates(ignore_columns=["scrape_date"], n_to_show=10).run(phishing_dataset)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2023 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"ba4f593a0cee4f749d3c2d94ce38b008": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_0c84b45238454a46b327cf06085090c8", "IPY_MODEL_fb0e88be1f5a42e5af0132cda40f94f6", "IPY_MODEL_7db076350e4a42beaeb039a877ddd7bb", "IPY_MODEL_e73ad267e0bc4014918f35687bb7c59e"], "layout": "IPY_MODEL_b0f8eec7913a49cfa10cec3de7d9ee62"}}, "0c84b45238454a46b327cf06085090c8": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_a09669c8e2b246f3a742ded8789d2a01", "style": "IPY_MODEL_46378c89811b4794910cf74d206b992c", "value": "<h4><b>Data Duplicates</b></h4>"}}, "a09669c8e2b246f3a742ded8789d2a01": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "46378c89811b4794910cf74d206b992c": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "fb0e88be1f5a42e5af0132cda40f94f6": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_6a818819ec5c40c6a7e17c4807ef2ac0", "style": "IPY_MODEL_aca78ecb7ce448199720c24a27330278", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.18/tabular/auto_checks/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "6a818819ec5c40c6a7e17c4807ef2ac0": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "aca78ecb7ce448199720c24a27330278": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "7db076350e4a42beaeb039a877ddd7bb": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_5b192de903cb45ed832de48f30bb80e5", "style": "IPY_MODEL_13db84d908aa4d1ebe9de8880472935b"}}, "5b192de903cb45ed832de48f30bb80e5": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "13db84d908aa4d1ebe9de8880472935b": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "e73ad267e0bc4014918f35687bb7c59e": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_910b30520c574c9d88a2366e57a9cdbe", "IPY_MODEL_55b50e9e1b744136a6a2befd2c7366a7", "IPY_MODEL_3c2a6ae9913f4f1da0fad6497337c074", "IPY_MODEL_8624aec8dda7412c83eb2b1cf695a948"], "layout": "IPY_MODEL_62c2312e318748318397ae31f0cc9a28"}}, "910b30520c574c9d88a2366e57a9cdbe": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_1ce01ba751b6470c8cc08fb602208f01", "style": "IPY_MODEL_e33a4b4f838c42508ce13f210151d14d", "value": "<h5><b>Additional Outputs</b></h5>"}}, "1ce01ba751b6470c8cc08fb602208f01": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "e33a4b4f838c42508ce13f210151d14d": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "55b50e9e1b744136a6a2befd2c7366a7": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_d43cb9e702e546569d6bce85069ea153", "style": "IPY_MODEL_7d11a17ea27c41c2bb6d39ba4f4a7abc", "value": "<div>0.22% of data samples are duplicates. </div>"}}, "d43cb9e702e546569d6bce85069ea153": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "7d11a17ea27c41c2bb6d39ba4f4a7abc": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "3c2a6ae9913f4f1da0fad6497337c074": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_2243433413594583bdfabf60f65ab7de", "style": "IPY_MODEL_d04863f31abf4034a8e23457c64f8e3b", "value": "<div>Each row in the table shows an example of duplicate data and the number of times it appears.</div>"}}, "2243433413594583bdfabf60f65ab7de": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "d04863f31abf4034a8e23457c64f8e3b": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "8624aec8dda7412c83eb2b1cf695a948": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_cad6e55af6bb47a09e7ae9fdcb27b5d3", "style": "IPY_MODEL_84a037e7f45b4c20b0982179e1761be8", "value": "<style type=\"text/css\">\n#T_9b4ca_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_9b4ca_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_9b4ca_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_9b4ca_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_9b4ca_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_9b4ca_\">\n  <thead>\n    <tr>\n      <th class=\"blank\" >&nbsp;</th>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >target</th>\n      <th class=\"col_heading level0 col1\" >month</th>\n      <th class=\"col_heading level0 col2\" >ext</th>\n      <th class=\"col_heading level0 col3\" >urlLength</th>\n      <th class=\"col_heading level0 col4\" >numDigits</th>\n      <th class=\"col_heading level0 col5\" >numParams</th>\n      <th class=\"col_heading level0 col6\" >num_%20</th>\n      <th class=\"col_heading level0 col7\" >num_@</th>\n      <th class=\"col_heading level0 col8\" >entropy</th>\n      <th class=\"col_heading level0 col9\" >has_ip</th>\n      <th class=\"col_heading level0 col10\" >hasHttp</th>\n      <th class=\"col_heading level0 col11\" >hasHttps</th>\n      <th class=\"col_heading level0 col12\" >urlIsLive</th>\n      <th class=\"col_heading level0 col13\" >dsr</th>\n      <th class=\"col_heading level0 col14\" >dse</th>\n      <th class=\"col_heading level0 col15\" >bodyLength</th>\n      <th class=\"col_heading level0 col16\" >numTitles</th>\n      <th class=\"col_heading level0 col17\" >numImages</th>\n      <th class=\"col_heading level0 col18\" >numLinks</th>\n      <th class=\"col_heading level0 col19\" >specialChars</th>\n      <th class=\"col_heading level0 col20\" >scriptLength</th>\n      <th class=\"col_heading level0 col21\" >sbr</th>\n      <th class=\"col_heading level0 col22\" >bscr</th>\n      <th class=\"col_heading level0 col23\" >sscr</th>\n    </tr>\n    <tr>\n      <th class=\"index_name level0\" >Instances</th>\n      <th class=\"index_name level1\" >Number of Duplicates</th>\n      <th class=\"blank col0\" >&nbsp;</th>\n      <th class=\"blank col1\" >&nbsp;</th>\n      <th class=\"blank col2\" >&nbsp;</th>\n      <th class=\"blank col3\" >&nbsp;</th>\n      <th class=\"blank col4\" >&nbsp;</th>\n      <th class=\"blank col5\" >&nbsp;</th>\n      <th class=\"blank col6\" >&nbsp;</th>\n      <th class=\"blank col7\" >&nbsp;</th>\n      <th class=\"blank col8\" >&nbsp;</th>\n      <th class=\"blank col9\" >&nbsp;</th>\n      <th class=\"blank col10\" >&nbsp;</th>\n      <th class=\"blank col11\" >&nbsp;</th>\n      <th class=\"blank col12\" >&nbsp;</th>\n      <th class=\"blank col13\" >&nbsp;</th>\n      <th class=\"blank col14\" >&nbsp;</th>\n      <th class=\"blank col15\" >&nbsp;</th>\n      <th class=\"blank col16\" >&nbsp;</th>\n      <th class=\"blank col17\" >&nbsp;</th>\n      <th class=\"blank col18\" >&nbsp;</th>\n      <th class=\"blank col19\" >&nbsp;</th>\n      <th class=\"blank col20\" >&nbsp;</th>\n      <th class=\"blank col21\" >&nbsp;</th>\n      <th class=\"blank col22\" >&nbsp;</th>\n      <th class=\"blank col23\" >&nbsp;</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_9b4ca_level0_row0\" class=\"row_heading level0 row0\" >4696, 5398, 4719</th>\n      <th id=\"T_9b4ca_level1_row0\" class=\"row_heading level1 row0\" >3</th>\n      <td id=\"T_9b4ca_row0_col0\" class=\"data row0 col0\" >0</td>\n      <td id=\"T_9b4ca_row0_col1\" class=\"data row0 col1\" >6</td>\n      <td id=\"T_9b4ca_row0_col2\" class=\"data row0 col2\" >other</td>\n      <td id=\"T_9b4ca_row0_col3\" class=\"data row0 col3\" >123</td>\n      <td id=\"T_9b4ca_row0_col4\" class=\"data row0 col4\" >28</td>\n      <td id=\"T_9b4ca_row0_col5\" class=\"data row0 col5\" >4</td>\n      <td id=\"T_9b4ca_row0_col6\" class=\"data row0 col6\" >0</td>\n      <td id=\"T_9b4ca_row0_col7\" class=\"data row0 col7\" >0</td>\n      <td id=\"T_9b4ca_row0_col8\" class=\"data row0 col8\" >-4.91</td>\n      <td id=\"T_9b4ca_row0_col9\" class=\"data row0 col9\" >0</td>\n      <td id=\"T_9b4ca_row0_col10\" class=\"data row0 col10\" >True</td>\n      <td id=\"T_9b4ca_row0_col11\" class=\"data row0 col11\" >False</td>\n      <td id=\"T_9b4ca_row0_col12\" class=\"data row0 col12\" >False</td>\n      <td id=\"T_9b4ca_row0_col13\" class=\"data row0 col13\" >0</td>\n      <td id=\"T_9b4ca_row0_col14\" class=\"data row0 col14\" >0</td>\n      <td id=\"T_9b4ca_row0_col15\" class=\"data row0 col15\" >0</td>\n      <td id=\"T_9b4ca_row0_col16\" class=\"data row0 col16\" >0</td>\n      <td id=\"T_9b4ca_row0_col17\" class=\"data row0 col17\" >0</td>\n      <td id=\"T_9b4ca_row0_col18\" class=\"data row0 col18\" >0</td>\n      <td id=\"T_9b4ca_row0_col19\" class=\"data row0 col19\" >0</td>\n      <td id=\"T_9b4ca_row0_col20\" class=\"data row0 col20\" >0</td>\n      <td id=\"T_9b4ca_row0_col21\" class=\"data row0 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row0_col22\" class=\"data row0 col22\" >0.00</td>\n      <td id=\"T_9b4ca_row0_col23\" class=\"data row0 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row1\" class=\"row_heading level0 row1\" >11342, 82</th>\n      <th id=\"T_9b4ca_level1_row1\" class=\"row_heading level1 row1\" >2</th>\n      <td id=\"T_9b4ca_row1_col0\" class=\"data row1 col0\" >0</td>\n      <td id=\"T_9b4ca_row1_col1\" class=\"data row1 col1\" >1</td>\n      <td id=\"T_9b4ca_row1_col2\" class=\"data row1 col2\" >html</td>\n      <td id=\"T_9b4ca_row1_col3\" class=\"data row1 col3\" >92</td>\n      <td id=\"T_9b4ca_row1_col4\" class=\"data row1 col4\" >2</td>\n      <td id=\"T_9b4ca_row1_col5\" class=\"data row1 col5\" >0</td>\n      <td id=\"T_9b4ca_row1_col6\" class=\"data row1 col6\" >0</td>\n      <td id=\"T_9b4ca_row1_col7\" class=\"data row1 col7\" >0</td>\n      <td id=\"T_9b4ca_row1_col8\" class=\"data row1 col8\" >-4.31</td>\n      <td id=\"T_9b4ca_row1_col9\" class=\"data row1 col9\" >0</td>\n      <td id=\"T_9b4ca_row1_col10\" class=\"data row1 col10\" >True</td>\n      <td id=\"T_9b4ca_row1_col11\" class=\"data row1 col11\" >False</td>\n      <td id=\"T_9b4ca_row1_col12\" class=\"data row1 col12\" >False</td>\n      <td id=\"T_9b4ca_row1_col13\" class=\"data row1 col13\" >0</td>\n      <td id=\"T_9b4ca_row1_col14\" class=\"data row1 col14\" >0</td>\n      <td id=\"T_9b4ca_row1_col15\" class=\"data row1 col15\" >149</td>\n      <td id=\"T_9b4ca_row1_col16\" class=\"data row1 col16\" >1</td>\n      <td id=\"T_9b4ca_row1_col17\" class=\"data row1 col17\" >0</td>\n      <td id=\"T_9b4ca_row1_col18\" class=\"data row1 col18\" >0</td>\n      <td id=\"T_9b4ca_row1_col19\" class=\"data row1 col19\" >25</td>\n      <td id=\"T_9b4ca_row1_col20\" class=\"data row1 col20\" >0</td>\n      <td id=\"T_9b4ca_row1_col21\" class=\"data row1 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row1_col22\" class=\"data row1 col22\" >0.17</td>\n      <td id=\"T_9b4ca_row1_col23\" class=\"data row1 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row2\" class=\"row_heading level0 row2\" >790, 250</th>\n      <th id=\"T_9b4ca_level1_row2\" class=\"row_heading level1 row2\" >2</th>\n      <td id=\"T_9b4ca_row2_col0\" class=\"data row2 col0\" >0</td>\n      <td id=\"T_9b4ca_row2_col1\" class=\"data row2 col1\" >1</td>\n      <td id=\"T_9b4ca_row2_col2\" class=\"data row2 col2\" >php</td>\n      <td id=\"T_9b4ca_row2_col3\" class=\"data row2 col3\" >107</td>\n      <td id=\"T_9b4ca_row2_col4\" class=\"data row2 col4\" >4</td>\n      <td id=\"T_9b4ca_row2_col5\" class=\"data row2 col5\" >8</td>\n      <td id=\"T_9b4ca_row2_col6\" class=\"data row2 col6\" >0</td>\n      <td id=\"T_9b4ca_row2_col7\" class=\"data row2 col7\" >0</td>\n      <td id=\"T_9b4ca_row2_col8\" class=\"data row2 col8\" >-4.53</td>\n      <td id=\"T_9b4ca_row2_col9\" class=\"data row2 col9\" >0</td>\n      <td id=\"T_9b4ca_row2_col10\" class=\"data row2 col10\" >True</td>\n      <td id=\"T_9b4ca_row2_col11\" class=\"data row2 col11\" >False</td>\n      <td id=\"T_9b4ca_row2_col12\" class=\"data row2 col12\" >False</td>\n      <td id=\"T_9b4ca_row2_col13\" class=\"data row2 col13\" >1381</td>\n      <td id=\"T_9b4ca_row2_col14\" class=\"data row2 col14\" >79</td>\n      <td id=\"T_9b4ca_row2_col15\" class=\"data row2 col15\" >0</td>\n      <td id=\"T_9b4ca_row2_col16\" class=\"data row2 col16\" >1</td>\n      <td id=\"T_9b4ca_row2_col17\" class=\"data row2 col17\" >0</td>\n      <td id=\"T_9b4ca_row2_col18\" class=\"data row2 col18\" >0</td>\n      <td id=\"T_9b4ca_row2_col19\" class=\"data row2 col19\" >0</td>\n      <td id=\"T_9b4ca_row2_col20\" class=\"data row2 col20\" >0</td>\n      <td id=\"T_9b4ca_row2_col21\" class=\"data row2 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row2_col22\" class=\"data row2 col22\" >0.00</td>\n      <td id=\"T_9b4ca_row2_col23\" class=\"data row2 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row3\" class=\"row_heading level0 row3\" >217, 6</th>\n      <th id=\"T_9b4ca_level1_row3\" class=\"row_heading level1 row3\" >2</th>\n      <td id=\"T_9b4ca_row3_col0\" class=\"data row3 col0\" >0</td>\n      <td id=\"T_9b4ca_row3_col1\" class=\"data row3 col1\" >1</td>\n      <td id=\"T_9b4ca_row3_col2\" class=\"data row3 col2\" >php</td>\n      <td id=\"T_9b4ca_row3_col3\" class=\"data row3 col3\" >107</td>\n      <td id=\"T_9b4ca_row3_col4\" class=\"data row3 col4\" >5</td>\n      <td id=\"T_9b4ca_row3_col5\" class=\"data row3 col5\" >8</td>\n      <td id=\"T_9b4ca_row3_col6\" class=\"data row3 col6\" >0</td>\n      <td id=\"T_9b4ca_row3_col7\" class=\"data row3 col7\" >0</td>\n      <td id=\"T_9b4ca_row3_col8\" class=\"data row3 col8\" >-4.52</td>\n      <td id=\"T_9b4ca_row3_col9\" class=\"data row3 col9\" >0</td>\n      <td id=\"T_9b4ca_row3_col10\" class=\"data row3 col10\" >True</td>\n      <td id=\"T_9b4ca_row3_col11\" class=\"data row3 col11\" >False</td>\n      <td id=\"T_9b4ca_row3_col12\" class=\"data row3 col12\" >False</td>\n      <td id=\"T_9b4ca_row3_col13\" class=\"data row3 col13\" >1381</td>\n      <td id=\"T_9b4ca_row3_col14\" class=\"data row3 col14\" >79</td>\n      <td id=\"T_9b4ca_row3_col15\" class=\"data row3 col15\" >0</td>\n      <td id=\"T_9b4ca_row3_col16\" class=\"data row3 col16\" >1</td>\n      <td id=\"T_9b4ca_row3_col17\" class=\"data row3 col17\" >0</td>\n      <td id=\"T_9b4ca_row3_col18\" class=\"data row3 col18\" >0</td>\n      <td id=\"T_9b4ca_row3_col19\" class=\"data row3 col19\" >0</td>\n      <td id=\"T_9b4ca_row3_col20\" class=\"data row3 col20\" >0</td>\n      <td id=\"T_9b4ca_row3_col21\" class=\"data row3 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row3_col22\" class=\"data row3 col22\" >0.00</td>\n      <td id=\"T_9b4ca_row3_col23\" class=\"data row3 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row4\" class=\"row_heading level0 row4\" >763, 609</th>\n      <th id=\"T_9b4ca_level1_row4\" class=\"row_heading level1 row4\" >2</th>\n      <td id=\"T_9b4ca_row4_col0\" class=\"data row4 col0\" >0</td>\n      <td id=\"T_9b4ca_row4_col1\" class=\"data row4 col1\" >1</td>\n      <td id=\"T_9b4ca_row4_col2\" class=\"data row4 col2\" >php</td>\n      <td id=\"T_9b4ca_row4_col3\" class=\"data row4 col3\" >113</td>\n      <td id=\"T_9b4ca_row4_col4\" class=\"data row4 col4\" >6</td>\n      <td id=\"T_9b4ca_row4_col5\" class=\"data row4 col5\" >8</td>\n      <td id=\"T_9b4ca_row4_col6\" class=\"data row4 col6\" >0</td>\n      <td id=\"T_9b4ca_row4_col7\" class=\"data row4 col7\" >0</td>\n      <td id=\"T_9b4ca_row4_col8\" class=\"data row4 col8\" >-4.63</td>\n      <td id=\"T_9b4ca_row4_col9\" class=\"data row4 col9\" >0</td>\n      <td id=\"T_9b4ca_row4_col10\" class=\"data row4 col10\" >True</td>\n      <td id=\"T_9b4ca_row4_col11\" class=\"data row4 col11\" >False</td>\n      <td id=\"T_9b4ca_row4_col12\" class=\"data row4 col12\" >False</td>\n      <td id=\"T_9b4ca_row4_col13\" class=\"data row4 col13\" >1381</td>\n      <td id=\"T_9b4ca_row4_col14\" class=\"data row4 col14\" >79</td>\n      <td id=\"T_9b4ca_row4_col15\" class=\"data row4 col15\" >0</td>\n      <td id=\"T_9b4ca_row4_col16\" class=\"data row4 col16\" >1</td>\n      <td id=\"T_9b4ca_row4_col17\" class=\"data row4 col17\" >0</td>\n      <td id=\"T_9b4ca_row4_col18\" class=\"data row4 col18\" >0</td>\n      <td id=\"T_9b4ca_row4_col19\" class=\"data row4 col19\" >0</td>\n      <td id=\"T_9b4ca_row4_col20\" class=\"data row4 col20\" >0</td>\n      <td id=\"T_9b4ca_row4_col21\" class=\"data row4 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row4_col22\" class=\"data row4 col22\" >0.00</td>\n      <td id=\"T_9b4ca_row4_col23\" class=\"data row4 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row5\" class=\"row_heading level0 row5\" >1557, 974</th>\n      <th id=\"T_9b4ca_level1_row5\" class=\"row_heading level1 row5\" >2</th>\n      <td id=\"T_9b4ca_row5_col0\" class=\"data row5 col0\" >0</td>\n      <td id=\"T_9b4ca_row5_col1\" class=\"data row5 col1\" >2</td>\n      <td id=\"T_9b4ca_row5_col2\" class=\"data row5 col2\" >html</td>\n      <td id=\"T_9b4ca_row5_col3\" class=\"data row5 col3\" >92</td>\n      <td id=\"T_9b4ca_row5_col4\" class=\"data row5 col4\" >2</td>\n      <td id=\"T_9b4ca_row5_col5\" class=\"data row5 col5\" >0</td>\n      <td id=\"T_9b4ca_row5_col6\" class=\"data row5 col6\" >0</td>\n      <td id=\"T_9b4ca_row5_col7\" class=\"data row5 col7\" >0</td>\n      <td id=\"T_9b4ca_row5_col8\" class=\"data row5 col8\" >-4.31</td>\n      <td id=\"T_9b4ca_row5_col9\" class=\"data row5 col9\" >0</td>\n      <td id=\"T_9b4ca_row5_col10\" class=\"data row5 col10\" >True</td>\n      <td id=\"T_9b4ca_row5_col11\" class=\"data row5 col11\" >False</td>\n      <td id=\"T_9b4ca_row5_col12\" class=\"data row5 col12\" >False</td>\n      <td id=\"T_9b4ca_row5_col13\" class=\"data row5 col13\" >0</td>\n      <td id=\"T_9b4ca_row5_col14\" class=\"data row5 col14\" >0</td>\n      <td id=\"T_9b4ca_row5_col15\" class=\"data row5 col15\" >149</td>\n      <td id=\"T_9b4ca_row5_col16\" class=\"data row5 col16\" >1</td>\n      <td id=\"T_9b4ca_row5_col17\" class=\"data row5 col17\" >0</td>\n      <td id=\"T_9b4ca_row5_col18\" class=\"data row5 col18\" >0</td>\n      <td id=\"T_9b4ca_row5_col19\" class=\"data row5 col19\" >25</td>\n      <td id=\"T_9b4ca_row5_col20\" class=\"data row5 col20\" >0</td>\n      <td id=\"T_9b4ca_row5_col21\" class=\"data row5 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row5_col22\" class=\"data row5 col22\" >0.17</td>\n      <td id=\"T_9b4ca_row5_col23\" class=\"data row5 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row6\" class=\"row_heading level0 row6\" >2360, 2150</th>\n      <th id=\"T_9b4ca_level1_row6\" class=\"row_heading level1 row6\" >2</th>\n      <td id=\"T_9b4ca_row6_col0\" class=\"data row6 col0\" >0</td>\n      <td id=\"T_9b4ca_row6_col1\" class=\"data row6 col1\" >3</td>\n      <td id=\"T_9b4ca_row6_col2\" class=\"data row6 col2\" >html</td>\n      <td id=\"T_9b4ca_row6_col3\" class=\"data row6 col3\" >92</td>\n      <td id=\"T_9b4ca_row6_col4\" class=\"data row6 col4\" >2</td>\n      <td id=\"T_9b4ca_row6_col5\" class=\"data row6 col5\" >0</td>\n      <td id=\"T_9b4ca_row6_col6\" class=\"data row6 col6\" >0</td>\n      <td id=\"T_9b4ca_row6_col7\" class=\"data row6 col7\" >0</td>\n      <td id=\"T_9b4ca_row6_col8\" class=\"data row6 col8\" >-4.31</td>\n      <td id=\"T_9b4ca_row6_col9\" class=\"data row6 col9\" >0</td>\n      <td id=\"T_9b4ca_row6_col10\" class=\"data row6 col10\" >True</td>\n      <td id=\"T_9b4ca_row6_col11\" class=\"data row6 col11\" >False</td>\n      <td id=\"T_9b4ca_row6_col12\" class=\"data row6 col12\" >False</td>\n      <td id=\"T_9b4ca_row6_col13\" class=\"data row6 col13\" >0</td>\n      <td id=\"T_9b4ca_row6_col14\" class=\"data row6 col14\" >0</td>\n      <td id=\"T_9b4ca_row6_col15\" class=\"data row6 col15\" >149</td>\n      <td id=\"T_9b4ca_row6_col16\" class=\"data row6 col16\" >1</td>\n      <td id=\"T_9b4ca_row6_col17\" class=\"data row6 col17\" >0</td>\n      <td id=\"T_9b4ca_row6_col18\" class=\"data row6 col18\" >0</td>\n      <td id=\"T_9b4ca_row6_col19\" class=\"data row6 col19\" >25</td>\n      <td id=\"T_9b4ca_row6_col20\" class=\"data row6 col20\" >0</td>\n      <td id=\"T_9b4ca_row6_col21\" class=\"data row6 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row6_col22\" class=\"data row6 col22\" >0.17</td>\n      <td id=\"T_9b4ca_row6_col23\" class=\"data row6 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row7\" class=\"row_heading level0 row7\" >2238, 2489</th>\n      <th id=\"T_9b4ca_level1_row7\" class=\"row_heading level1 row7\" >2</th>\n      <td id=\"T_9b4ca_row7_col0\" class=\"data row7 col0\" >0</td>\n      <td id=\"T_9b4ca_row7_col1\" class=\"data row7 col1\" >3</td>\n      <td id=\"T_9b4ca_row7_col2\" class=\"data row7 col2\" >php</td>\n      <td id=\"T_9b4ca_row7_col3\" class=\"data row7 col3\" >108</td>\n      <td id=\"T_9b4ca_row7_col4\" class=\"data row7 col4\" >3</td>\n      <td id=\"T_9b4ca_row7_col5\" class=\"data row7 col5\" >8</td>\n      <td id=\"T_9b4ca_row7_col6\" class=\"data row7 col6\" >0</td>\n      <td id=\"T_9b4ca_row7_col7\" class=\"data row7 col7\" >0</td>\n      <td id=\"T_9b4ca_row7_col8\" class=\"data row7 col8\" >-4.51</td>\n      <td id=\"T_9b4ca_row7_col9\" class=\"data row7 col9\" >0</td>\n      <td id=\"T_9b4ca_row7_col10\" class=\"data row7 col10\" >True</td>\n      <td id=\"T_9b4ca_row7_col11\" class=\"data row7 col11\" >False</td>\n      <td id=\"T_9b4ca_row7_col12\" class=\"data row7 col12\" >False</td>\n      <td id=\"T_9b4ca_row7_col13\" class=\"data row7 col13\" >1381</td>\n      <td id=\"T_9b4ca_row7_col14\" class=\"data row7 col14\" >79</td>\n      <td id=\"T_9b4ca_row7_col15\" class=\"data row7 col15\" >0</td>\n      <td id=\"T_9b4ca_row7_col16\" class=\"data row7 col16\" >1</td>\n      <td id=\"T_9b4ca_row7_col17\" class=\"data row7 col17\" >0</td>\n      <td id=\"T_9b4ca_row7_col18\" class=\"data row7 col18\" >0</td>\n      <td id=\"T_9b4ca_row7_col19\" class=\"data row7 col19\" >0</td>\n      <td id=\"T_9b4ca_row7_col20\" class=\"data row7 col20\" >0</td>\n      <td id=\"T_9b4ca_row7_col21\" class=\"data row7 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row7_col22\" class=\"data row7 col22\" >0.00</td>\n      <td id=\"T_9b4ca_row7_col23\" class=\"data row7 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row8\" class=\"row_heading level0 row8\" >3192, 3444</th>\n      <th id=\"T_9b4ca_level1_row8\" class=\"row_heading level1 row8\" >2</th>\n      <td id=\"T_9b4ca_row8_col0\" class=\"data row8 col0\" >0</td>\n      <td id=\"T_9b4ca_row8_col1\" class=\"data row8 col1\" >4</td>\n      <td id=\"T_9b4ca_row8_col2\" class=\"data row8 col2\" >other</td>\n      <td id=\"T_9b4ca_row8_col3\" class=\"data row8 col3\" >123</td>\n      <td id=\"T_9b4ca_row8_col4\" class=\"data row8 col4\" >28</td>\n      <td id=\"T_9b4ca_row8_col5\" class=\"data row8 col5\" >4</td>\n      <td id=\"T_9b4ca_row8_col6\" class=\"data row8 col6\" >0</td>\n      <td id=\"T_9b4ca_row8_col7\" class=\"data row8 col7\" >0</td>\n      <td id=\"T_9b4ca_row8_col8\" class=\"data row8 col8\" >-4.92</td>\n      <td id=\"T_9b4ca_row8_col9\" class=\"data row8 col9\" >0</td>\n      <td id=\"T_9b4ca_row8_col10\" class=\"data row8 col10\" >True</td>\n      <td id=\"T_9b4ca_row8_col11\" class=\"data row8 col11\" >False</td>\n      <td id=\"T_9b4ca_row8_col12\" class=\"data row8 col12\" >False</td>\n      <td id=\"T_9b4ca_row8_col13\" class=\"data row8 col13\" >0</td>\n      <td id=\"T_9b4ca_row8_col14\" class=\"data row8 col14\" >0</td>\n      <td id=\"T_9b4ca_row8_col15\" class=\"data row8 col15\" >0</td>\n      <td id=\"T_9b4ca_row8_col16\" class=\"data row8 col16\" >0</td>\n      <td id=\"T_9b4ca_row8_col17\" class=\"data row8 col17\" >0</td>\n      <td id=\"T_9b4ca_row8_col18\" class=\"data row8 col18\" >0</td>\n      <td id=\"T_9b4ca_row8_col19\" class=\"data row8 col19\" >0</td>\n      <td id=\"T_9b4ca_row8_col20\" class=\"data row8 col20\" >0</td>\n      <td id=\"T_9b4ca_row8_col21\" class=\"data row8 col21\" >0.00</td>\n      <td id=\"T_9b4ca_row8_col22\" class=\"data row8 col22\" >0.00</td>\n      <td id=\"T_9b4ca_row8_col23\" class=\"data row8 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_9b4ca_level0_row9\" class=\"row_heading level0 row9\" >3498, 3277</th>\n      <th id=\"T_9b4ca_level1_row9\" class=\"row_heading level1 row9\" >2</th>\n      <td id=\"T_9b4ca_row9_col0\" class=\"data row9 col0\" >0</td>\n      <td id=\"T_9b4ca_row9_col1\" class=\"data row9 col1\" >4</td>\n      <td id=\"T_9b4ca_row9_col2\" class=\"data row9 col2\" >php</td>\n      <td id=\"T_9b4ca_row9_col3\" class=\"data row9 col3\" >93</td>\n      <td id=\"T_9b4ca_row9_col4\" class=\"data row9 col4\" >31</td>\n      <td id=\"T_9b4ca_row9_col5\" class=\"data row9 col5\" >1</td>\n      <td id=\"T_9b4ca_row9_col6\" class=\"data row9 col6\" >0</td>\n      <td id=\"T_9b4ca_row9_col7\" class=\"data row9 col7\" >0</td>\n      <td id=\"T_9b4ca_row9_col8\" class=\"data row9 col8\" >-4.93</td>\n      <td id=\"T_9b4ca_row9_col9\" class=\"data row9 col9\" >0</td>\n      <td id=\"T_9b4ca_row9_col10\" class=\"data row9 col10\" >True</td>\n      <td id=\"T_9b4ca_row9_col11\" class=\"data row9 col11\" >False</td>\n      <td id=\"T_9b4ca_row9_col12\" class=\"data row9 col12\" >False</td>\n      <td id=\"T_9b4ca_row9_col13\" class=\"data row9 col13\" >0</td>\n      <td id=\"T_9b4ca_row9_col14\" class=\"data row9 col14\" >0</td>\n      <td id=\"T_9b4ca_row9_col15\" class=\"data row9 col15\" >281</td>\n      <td id=\"T_9b4ca_row9_col16\" class=\"data row9 col16\" >0</td>\n      <td id=\"T_9b4ca_row9_col17\" class=\"data row9 col17\" >0</td>\n      <td id=\"T_9b4ca_row9_col18\" class=\"data row9 col18\" >0</td>\n      <td id=\"T_9b4ca_row9_col19\" class=\"data row9 col19\" >74</td>\n      <td id=\"T_9b4ca_row9_col20\" class=\"data row9 col20\" >142</td>\n      <td id=\"T_9b4ca_row9_col21\" class=\"data row9 col21\" >0.51</td>\n      <td id=\"T_9b4ca_row9_col22\" class=\"data row9 col22\" >0.26</td>\n      <td id=\"T_9b4ca_row9_col23\" class=\"data row9 col23\" >1.92</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "cad6e55af6bb47a09e7ae9fdcb27b5d3": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "84a037e7f45b4c20b0982179e1761be8": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "62c2312e318748318397ae31f0cc9a28": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "b0f8eec7913a49cfa10cec3de7d9ee62": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "ba4f593a0cee4f749d3c2d94ce38b008"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 63-68

Define a Condition
==================
Now, we define a condition that enforce the ratio of duplicates to be 0. A condition
is deepchecks' way to validate model and data quality, and let you know if anything
goes wrong.

.. GENERATED FROM PYTHON SOURCE LINES 68-73

.. code-block:: default


    check = DataDuplicates()
    check.add_condition_ratio_less_or_equal(0)
    result = check.run(phishing_dataset)
    result.show(show_additional_outputs=False)





.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2023 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"15cb02e5bf614cac959cb06feef4f347": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_ce2c7b0acb294551ab15cef28e9d3009", "IPY_MODEL_d9ed3c957a83419daed187884fb3d0a5", "IPY_MODEL_7ecb94fc360240699042ff85645ee193"], "layout": "IPY_MODEL_9d029bc4fa694dfb83fbb52d881c2532"}}, "ce2c7b0acb294551ab15cef28e9d3009": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_14a95ad729e542ebb03cda32c235f79e", "style": "IPY_MODEL_e6fb8d4eae6e4451bb4976bf03aff197", "value": "<h4><b>Data Duplicates</b></h4>"}}, "14a95ad729e542ebb03cda32c235f79e": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "e6fb8d4eae6e4451bb4976bf03aff197": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "d9ed3c957a83419daed187884fb3d0a5": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_af6f603881cf43078c3cf419b9b1c3d1", "style": "IPY_MODEL_692c1c5eae6b4730a47649b5e31e25b9", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.18/tabular/auto_checks/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "af6f603881cf43078c3cf419b9b1c3d1": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "692c1c5eae6b4730a47649b5e31e25b9": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "7ecb94fc360240699042ff85645ee193": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_6bc477331cab4d358c6483837b757cc1", "style": "IPY_MODEL_3f39984573b94ce2880d66f967a1c578", "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_481d5_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_481d5_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_481d5_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_481d5_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_481d5_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_481d5_\">\n  <thead>\n    <tr>\n      <th class=\"col_heading level0 col0\" >Status</th>\n      <th class=\"col_heading level0 col1\" >Condition</th>\n      <th class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_481d5_row0_col0\" class=\"data row0 col0\" ><div style=\"color: orange;text-align: center;font-weight:bold\">!</div></td>\n      <td id=\"T_481d5_row0_col1\" class=\"data row0 col1\" >Duplicate data ratio is less or equal to 0%</td>\n      <td id=\"T_481d5_row0_col2\" class=\"data row0 col2\" >Found 0.0088% duplicate data</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "6bc477331cab4d358c6483837b757cc1": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "3f39984573b94ce2880d66f967a1c578": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "9d029bc4fa694dfb83fbb52d881c2532": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "15cb02e5bf614cac959cb06feef4f347"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 2.108 seconds)


.. _sphx_glr_download_tabular_auto_checks_data_integrity_plot_data_duplicates.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example




    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_data_duplicates.py <plot_data_duplicates.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_data_duplicates.ipynb <plot_data_duplicates.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_