.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tabular/auto_checks/data_integrity/plot_data_duplicates.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_tabular_auto_checks_data_integrity_plot_data_duplicates.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_tabular_auto_checks_data_integrity_plot_data_duplicates.py:


.. _tabular__data_duplicates:

Data Duplicates
***************

This notebook provides an overview for using and understanding the data duplicates check:

**Structure:**

* `Why data duplicates? <#why-data-duplicates>`__
* `Load Data <#load-data>`__
* `Run the Check <#run-the-check>`__
* `Define a Condition <#define-a-condition>`__

.. GENERATED FROM PYTHON SOURCE LINES 19-26

.. code-block:: default


    from datetime import datetime

    import pandas as pd

    from deepchecks.tabular.datasets.classification.phishing import load_data








.. GENERATED FROM PYTHON SOURCE LINES 27-38

Why data duplicates?
====================
The ``DataDuplicates`` check finds multiple instances of identical samples in the
Dataset. Duplicate samples increase the weight the model gives to those samples.
If these duplicates are there intentionally (e.g. as a result of intentional
oversampling, or due to the dataset's nature it has identical-looking samples)
this may be valid, however if this is an hidden issue we're not expecting to occur,
it may be an indicator for a problem in the data pipeline that requires attention.

Load Data
=========

.. GENERATED FROM PYTHON SOURCE LINES 38-43

.. code-block:: default



    phishing_dataset = load_data(as_train_test=False, data_format='DataFrame')
    phishing_dataset






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>target</th>
          <th>month</th>
          <th>scrape_date</th>
          <th>ext</th>
          <th>urlLength</th>
          <th>numDigits</th>
          <th>numParams</th>
          <th>num_%20</th>
          <th>num_@</th>
          <th>entropy</th>
          <th>has_ip</th>
          <th>hasHttp</th>
          <th>hasHttps</th>
          <th>urlIsLive</th>
          <th>dsr</th>
          <th>dse</th>
          <th>bodyLength</th>
          <th>numTitles</th>
          <th>numImages</th>
          <th>numLinks</th>
          <th>specialChars</th>
          <th>scriptLength</th>
          <th>sbr</th>
          <th>bscr</th>
          <th>sscr</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>net</td>
          <td>102</td>
          <td>8</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.384032</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>4921</td>
          <td>191</td>
          <td>32486</td>
          <td>3</td>
          <td>5</td>
          <td>330</td>
          <td>9419</td>
          <td>23919</td>
          <td>0.736286</td>
          <td>0.289940</td>
          <td>2.539442</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>country</td>
          <td>154</td>
          <td>60</td>
          <td>0</td>
          <td>2</td>
          <td>0</td>
          <td>-3.566515</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>0</td>
          <td>0</td>
          <td>16199</td>
          <td>0</td>
          <td>4</td>
          <td>39</td>
          <td>2735</td>
          <td>794</td>
          <td>0.049015</td>
          <td>0.168838</td>
          <td>0.290311</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>net</td>
          <td>171</td>
          <td>5</td>
          <td>11</td>
          <td>0</td>
          <td>0</td>
          <td>-4.608755</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>5374</td>
          <td>104</td>
          <td>103344</td>
          <td>18</td>
          <td>9</td>
          <td>302</td>
          <td>27798</td>
          <td>83817</td>
          <td>0.811049</td>
          <td>0.268985</td>
          <td>2.412174</td>
        </tr>
        <tr>
          <th>3</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>com</td>
          <td>94</td>
          <td>10</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.548921</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>6107</td>
          <td>466</td>
          <td>34093</td>
          <td>11</td>
          <td>43</td>
          <td>199</td>
          <td>9087</td>
          <td>19427</td>
          <td>0.569824</td>
          <td>0.266536</td>
          <td>2.137889</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>other</td>
          <td>95</td>
          <td>11</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.717188</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>3819</td>
          <td>928</td>
          <td>202</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>39</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.193069</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>11345</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>country</td>
          <td>89</td>
          <td>7</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.254491</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>0</td>
          <td>0</td>
          <td>4117</td>
          <td>5</td>
          <td>0</td>
          <td>1</td>
          <td>971</td>
          <td>1866</td>
          <td>0.625302</td>
          <td>0.213266</td>
          <td>2.932029</td>
        </tr>
        <tr>
          <th>11346</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>other</td>
          <td>107</td>
          <td>13</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.758879</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>9073</td>
          <td>1882</td>
          <td>17788</td>
          <td>47</td>
          <td>58</td>
          <td>645</td>
          <td>3185</td>
          <td>4228</td>
          <td>0.291069</td>
          <td>0.214348</td>
          <td>1.357928</td>
        </tr>
        <tr>
          <th>11347</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>com</td>
          <td>112</td>
          <td>10</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.723014</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>2640</td>
          <td>1011</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>11348</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>html</td>
          <td>111</td>
          <td>3</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.289384</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>2291</td>
          <td>265</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>11349</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>html</td>
          <td>97</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>-4.304523</td>
          <td>0</td>
          <td>True</td>
          <td>False</td>
          <td>False</td>
          <td>6273</td>
          <td>298</td>
          <td>149</td>
          <td>1</td>
          <td>0</td>
          <td>0</td>
          <td>25</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.167785</td>
          <td>0.000000</td>
        </tr>
      </tbody>
    </table>
    <p>11350 rows × 25 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 44-46

Run the Check
=============

.. GENERATED FROM PYTHON SOURCE LINES 46-58

.. code-block:: default


    from deepchecks.tabular.checks import DataDuplicates

    DataDuplicates().run(phishing_dataset)

    # With Check Parameters
    # ---------------------
    # ``DataDuplicates`` check can also use a specific subset of columns (or alternatively
    # use all columns except specific ignore_columns to check duplication):

    DataDuplicates(columns=["entropy", "numParams"]).run(phishing_dataset)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2023 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"b58a720c168d49d2981b04e40637de33": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_54075ec4f8e246ae809549b5bb5e9a19", "IPY_MODEL_88bbc431d9bc4ef49dc36cb0d4a19373", "IPY_MODEL_f63c48e0f59e4dca9bc3a411e3029019", "IPY_MODEL_cbf6f3c85fdc467c9756820a7f6180ca"], "layout": "IPY_MODEL_eac19af5051248a3aef7382547bdcfbc"}}, "54075ec4f8e246ae809549b5bb5e9a19": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_4bec096fb35f45779d16d4be93af5ff4", "style": "IPY_MODEL_a746bc8e690248148b44e52583dc2c5f", "value": "<h4><b>Data Duplicates</b></h4>"}}, "4bec096fb35f45779d16d4be93af5ff4": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "a746bc8e690248148b44e52583dc2c5f": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "88bbc431d9bc4ef49dc36cb0d4a19373": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_ee99905e40ea4d9289a917b85468b08d", "style": "IPY_MODEL_e9886fd47d274057a12aeb852fcc87df", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.17/tabular/auto_checks/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "ee99905e40ea4d9289a917b85468b08d": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "e9886fd47d274057a12aeb852fcc87df": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "f63c48e0f59e4dca9bc3a411e3029019": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_b6d2bffbc90240b1afabfcfbd377498b", "style": "IPY_MODEL_246825797b7a4b5eb6d077620ab3a7ae"}}, "b6d2bffbc90240b1afabfcfbd377498b": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "246825797b7a4b5eb6d077620ab3a7ae": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "cbf6f3c85fdc467c9756820a7f6180ca": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_6203a4742ed048b0873a3e8ad51f24c8", "IPY_MODEL_60c5011bbf9144c2aeb74c6584c8f056", "IPY_MODEL_daaf7cd1dd8842d9833c405fa38bed3c", "IPY_MODEL_c81897c5b8aa4dd4b501ebd8a2a3e5f4"], "layout": "IPY_MODEL_491546c7cc0046129ef3edfa9a3cdaa3"}}, "6203a4742ed048b0873a3e8ad51f24c8": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_fd45320d7e5b4ea0b65ce9dae8641488", "style": "IPY_MODEL_f9c97a2bfac648d49ae54f33f74a4c2d", "value": "<h5><b>Additional Outputs</b></h5>"}}, "fd45320d7e5b4ea0b65ce9dae8641488": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "f9c97a2bfac648d49ae54f33f74a4c2d": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "60c5011bbf9144c2aeb74c6584c8f056": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_8bd70aa6ed1c43f1a896f372deb1a62f", "style": "IPY_MODEL_087f960a421e422fbcfe9cd3a7bca532", "value": "<div>4.11% of data samples are duplicates. </div>"}}, "8bd70aa6ed1c43f1a896f372deb1a62f": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "087f960a421e422fbcfe9cd3a7bca532": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "daaf7cd1dd8842d9833c405fa38bed3c": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_3ac0ff339f3b422da191bcf3877137f6", "style": "IPY_MODEL_33359e7c31dc4ff9b080fe67be086c1d", "value": "<div>Each row in the table shows an example of duplicate data and the number of times it appears.</div>"}}, "3ac0ff339f3b422da191bcf3877137f6": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "33359e7c31dc4ff9b080fe67be086c1d": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "c81897c5b8aa4dd4b501ebd8a2a3e5f4": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_923857d75c9242f2b6085d14cbfa978a", "style": "IPY_MODEL_2153d644064f4398bafcb5afb68fc7c7", "value": "<style type=\"text/css\">\n#T_aaa89_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_aaa89_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_aaa89_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_aaa89_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_aaa89_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_aaa89_\">\n  <thead>\n    <tr>\n      <th class=\"blank\" >&nbsp;</th>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >entropy</th>\n      <th class=\"col_heading level0 col1\" >numParams</th>\n    </tr>\n    <tr>\n      <th class=\"index_name level0\" >Instances</th>\n      <th class=\"index_name level1\" >Number of Duplicates</th>\n      <th class=\"blank col0\" >&nbsp;</th>\n      <th class=\"blank col1\" >&nbsp;</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_aaa89_level0_row0\" class=\"row_heading level0 row0\" >2360, 6560, 11342, 2150, 3528, 1557, 810...</th>\n      <th id=\"T_aaa89_level1_row0\" class=\"row_heading level1 row0\" >13</th>\n      <td id=\"T_aaa89_row0_col0\" class=\"data row0 col0\" >-4.31</td>\n      <td id=\"T_aaa89_row0_col1\" class=\"data row0 col1\" >0</td>\n    </tr>\n    <tr>\n      <th id=\"T_aaa89_level0_row1\" class=\"row_heading level0 row1\" >4638, 4412, 1729, 2213, 1641, 2234, 6328...</th>\n      <th id=\"T_aaa89_level1_row1\" class=\"row_heading level1 row1\" >8</th>\n      <td id=\"T_aaa89_row1_col0\" class=\"data row1 col0\" >-4.57</td>\n      <td id=\"T_aaa89_row1_col1\" class=\"data row1 col1\" >4</td>\n    </tr>\n    <tr>\n      <th id=\"T_aaa89_level0_row2\" class=\"row_heading level0 row2\" >6774, 7528, 9592, 4634, 10046, 6504, 271...</th>\n      <th id=\"T_aaa89_level1_row2\" class=\"row_heading level1 row2\" >8</th>\n      <td id=\"T_aaa89_row2_col0\" class=\"data row2 col0\" >-4.49</td>\n      <td id=\"T_aaa89_row2_col1\" class=\"data row2 col1\" >8</td>\n    </tr>\n    <tr>\n      <th id=\"T_aaa89_level0_row3\" class=\"row_heading level0 row3\" >4047, 8391, 2499, 9932, 929, 11345, 9348...</th>\n      <th id=\"T_aaa89_level1_row3\" class=\"row_heading level1 row3\" >8</th>\n      <td id=\"T_aaa89_row3_col0\" class=\"data row3 col0\" >-4.25</td>\n      <td id=\"T_aaa89_row3_col1\" class=\"data row3 col1\" >0</td>\n    </tr>\n    <tr>\n      <th id=\"T_aaa89_level0_row4\" class=\"row_heading level0 row4\" >2984, 1670, 1020, 6666, 10923, 9138, 180...</th>\n      <th id=\"T_aaa89_level1_row4\" class=\"row_heading level1 row4\" >7</th>\n      <td id=\"T_aaa89_row4_col0\" class=\"data row4 col0\" >-4.65</td>\n      <td id=\"T_aaa89_row4_col1\" class=\"data row4 col1\" >5</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "923857d75c9242f2b6085d14cbfa978a": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "2153d644064f4398bafcb5afb68fc7c7": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "491546c7cc0046129ef3edfa9a3cdaa3": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "eac19af5051248a3aef7382547bdcfbc": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "b58a720c168d49d2981b04e40637de33"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 59-62

.. code-block:: default


    DataDuplicates(ignore_columns=["scrape_date"], n_to_show=10).run(phishing_dataset)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2023 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"635d5ef0134f40a196cf49a646a67811": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_f7ee226a22764788b8bb7d5983993c30", "IPY_MODEL_0c885c2ea8eb40d4904d3609997f5281", "IPY_MODEL_9fdaa575638244539a21032f51a52dec", "IPY_MODEL_ea8001fb6f5a4dfa907a814680414419"], "layout": "IPY_MODEL_10544a53c6be43cb91c1d9d5809c0ffc"}}, "f7ee226a22764788b8bb7d5983993c30": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_cbdca456118a45df9aba5333636b820d", "style": "IPY_MODEL_f367259d2d57448a8813cbb97fe1921c", "value": "<h4><b>Data Duplicates</b></h4>"}}, "cbdca456118a45df9aba5333636b820d": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "f367259d2d57448a8813cbb97fe1921c": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "0c885c2ea8eb40d4904d3609997f5281": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_ac71c8f4ef3d4614a887b12efcf98556", "style": "IPY_MODEL_5f7d1814909644b4a43abb731dd00d19", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.17/tabular/auto_checks/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "ac71c8f4ef3d4614a887b12efcf98556": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "5f7d1814909644b4a43abb731dd00d19": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "9fdaa575638244539a21032f51a52dec": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_c97501e229f343458b448f2b75f0bfeb", "style": "IPY_MODEL_830c62c1a5604b24b01a1051f8486997"}}, "c97501e229f343458b448f2b75f0bfeb": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "830c62c1a5604b24b01a1051f8486997": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "ea8001fb6f5a4dfa907a814680414419": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_45eb1fef71b94ee3a9c2978099be93a4", "IPY_MODEL_1e1a6ed784d742fd8488b976d84715e7", "IPY_MODEL_8856af00851c418cba98c0538779e399", "IPY_MODEL_5b6cc515766a4e0ba62a146c1bbfb4d4"], "layout": "IPY_MODEL_bbc38e2b47e745f3a39e69b1624dec9a"}}, "45eb1fef71b94ee3a9c2978099be93a4": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_19bfe74fae604d3e881bca11ade97428", "style": "IPY_MODEL_ecd035efb96c4d8bbf75ed639311c809", "value": "<h5><b>Additional Outputs</b></h5>"}}, "19bfe74fae604d3e881bca11ade97428": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "ecd035efb96c4d8bbf75ed639311c809": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "1e1a6ed784d742fd8488b976d84715e7": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_6079073a18f44161a1a81311af71f62f", "style": "IPY_MODEL_411e9dfa94fd47bd9c2975d78126861b", "value": "<div>0.22% of data samples are duplicates. </div>"}}, "6079073a18f44161a1a81311af71f62f": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "411e9dfa94fd47bd9c2975d78126861b": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "8856af00851c418cba98c0538779e399": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_0e37aceb06a849868812b7ab9b2792f6", "style": "IPY_MODEL_45abf2b251734ab89ea0f48a165a5afa", "value": "<div>Each row in the table shows an example of duplicate data and the number of times it appears.</div>"}}, "0e37aceb06a849868812b7ab9b2792f6": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "45abf2b251734ab89ea0f48a165a5afa": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "5b6cc515766a4e0ba62a146c1bbfb4d4": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_e856e5c2e5f040da913578717271f848", "style": "IPY_MODEL_a781c85267fa410498a4893288497983", "value": "<style type=\"text/css\">\n#T_73f7e_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_73f7e_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_73f7e_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_73f7e_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_73f7e_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_73f7e_\">\n  <thead>\n    <tr>\n      <th class=\"blank\" >&nbsp;</th>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >target</th>\n      <th class=\"col_heading level0 col1\" >month</th>\n      <th class=\"col_heading level0 col2\" >ext</th>\n      <th class=\"col_heading level0 col3\" >urlLength</th>\n      <th class=\"col_heading level0 col4\" >numDigits</th>\n      <th class=\"col_heading level0 col5\" >numParams</th>\n      <th class=\"col_heading level0 col6\" >num_%20</th>\n      <th class=\"col_heading level0 col7\" >num_@</th>\n      <th class=\"col_heading level0 col8\" >entropy</th>\n      <th class=\"col_heading level0 col9\" >has_ip</th>\n      <th class=\"col_heading level0 col10\" >hasHttp</th>\n      <th class=\"col_heading level0 col11\" >hasHttps</th>\n      <th class=\"col_heading level0 col12\" >urlIsLive</th>\n      <th class=\"col_heading level0 col13\" >dsr</th>\n      <th class=\"col_heading level0 col14\" >dse</th>\n      <th class=\"col_heading level0 col15\" >bodyLength</th>\n      <th class=\"col_heading level0 col16\" >numTitles</th>\n      <th class=\"col_heading level0 col17\" >numImages</th>\n      <th class=\"col_heading level0 col18\" >numLinks</th>\n      <th class=\"col_heading level0 col19\" >specialChars</th>\n      <th class=\"col_heading level0 col20\" >scriptLength</th>\n      <th class=\"col_heading level0 col21\" >sbr</th>\n      <th class=\"col_heading level0 col22\" >bscr</th>\n      <th class=\"col_heading level0 col23\" >sscr</th>\n    </tr>\n    <tr>\n      <th class=\"index_name level0\" >Instances</th>\n      <th class=\"index_name level1\" >Number of Duplicates</th>\n      <th class=\"blank col0\" >&nbsp;</th>\n      <th class=\"blank col1\" >&nbsp;</th>\n      <th class=\"blank col2\" >&nbsp;</th>\n      <th class=\"blank col3\" >&nbsp;</th>\n      <th class=\"blank col4\" >&nbsp;</th>\n      <th class=\"blank col5\" >&nbsp;</th>\n      <th class=\"blank col6\" >&nbsp;</th>\n      <th class=\"blank col7\" >&nbsp;</th>\n      <th class=\"blank col8\" >&nbsp;</th>\n      <th class=\"blank col9\" >&nbsp;</th>\n      <th class=\"blank col10\" >&nbsp;</th>\n      <th class=\"blank col11\" >&nbsp;</th>\n      <th class=\"blank col12\" >&nbsp;</th>\n      <th class=\"blank col13\" >&nbsp;</th>\n      <th class=\"blank col14\" >&nbsp;</th>\n      <th class=\"blank col15\" >&nbsp;</th>\n      <th class=\"blank col16\" >&nbsp;</th>\n      <th class=\"blank col17\" >&nbsp;</th>\n      <th class=\"blank col18\" >&nbsp;</th>\n      <th class=\"blank col19\" >&nbsp;</th>\n      <th class=\"blank col20\" >&nbsp;</th>\n      <th class=\"blank col21\" >&nbsp;</th>\n      <th class=\"blank col22\" >&nbsp;</th>\n      <th class=\"blank col23\" >&nbsp;</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_73f7e_level0_row0\" class=\"row_heading level0 row0\" >4696, 5398, 4719</th>\n      <th id=\"T_73f7e_level1_row0\" class=\"row_heading level1 row0\" >3</th>\n      <td id=\"T_73f7e_row0_col0\" class=\"data row0 col0\" >0</td>\n      <td id=\"T_73f7e_row0_col1\" class=\"data row0 col1\" >6</td>\n      <td id=\"T_73f7e_row0_col2\" class=\"data row0 col2\" >other</td>\n      <td id=\"T_73f7e_row0_col3\" class=\"data row0 col3\" >123</td>\n      <td id=\"T_73f7e_row0_col4\" class=\"data row0 col4\" >28</td>\n      <td id=\"T_73f7e_row0_col5\" class=\"data row0 col5\" >4</td>\n      <td id=\"T_73f7e_row0_col6\" class=\"data row0 col6\" >0</td>\n      <td id=\"T_73f7e_row0_col7\" class=\"data row0 col7\" >0</td>\n      <td id=\"T_73f7e_row0_col8\" class=\"data row0 col8\" >-4.91</td>\n      <td id=\"T_73f7e_row0_col9\" class=\"data row0 col9\" >0</td>\n      <td id=\"T_73f7e_row0_col10\" class=\"data row0 col10\" >True</td>\n      <td id=\"T_73f7e_row0_col11\" class=\"data row0 col11\" >False</td>\n      <td id=\"T_73f7e_row0_col12\" class=\"data row0 col12\" >False</td>\n      <td id=\"T_73f7e_row0_col13\" class=\"data row0 col13\" >0</td>\n      <td id=\"T_73f7e_row0_col14\" class=\"data row0 col14\" >0</td>\n      <td id=\"T_73f7e_row0_col15\" class=\"data row0 col15\" >0</td>\n      <td id=\"T_73f7e_row0_col16\" class=\"data row0 col16\" >0</td>\n      <td id=\"T_73f7e_row0_col17\" class=\"data row0 col17\" >0</td>\n      <td id=\"T_73f7e_row0_col18\" class=\"data row0 col18\" >0</td>\n      <td id=\"T_73f7e_row0_col19\" class=\"data row0 col19\" >0</td>\n      <td id=\"T_73f7e_row0_col20\" class=\"data row0 col20\" >0</td>\n      <td id=\"T_73f7e_row0_col21\" class=\"data row0 col21\" >0.00</td>\n      <td id=\"T_73f7e_row0_col22\" class=\"data row0 col22\" >0.00</td>\n      <td id=\"T_73f7e_row0_col23\" class=\"data row0 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row1\" class=\"row_heading level0 row1\" >11342, 82</th>\n      <th id=\"T_73f7e_level1_row1\" class=\"row_heading level1 row1\" >2</th>\n      <td id=\"T_73f7e_row1_col0\" class=\"data row1 col0\" >0</td>\n      <td id=\"T_73f7e_row1_col1\" class=\"data row1 col1\" >1</td>\n      <td id=\"T_73f7e_row1_col2\" class=\"data row1 col2\" >html</td>\n      <td id=\"T_73f7e_row1_col3\" class=\"data row1 col3\" >92</td>\n      <td id=\"T_73f7e_row1_col4\" class=\"data row1 col4\" >2</td>\n      <td id=\"T_73f7e_row1_col5\" class=\"data row1 col5\" >0</td>\n      <td id=\"T_73f7e_row1_col6\" class=\"data row1 col6\" >0</td>\n      <td id=\"T_73f7e_row1_col7\" class=\"data row1 col7\" >0</td>\n      <td id=\"T_73f7e_row1_col8\" class=\"data row1 col8\" >-4.31</td>\n      <td id=\"T_73f7e_row1_col9\" class=\"data row1 col9\" >0</td>\n      <td id=\"T_73f7e_row1_col10\" class=\"data row1 col10\" >True</td>\n      <td id=\"T_73f7e_row1_col11\" class=\"data row1 col11\" >False</td>\n      <td id=\"T_73f7e_row1_col12\" class=\"data row1 col12\" >False</td>\n      <td id=\"T_73f7e_row1_col13\" class=\"data row1 col13\" >0</td>\n      <td id=\"T_73f7e_row1_col14\" class=\"data row1 col14\" >0</td>\n      <td id=\"T_73f7e_row1_col15\" class=\"data row1 col15\" >149</td>\n      <td id=\"T_73f7e_row1_col16\" class=\"data row1 col16\" >1</td>\n      <td id=\"T_73f7e_row1_col17\" class=\"data row1 col17\" >0</td>\n      <td id=\"T_73f7e_row1_col18\" class=\"data row1 col18\" >0</td>\n      <td id=\"T_73f7e_row1_col19\" class=\"data row1 col19\" >25</td>\n      <td id=\"T_73f7e_row1_col20\" class=\"data row1 col20\" >0</td>\n      <td id=\"T_73f7e_row1_col21\" class=\"data row1 col21\" >0.00</td>\n      <td id=\"T_73f7e_row1_col22\" class=\"data row1 col22\" >0.17</td>\n      <td id=\"T_73f7e_row1_col23\" class=\"data row1 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row2\" class=\"row_heading level0 row2\" >790, 250</th>\n      <th id=\"T_73f7e_level1_row2\" class=\"row_heading level1 row2\" >2</th>\n      <td id=\"T_73f7e_row2_col0\" class=\"data row2 col0\" >0</td>\n      <td id=\"T_73f7e_row2_col1\" class=\"data row2 col1\" >1</td>\n      <td id=\"T_73f7e_row2_col2\" class=\"data row2 col2\" >php</td>\n      <td id=\"T_73f7e_row2_col3\" class=\"data row2 col3\" >107</td>\n      <td id=\"T_73f7e_row2_col4\" class=\"data row2 col4\" >4</td>\n      <td id=\"T_73f7e_row2_col5\" class=\"data row2 col5\" >8</td>\n      <td id=\"T_73f7e_row2_col6\" class=\"data row2 col6\" >0</td>\n      <td id=\"T_73f7e_row2_col7\" class=\"data row2 col7\" >0</td>\n      <td id=\"T_73f7e_row2_col8\" class=\"data row2 col8\" >-4.53</td>\n      <td id=\"T_73f7e_row2_col9\" class=\"data row2 col9\" >0</td>\n      <td id=\"T_73f7e_row2_col10\" class=\"data row2 col10\" >True</td>\n      <td id=\"T_73f7e_row2_col11\" class=\"data row2 col11\" >False</td>\n      <td id=\"T_73f7e_row2_col12\" class=\"data row2 col12\" >False</td>\n      <td id=\"T_73f7e_row2_col13\" class=\"data row2 col13\" >1381</td>\n      <td id=\"T_73f7e_row2_col14\" class=\"data row2 col14\" >79</td>\n      <td id=\"T_73f7e_row2_col15\" class=\"data row2 col15\" >0</td>\n      <td id=\"T_73f7e_row2_col16\" class=\"data row2 col16\" >1</td>\n      <td id=\"T_73f7e_row2_col17\" class=\"data row2 col17\" >0</td>\n      <td id=\"T_73f7e_row2_col18\" class=\"data row2 col18\" >0</td>\n      <td id=\"T_73f7e_row2_col19\" class=\"data row2 col19\" >0</td>\n      <td id=\"T_73f7e_row2_col20\" class=\"data row2 col20\" >0</td>\n      <td id=\"T_73f7e_row2_col21\" class=\"data row2 col21\" >0.00</td>\n      <td id=\"T_73f7e_row2_col22\" class=\"data row2 col22\" >0.00</td>\n      <td id=\"T_73f7e_row2_col23\" class=\"data row2 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row3\" class=\"row_heading level0 row3\" >217, 6</th>\n      <th id=\"T_73f7e_level1_row3\" class=\"row_heading level1 row3\" >2</th>\n      <td id=\"T_73f7e_row3_col0\" class=\"data row3 col0\" >0</td>\n      <td id=\"T_73f7e_row3_col1\" class=\"data row3 col1\" >1</td>\n      <td id=\"T_73f7e_row3_col2\" class=\"data row3 col2\" >php</td>\n      <td id=\"T_73f7e_row3_col3\" class=\"data row3 col3\" >107</td>\n      <td id=\"T_73f7e_row3_col4\" class=\"data row3 col4\" >5</td>\n      <td id=\"T_73f7e_row3_col5\" class=\"data row3 col5\" >8</td>\n      <td id=\"T_73f7e_row3_col6\" class=\"data row3 col6\" >0</td>\n      <td id=\"T_73f7e_row3_col7\" class=\"data row3 col7\" >0</td>\n      <td id=\"T_73f7e_row3_col8\" class=\"data row3 col8\" >-4.52</td>\n      <td id=\"T_73f7e_row3_col9\" class=\"data row3 col9\" >0</td>\n      <td id=\"T_73f7e_row3_col10\" class=\"data row3 col10\" >True</td>\n      <td id=\"T_73f7e_row3_col11\" class=\"data row3 col11\" >False</td>\n      <td id=\"T_73f7e_row3_col12\" class=\"data row3 col12\" >False</td>\n      <td id=\"T_73f7e_row3_col13\" class=\"data row3 col13\" >1381</td>\n      <td id=\"T_73f7e_row3_col14\" class=\"data row3 col14\" >79</td>\n      <td id=\"T_73f7e_row3_col15\" class=\"data row3 col15\" >0</td>\n      <td id=\"T_73f7e_row3_col16\" class=\"data row3 col16\" >1</td>\n      <td id=\"T_73f7e_row3_col17\" class=\"data row3 col17\" >0</td>\n      <td id=\"T_73f7e_row3_col18\" class=\"data row3 col18\" >0</td>\n      <td id=\"T_73f7e_row3_col19\" class=\"data row3 col19\" >0</td>\n      <td id=\"T_73f7e_row3_col20\" class=\"data row3 col20\" >0</td>\n      <td id=\"T_73f7e_row3_col21\" class=\"data row3 col21\" >0.00</td>\n      <td id=\"T_73f7e_row3_col22\" class=\"data row3 col22\" >0.00</td>\n      <td id=\"T_73f7e_row3_col23\" class=\"data row3 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row4\" class=\"row_heading level0 row4\" >763, 609</th>\n      <th id=\"T_73f7e_level1_row4\" class=\"row_heading level1 row4\" >2</th>\n      <td id=\"T_73f7e_row4_col0\" class=\"data row4 col0\" >0</td>\n      <td id=\"T_73f7e_row4_col1\" class=\"data row4 col1\" >1</td>\n      <td id=\"T_73f7e_row4_col2\" class=\"data row4 col2\" >php</td>\n      <td id=\"T_73f7e_row4_col3\" class=\"data row4 col3\" >113</td>\n      <td id=\"T_73f7e_row4_col4\" class=\"data row4 col4\" >6</td>\n      <td id=\"T_73f7e_row4_col5\" class=\"data row4 col5\" >8</td>\n      <td id=\"T_73f7e_row4_col6\" class=\"data row4 col6\" >0</td>\n      <td id=\"T_73f7e_row4_col7\" class=\"data row4 col7\" >0</td>\n      <td id=\"T_73f7e_row4_col8\" class=\"data row4 col8\" >-4.63</td>\n      <td id=\"T_73f7e_row4_col9\" class=\"data row4 col9\" >0</td>\n      <td id=\"T_73f7e_row4_col10\" class=\"data row4 col10\" >True</td>\n      <td id=\"T_73f7e_row4_col11\" class=\"data row4 col11\" >False</td>\n      <td id=\"T_73f7e_row4_col12\" class=\"data row4 col12\" >False</td>\n      <td id=\"T_73f7e_row4_col13\" class=\"data row4 col13\" >1381</td>\n      <td id=\"T_73f7e_row4_col14\" class=\"data row4 col14\" >79</td>\n      <td id=\"T_73f7e_row4_col15\" class=\"data row4 col15\" >0</td>\n      <td id=\"T_73f7e_row4_col16\" class=\"data row4 col16\" >1</td>\n      <td id=\"T_73f7e_row4_col17\" class=\"data row4 col17\" >0</td>\n      <td id=\"T_73f7e_row4_col18\" class=\"data row4 col18\" >0</td>\n      <td id=\"T_73f7e_row4_col19\" class=\"data row4 col19\" >0</td>\n      <td id=\"T_73f7e_row4_col20\" class=\"data row4 col20\" >0</td>\n      <td id=\"T_73f7e_row4_col21\" class=\"data row4 col21\" >0.00</td>\n      <td id=\"T_73f7e_row4_col22\" class=\"data row4 col22\" >0.00</td>\n      <td id=\"T_73f7e_row4_col23\" class=\"data row4 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row5\" class=\"row_heading level0 row5\" >1557, 974</th>\n      <th id=\"T_73f7e_level1_row5\" class=\"row_heading level1 row5\" >2</th>\n      <td id=\"T_73f7e_row5_col0\" class=\"data row5 col0\" >0</td>\n      <td id=\"T_73f7e_row5_col1\" class=\"data row5 col1\" >2</td>\n      <td id=\"T_73f7e_row5_col2\" class=\"data row5 col2\" >html</td>\n      <td id=\"T_73f7e_row5_col3\" class=\"data row5 col3\" >92</td>\n      <td id=\"T_73f7e_row5_col4\" class=\"data row5 col4\" >2</td>\n      <td id=\"T_73f7e_row5_col5\" class=\"data row5 col5\" >0</td>\n      <td id=\"T_73f7e_row5_col6\" class=\"data row5 col6\" >0</td>\n      <td id=\"T_73f7e_row5_col7\" class=\"data row5 col7\" >0</td>\n      <td id=\"T_73f7e_row5_col8\" class=\"data row5 col8\" >-4.31</td>\n      <td id=\"T_73f7e_row5_col9\" class=\"data row5 col9\" >0</td>\n      <td id=\"T_73f7e_row5_col10\" class=\"data row5 col10\" >True</td>\n      <td id=\"T_73f7e_row5_col11\" class=\"data row5 col11\" >False</td>\n      <td id=\"T_73f7e_row5_col12\" class=\"data row5 col12\" >False</td>\n      <td id=\"T_73f7e_row5_col13\" class=\"data row5 col13\" >0</td>\n      <td id=\"T_73f7e_row5_col14\" class=\"data row5 col14\" >0</td>\n      <td id=\"T_73f7e_row5_col15\" class=\"data row5 col15\" >149</td>\n      <td id=\"T_73f7e_row5_col16\" class=\"data row5 col16\" >1</td>\n      <td id=\"T_73f7e_row5_col17\" class=\"data row5 col17\" >0</td>\n      <td id=\"T_73f7e_row5_col18\" class=\"data row5 col18\" >0</td>\n      <td id=\"T_73f7e_row5_col19\" class=\"data row5 col19\" >25</td>\n      <td id=\"T_73f7e_row5_col20\" class=\"data row5 col20\" >0</td>\n      <td id=\"T_73f7e_row5_col21\" class=\"data row5 col21\" >0.00</td>\n      <td id=\"T_73f7e_row5_col22\" class=\"data row5 col22\" >0.17</td>\n      <td id=\"T_73f7e_row5_col23\" class=\"data row5 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row6\" class=\"row_heading level0 row6\" >2360, 2150</th>\n      <th id=\"T_73f7e_level1_row6\" class=\"row_heading level1 row6\" >2</th>\n      <td id=\"T_73f7e_row6_col0\" class=\"data row6 col0\" >0</td>\n      <td id=\"T_73f7e_row6_col1\" class=\"data row6 col1\" >3</td>\n      <td id=\"T_73f7e_row6_col2\" class=\"data row6 col2\" >html</td>\n      <td id=\"T_73f7e_row6_col3\" class=\"data row6 col3\" >92</td>\n      <td id=\"T_73f7e_row6_col4\" class=\"data row6 col4\" >2</td>\n      <td id=\"T_73f7e_row6_col5\" class=\"data row6 col5\" >0</td>\n      <td id=\"T_73f7e_row6_col6\" class=\"data row6 col6\" >0</td>\n      <td id=\"T_73f7e_row6_col7\" class=\"data row6 col7\" >0</td>\n      <td id=\"T_73f7e_row6_col8\" class=\"data row6 col8\" >-4.31</td>\n      <td id=\"T_73f7e_row6_col9\" class=\"data row6 col9\" >0</td>\n      <td id=\"T_73f7e_row6_col10\" class=\"data row6 col10\" >True</td>\n      <td id=\"T_73f7e_row6_col11\" class=\"data row6 col11\" >False</td>\n      <td id=\"T_73f7e_row6_col12\" class=\"data row6 col12\" >False</td>\n      <td id=\"T_73f7e_row6_col13\" class=\"data row6 col13\" >0</td>\n      <td id=\"T_73f7e_row6_col14\" class=\"data row6 col14\" >0</td>\n      <td id=\"T_73f7e_row6_col15\" class=\"data row6 col15\" >149</td>\n      <td id=\"T_73f7e_row6_col16\" class=\"data row6 col16\" >1</td>\n      <td id=\"T_73f7e_row6_col17\" class=\"data row6 col17\" >0</td>\n      <td id=\"T_73f7e_row6_col18\" class=\"data row6 col18\" >0</td>\n      <td id=\"T_73f7e_row6_col19\" class=\"data row6 col19\" >25</td>\n      <td id=\"T_73f7e_row6_col20\" class=\"data row6 col20\" >0</td>\n      <td id=\"T_73f7e_row6_col21\" class=\"data row6 col21\" >0.00</td>\n      <td id=\"T_73f7e_row6_col22\" class=\"data row6 col22\" >0.17</td>\n      <td id=\"T_73f7e_row6_col23\" class=\"data row6 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row7\" class=\"row_heading level0 row7\" >2238, 2489</th>\n      <th id=\"T_73f7e_level1_row7\" class=\"row_heading level1 row7\" >2</th>\n      <td id=\"T_73f7e_row7_col0\" class=\"data row7 col0\" >0</td>\n      <td id=\"T_73f7e_row7_col1\" class=\"data row7 col1\" >3</td>\n      <td id=\"T_73f7e_row7_col2\" class=\"data row7 col2\" >php</td>\n      <td id=\"T_73f7e_row7_col3\" class=\"data row7 col3\" >108</td>\n      <td id=\"T_73f7e_row7_col4\" class=\"data row7 col4\" >3</td>\n      <td id=\"T_73f7e_row7_col5\" class=\"data row7 col5\" >8</td>\n      <td id=\"T_73f7e_row7_col6\" class=\"data row7 col6\" >0</td>\n      <td id=\"T_73f7e_row7_col7\" class=\"data row7 col7\" >0</td>\n      <td id=\"T_73f7e_row7_col8\" class=\"data row7 col8\" >-4.51</td>\n      <td id=\"T_73f7e_row7_col9\" class=\"data row7 col9\" >0</td>\n      <td id=\"T_73f7e_row7_col10\" class=\"data row7 col10\" >True</td>\n      <td id=\"T_73f7e_row7_col11\" class=\"data row7 col11\" >False</td>\n      <td id=\"T_73f7e_row7_col12\" class=\"data row7 col12\" >False</td>\n      <td id=\"T_73f7e_row7_col13\" class=\"data row7 col13\" >1381</td>\n      <td id=\"T_73f7e_row7_col14\" class=\"data row7 col14\" >79</td>\n      <td id=\"T_73f7e_row7_col15\" class=\"data row7 col15\" >0</td>\n      <td id=\"T_73f7e_row7_col16\" class=\"data row7 col16\" >1</td>\n      <td id=\"T_73f7e_row7_col17\" class=\"data row7 col17\" >0</td>\n      <td id=\"T_73f7e_row7_col18\" class=\"data row7 col18\" >0</td>\n      <td id=\"T_73f7e_row7_col19\" class=\"data row7 col19\" >0</td>\n      <td id=\"T_73f7e_row7_col20\" class=\"data row7 col20\" >0</td>\n      <td id=\"T_73f7e_row7_col21\" class=\"data row7 col21\" >0.00</td>\n      <td id=\"T_73f7e_row7_col22\" class=\"data row7 col22\" >0.00</td>\n      <td id=\"T_73f7e_row7_col23\" class=\"data row7 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row8\" class=\"row_heading level0 row8\" >3192, 3444</th>\n      <th id=\"T_73f7e_level1_row8\" class=\"row_heading level1 row8\" >2</th>\n      <td id=\"T_73f7e_row8_col0\" class=\"data row8 col0\" >0</td>\n      <td id=\"T_73f7e_row8_col1\" class=\"data row8 col1\" >4</td>\n      <td id=\"T_73f7e_row8_col2\" class=\"data row8 col2\" >other</td>\n      <td id=\"T_73f7e_row8_col3\" class=\"data row8 col3\" >123</td>\n      <td id=\"T_73f7e_row8_col4\" class=\"data row8 col4\" >28</td>\n      <td id=\"T_73f7e_row8_col5\" class=\"data row8 col5\" >4</td>\n      <td id=\"T_73f7e_row8_col6\" class=\"data row8 col6\" >0</td>\n      <td id=\"T_73f7e_row8_col7\" class=\"data row8 col7\" >0</td>\n      <td id=\"T_73f7e_row8_col8\" class=\"data row8 col8\" >-4.92</td>\n      <td id=\"T_73f7e_row8_col9\" class=\"data row8 col9\" >0</td>\n      <td id=\"T_73f7e_row8_col10\" class=\"data row8 col10\" >True</td>\n      <td id=\"T_73f7e_row8_col11\" class=\"data row8 col11\" >False</td>\n      <td id=\"T_73f7e_row8_col12\" class=\"data row8 col12\" >False</td>\n      <td id=\"T_73f7e_row8_col13\" class=\"data row8 col13\" >0</td>\n      <td id=\"T_73f7e_row8_col14\" class=\"data row8 col14\" >0</td>\n      <td id=\"T_73f7e_row8_col15\" class=\"data row8 col15\" >0</td>\n      <td id=\"T_73f7e_row8_col16\" class=\"data row8 col16\" >0</td>\n      <td id=\"T_73f7e_row8_col17\" class=\"data row8 col17\" >0</td>\n      <td id=\"T_73f7e_row8_col18\" class=\"data row8 col18\" >0</td>\n      <td id=\"T_73f7e_row8_col19\" class=\"data row8 col19\" >0</td>\n      <td id=\"T_73f7e_row8_col20\" class=\"data row8 col20\" >0</td>\n      <td id=\"T_73f7e_row8_col21\" class=\"data row8 col21\" >0.00</td>\n      <td id=\"T_73f7e_row8_col22\" class=\"data row8 col22\" >0.00</td>\n      <td id=\"T_73f7e_row8_col23\" class=\"data row8 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_73f7e_level0_row9\" class=\"row_heading level0 row9\" >3498, 3277</th>\n      <th id=\"T_73f7e_level1_row9\" class=\"row_heading level1 row9\" >2</th>\n      <td id=\"T_73f7e_row9_col0\" class=\"data row9 col0\" >0</td>\n      <td id=\"T_73f7e_row9_col1\" class=\"data row9 col1\" >4</td>\n      <td id=\"T_73f7e_row9_col2\" class=\"data row9 col2\" >php</td>\n      <td id=\"T_73f7e_row9_col3\" class=\"data row9 col3\" >93</td>\n      <td id=\"T_73f7e_row9_col4\" class=\"data row9 col4\" >31</td>\n      <td id=\"T_73f7e_row9_col5\" class=\"data row9 col5\" >1</td>\n      <td id=\"T_73f7e_row9_col6\" class=\"data row9 col6\" >0</td>\n      <td id=\"T_73f7e_row9_col7\" class=\"data row9 col7\" >0</td>\n      <td id=\"T_73f7e_row9_col8\" class=\"data row9 col8\" >-4.93</td>\n      <td id=\"T_73f7e_row9_col9\" class=\"data row9 col9\" >0</td>\n      <td id=\"T_73f7e_row9_col10\" class=\"data row9 col10\" >True</td>\n      <td id=\"T_73f7e_row9_col11\" class=\"data row9 col11\" >False</td>\n      <td id=\"T_73f7e_row9_col12\" class=\"data row9 col12\" >False</td>\n      <td id=\"T_73f7e_row9_col13\" class=\"data row9 col13\" >0</td>\n      <td id=\"T_73f7e_row9_col14\" class=\"data row9 col14\" >0</td>\n      <td id=\"T_73f7e_row9_col15\" class=\"data row9 col15\" >281</td>\n      <td id=\"T_73f7e_row9_col16\" class=\"data row9 col16\" >0</td>\n      <td id=\"T_73f7e_row9_col17\" class=\"data row9 col17\" >0</td>\n      <td id=\"T_73f7e_row9_col18\" class=\"data row9 col18\" >0</td>\n      <td id=\"T_73f7e_row9_col19\" class=\"data row9 col19\" >74</td>\n      <td id=\"T_73f7e_row9_col20\" class=\"data row9 col20\" >142</td>\n      <td id=\"T_73f7e_row9_col21\" class=\"data row9 col21\" >0.51</td>\n      <td id=\"T_73f7e_row9_col22\" class=\"data row9 col22\" >0.26</td>\n      <td id=\"T_73f7e_row9_col23\" class=\"data row9 col23\" >1.92</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "e856e5c2e5f040da913578717271f848": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "a781c85267fa410498a4893288497983": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "bbc38e2b47e745f3a39e69b1624dec9a": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "10544a53c6be43cb91c1d9d5809c0ffc": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "635d5ef0134f40a196cf49a646a67811"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 63-68

Define a Condition
==================
Now, we define a condition that enforce the ratio of duplicates to be 0. A condition
is deepchecks' way to validate model and data quality, and let you know if anything
goes wrong.

.. GENERATED FROM PYTHON SOURCE LINES 68-73

.. code-block:: default


    check = DataDuplicates()
    check.add_condition_ratio_less_or_equal(0)
    result = check.run(phishing_dataset)
    result.show(show_additional_outputs=False)





.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2023 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"052ca3fee26e4d0fa6c231f9c6ade2bd": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_4fc9f3515a1b475eb958008a1f8588aa", "IPY_MODEL_3a28841e5fad40dfbb522827ed531cfa", "IPY_MODEL_e50e1e02efbe46229a1797a531b110d8"], "layout": "IPY_MODEL_788aa3723d004ad5bf2a3b87ff82e442"}}, "4fc9f3515a1b475eb958008a1f8588aa": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_e6c1b74aad7a46729d68c8900bea30f7", "style": "IPY_MODEL_d2ad966371b84718877b470828a38e3f", "value": "<h4><b>Data Duplicates</b></h4>"}}, "e6c1b74aad7a46729d68c8900bea30f7": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "d2ad966371b84718877b470828a38e3f": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "3a28841e5fad40dfbb522827ed531cfa": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_4434633618a84f889c437eab428fe5c8", "style": "IPY_MODEL_d098d63d26044d9b8a01767dae2508ff", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.17/tabular/auto_checks/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "4434633618a84f889c437eab428fe5c8": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "d098d63d26044d9b8a01767dae2508ff": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "e50e1e02efbe46229a1797a531b110d8": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_b074cd691cb64492b0bb19f97004d705", "style": "IPY_MODEL_6090cffd543a400ba78a972486928e2b", "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_de518_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_de518_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_de518_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_de518_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_de518_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_de518_\">\n  <thead>\n    <tr>\n      <th class=\"col_heading level0 col0\" >Status</th>\n      <th class=\"col_heading level0 col1\" >Condition</th>\n      <th class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_de518_row0_col0\" class=\"data row0 col0\" ><div style=\"color: orange;text-align: center;font-weight:bold\">!</div></td>\n      <td id=\"T_de518_row0_col1\" class=\"data row0 col1\" >Duplicate data ratio is less or equal to 0%</td>\n      <td id=\"T_de518_row0_col2\" class=\"data row0 col2\" >Found 0.0088% duplicate data</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "b074cd691cb64492b0bb19f97004d705": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "6090cffd543a400ba78a972486928e2b": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "788aa3723d004ad5bf2a3b87ff82e442": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "052ca3fee26e4d0fa6c231f9c6ade2bd"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 2.404 seconds)


.. _sphx_glr_download_tabular_auto_checks_data_integrity_plot_data_duplicates.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example




    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_data_duplicates.py <plot_data_duplicates.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_data_duplicates.ipynb <plot_data_duplicates.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_