.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "checks_gallery/tabular/data_integrity/plot_data_duplicates.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_checks_gallery_tabular_data_integrity_plot_data_duplicates.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_checks_gallery_tabular_data_integrity_plot_data_duplicates.py:


.. _plot_tabular_data_duplicates:

Data Duplicates
***************

This notebook provides an overview for using and understanding the data duplicates check:

**Structure:**

* `Why data duplicates? <#why-data-duplicates>`__
* `Load Data <#load-data>`__
* `Run the Check <#run-the-check>`__
* `Define a Condition <#define-a-condition>`__

.. GENERATED FROM PYTHON SOURCE LINES 19-26

.. code-block:: default


    from datetime import datetime

    import pandas as pd

    from deepchecks.tabular.datasets.classification.phishing import load_data








.. GENERATED FROM PYTHON SOURCE LINES 27-38

Why data duplicates?
====================
The ``DataDuplicates`` check finds multiple instances of identical samples in the
Dataset. Duplicate samples increase the weight the model gives to those samples.
If these duplicates are there intentionally (e.g. as a result of intentional
oversampling, or due to the dataset's nature it has identical-looking samples)
this may be valid, however if this is an hidden issue we're not expecting to occur,
it may be an indicator for a problem in the data pipeline that requires attention.

Load Data
=========

.. GENERATED FROM PYTHON SOURCE LINES 38-43

.. code-block:: default



    phishing_dataset = load_data(as_train_test=False, data_format='DataFrame')
    phishing_dataset






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>target</th>
          <th>month</th>
          <th>scrape_date</th>
          <th>ext</th>
          <th>urlLength</th>
          <th>...</th>
          <th>specialChars</th>
          <th>scriptLength</th>
          <th>sbr</th>
          <th>bscr</th>
          <th>sscr</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>net</td>
          <td>102</td>
          <td>...</td>
          <td>9419</td>
          <td>23919</td>
          <td>0.736286</td>
          <td>0.289940</td>
          <td>2.539442</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>country</td>
          <td>154</td>
          <td>...</td>
          <td>2735</td>
          <td>794</td>
          <td>0.049015</td>
          <td>0.168838</td>
          <td>0.290311</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>net</td>
          <td>171</td>
          <td>...</td>
          <td>27798</td>
          <td>83817</td>
          <td>0.811049</td>
          <td>0.268985</td>
          <td>2.412174</td>
        </tr>
        <tr>
          <th>3</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>com</td>
          <td>94</td>
          <td>...</td>
          <td>9087</td>
          <td>19427</td>
          <td>0.569824</td>
          <td>0.266536</td>
          <td>2.137889</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>other</td>
          <td>95</td>
          <td>...</td>
          <td>39</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.193069</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>11345</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>country</td>
          <td>89</td>
          <td>...</td>
          <td>971</td>
          <td>1866</td>
          <td>0.625302</td>
          <td>0.213266</td>
          <td>2.932029</td>
        </tr>
        <tr>
          <th>11346</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>other</td>
          <td>107</td>
          <td>...</td>
          <td>3185</td>
          <td>4228</td>
          <td>0.291069</td>
          <td>0.214348</td>
          <td>1.357928</td>
        </tr>
        <tr>
          <th>11347</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>com</td>
          <td>112</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>11348</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>html</td>
          <td>111</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>11349</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>html</td>
          <td>97</td>
          <td>...</td>
          <td>25</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.167785</td>
          <td>0.000000</td>
        </tr>
      </tbody>
    </table>
    <p>11350 rows × 25 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 44-46

Run the Check
=============

.. GENERATED FROM PYTHON SOURCE LINES 46-58

.. code-block:: default


    from deepchecks.tabular.checks import DataDuplicates

    DataDuplicates().run(phishing_dataset)

    # With Check Parameters
    # ---------------------
    # ``DataDuplicates`` check can also use a specific subset of columns (or alternatively
    # use all columns except specific ignore_columns to check duplication):

    DataDuplicates(columns=["entropy", "numParams"]).run(phishing_dataset)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"081603f316ba42d1a5fb18b2f7e3874f": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_bbc281861853458d8bb00841ea4adee3", "IPY_MODEL_57df160664d84c56bc6923e20565b5f8", "IPY_MODEL_4ae1771daddd4175a5be80dbb7c11120", "IPY_MODEL_b75a51cfff7548bca789a1ee692e7ed4"], "layout": "IPY_MODEL_006d515650a64914a40bd41817687a4e"}}, "bbc281861853458d8bb00841ea4adee3": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_3800009cfcd0442f837fe5bab84b6c13", "style": "IPY_MODEL_1e9e72f80d7248ac92e3c0556dc1ecfd", "value": "<h4><b>Data Duplicates</b></h4>"}}, "3800009cfcd0442f837fe5bab84b6c13": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "1e9e72f80d7248ac92e3c0556dc1ecfd": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "57df160664d84c56bc6923e20565b5f8": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_231686b5746e4490b88ea41cf6c15dfa", "style": "IPY_MODEL_a609597a1f404e358d7b9839f634f2fc", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.9/checks_gallery/tabular/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "231686b5746e4490b88ea41cf6c15dfa": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "a609597a1f404e358d7b9839f634f2fc": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "4ae1771daddd4175a5be80dbb7c11120": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_53333126be874916882d182539224197", "style": "IPY_MODEL_588696af06e744168fc0a80a375d002a"}}, "53333126be874916882d182539224197": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "588696af06e744168fc0a80a375d002a": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "b75a51cfff7548bca789a1ee692e7ed4": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_b956dcf53b944bfda5c256b349d6383a", "IPY_MODEL_1524bc415f524c749e8a79c48d0fd11d", "IPY_MODEL_c7d5d693f271452dae1c694727006a73", "IPY_MODEL_af37828d2bd1484aaf549445be2028c4"], "layout": "IPY_MODEL_b4e8e14734bd402dac741587f86e7876"}}, "b956dcf53b944bfda5c256b349d6383a": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_af90f05743b64823b9095f5e5d285552", "style": "IPY_MODEL_95ac234811fb4e58b3314a415cc00770", "value": "<h5><b>Additional Outputs</b></h5>"}}, "af90f05743b64823b9095f5e5d285552": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "95ac234811fb4e58b3314a415cc00770": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "1524bc415f524c749e8a79c48d0fd11d": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_af3c03af4a7940f5974a3f1660b6f548", "style": "IPY_MODEL_4a60b95cfd314e51aec99d11f8d77c98", "value": "<div>4.11% of data samples are duplicates. </div>"}}, "af3c03af4a7940f5974a3f1660b6f548": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "4a60b95cfd314e51aec99d11f8d77c98": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "c7d5d693f271452dae1c694727006a73": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_a22ac83728b24648b17b8e5a9c57a53a", "style": "IPY_MODEL_ca518b382279463fa2434c8da2e3a98e", "value": "<div>Each row in the table shows an example of duplicate data and the number of times it appears.</div>"}}, "a22ac83728b24648b17b8e5a9c57a53a": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "ca518b382279463fa2434c8da2e3a98e": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "af37828d2bd1484aaf549445be2028c4": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_f1d49b359b234f15986bc41e6fb6820f", "style": "IPY_MODEL_770b6b42e45248d5ad50522d1929b8d5", "value": "<style type=\"text/css\">\n#T_e6b13_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_e6b13_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_e6b13_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_e6b13_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_e6b13_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_e6b13_\">\n  <thead>\n    <tr>\n      <th class=\"blank\" >&nbsp;</th>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >entropy</th>\n      <th class=\"col_heading level0 col1\" >numParams</th>\n    </tr>\n    <tr>\n      <th class=\"index_name level0\" >Instances</th>\n      <th class=\"index_name level1\" >Number of Duplicates</th>\n      <th class=\"blank col0\" >&nbsp;</th>\n      <th class=\"blank col1\" >&nbsp;</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_e6b13_level0_row0\" class=\"row_heading level0 row0\" >2360, 6560, 11342, 2150, 3528, 1557, 810...</th>\n      <th id=\"T_e6b13_level1_row0\" class=\"row_heading level1 row0\" >13</th>\n      <td id=\"T_e6b13_row0_col0\" class=\"data row0 col0\" >-4.31</td>\n      <td id=\"T_e6b13_row0_col1\" class=\"data row0 col1\" >0</td>\n    </tr>\n    <tr>\n      <th id=\"T_e6b13_level0_row1\" class=\"row_heading level0 row1\" >4638, 4412, 1729, 2213, 1641, 2234, 6328...</th>\n      <th id=\"T_e6b13_level1_row1\" class=\"row_heading level1 row1\" >8</th>\n      <td id=\"T_e6b13_row1_col0\" class=\"data row1 col0\" >-4.57</td>\n      <td id=\"T_e6b13_row1_col1\" class=\"data row1 col1\" >4</td>\n    </tr>\n    <tr>\n      <th id=\"T_e6b13_level0_row2\" class=\"row_heading level0 row2\" >6774, 7528, 9592, 4634, 10046, 6504, 271...</th>\n      <th id=\"T_e6b13_level1_row2\" class=\"row_heading level1 row2\" >8</th>\n      <td id=\"T_e6b13_row2_col0\" class=\"data row2 col0\" >-4.49</td>\n      <td id=\"T_e6b13_row2_col1\" class=\"data row2 col1\" >8</td>\n    </tr>\n    <tr>\n      <th id=\"T_e6b13_level0_row3\" class=\"row_heading level0 row3\" >4047, 8391, 2499, 9932, 929, 11345, 9348...</th>\n      <th id=\"T_e6b13_level1_row3\" class=\"row_heading level1 row3\" >8</th>\n      <td id=\"T_e6b13_row3_col0\" class=\"data row3 col0\" >-4.25</td>\n      <td id=\"T_e6b13_row3_col1\" class=\"data row3 col1\" >0</td>\n    </tr>\n    <tr>\n      <th id=\"T_e6b13_level0_row4\" class=\"row_heading level0 row4\" >2984, 1670, 1020, 6666, 10923, 9138, 180...</th>\n      <th id=\"T_e6b13_level1_row4\" class=\"row_heading level1 row4\" >7</th>\n      <td id=\"T_e6b13_row4_col0\" class=\"data row4 col0\" >-4.65</td>\n      <td id=\"T_e6b13_row4_col1\" class=\"data row4 col1\" >5</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "f1d49b359b234f15986bc41e6fb6820f": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "770b6b42e45248d5ad50522d1929b8d5": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "b4e8e14734bd402dac741587f86e7876": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "006d515650a64914a40bd41817687a4e": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "081603f316ba42d1a5fb18b2f7e3874f"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 59-62

.. code-block:: default


    DataDuplicates(ignore_columns=["scrape_date"], n_to_show=10).run(phishing_dataset)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"9f9697ff6b3f4eb485bc3e729748d920": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_47122619d65d4620980a131c1b33c605", "IPY_MODEL_6feaace1615e42f6a4f5a12054fc3720", "IPY_MODEL_4d43b284e3be4d8699f3162487baa262", "IPY_MODEL_f4862a3454e74245a44c86bf84b312a1"], "layout": "IPY_MODEL_daf5465d4bde45d386aa42c25b2b325d"}}, "47122619d65d4620980a131c1b33c605": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_bf1d0b60ef9a4b3ca8f8fbe1819cc020", "style": "IPY_MODEL_47216f41a6e842c9a05b15fa5d4389bd", "value": "<h4><b>Data Duplicates</b></h4>"}}, "bf1d0b60ef9a4b3ca8f8fbe1819cc020": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "47216f41a6e842c9a05b15fa5d4389bd": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "6feaace1615e42f6a4f5a12054fc3720": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_cdaba4eae62e4bd68a03116f0d78ccaa", "style": "IPY_MODEL_8225e2cd70c94f65a53bfa76af5af7c9", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.9/checks_gallery/tabular/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "cdaba4eae62e4bd68a03116f0d78ccaa": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "8225e2cd70c94f65a53bfa76af5af7c9": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "4d43b284e3be4d8699f3162487baa262": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_e09303614bb74db1bfaf2b63380b9397", "style": "IPY_MODEL_36e9b75d6ab1421ab7abf80ea65cbb61"}}, "e09303614bb74db1bfaf2b63380b9397": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "36e9b75d6ab1421ab7abf80ea65cbb61": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "f4862a3454e74245a44c86bf84b312a1": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_7ca99688332048b698eaa9944ac0c63a", "IPY_MODEL_f0d18c294cc54a30a2d33165eac7814a", "IPY_MODEL_c060d0b7621d40e28fa806f83f2c32fd", "IPY_MODEL_d7e6610fd2e34ccaa9c1decc809523d4"], "layout": "IPY_MODEL_091beb37da6b4a5285a58a1e0f11c17b"}}, "7ca99688332048b698eaa9944ac0c63a": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_63aea184008148d4957a683cb62fe016", "style": "IPY_MODEL_3e51aea584d5430a9fdd8baacffc7601", "value": "<h5><b>Additional Outputs</b></h5>"}}, "63aea184008148d4957a683cb62fe016": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "3e51aea584d5430a9fdd8baacffc7601": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "f0d18c294cc54a30a2d33165eac7814a": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_981388ea2d9c454da8796f17e22b9bdf", "style": "IPY_MODEL_2a4dd66c21984ab496982bfb4a26f692", "value": "<div>0.22% of data samples are duplicates. </div>"}}, "981388ea2d9c454da8796f17e22b9bdf": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "2a4dd66c21984ab496982bfb4a26f692": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "c060d0b7621d40e28fa806f83f2c32fd": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_af3387b35b09472a99603fce460125d5", "style": "IPY_MODEL_45ea2986be634a1092d8f430d0e2a3cf", "value": "<div>Each row in the table shows an example of duplicate data and the number of times it appears.</div>"}}, "af3387b35b09472a99603fce460125d5": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "45ea2986be634a1092d8f430d0e2a3cf": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "d7e6610fd2e34ccaa9c1decc809523d4": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_f6fdff5f9fa9412eb0e7580a9eab326b", "style": "IPY_MODEL_14f41f5df64c4b8f9fcba2b3c6b13cd0", "value": "<style type=\"text/css\">\n#T_c177b_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_c177b_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_c177b_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_c177b_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_c177b_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_c177b_\">\n  <thead>\n    <tr>\n      <th class=\"blank\" >&nbsp;</th>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >target</th>\n      <th class=\"col_heading level0 col1\" >month</th>\n      <th class=\"col_heading level0 col2\" >ext</th>\n      <th class=\"col_heading level0 col3\" >urlLength</th>\n      <th class=\"col_heading level0 col4\" >numDigits</th>\n      <th class=\"col_heading level0 col5\" >numParams</th>\n      <th class=\"col_heading level0 col6\" >num_%20</th>\n      <th class=\"col_heading level0 col7\" >num_@</th>\n      <th class=\"col_heading level0 col8\" >entropy</th>\n      <th class=\"col_heading level0 col9\" >has_ip</th>\n      <th class=\"col_heading level0 col10\" >hasHttp</th>\n      <th class=\"col_heading level0 col11\" >hasHttps</th>\n      <th class=\"col_heading level0 col12\" >urlIsLive</th>\n      <th class=\"col_heading level0 col13\" >dsr</th>\n      <th class=\"col_heading level0 col14\" >dse</th>\n      <th class=\"col_heading level0 col15\" >bodyLength</th>\n      <th class=\"col_heading level0 col16\" >numTitles</th>\n      <th class=\"col_heading level0 col17\" >numImages</th>\n      <th class=\"col_heading level0 col18\" >numLinks</th>\n      <th class=\"col_heading level0 col19\" >specialChars</th>\n      <th class=\"col_heading level0 col20\" >scriptLength</th>\n      <th class=\"col_heading level0 col21\" >sbr</th>\n      <th class=\"col_heading level0 col22\" >bscr</th>\n      <th class=\"col_heading level0 col23\" >sscr</th>\n    </tr>\n    <tr>\n      <th class=\"index_name level0\" >Instances</th>\n      <th class=\"index_name level1\" >Number of Duplicates</th>\n      <th class=\"blank col0\" >&nbsp;</th>\n      <th class=\"blank col1\" >&nbsp;</th>\n      <th class=\"blank col2\" >&nbsp;</th>\n      <th class=\"blank col3\" >&nbsp;</th>\n      <th class=\"blank col4\" >&nbsp;</th>\n      <th class=\"blank col5\" >&nbsp;</th>\n      <th class=\"blank col6\" >&nbsp;</th>\n      <th class=\"blank col7\" >&nbsp;</th>\n      <th class=\"blank col8\" >&nbsp;</th>\n      <th class=\"blank col9\" >&nbsp;</th>\n      <th class=\"blank col10\" >&nbsp;</th>\n      <th class=\"blank col11\" >&nbsp;</th>\n      <th class=\"blank col12\" >&nbsp;</th>\n      <th class=\"blank col13\" >&nbsp;</th>\n      <th class=\"blank col14\" >&nbsp;</th>\n      <th class=\"blank col15\" >&nbsp;</th>\n      <th class=\"blank col16\" >&nbsp;</th>\n      <th class=\"blank col17\" >&nbsp;</th>\n      <th class=\"blank col18\" >&nbsp;</th>\n      <th class=\"blank col19\" >&nbsp;</th>\n      <th class=\"blank col20\" >&nbsp;</th>\n      <th class=\"blank col21\" >&nbsp;</th>\n      <th class=\"blank col22\" >&nbsp;</th>\n      <th class=\"blank col23\" >&nbsp;</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_c177b_level0_row0\" class=\"row_heading level0 row0\" >4696, 5398, 4719</th>\n      <th id=\"T_c177b_level1_row0\" class=\"row_heading level1 row0\" >3</th>\n      <td id=\"T_c177b_row0_col0\" class=\"data row0 col0\" >0</td>\n      <td id=\"T_c177b_row0_col1\" class=\"data row0 col1\" >6</td>\n      <td id=\"T_c177b_row0_col2\" class=\"data row0 col2\" >other</td>\n      <td id=\"T_c177b_row0_col3\" class=\"data row0 col3\" >123</td>\n      <td id=\"T_c177b_row0_col4\" class=\"data row0 col4\" >28</td>\n      <td id=\"T_c177b_row0_col5\" class=\"data row0 col5\" >4</td>\n      <td id=\"T_c177b_row0_col6\" class=\"data row0 col6\" >0</td>\n      <td id=\"T_c177b_row0_col7\" class=\"data row0 col7\" >0</td>\n      <td id=\"T_c177b_row0_col8\" class=\"data row0 col8\" >-4.91</td>\n      <td id=\"T_c177b_row0_col9\" class=\"data row0 col9\" >0</td>\n      <td id=\"T_c177b_row0_col10\" class=\"data row0 col10\" >True</td>\n      <td id=\"T_c177b_row0_col11\" class=\"data row0 col11\" >False</td>\n      <td id=\"T_c177b_row0_col12\" class=\"data row0 col12\" >False</td>\n      <td id=\"T_c177b_row0_col13\" class=\"data row0 col13\" >0</td>\n      <td id=\"T_c177b_row0_col14\" class=\"data row0 col14\" >0</td>\n      <td id=\"T_c177b_row0_col15\" class=\"data row0 col15\" >0</td>\n      <td id=\"T_c177b_row0_col16\" class=\"data row0 col16\" >0</td>\n      <td id=\"T_c177b_row0_col17\" class=\"data row0 col17\" >0</td>\n      <td id=\"T_c177b_row0_col18\" class=\"data row0 col18\" >0</td>\n      <td id=\"T_c177b_row0_col19\" class=\"data row0 col19\" >0</td>\n      <td id=\"T_c177b_row0_col20\" class=\"data row0 col20\" >0</td>\n      <td id=\"T_c177b_row0_col21\" class=\"data row0 col21\" >0.00</td>\n      <td id=\"T_c177b_row0_col22\" class=\"data row0 col22\" >0.00</td>\n      <td id=\"T_c177b_row0_col23\" class=\"data row0 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row1\" class=\"row_heading level0 row1\" >11342, 82</th>\n      <th id=\"T_c177b_level1_row1\" class=\"row_heading level1 row1\" >2</th>\n      <td id=\"T_c177b_row1_col0\" class=\"data row1 col0\" >0</td>\n      <td id=\"T_c177b_row1_col1\" class=\"data row1 col1\" >1</td>\n      <td id=\"T_c177b_row1_col2\" class=\"data row1 col2\" >html</td>\n      <td id=\"T_c177b_row1_col3\" class=\"data row1 col3\" >92</td>\n      <td id=\"T_c177b_row1_col4\" class=\"data row1 col4\" >2</td>\n      <td id=\"T_c177b_row1_col5\" class=\"data row1 col5\" >0</td>\n      <td id=\"T_c177b_row1_col6\" class=\"data row1 col6\" >0</td>\n      <td id=\"T_c177b_row1_col7\" class=\"data row1 col7\" >0</td>\n      <td id=\"T_c177b_row1_col8\" class=\"data row1 col8\" >-4.31</td>\n      <td id=\"T_c177b_row1_col9\" class=\"data row1 col9\" >0</td>\n      <td id=\"T_c177b_row1_col10\" class=\"data row1 col10\" >True</td>\n      <td id=\"T_c177b_row1_col11\" class=\"data row1 col11\" >False</td>\n      <td id=\"T_c177b_row1_col12\" class=\"data row1 col12\" >False</td>\n      <td id=\"T_c177b_row1_col13\" class=\"data row1 col13\" >0</td>\n      <td id=\"T_c177b_row1_col14\" class=\"data row1 col14\" >0</td>\n      <td id=\"T_c177b_row1_col15\" class=\"data row1 col15\" >149</td>\n      <td id=\"T_c177b_row1_col16\" class=\"data row1 col16\" >1</td>\n      <td id=\"T_c177b_row1_col17\" class=\"data row1 col17\" >0</td>\n      <td id=\"T_c177b_row1_col18\" class=\"data row1 col18\" >0</td>\n      <td id=\"T_c177b_row1_col19\" class=\"data row1 col19\" >25</td>\n      <td id=\"T_c177b_row1_col20\" class=\"data row1 col20\" >0</td>\n      <td id=\"T_c177b_row1_col21\" class=\"data row1 col21\" >0.00</td>\n      <td id=\"T_c177b_row1_col22\" class=\"data row1 col22\" >0.17</td>\n      <td id=\"T_c177b_row1_col23\" class=\"data row1 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row2\" class=\"row_heading level0 row2\" >790, 250</th>\n      <th id=\"T_c177b_level1_row2\" class=\"row_heading level1 row2\" >2</th>\n      <td id=\"T_c177b_row2_col0\" class=\"data row2 col0\" >0</td>\n      <td id=\"T_c177b_row2_col1\" class=\"data row2 col1\" >1</td>\n      <td id=\"T_c177b_row2_col2\" class=\"data row2 col2\" >php</td>\n      <td id=\"T_c177b_row2_col3\" class=\"data row2 col3\" >107</td>\n      <td id=\"T_c177b_row2_col4\" class=\"data row2 col4\" >4</td>\n      <td id=\"T_c177b_row2_col5\" class=\"data row2 col5\" >8</td>\n      <td id=\"T_c177b_row2_col6\" class=\"data row2 col6\" >0</td>\n      <td id=\"T_c177b_row2_col7\" class=\"data row2 col7\" >0</td>\n      <td id=\"T_c177b_row2_col8\" class=\"data row2 col8\" >-4.53</td>\n      <td id=\"T_c177b_row2_col9\" class=\"data row2 col9\" >0</td>\n      <td id=\"T_c177b_row2_col10\" class=\"data row2 col10\" >True</td>\n      <td id=\"T_c177b_row2_col11\" class=\"data row2 col11\" >False</td>\n      <td id=\"T_c177b_row2_col12\" class=\"data row2 col12\" >False</td>\n      <td id=\"T_c177b_row2_col13\" class=\"data row2 col13\" >1381</td>\n      <td id=\"T_c177b_row2_col14\" class=\"data row2 col14\" >79</td>\n      <td id=\"T_c177b_row2_col15\" class=\"data row2 col15\" >0</td>\n      <td id=\"T_c177b_row2_col16\" class=\"data row2 col16\" >1</td>\n      <td id=\"T_c177b_row2_col17\" class=\"data row2 col17\" >0</td>\n      <td id=\"T_c177b_row2_col18\" class=\"data row2 col18\" >0</td>\n      <td id=\"T_c177b_row2_col19\" class=\"data row2 col19\" >0</td>\n      <td id=\"T_c177b_row2_col20\" class=\"data row2 col20\" >0</td>\n      <td id=\"T_c177b_row2_col21\" class=\"data row2 col21\" >0.00</td>\n      <td id=\"T_c177b_row2_col22\" class=\"data row2 col22\" >0.00</td>\n      <td id=\"T_c177b_row2_col23\" class=\"data row2 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row3\" class=\"row_heading level0 row3\" >217, 6</th>\n      <th id=\"T_c177b_level1_row3\" class=\"row_heading level1 row3\" >2</th>\n      <td id=\"T_c177b_row3_col0\" class=\"data row3 col0\" >0</td>\n      <td id=\"T_c177b_row3_col1\" class=\"data row3 col1\" >1</td>\n      <td id=\"T_c177b_row3_col2\" class=\"data row3 col2\" >php</td>\n      <td id=\"T_c177b_row3_col3\" class=\"data row3 col3\" >107</td>\n      <td id=\"T_c177b_row3_col4\" class=\"data row3 col4\" >5</td>\n      <td id=\"T_c177b_row3_col5\" class=\"data row3 col5\" >8</td>\n      <td id=\"T_c177b_row3_col6\" class=\"data row3 col6\" >0</td>\n      <td id=\"T_c177b_row3_col7\" class=\"data row3 col7\" >0</td>\n      <td id=\"T_c177b_row3_col8\" class=\"data row3 col8\" >-4.52</td>\n      <td id=\"T_c177b_row3_col9\" class=\"data row3 col9\" >0</td>\n      <td id=\"T_c177b_row3_col10\" class=\"data row3 col10\" >True</td>\n      <td id=\"T_c177b_row3_col11\" class=\"data row3 col11\" >False</td>\n      <td id=\"T_c177b_row3_col12\" class=\"data row3 col12\" >False</td>\n      <td id=\"T_c177b_row3_col13\" class=\"data row3 col13\" >1381</td>\n      <td id=\"T_c177b_row3_col14\" class=\"data row3 col14\" >79</td>\n      <td id=\"T_c177b_row3_col15\" class=\"data row3 col15\" >0</td>\n      <td id=\"T_c177b_row3_col16\" class=\"data row3 col16\" >1</td>\n      <td id=\"T_c177b_row3_col17\" class=\"data row3 col17\" >0</td>\n      <td id=\"T_c177b_row3_col18\" class=\"data row3 col18\" >0</td>\n      <td id=\"T_c177b_row3_col19\" class=\"data row3 col19\" >0</td>\n      <td id=\"T_c177b_row3_col20\" class=\"data row3 col20\" >0</td>\n      <td id=\"T_c177b_row3_col21\" class=\"data row3 col21\" >0.00</td>\n      <td id=\"T_c177b_row3_col22\" class=\"data row3 col22\" >0.00</td>\n      <td id=\"T_c177b_row3_col23\" class=\"data row3 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row4\" class=\"row_heading level0 row4\" >763, 609</th>\n      <th id=\"T_c177b_level1_row4\" class=\"row_heading level1 row4\" >2</th>\n      <td id=\"T_c177b_row4_col0\" class=\"data row4 col0\" >0</td>\n      <td id=\"T_c177b_row4_col1\" class=\"data row4 col1\" >1</td>\n      <td id=\"T_c177b_row4_col2\" class=\"data row4 col2\" >php</td>\n      <td id=\"T_c177b_row4_col3\" class=\"data row4 col3\" >113</td>\n      <td id=\"T_c177b_row4_col4\" class=\"data row4 col4\" >6</td>\n      <td id=\"T_c177b_row4_col5\" class=\"data row4 col5\" >8</td>\n      <td id=\"T_c177b_row4_col6\" class=\"data row4 col6\" >0</td>\n      <td id=\"T_c177b_row4_col7\" class=\"data row4 col7\" >0</td>\n      <td id=\"T_c177b_row4_col8\" class=\"data row4 col8\" >-4.63</td>\n      <td id=\"T_c177b_row4_col9\" class=\"data row4 col9\" >0</td>\n      <td id=\"T_c177b_row4_col10\" class=\"data row4 col10\" >True</td>\n      <td id=\"T_c177b_row4_col11\" class=\"data row4 col11\" >False</td>\n      <td id=\"T_c177b_row4_col12\" class=\"data row4 col12\" >False</td>\n      <td id=\"T_c177b_row4_col13\" class=\"data row4 col13\" >1381</td>\n      <td id=\"T_c177b_row4_col14\" class=\"data row4 col14\" >79</td>\n      <td id=\"T_c177b_row4_col15\" class=\"data row4 col15\" >0</td>\n      <td id=\"T_c177b_row4_col16\" class=\"data row4 col16\" >1</td>\n      <td id=\"T_c177b_row4_col17\" class=\"data row4 col17\" >0</td>\n      <td id=\"T_c177b_row4_col18\" class=\"data row4 col18\" >0</td>\n      <td id=\"T_c177b_row4_col19\" class=\"data row4 col19\" >0</td>\n      <td id=\"T_c177b_row4_col20\" class=\"data row4 col20\" >0</td>\n      <td id=\"T_c177b_row4_col21\" class=\"data row4 col21\" >0.00</td>\n      <td id=\"T_c177b_row4_col22\" class=\"data row4 col22\" >0.00</td>\n      <td id=\"T_c177b_row4_col23\" class=\"data row4 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row5\" class=\"row_heading level0 row5\" >1557, 974</th>\n      <th id=\"T_c177b_level1_row5\" class=\"row_heading level1 row5\" >2</th>\n      <td id=\"T_c177b_row5_col0\" class=\"data row5 col0\" >0</td>\n      <td id=\"T_c177b_row5_col1\" class=\"data row5 col1\" >2</td>\n      <td id=\"T_c177b_row5_col2\" class=\"data row5 col2\" >html</td>\n      <td id=\"T_c177b_row5_col3\" class=\"data row5 col3\" >92</td>\n      <td id=\"T_c177b_row5_col4\" class=\"data row5 col4\" >2</td>\n      <td id=\"T_c177b_row5_col5\" class=\"data row5 col5\" >0</td>\n      <td id=\"T_c177b_row5_col6\" class=\"data row5 col6\" >0</td>\n      <td id=\"T_c177b_row5_col7\" class=\"data row5 col7\" >0</td>\n      <td id=\"T_c177b_row5_col8\" class=\"data row5 col8\" >-4.31</td>\n      <td id=\"T_c177b_row5_col9\" class=\"data row5 col9\" >0</td>\n      <td id=\"T_c177b_row5_col10\" class=\"data row5 col10\" >True</td>\n      <td id=\"T_c177b_row5_col11\" class=\"data row5 col11\" >False</td>\n      <td id=\"T_c177b_row5_col12\" class=\"data row5 col12\" >False</td>\n      <td id=\"T_c177b_row5_col13\" class=\"data row5 col13\" >0</td>\n      <td id=\"T_c177b_row5_col14\" class=\"data row5 col14\" >0</td>\n      <td id=\"T_c177b_row5_col15\" class=\"data row5 col15\" >149</td>\n      <td id=\"T_c177b_row5_col16\" class=\"data row5 col16\" >1</td>\n      <td id=\"T_c177b_row5_col17\" class=\"data row5 col17\" >0</td>\n      <td id=\"T_c177b_row5_col18\" class=\"data row5 col18\" >0</td>\n      <td id=\"T_c177b_row5_col19\" class=\"data row5 col19\" >25</td>\n      <td id=\"T_c177b_row5_col20\" class=\"data row5 col20\" >0</td>\n      <td id=\"T_c177b_row5_col21\" class=\"data row5 col21\" >0.00</td>\n      <td id=\"T_c177b_row5_col22\" class=\"data row5 col22\" >0.17</td>\n      <td id=\"T_c177b_row5_col23\" class=\"data row5 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row6\" class=\"row_heading level0 row6\" >2360, 2150</th>\n      <th id=\"T_c177b_level1_row6\" class=\"row_heading level1 row6\" >2</th>\n      <td id=\"T_c177b_row6_col0\" class=\"data row6 col0\" >0</td>\n      <td id=\"T_c177b_row6_col1\" class=\"data row6 col1\" >3</td>\n      <td id=\"T_c177b_row6_col2\" class=\"data row6 col2\" >html</td>\n      <td id=\"T_c177b_row6_col3\" class=\"data row6 col3\" >92</td>\n      <td id=\"T_c177b_row6_col4\" class=\"data row6 col4\" >2</td>\n      <td id=\"T_c177b_row6_col5\" class=\"data row6 col5\" >0</td>\n      <td id=\"T_c177b_row6_col6\" class=\"data row6 col6\" >0</td>\n      <td id=\"T_c177b_row6_col7\" class=\"data row6 col7\" >0</td>\n      <td id=\"T_c177b_row6_col8\" class=\"data row6 col8\" >-4.31</td>\n      <td id=\"T_c177b_row6_col9\" class=\"data row6 col9\" >0</td>\n      <td id=\"T_c177b_row6_col10\" class=\"data row6 col10\" >True</td>\n      <td id=\"T_c177b_row6_col11\" class=\"data row6 col11\" >False</td>\n      <td id=\"T_c177b_row6_col12\" class=\"data row6 col12\" >False</td>\n      <td id=\"T_c177b_row6_col13\" class=\"data row6 col13\" >0</td>\n      <td id=\"T_c177b_row6_col14\" class=\"data row6 col14\" >0</td>\n      <td id=\"T_c177b_row6_col15\" class=\"data row6 col15\" >149</td>\n      <td id=\"T_c177b_row6_col16\" class=\"data row6 col16\" >1</td>\n      <td id=\"T_c177b_row6_col17\" class=\"data row6 col17\" >0</td>\n      <td id=\"T_c177b_row6_col18\" class=\"data row6 col18\" >0</td>\n      <td id=\"T_c177b_row6_col19\" class=\"data row6 col19\" >25</td>\n      <td id=\"T_c177b_row6_col20\" class=\"data row6 col20\" >0</td>\n      <td id=\"T_c177b_row6_col21\" class=\"data row6 col21\" >0.00</td>\n      <td id=\"T_c177b_row6_col22\" class=\"data row6 col22\" >0.17</td>\n      <td id=\"T_c177b_row6_col23\" class=\"data row6 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row7\" class=\"row_heading level0 row7\" >2238, 2489</th>\n      <th id=\"T_c177b_level1_row7\" class=\"row_heading level1 row7\" >2</th>\n      <td id=\"T_c177b_row7_col0\" class=\"data row7 col0\" >0</td>\n      <td id=\"T_c177b_row7_col1\" class=\"data row7 col1\" >3</td>\n      <td id=\"T_c177b_row7_col2\" class=\"data row7 col2\" >php</td>\n      <td id=\"T_c177b_row7_col3\" class=\"data row7 col3\" >108</td>\n      <td id=\"T_c177b_row7_col4\" class=\"data row7 col4\" >3</td>\n      <td id=\"T_c177b_row7_col5\" class=\"data row7 col5\" >8</td>\n      <td id=\"T_c177b_row7_col6\" class=\"data row7 col6\" >0</td>\n      <td id=\"T_c177b_row7_col7\" class=\"data row7 col7\" >0</td>\n      <td id=\"T_c177b_row7_col8\" class=\"data row7 col8\" >-4.51</td>\n      <td id=\"T_c177b_row7_col9\" class=\"data row7 col9\" >0</td>\n      <td id=\"T_c177b_row7_col10\" class=\"data row7 col10\" >True</td>\n      <td id=\"T_c177b_row7_col11\" class=\"data row7 col11\" >False</td>\n      <td id=\"T_c177b_row7_col12\" class=\"data row7 col12\" >False</td>\n      <td id=\"T_c177b_row7_col13\" class=\"data row7 col13\" >1381</td>\n      <td id=\"T_c177b_row7_col14\" class=\"data row7 col14\" >79</td>\n      <td id=\"T_c177b_row7_col15\" class=\"data row7 col15\" >0</td>\n      <td id=\"T_c177b_row7_col16\" class=\"data row7 col16\" >1</td>\n      <td id=\"T_c177b_row7_col17\" class=\"data row7 col17\" >0</td>\n      <td id=\"T_c177b_row7_col18\" class=\"data row7 col18\" >0</td>\n      <td id=\"T_c177b_row7_col19\" class=\"data row7 col19\" >0</td>\n      <td id=\"T_c177b_row7_col20\" class=\"data row7 col20\" >0</td>\n      <td id=\"T_c177b_row7_col21\" class=\"data row7 col21\" >0.00</td>\n      <td id=\"T_c177b_row7_col22\" class=\"data row7 col22\" >0.00</td>\n      <td id=\"T_c177b_row7_col23\" class=\"data row7 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row8\" class=\"row_heading level0 row8\" >3192, 3444</th>\n      <th id=\"T_c177b_level1_row8\" class=\"row_heading level1 row8\" >2</th>\n      <td id=\"T_c177b_row8_col0\" class=\"data row8 col0\" >0</td>\n      <td id=\"T_c177b_row8_col1\" class=\"data row8 col1\" >4</td>\n      <td id=\"T_c177b_row8_col2\" class=\"data row8 col2\" >other</td>\n      <td id=\"T_c177b_row8_col3\" class=\"data row8 col3\" >123</td>\n      <td id=\"T_c177b_row8_col4\" class=\"data row8 col4\" >28</td>\n      <td id=\"T_c177b_row8_col5\" class=\"data row8 col5\" >4</td>\n      <td id=\"T_c177b_row8_col6\" class=\"data row8 col6\" >0</td>\n      <td id=\"T_c177b_row8_col7\" class=\"data row8 col7\" >0</td>\n      <td id=\"T_c177b_row8_col8\" class=\"data row8 col8\" >-4.92</td>\n      <td id=\"T_c177b_row8_col9\" class=\"data row8 col9\" >0</td>\n      <td id=\"T_c177b_row8_col10\" class=\"data row8 col10\" >True</td>\n      <td id=\"T_c177b_row8_col11\" class=\"data row8 col11\" >False</td>\n      <td id=\"T_c177b_row8_col12\" class=\"data row8 col12\" >False</td>\n      <td id=\"T_c177b_row8_col13\" class=\"data row8 col13\" >0</td>\n      <td id=\"T_c177b_row8_col14\" class=\"data row8 col14\" >0</td>\n      <td id=\"T_c177b_row8_col15\" class=\"data row8 col15\" >0</td>\n      <td id=\"T_c177b_row8_col16\" class=\"data row8 col16\" >0</td>\n      <td id=\"T_c177b_row8_col17\" class=\"data row8 col17\" >0</td>\n      <td id=\"T_c177b_row8_col18\" class=\"data row8 col18\" >0</td>\n      <td id=\"T_c177b_row8_col19\" class=\"data row8 col19\" >0</td>\n      <td id=\"T_c177b_row8_col20\" class=\"data row8 col20\" >0</td>\n      <td id=\"T_c177b_row8_col21\" class=\"data row8 col21\" >0.00</td>\n      <td id=\"T_c177b_row8_col22\" class=\"data row8 col22\" >0.00</td>\n      <td id=\"T_c177b_row8_col23\" class=\"data row8 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_c177b_level0_row9\" class=\"row_heading level0 row9\" >3498, 3277</th>\n      <th id=\"T_c177b_level1_row9\" class=\"row_heading level1 row9\" >2</th>\n      <td id=\"T_c177b_row9_col0\" class=\"data row9 col0\" >0</td>\n      <td id=\"T_c177b_row9_col1\" class=\"data row9 col1\" >4</td>\n      <td id=\"T_c177b_row9_col2\" class=\"data row9 col2\" >php</td>\n      <td id=\"T_c177b_row9_col3\" class=\"data row9 col3\" >93</td>\n      <td id=\"T_c177b_row9_col4\" class=\"data row9 col4\" >31</td>\n      <td id=\"T_c177b_row9_col5\" class=\"data row9 col5\" >1</td>\n      <td id=\"T_c177b_row9_col6\" class=\"data row9 col6\" >0</td>\n      <td id=\"T_c177b_row9_col7\" class=\"data row9 col7\" >0</td>\n      <td id=\"T_c177b_row9_col8\" class=\"data row9 col8\" >-4.93</td>\n      <td id=\"T_c177b_row9_col9\" class=\"data row9 col9\" >0</td>\n      <td id=\"T_c177b_row9_col10\" class=\"data row9 col10\" >True</td>\n      <td id=\"T_c177b_row9_col11\" class=\"data row9 col11\" >False</td>\n      <td id=\"T_c177b_row9_col12\" class=\"data row9 col12\" >False</td>\n      <td id=\"T_c177b_row9_col13\" class=\"data row9 col13\" >0</td>\n      <td id=\"T_c177b_row9_col14\" class=\"data row9 col14\" >0</td>\n      <td id=\"T_c177b_row9_col15\" class=\"data row9 col15\" >281</td>\n      <td id=\"T_c177b_row9_col16\" class=\"data row9 col16\" >0</td>\n      <td id=\"T_c177b_row9_col17\" class=\"data row9 col17\" >0</td>\n      <td id=\"T_c177b_row9_col18\" class=\"data row9 col18\" >0</td>\n      <td id=\"T_c177b_row9_col19\" class=\"data row9 col19\" >74</td>\n      <td id=\"T_c177b_row9_col20\" class=\"data row9 col20\" >142</td>\n      <td id=\"T_c177b_row9_col21\" class=\"data row9 col21\" >0.51</td>\n      <td id=\"T_c177b_row9_col22\" class=\"data row9 col22\" >0.26</td>\n      <td id=\"T_c177b_row9_col23\" class=\"data row9 col23\" >1.92</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "f6fdff5f9fa9412eb0e7580a9eab326b": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "14f41f5df64c4b8f9fcba2b3c6b13cd0": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "091beb37da6b4a5285a58a1e0f11c17b": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "daf5465d4bde45d386aa42c25b2b325d": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "9f9697ff6b3f4eb485bc3e729748d920"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 63-68

Define a Condition
==================
Now, we define a condition that enforce the ratio of duplicates to be 0. A condition
is deepchecks' way to validate model and data quality, and let you know if anything
goes wrong.

.. GENERATED FROM PYTHON SOURCE LINES 68-73

.. code-block:: default


    check = DataDuplicates()
    check.add_condition_ratio_less_or_equal(0)
    result = check.run(phishing_dataset)
    result.show(show_additional_outputs=False)





.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <link rel="icon" type="image/x-icon" href="https://deepchecks.com/wp-content/themes/deepchecks/assets/favicons/favicon.ico">
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"425101adcd824d819d5744e0433b597f": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_5663d9f242cd49e98de8eeeec6ddea6d", "IPY_MODEL_8fcad4a9db1b4f7f9379081e30039b39", "IPY_MODEL_11a5282f935f4026be2a42231bb61973"], "layout": "IPY_MODEL_a48196d9b7414512a3dae20d756bee15"}}, "5663d9f242cd49e98de8eeeec6ddea6d": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_8d3d3f68032b425ea4cd2dbd9e35d891", "style": "IPY_MODEL_65ac189dac974a13bb2c555ea988e770", "value": "<h4><b>Data Duplicates</b></h4>"}}, "8d3d3f68032b425ea4cd2dbd9e35d891": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "65ac189dac974a13bb2c555ea988e770": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "8fcad4a9db1b4f7f9379081e30039b39": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_a78dbd42947248a9ae8a5cff8b6a26ed", "style": "IPY_MODEL_2bf47a4cd37841ca9fa3a66103d8ae81", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.9/checks_gallery/tabular/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "a78dbd42947248a9ae8a5cff8b6a26ed": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "2bf47a4cd37841ca9fa3a66103d8ae81": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "11a5282f935f4026be2a42231bb61973": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_fadb3c8a4ed74bc79ba921598a794cad", "style": "IPY_MODEL_dc50cb86b62047d78573b0c0a9645cfd", "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_bf019_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_bf019_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_bf019_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_bf019_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_bf019_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_bf019_\">\n  <thead>\n    <tr>\n      <th class=\"col_heading level0 col0\" >Status</th>\n      <th class=\"col_heading level0 col1\" >Condition</th>\n      <th class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_bf019_row0_col0\" class=\"data row0 col0\" ><div style=\"color: orange;text-align: center;font-weight:bold\">!</div></td>\n      <td id=\"T_bf019_row0_col1\" class=\"data row0 col1\" >Duplicate data ratio is less or equal to 0%</td>\n      <td id=\"T_bf019_row0_col2\" class=\"data row0 col2\" >Found 0.0088% duplicate data</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "fadb3c8a4ed74bc79ba921598a794cad": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "dc50cb86b62047d78573b0c0a9645cfd": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "a48196d9b7414512a3dae20d756bee15": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "425101adcd824d819d5744e0433b597f"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  2.432 seconds)


.. _sphx_glr_download_checks_gallery_tabular_data_integrity_plot_data_duplicates.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_data_duplicates.py <plot_data_duplicates.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_data_duplicates.ipynb <plot_data_duplicates.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_